InnerProduct16bLayer functions

Functions to acceleratе InnerProduct16bLayer in Synet Framework. More...

Functions
SIMD_API void *	SimdSynetInnerProduct16bInit (size_t M, size_t N, size_t K, SimdTensorDataType typeA, SimdTensorDataType typeB, SimdTensorDataType typeC, SimdBool transB, SimdBool constB, SimdBool bias)
	Initilizes BF16 inner product (matrix mutiplication) algorithm. More...
SIMD_API size_t	SimdSynetInnerProduct16bInternalBufferSize (const void *context)
	Gets size in bytes of internal buffer used inside BF16 inner product algorithm. More...
SIMD_API size_t	SimdSynetInnerProduct16bExternalBufferSize (const void *context)
	Gets size in bytes of external buffer used in BF16 inner product algorithm. More...
SIMD_API const char *	SimdSynetInnerProduct16bInfo (const void *context)
	Gets string with description of internal implementation of BF16 inner product algorithm. More...
SIMD_API void	SimdSynetInnerProduct16bForward (void *context, const uint8_t *A, const uint8_t *B, uint8_t *buf, uint8_t *C)
	Performs forward propagation of BF16 inner product algorithm. More...

Detailed Description

Functions to acceleratе InnerProduct16bLayer in Synet Framework.

Function Documentation

SimdSynetInnerProduct16bInit()

void * SimdSynetInnerProduct16bInit	(	size_t	M,
		size_t	N,
		size_t	K,
		SimdTensorDataType	typeA,
		SimdTensorDataType	typeB,
		SimdTensorDataType	typeC,
		SimdBool	transB,
		SimdBool	constB,
		SimdBool	bias
	)

Initilizes BF16 inner product (matrix mutiplication) algorithm.

Algorithm's details (transpA = false, bias = true):

for(i = 0; i < M; ++i)
    for(j = 0; j < N; ++j)
    {
        C[i,j] = bias[j];
        for(k = 0; k < K; ++k)
            C[i,j] += A[i,k] * B[k,j];
    }

Parameters

[in]	M	- a height of A and height of C matrices.
[in]	N	- a width of B and width of C matrices.
[in]	K	- a width of A and height of B matrices.
[in]	typeA	- a type of A matrix. It can be FP32 or BF16.
[in]	typeB	- a type of B matrix. It can be FP32 or BF16.
[in]	typeC	- a type of C matrix. It can be FP32 or BF16.
[in]	transB	- a transpose matrix B before multiplication.
[in]	constB	- a matrix B is constant.
[in]	bias	- a flag to add bias to output matrix C.

Returns

a pointer to BF32 inner product context. On error it returns NULL. It must be released with using of function SimdRelease. This pointer is used in functions SimdSynetInnerProduct16bInternalBufferSize, SimdSynetInnerProduct16bExternalBufferSize, SimdSynetInnerProduct16bInfo, SimdSynetInnerProduct16bSetParams and SimdSynetInnerProduct16bForward.

SimdSynetInnerProduct16bInternalBufferSize()

size_t SimdSynetInnerProduct16bInternalBufferSize

(

const void *

context

)

Gets size in bytes of internal buffer used inside BF16 inner product algorithm.

Parameters

[in]

context

- a pointer to BF16 inner product context. It must be created by function SimdSynetInnerProduct16bInit and released by function SimdRelease.

Returns

size in bytes of internal buffer used inside BF16 inner product algorithm.

SimdSynetInnerProduct16bExternalBufferSize()

size_t SimdSynetInnerProduct16bExternalBufferSize

(

const void *

context

)

Gets size in bytes of external buffer used in BF16 inner product algorithm.

Parameters

[in]

context

- a pointer to BF16 inner product context. It must be created by function SimdSynetInnerProduct16bInit and released by function SimdRelease.

Returns

size in bytes of external buffer used in BF16 inner product algorithm.

SimdSynetInnerProduct16bInfo()

const char * SimdSynetInnerProduct16bInfo

(

const void *

context

)

Gets string with description of internal implementation of BF16 inner product algorithm.

Parameters

[in]

context

- a pointer to BF16 inner product context. It must be created by function SimdSynetInnerProduct16bInit and released by function SimdRelease.

Returns

string with description of internal implementation of BF16 inner product algorithm.

SimdSynetInnerProduct16bForward()

void SimdSynetInnerProduct16bForward	(	void *	context,
		const uint8_t *	A,
		const uint8_t *	B,
		uint8_t *	buf,
		uint8_t *	C
	)

Performs forward propagation of BF16 inner product algorithm.

Parameters

[in]	context	- a pointer to BF16 inner product context. It must be created by function SimdSynetInnerProduct16bInit and released by function SimdRelease.
[in]	A	- a pointer to A matrix.
[in]	B	- a pointer to B matrix. Can be NULL if B is constant matrix. In that case you have to set B in function SimdSynetInnerProduct16bSetParams.
[out]	buf	- a pointer to external buffer. The size of the external temporary buffer is determined by function SimdSynetInnerProduct16bExternalBufferSize. Can be NULL (it causes usage of internal buffer).
[out]	C	- a pointer to output matrix.