DynamicQuantizedInnerProductLayer functions

A framework to accelerate DynamicQuantizedInnerProductLayer in Synet Framework. More...

Functions
SIMD_API void *	SimdSynetDynamicQuantizedInnerProductInit (size_t M, size_t N, size_t K, SimdBool bias, SimdConvolutionActivationType activation)
	Initializes dynamic quantized inner product (matrix multiplication) algorithm for FP32 input, INT8 weight and FP32 output. More...
SIMD_API size_t	SimdSynetDynamicQuantizedInnerProductInternalBufferSize (const void *context)
	Gets size in bytes of internal buffers allocated by dynamic quantized inner product context. More...
SIMD_API size_t	SimdSynetDynamicQuantizedInnerProductExternalBufferSize (const void *context)
	Gets size in bytes of external temporary buffer required for dynamic quantized inner product. More...
SIMD_API const char *	SimdSynetDynamicQuantizedInnerProductInfo (const void *context)
	Gets description of selected dynamic quantized inner product implementation. More...
SIMD_API void	SimdSynetDynamicQuantizedInnerProductSetParams (void *context, const int8_t *weight, const float *scale, const float *bias, const float *params)
	Sets constant matrix B, bias and dynamic quantization parameters for quantized inner product. More...

Detailed Description

A framework to accelerate DynamicQuantizedInnerProductLayer in Synet Framework.

Function Documentation

SimdSynetDynamicQuantizedInnerProductInit()

void * SimdSynetDynamicQuantizedInnerProductInit	(	size_t	M,
		size_t	N,
		size_t	K,
		SimdBool	bias,
		SimdConvolutionActivationType	activation
	)

Initializes dynamic quantized inner product (matrix multiplication) algorithm for FP32 input, INT8 weight and FP32 output.

Note

This function has a C++ wrapper: Simd::SynetDynamicQuantizedInnerProduct.

Parameters

[in]	M	- a height of A and height of C matrices.
[in]	N	- a width of B and width of C matrices.
[in]	K	- a width of A and height of B matrices.
[in]	bias	- a flag to add bias to output matrix C.
[in]	activation	- an activation function type applied after the rest operations.

Returns

a pointer to quantized inner product context. On error it returns NULL. It must be released with using of function SimdRelease. This pointer is used in functions SimdSynetDynamicQuantizedInnerProductInternalBufferSize, SimdSynetDynamicQuantizedInnerProductExternalBufferSize, SimdSynetDynamicQuantizedInnerProductInfo, SimdSynetDynamicQuantizedInnerProductSetParams and SimdSynetDynamicQuantizedInnerProductForward.

SimdSynetDynamicQuantizedInnerProductInternalBufferSize()

size_t SimdSynetDynamicQuantizedInnerProductInternalBufferSize

(

const void *

context

)

Gets size in bytes of internal buffers allocated by dynamic quantized inner product context.

Note

This function has a C++ wrapper: Simd::SynetDynamicQuantizedInnerProduct.

Parameters

[in]

context

- a pointer to dynamic quantized inner product context. It must be created by function SimdSynetDynamicQuantizedInnerProductInit and released by function SimdRelease.

Returns

size in bytes of internal buffers used inside the algorithm.

SimdSynetDynamicQuantizedInnerProductExternalBufferSize()

size_t SimdSynetDynamicQuantizedInnerProductExternalBufferSize

(

const void *

context

)

Gets size in bytes of external temporary buffer required for dynamic quantized inner product.

Note

This function has a C++ wrapper: Simd::SynetDynamicQuantizedInnerProduct.

Parameters

[in]

context

- a pointer to dynamic quantized inner product context. It must be created by function SimdSynetDynamicQuantizedInnerProductInit and released by function SimdRelease.

Returns

size in bytes of external temporary buffer required by SimdSynetDynamicQuantizedInnerProductForward.

SimdSynetDynamicQuantizedInnerProductInfo()

const char * SimdSynetDynamicQuantizedInnerProductInfo

(

const void *

context

)

Gets description of selected dynamic quantized inner product implementation.

Note

This function has a C++ wrapper: Simd::SynetDynamicQuantizedInnerProduct.

Parameters

[in]

context

- a pointer to dynamic quantized inner product context. It must be created by function SimdSynetDynamicQuantizedInnerProductInit and released by function SimdRelease.

Returns

string with description of selected implementation (extension and algorithm name).

SimdSynetDynamicQuantizedInnerProductSetParams()

void SimdSynetDynamicQuantizedInnerProductSetParams	(	void *	context,
		const int8_t *	weight,
		const float *	scale,
		const float *	bias,
		const float *	params
	)

Sets constant matrix B, bias and dynamic quantization parameters for quantized inner product.

Note

This function has a C++ wrapper: Simd::SynetDynamicQuantizedInnerProduct.

Parameters

[in,out]	context	- a pointer to dynamic quantized inner product context. It must be created by function SimdSynetDynamicQuantizedInnerProductInit and released by function SimdRelease.
[in]	weight	- a pointer to constant INT8 B matrix. The size of the array must be equal to K * N;
[in]	scale	- a pointer to per-output-channel FP32 scales of B matrix. The size of the array must be equal to N.
[in]	bias	- a pointer to per-output-channel FP32 bias. Can be NULL. Otherwise the size of the array must be equal to N.
[in]	params	- a pointer to FP32 activation parameters (see SimdConvolutionActivationType). Can be NULL.