Quantized merged convolution framework

A framework to accelerate Quantized merged convolution in Synet Framework. More...

Functions
SIMD_API void *	SimdSynetQuantizedMergedConvolutionInit (size_t batch, const SimdConvolutionParameters *convs, size_t count, int add)
	Initializes a chain of merged UINT8-to-UINT8 quantized convolutions. More...
SIMD_API size_t	SimdSynetQuantizedMergedConvolutionExternalBufferSize (const void *context)
	Gets size in bytes of external temporary buffer required for quantized merged convolution. More...
SIMD_API size_t	SimdSynetQuantizedMergedConvolutionInternalBufferSize (const void *context)
	Gets size in bytes of internal buffers allocated by quantized merged convolution context. More...
SIMD_API const char *	SimdSynetQuantizedMergedConvolutionInfo (const void *context)
	Gets description of selected quantized merged convolution implementation. More...
SIMD_API void	SimdSynetQuantizedMergedConvolutionSetParams (void *context, const float *ioScale, const uint8_t *ioZero, const int8_t *const *weight, const float *const *weightScale, const int32_t *const *bias)
	Sets weights, biases and quantization parameters for quantized merged convolution. More...
SIMD_API void	SimdSynetQuantizedMergedConvolutionForward (void *context, const uint8_t *src, uint8_t *buf, uint8_t *dst)
	Performs forward propagation of quantized merged convolution. More...

Detailed Description

A framework to accelerate Quantized merged convolution in Synet Framework.

Function Documentation

SimdSynetQuantizedMergedConvolutionInit()

void * SimdSynetQuantizedMergedConvolutionInit	(	size_t	batch,
		const SimdConvolutionParameters *	convs,
		size_t	count,
		int	add
	)

Initializes a chain of merged UINT8-to-UINT8 quantized convolutions.

The merged chain contains 2 or 3 NHWC convolutions with UINT8 source and destination tensors, INT8 weights and per-layer quantization parameters. Supported patterns are pointwise-depthwise, depthwise-pointwise and pointwise-depthwise-pointwise. If add is non-zero for a 3-convolution chain, the final output is requantized residual sum of the convolution output and the original input.

Parameters

[in]	batch	- a batch size.
[in]	convs	- an array with convolution parameters. The array size must be equal to count.
[in]	count	- a number of merged convolutions. It must be 2 or 3.
[in]	add	- a residual addition mode: 0 disables addition, 1 adds output to source, 2 adds source to output.

Returns

a pointer to Quantized merged convolution context. On error it returns NULL. It must be released with using of function SimdRelease. This pointer is used in functions SimdSynetQuantizedMergedConvolutionExternalBufferSize, SimdSynetQuantizedMergedConvolutionInternalBufferSize, SimdSynetQuantizedMergedConvolutionInfo, SimdSynetQuantizedMergedConvolutionSetParams and SimdSynetQuantizedMergedConvolutionForward.

SimdSynetQuantizedMergedConvolutionExternalBufferSize()

size_t SimdSynetQuantizedMergedConvolutionExternalBufferSize

(

const void *

context

)

Gets size in bytes of external temporary buffer required for quantized merged convolution.

Parameters

[in]

context

- a pointer to Quantized merged convolution context. It must be created by function SimdSynetQuantizedMergedConvolutionInit and released by function SimdRelease.

Returns

size in bytes of external temporary buffer required by SimdSynetQuantizedMergedConvolutionForward.

SimdSynetQuantizedMergedConvolutionInternalBufferSize()

size_t SimdSynetQuantizedMergedConvolutionInternalBufferSize

(

const void *

context

)

Gets size in bytes of internal buffers allocated by quantized merged convolution context.

Parameters

[in]

context

- a pointer to Quantized merged convolution context. It must be created by function SimdSynetQuantizedMergedConvolutionInit and released by function SimdRelease.

Returns

size in bytes of internal buffers used to store reordered weights, biases, norms, zero points and an optional fallback temporary buffer.

SimdSynetQuantizedMergedConvolutionInfo()

const char * SimdSynetQuantizedMergedConvolutionInfo

(

const void *

context

)

Gets description of selected quantized merged convolution implementation.

Parameters

[in]

context

- a pointer to Quantized merged convolution context. It must be created by function SimdSynetQuantizedMergedConvolutionInit and released by function SimdRelease.

Returns

string with description of selected implementation (extension and algorithm name).

SimdSynetQuantizedMergedConvolutionSetParams()

void SimdSynetQuantizedMergedConvolutionSetParams	(	void *	context,
		const float *	ioScale,
		const uint8_t *	ioZero,
		const int8_t *const *	weight,
		const float *const *	weightScale,
		const int32_t *const *	bias
	)

Sets weights, biases and quantization parameters for quantized merged convolution.

Arrays weight, weightScale and bias contain one pointer per merged convolution. The ioScale and ioZero arrays contain quantization parameters for every edge between convolutions: input, intermediate outputs and final output. When residual addition is enabled, one additional scale and zero point are used for the residual-sum output.

Parameters

[in,out]	context	- a pointer to Quantized merged convolution context. It must be created by function SimdSynetQuantizedMergedConvolutionInit and released by function SimdRelease.
[in]	ioScale	- a pointer to FP32 input/intermediate/output tensor scales.
[in]	ioZero	- a pointer to UINT8 input/intermediate/output tensor zero points.
[in]	weight	- an array of pointers to INT8 convolution weights.
[in]	weightScale	- an array of pointers to per-output-channel FP32 weight scales.
[in]	bias	- an array of pointers to per-output-channel INT32 biases. Individual pointers can be NULL.

SimdSynetQuantizedMergedConvolutionForward()

void SimdSynetQuantizedMergedConvolutionForward	(	void *	context,
		const uint8_t *	src,
		uint8_t *	buf,
		uint8_t *	dst
	)

Performs forward propagation of quantized merged convolution.

Parameters

[in]	context	- a pointer to Quantized merged convolution context. It must be created by function SimdSynetQuantizedMergedConvolutionInit and released by function SimdRelease.
[in]	src	- a pointer to UINT8 input tensor of the first convolution.
[out]	buf	- a pointer to external temporary buffer. Its size is determined by function SimdSynetQuantizedMergedConvolutionExternalBufferSize. Can be NULL (then context uses an internal buffer).
[out]	dst	- a pointer to UINT8 output tensor of the last convolution or residual sum.