A framework to accelerate INT8 merged convolution in Synet Framework. More...

Functions
SIMD_API void *	SimdSynetMergedConvolution8iInit (size_t batch, const SimdConvolutionParameters *convs, size_t count, SimdSynetCompatibilityType compatibility)
	Initializes an INT8 merged convolution context. More...

SIMD_API size_t	SimdSynetMergedConvolution8iExternalBufferSize (const void *context)
	Gets the size in bytes of the optional external temporary buffer for INT8 merged convolution. More...

SIMD_API size_t	SimdSynetMergedConvolution8iInternalBufferSize (const void *context)
	Gets the size in bytes of internal storage used by an INT8 merged convolution context. More...

SIMD_API const char *	SimdSynetMergedConvolution8iInfo (const void *context)
	Gets a textual description of the selected INT8 merged convolution implementation. More...

SIMD_API void	SimdSynetMergedConvolution8iSetParams (void context, const float const weight, SimdBool internal, const float const bias, const float const params, const float const stats)
	Sets FP32 weights, biases, activation parameters and quantization statistics for INT8 merged convolution. More...

SIMD_API void	SimdSynetMergedConvolution8iForward (void context, const uint8_t src, uint8_t buf, uint8_t dst)
	Performs forward propagation through the fused INT8 merged convolution sequence. More...

Detailed Description

A framework to accelerate INT8 merged convolution in Synet Framework.

Function Documentation

◆ SimdSynetMergedConvolution8iInit()

void * SimdSynetMergedConvolution8iInit	(	size_t	batch,
		const SimdConvolutionParameters *	convs,
		size_t	count,
		SimdSynetCompatibilityType	compatibility
	)

Initializes an INT8 merged convolution context.

The context fuses a sequence of two or three NHWC convolutions into one forward call: convolution + depthwise convolution, depthwise convolution + convolution, or convolution + depthwise convolution + convolution. Source and destination tensors can be FP32 or UINT8 according to the corresponding SimdConvolutionParameters fields. Ordinary convolutions use 1x1 or 3x3 kernels, depthwise convolutions use 3x3, 5x5 or 7x7 kernels; kernels and strides must be square, dilation must be 1 and stride must be 1, 2 or 3. Ordinary convolution weights are quantized to INT8 by SimdSynetMergedConvolution8iSetParams.

Parameters

[in]	batch	- a batch size.
[in]	convs	- an array with convolution parameters in execution order.
[in]	count	- a number of merged convolutions. It must be 2 or 3.
[in]	compatibility	- calculation compatibility flags (see SimdSynetCompatibilityType).

Returns: a pointer to INT8 merged convolution context. On error it returns NULL. It must be released with function SimdRelease. This pointer is used in functions SimdSynetMergedConvolution8iExternalBufferSize, SimdSynetMergedConvolution8iInternalBufferSize, SimdSynetMergedConvolution8iInfo, SimdSynetMergedConvolution8iSetParams and SimdSynetMergedConvolution8iForward.

◆ SimdSynetMergedConvolution8iExternalBufferSize()

size_t SimdSynetMergedConvolution8iExternalBufferSize ( const void * context )

Gets the size in bytes of the optional external temporary buffer for INT8 merged convolution.

Parameters

[in] context - a pointer to INT8 merged convolution context. It must be created by function SimdSynetMergedConvolution8iInit and released by function SimdRelease.

Returns: size in bytes of the external temporary buffer passed to SimdSynetMergedConvolution8iForward.

◆ SimdSynetMergedConvolution8iInternalBufferSize()

size_t SimdSynetMergedConvolution8iInternalBufferSize ( const void * context )

Gets the size in bytes of internal storage used by an INT8 merged convolution context.

Parameters

[in] context - a pointer to INT8 merged convolution context. It must be created by function SimdSynetMergedConvolution8iInit and released by function SimdRelease.

Returns: size in bytes of internal temporary storage, quantized/reordered weights, conversion parameters, biases and activation parameters.

◆ SimdSynetMergedConvolution8iInfo()

const char * SimdSynetMergedConvolution8iInfo ( const void * context )

Gets a textual description of the selected INT8 merged convolution implementation.

Parameters

[in] context - a pointer to INT8 merged convolution context. It must be created by function SimdSynetMergedConvolution8iInit and released by function SimdRelease.

Returns: a zero-terminated string with the selected implementation name.

◆ SimdSynetMergedConvolution8iSetParams()

void SimdSynetMergedConvolution8iSetParams	(	void *	context,
		const float const	weight,
		SimdBool *	internal,
		const float const	bias,
		const float const	params,
		const float const	stats
	)

Sets FP32 weights, biases, activation parameters and quantization statistics for INT8 merged convolution.

Parameters

[in,out]	context	- a pointer to INT8 merged convolution context. It must be created by function SimdSynetMergedConvolution8iInit and released by function SimdRelease.
[in]	weight	- an array of pointers to FP32 convolution weights. The array size must be equal to the number of merged convolutions.
[out]	internal	- an array of flags set to SimdTrue when the corresponding weights are stored in the context after quantization/reordering. The array size must be equal to the number of merged convolutions. Can be NULL.
[in]	bias	- an array of pointers to FP32 bias arrays, one per convolution. Each pointer can be NULL.
[in]	params	- an array of pointers to activation parameters (see SimdConvolutionActivationType), one per convolution. Each pointer can be NULL for activations that do not use parameters.
[in]	stats	- an array of six pointers to FP32 per-channel statistics: input min/max (stats[0], stats[1]), intermediate min/max before the last convolution (stats[2], stats[3]) and output min/max (stats[4], stats[5]).

◆ SimdSynetMergedConvolution8iForward()

void SimdSynetMergedConvolution8iForward	(	void *	context,
		const uint8_t *	src,
		uint8_t *	buf,
		uint8_t *	dst
	)

Performs forward propagation through the fused INT8 merged convolution sequence.

Parameters

[in]	context	- a pointer to INT8 merged convolution context. It must be created by function SimdSynetMergedConvolution8iInit and released by function SimdRelease.
[in]	src	- a pointer to the input tensor bytes. The tensor type is determined by convs[0].srcT (FP32 or UINT8).
[out]	buf	- a pointer to an external temporary byte buffer. Its size in bytes is determined by function SimdSynetMergedConvolution8iExternalBufferSize. Can be NULL (it causes usage of internal buffer).
[out]	dst	- a pointer to the output tensor bytes. The tensor type is determined by convs[count - 1].dstT (FP32 or UINT8).

Simd Library Documentation.

Functions

Detailed Description

Function Documentation

◆ SimdSynetMergedConvolution8iInit()

◆ SimdSynetMergedConvolution8iExternalBufferSize()

◆ SimdSynetMergedConvolution8iInternalBufferSize()

◆ SimdSynetMergedConvolution8iInfo()

◆ SimdSynetMergedConvolution8iSetParams()

◆ SimdSynetMergedConvolution8iForward()