SynetConvolution8i Class Reference

The SynetConvolution8i class is a C++ wrapper of INT8 convolution. More...

#include <SimdSynet.hpp>

Public Member Functions
	SynetConvolution8i ()
virtual	~SynetConvolution8i ()
SIMD_INLINE void	Init (size_t batch, const SimdConvolutionParameters *conv, SimdSynetCompatibilityType compatibility=SimdSynetCompatibilityDefault)
SIMD_INLINE bool	Enable () const
SIMD_INLINE size_t	ExternalBufferSize () const
SIMD_INLINE size_t	InternalBufferSize () const
SIMD_INLINE const char *	Info () const
SIMD_INLINE void	SetParams (const float *weight, const float *bias, const float *params, const float *const *stats)
SIMD_INLINE void	Forward (const uint8_t *src, uint8_t *buf, uint8_t *dst)
SIMD_INLINE void	Clear ()

Detailed Description

The SynetConvolution8i class is a C++ wrapper of INT8 convolution.

The class wraps C API functions SimdSynetConvolution8iInit, SimdSynetConvolution8iExternalBufferSize, SimdSynetConvolution8iInternalBufferSize, SimdSynetConvolution8iInfo, SimdSynetConvolution8iSetParams and SimdSynetConvolution8iForward. It convolves each image in the batch with INT8 weights, optionally adds bias and applies activation. Source and destination tensors can be FP32 or UINT8:

if(srcT == SimdTensorData32f)
    src8u = restrict(round(src32f*srcScale[c] + srcShift[c]), srcLower, srcUpper);
sum = convolution_int32(src8u, weight8i, zero);
value = Activate(sum*norm[dc] + bias[dc], activation, params);
dst[outputOffset] = dstT == SimdTensorData8u ?
    restrict(round(value*dstScale[dc] + dstShift[dc]), dstLower, dstUpper) : value;

The exact offsets depend on tensor format, padding, dilation, stride and group. The current implementation supports FP32 or UINT8 source and destination tensors with matching NCHW or NHWC format. The destination spatial size must match convolution parameters:

dstH = (srcH + padY + padH - (dilationY*(kernelY - 1) + 1)) / strideY + 1
dstW = (srcW + padX + padW - (dilationX*(kernelX - 1) + 1)) / strideX + 1

Compatibility flags select precise, overflow or narrowed INT8 calculation mode. Narrowed mode uses unsigned range [0, 180] and signed range [-90, 90]; otherwise ranges are [0, 255] and [-128, 127].

Call Init() and SetParams() before Forward(). Use Enable() to check that a context was created. The context is released by Clear() or by the destructor.

Using example:

#include "Simd/SimdSynet.hpp"

int main()
{
    const size_t batch = 1, srcC = 4, srcH = 8, srcW = 8, dstC = 8;
    const SimdSynetCompatibilityType compatibility = (SimdSynetCompatibilityType)(SimdSynetCompatibility8iNarrowed | SimdSynetCompatibilityFmaUse);
    SimdConvolutionParameters conv = {};
    conv.srcC = srcC;
    conv.srcH = srcH;
    conv.srcW = srcW;
    conv.srcT = SimdTensorData32f;
    conv.srcF = SimdTensorFormatNhwc;
    conv.dstC = dstC;
    conv.kernelY = 3;
    conv.kernelX = 3;
    conv.dilationY = 1;
    conv.dilationX = 1;
    conv.strideY = 1;
    conv.strideX = 1;
    conv.padY = 1;
    conv.padX = 1;
    conv.padH = 1;
    conv.padW = 1;
    conv.group = 1;
    conv.activation = SimdConvolutionActivationIdentity;
    conv.dstH = (conv.srcH + conv.padY + conv.padH - (conv.dilationY * (conv.kernelY - 1) + 1)) / conv.strideY + 1;
    conv.dstW = (conv.srcW + conv.padX + conv.padW - (conv.dilationX * (conv.kernelX - 1) + 1)) / conv.strideX + 1;
    conv.dstT = SimdTensorData32f;
    conv.dstF = SimdTensorFormatNhwc;

    std::vector<float> src(batch * srcH * srcW * srcC);
    std::vector<float> weight(conv.kernelY * conv.kernelX * srcC * dstC / conv.group);
    std::vector<float> bias(dstC, 0.0f);
    std::vector<float> srcMin(srcC, -1.0f), srcMax(srcC, 1.0f);
    std::vector<float> dstMin(dstC, -1.0f), dstMax(dstC, 1.0f);
    std::vector<float> dst(batch * conv.dstH * conv.dstW * dstC, 0.0f);
    const float * stats[4] = { srcMin.data(), srcMax.data(), dstMin.data(), dstMax.data() };
    for (size_t i = 0; i < src.size(); ++i)
        src[i] = float(i) * 0.01f;
    for (size_t i = 0; i < weight.size(); ++i)
        weight[i] = float(i) * 0.02f;

    Simd::SynetConvolution8i convolution;
    convolution.Init(batch, &conv, compatibility);
    if (convolution.Enable())
    {
        convolution.SetParams(weight.data(), bias.data(), NULL, stats);
        convolution.Forward((const uint8_t*)src.data(), NULL, (uint8_t*)dst.data());
    }

    return 0;
}

Constructor & Destructor Documentation

SynetConvolution8i()

SynetConvolution8i

(

)

Creates a new empty SynetConvolution8i class.

~SynetConvolution8i()

virtual ~SynetConvolution8i ( )

virtual

SynetConvolution8i class destructor. Releases internal context.

Member Function Documentation

Init()

SIMD_INLINE void Init	(	size_t	batch,
		const SimdConvolutionParameters *	conv,
		SimdSynetCompatibilityType	compatibility = SimdSynetCompatibilityDefault
	)

Initializes (or re-initializes) an INT8 convolution context.

Creates an internal context with using of function SimdSynetConvolution8iInit. The context is recreated only if batch size, convolution parameters or compatibility flags were changed.

Note

This function is a C++ wrapper for function SimdSynetConvolution8iInit.

Parameters

[in]	batch	- a batch size.
[in]	conv	- a pointer to convolution parameters. Source and destination tensor types must be FP32 or UINT8.
[in]	compatibility	- calculation compatibility flags. They select precise, overflow or narrowed INT8 calculation mode. Narrowed mode uses unsigned range [0, 180] and signed range [-90, 90]; otherwise ranges are [0, 255] and [-128, 127].

Enable()

SIMD_INLINE bool Enable

(

)

const

Checks that the internal convolution context was created.

Returns

true if the context exists and Forward() can be called.

ExternalBufferSize()

SIMD_INLINE size_t ExternalBufferSize

(

)

const

Gets the size in bytes of caller-provided temporary buffer for INT8 convolution.

The returned value is a number of bytes. It depends on the implementation selected during initialization and can be used when allocating the buf argument of Forward(). The buffer can contain temporary UINT8 source conversion data, im2col/padded input data, INT32 sums and temporary FP32 output data.

Note

This function is a C++ wrapper for function SimdSynetConvolution8iExternalBufferSize.

Returns

a number of bytes required for external temporary buffer.

InternalBufferSize()

SIMD_INLINE size_t InternalBufferSize

(

)

const

Gets the size in bytes of internal storage used by the convolution context.

The returned value reports internal storage tracked by the selected implementation, including internal temporary buffers, quantized/reordered INT8 weights, source and destination conversion parameters, normalization, bias and activation parameters.

Note

This function is a C++ wrapper for function SimdSynetConvolution8iInternalBufferSize.

Returns

a number of bytes used by internal buffers.

Info()

SIMD_INLINE const char * Info

(

)

const

Gets a short description of the selected INT8 convolution implementation.

The returned string contains the implementation extension and algorithm name, for example a GEMM, NHWC direct or NHWC depthwise variant, with a suffix for precise, overflow or narrowed mode when applicable. The returned pointer is owned by the context and remains valid until the next call of this function or until the context is released.

Note

This function is a C++ wrapper for function SimdSynetConvolution8iInfo.

Returns

a string with description of internal implementation. NULL if the context was not created.

SetParams()

SIMD_INLINE void SetParams	(	const float *	weight,
		const float *	bias,
		const float *	params,
		const float *const *	stats
	)

Sets weights, bias, activation parameters and tensor statistics for INT8 convolution.

This function must be called before Forward(). The weight array contains FP32 convolution weights with kernelY*kernelX*srcC*dstC/group elements. Source statistics (stats[0], stats[1], each with srcC elements) define per-channel source quantization parameters; destination statistics (stats[2], stats[3], each with dstC elements) define per-channel output quantization parameters. The selected implementation converts weights to INT8, may reorder them, and computes per-output-channel normalization and bias terms used to convert INT32 sums back to FP32. Activation parameters are copied or expanded internally according to SimdConvolutionActivationType.

Note

This function is a C++ wrapper for function SimdSynetConvolution8iSetParams.

Parameters

[in]	weight	- a pointer to FP32 convolution weights.
[in]	bias	- a pointer to FP32 bias array with dstC elements. Can be NULL.
[in]	params	- a pointer to FP32 parameters of activation function (see SimdConvolutionActivationType). Can be NULL when activation does not require parameters.
[in]	stats	- a pointer to pointers with per-channel tensor statistics: source minimum stats[0], source maximum stats[1], destination minimum stats[2], destination maximum stats[3].

Forward()

SIMD_INLINE void Forward	(	const uint8_t *	src,
		uint8_t *	buf,
		uint8_t *	dst
	)

Performs INT8 convolution forward propagation.

The function converts FP32 input to UINT8 when the context source type is FP32, uses UINT8 input directly when the source type is UINT8, accumulates convolution sums in INT32 with INT8 weights, converts sums to FP32 using internal normalization and bias, applies activation, and writes FP32 or UINT8 output according to the context destination type. The buf argument can be NULL (it causes usage of internal buffer).

Note

This function is a C++ wrapper for function SimdSynetConvolution8iForward.

Parameters

[in]	src	- a pointer to input tensor. Actual element type is defined by srcT in convolution parameters.
[out]	buf	- a pointer to external temporary byte buffer. Can be NULL.
[out]	dst	- a pointer to output tensor. Actual element type is defined by dstT in convolution parameters.

Clear()

SIMD_INLINE void Clear

(

)

Releases internal context and clears stored convolution parameters.