SynetMergedConvolution16b Class Reference

The SynetMergedConvolution16b class is a C++ wrapper of BF16/FP32 merged convolution. More...

#include <SimdSynet.hpp>

Public Member Functions
	SynetMergedConvolution16b ()
virtual	~SynetMergedConvolution16b ()
SIMD_INLINE void	Init (size_t batch, const SimdConvolutionParameters *convs, size_t count, SimdBool add)
SIMD_INLINE bool	Enable () const
SIMD_INLINE size_t	ExternalBufferSize () const
SIMD_INLINE size_t	InternalBufferSize () const
SIMD_INLINE const char *	Info () const
SIMD_INLINE void	SetParams (const float *const *weight, const float *const *bias, const float *const *params)
SIMD_INLINE void	Forward (const uint8_t *src, uint8_t *buf, uint8_t *dst)
SIMD_INLINE void	Clear ()

Detailed Description

The SynetMergedConvolution16b class is a C++ wrapper of BF16/FP32 merged convolution.

The class wraps C API functions SimdSynetMergedConvolution16bInit, SimdSynetMergedConvolution16bExternalBufferSize, SimdSynetMergedConvolution16bInternalBufferSize, SimdSynetMergedConvolution16bInfo, SimdSynetMergedConvolution16bSetParams and SimdSynetMergedConvolution16bForward. It fuses a sequence of two or three NHWC convolutions into one forward call: convolution + depthwise convolution, depthwise convolution + convolution, or convolution + depthwise convolution + convolution. Internal convolution data is processed in BF16. Source and destination tensors can be FP32 or BF16 according to the corresponding SimdConvolutionParameters fields. Supported kernels are 1x1 or 3x3 for ordinary convolutions, 3x3, 5x5 or 7x7 for depthwise convolutions; dilation must be 1 and stride must be 1, 2 or 3. If add is SimdTrue for a three-convolution sequence, the source tensor is added to the final output and therefore must have the same shape as the final destination tensor.

Call Init() and SetParams() before Forward(). Use Enable() to check that a context was created. The context is released by Clear() or by the destructor.

Using example:

#include "Simd/SimdSynet.hpp"

int main()
{
    const size_t batch = 1, srcC = 4, srcH = 8, srcW = 8, midC = 8, count = 2;
    SimdConvolutionParameters convs[2] = {};

    convs[0].srcC = srcC;
    convs[0].srcH = srcH;
    convs[0].srcW = srcW;
    convs[0].srcT = SimdTensorData32f;
    convs[0].srcF = SimdTensorFormatNhwc;
    convs[0].dstC = midC;
    convs[0].kernelY = 1;
    convs[0].kernelX = 1;
    convs[0].dilationY = 1;
    convs[0].dilationX = 1;
    convs[0].strideY = 1;
    convs[0].strideX = 1;
    convs[0].padY = 0;
    convs[0].padX = 0;
    convs[0].padH = 0;
    convs[0].padW = 0;
    convs[0].group = 1;
    convs[0].activation = SimdConvolutionActivationIdentity;
    convs[0].dstH = srcH;
    convs[0].dstW = srcW;
    convs[0].dstT = SimdTensorData32f;
    convs[0].dstF = SimdTensorFormatNhwc;

    convs[1].srcC = midC;
    convs[1].srcH = convs[0].dstH;
    convs[1].srcW = convs[0].dstW;
    convs[1].srcT = SimdTensorData32f;
    convs[1].srcF = SimdTensorFormatNhwc;
    convs[1].dstC = midC;
    convs[1].kernelY = 3;
    convs[1].kernelX = 3;
    convs[1].dilationY = 1;
    convs[1].dilationX = 1;
    convs[1].strideY = 1;
    convs[1].strideX = 1;
    convs[1].padY = 1;
    convs[1].padX = 1;
    convs[1].padH = 1;
    convs[1].padW = 1;
    convs[1].group = midC;
    convs[1].activation = SimdConvolutionActivationIdentity;
    convs[1].dstH = convs[1].srcH;
    convs[1].dstW = convs[1].srcW;
    convs[1].dstT = SimdTensorData32f;
    convs[1].dstF = SimdTensorFormatNhwc;

    std::vector<float> src(batch * srcH * srcW * srcC);
    std::vector<float> weight0(convs[0].kernelY * convs[0].kernelX * convs[0].srcC * convs[0].dstC);
    std::vector<float> weight1(convs[1].kernelY * convs[1].kernelX * convs[1].dstC);
    std::vector<float> bias0(convs[0].dstC, 0.0f), bias1(convs[1].dstC, 0.0f);
    std::vector<float> dst(batch * convs[1].dstH * convs[1].dstW * convs[1].dstC, 0.0f);
    const float * weight[2] = { weight0.data(), weight1.data() };
    const float * bias[2] = { bias0.data(), bias1.data() };
    const float * params[2] = { NULL, NULL };
    for (size_t i = 0; i < src.size(); ++i)
        src[i] = float(i) * 0.01f;
    for (size_t i = 0; i < weight0.size(); ++i)
        weight0[i] = float(i) * 0.02f;
    for (size_t i = 0; i < weight1.size(); ++i)
        weight1[i] = float(i) * 0.03f;

    Simd::SynetMergedConvolution16b mergedConvolution;
    mergedConvolution.Init(batch, convs, count, SimdFalse);
    if (mergedConvolution.Enable())
    {
        mergedConvolution.SetParams(weight, bias, params);
        mergedConvolution.Forward((const uint8_t*)src.data(), NULL, (uint8_t*)dst.data());
    }

    return 0;
}

Constructor & Destructor Documentation

SynetMergedConvolution16b()

SynetMergedConvolution16b

(

)

Creates a new empty SynetMergedConvolution16b class.

~SynetMergedConvolution16b()

virtual ~SynetMergedConvolution16b ( )

virtual

SynetMergedConvolution16b class destructor. Releases internal context.

Member Function Documentation

Init()

SIMD_INLINE void Init	(	size_t	batch,
		const SimdConvolutionParameters *	convs,
		size_t	count,
		SimdBool	add
	)

Initializes (or re-initializes) a BF16/FP32 merged convolution context.

Creates an internal context with using of function SimdSynetMergedConvolution16bInit. The context is recreated only if batch size, convolution count, residual-add flag or any convolution parameters were changed.

Note

This function is a C++ wrapper for function SimdSynetMergedConvolution16bInit.

Parameters

[in]	batch	- a batch size.
[in]	convs	- an array with convolution parameters in execution order.
[in]	count	- a number of merged convolutions. It must be 2 or 3.
[in]	add	- a flag that enables adding the source tensor to the final output tensor.

Enable()

SIMD_INLINE bool Enable

(

)

const

Checks that the internal merged convolution context was created.

Returns

true if the context exists and Forward() can be called.

ExternalBufferSize()

SIMD_INLINE size_t ExternalBufferSize

(

)

const

Gets the size in bytes of caller-provided temporary buffer for BF16 merged convolution.

The returned value is a number of bytes. It depends on the implementation selected during initialization and can be used when allocating the buf argument of Forward(). Some implementations return 1 when they do not need external temporary storage.

Note

This function is a C++ wrapper for function SimdSynetMergedConvolution16bExternalBufferSize.

Returns

a number of bytes required for external temporary buffer.

InternalBufferSize()

SIMD_INLINE size_t InternalBufferSize

(

)

const

Gets the size in bytes of internal storage used by the merged convolution context.

The returned value reports internal temporary buffers and implementation-specific reordered weights, bias or activation parameters already allocated by the context.

Note

This function is a C++ wrapper for function SimdSynetMergedConvolution16bInternalBufferSize.

Returns

a number of bytes used by internal buffers.

Info()

SIMD_INLINE const char * Info

(

)

const

Gets a short description of the selected BF16 merged convolution implementation.

The returned string contains the implementation extension and algorithm name. The returned pointer is owned by the context and remains valid until the next call of this function or until the context is released.

Note

This function is a C++ wrapper for function SimdSynetMergedConvolution16bInfo.

Returns

a string with description of internal implementation. NULL if the context was not created.

SetParams()

SIMD_INLINE void SetParams	(	const float *const *	weight,
		const float *const *	bias,
		const float *const *	params
	)

Sets FP32 weights, biases and activation parameters for BF16 merged convolution.

This function must be called before Forward(). The weight array contains pointers to FP32 convolution weights, one per merged convolution. The selected implementation transforms weights to its internal BF16/reordered representation. Bias is copied to an internal FP32 array; when a bias pointer is NULL, zeros are used. Activation parameters are copied or expanded to the internal FP32 array according to SimdConvolutionActivationType.

Note

This function is a C++ wrapper for function SimdSynetMergedConvolution16bSetParams.

Parameters

[in]	weight	- an array of pointers to FP32 convolution weights. The array size must be equal to the number of merged convolutions.
[in]	bias	- an array of pointers to FP32 bias arrays, one per convolution. Each pointer can be NULL.
[in]	params	- an array of pointers to activation parameters (see SimdConvolutionActivationType), one per convolution. The array itself must be valid; each element can be NULL for activations that do not use parameters.

Forward()

SIMD_INLINE void Forward	(	const uint8_t *	src,
		uint8_t *	buf,
		uint8_t *	dst
	)

Performs BF16/FP32 merged convolution forward propagation.

The function converts FP32 input to BF16 when the context source type is FP32, uses BF16 input directly when the source type is BF16, applies the fused convolution sequence stored in the context created by Init() and SetParams(), and writes FP32 or BF16 output according to the last convolution destination type. The buf argument can be NULL (it causes usage of internal buffer). If Init() was called with add equal to SimdTrue, the source tensor is added to the final output.

Note

This function is a C++ wrapper for function SimdSynetMergedConvolution16bForward.

Parameters

[in]	src	- a pointer to the input tensor bytes. The tensor type is determined by convs[0].srcT (FP32 or BF16).
[out]	buf	- a pointer to an external temporary byte buffer. Can be NULL.
[out]	dst	- a pointer to the output tensor bytes. The tensor type is determined by convs[count - 1].dstT (FP32 or BF16).

Clear()

SIMD_INLINE void Clear

(

)

Releases internal context and clears stored merged convolution parameters.