SynetInnerProduct16b Class Reference

The SynetInnerProduct16b class is a C++ wrapper of BF16/FP32 inner product (matrix multiplication). More...

#include <SimdSynet.hpp>

Public Member Functions
	SynetInnerProduct16b ()
virtual	~SynetInnerProduct16b ()
SIMD_INLINE void	Init (size_t M, size_t N, size_t K, SimdTensorDataType typeA, SimdTensorDataType typeB, SimdTensorDataType typeC, SimdBool transB, SimdBool constB, SimdBool bias, SimdConvolutionActivationType activation)
SIMD_INLINE bool	Enable () const
SIMD_INLINE size_t	InternalBufferSize () const
SIMD_INLINE size_t	ExternalBufferSize () const
SIMD_INLINE const char *	Info () const
SIMD_INLINE void	SetParams (const float *weight, const float *bias, const float *params)
SIMD_INLINE void	Forward (const uint8_t *A, const uint8_t *B, uint8_t *buf, uint8_t *C)
SIMD_INLINE void	Clear ()

Detailed Description

The SynetInnerProduct16b class is a C++ wrapper of BF16/FP32 inner product (matrix multiplication).

The class wraps C API functions SimdSynetInnerProduct16bInit, SimdSynetInnerProduct16bInternalBufferSize, SimdSynetInnerProduct16bExternalBufferSize, SimdSynetInnerProduct16bInfo, SimdSynetInnerProduct16bSetParams and SimdSynetInnerProduct16bForward. It computes C = A*B with FP32 accumulation, optionally adds bias and applies activation. A, B and C can be FP32 or BF16 according to typeA, typeB and typeC:

for(i = 0; i < M; ++i)
    for(j = 0; j < N; ++j)
    {
        sum = bias ? bias[j] : 0;
        for(k = 0; k < K; ++k)
            sum += A[i, k] * (transB ? B[j, k] : B[k, j]);
        C[i, j] = ConvertToTypeC(Activate(sum, activation, params));
    }

When constB is SimdTrue, matrix B must be supplied to SetParams() in FP32 form and is converted or reordered into internal storage. Call Init() and SetParams() before Forward(). Use Enable() to check that a context was created. The context is released by Clear() or by the destructor.

Using example:

#include "Simd/SimdSynet.hpp"

int main()
{
    const size_t M = 4, N = 8, K = 16;
    std::vector<float> A(M * K), B(K * N), C(M * N), bias(N, 0.0f);
    for (size_t i = 0; i < A.size(); ++i)
        A[i] = float(i) * 0.01f;
    for (size_t i = 0; i < B.size(); ++i)
        B[i] = float(i) * 0.02f;

    Simd::SynetInnerProduct16b innerProduct;
    innerProduct.Init(M, N, K, SimdTensorData32f, SimdTensorData32f, SimdTensorData32f,
        SimdFalse, SimdTrue, SimdTrue, SimdConvolutionActivationIdentity);
    if (innerProduct.Enable())
    {
        innerProduct.SetParams(B.data(), bias.data(), NULL);
        innerProduct.Forward((const uint8_t*)A.data(), NULL, NULL, (uint8_t*)C.data());
    }

    return 0;
}

Constructor & Destructor Documentation

SynetInnerProduct16b()

SynetInnerProduct16b

(

)

Creates a new empty SynetInnerProduct16b class.

~SynetInnerProduct16b()

virtual ~SynetInnerProduct16b ( )

virtual

SynetInnerProduct16b class destructor. Releases internal context.

Member Function Documentation

Init()

SIMD_INLINE void Init	(	size_t	M,
		size_t	N,
		size_t	K,
		SimdTensorDataType	typeA,
		SimdTensorDataType	typeB,
		SimdTensorDataType	typeC,
		SimdBool	transB,
		SimdBool	constB,
		SimdBool	bias,
		SimdConvolutionActivationType	activation
	)

Initializes (or re-initializes) a BF16/FP32 inner-product context.

Creates an internal context with using of function SimdSynetInnerProduct16bInit. The context is recreated only if matrix sizes, tensor types or inner-product flags were changed.

Note

This function is a C++ wrapper for function SimdSynetInnerProduct16bInit.

Parameters

[in]	M	- a height of A and C matrices.
[in]	N	- a width of B and C matrices.
[in]	K	- a width of A and height of B matrices.
[in]	typeA	- a type of A matrix. It can be SimdTensorData32f or SimdTensorData16b.
[in]	typeB	- a type of B matrix. It can be SimdTensorData32f or SimdTensorData16b.
[in]	typeC	- a type of C matrix. It can be SimdTensorData32f or SimdTensorData16b.
[in]	transB	- a flag indicating that B is stored as N*K instead of K*N.
[in]	constB	- a flag indicating that matrix B is constant and can be set once.
[in]	bias	- a flag to add bias to output matrix C.
[in]	activation	- an activation function type used after inner product.

Enable()

SIMD_INLINE bool Enable

(

)

const

Checks that the internal inner-product context was created.

Returns

true if the context exists and Forward() can be called.

InternalBufferSize()

SIMD_INLINE size_t InternalBufferSize

(

)

const

Gets the size in bytes of internal storage used by the inner-product context.

The returned value reports internal temporary storage, reordered constant weights, copied bias and copied activation parameters.

Note

This function is a C++ wrapper for function SimdSynetInnerProduct16bInternalBufferSize.

Returns

a number of bytes used by internal buffers.

ExternalBufferSize()

SIMD_INLINE size_t ExternalBufferSize

(

)

const

Gets the size in bytes of caller-provided temporary buffer for BF16/FP32 inner product.

The returned value depends on matrix types and implementation. It covers temporary BF16 copies of FP32 inputs, packed non-constant B matrices, FP32 accumulation buffers and optional post-processing buffers. It can be used when allocating the buf argument of Forward().

Note

This function is a C++ wrapper for function SimdSynetInnerProduct16bExternalBufferSize.

Returns

a number of bytes required for external temporary buffer.

Info()

SIMD_INLINE const char * Info

(

)

const

Gets a short description of the selected BF16/FP32 inner-product implementation.

The returned string contains the implementation extension, algorithm name and parameter summary. The returned pointer is owned by the context and remains valid until the next call of this function or until the context is released.

Note

This function is a C++ wrapper for function SimdSynetInnerProduct16bInfo.

Returns

a string with description of internal implementation. NULL if the context was not created.

SetParams()

SIMD_INLINE void SetParams	(	const float *	weight,
		const float *	bias,
		const float *	params
	)

Sets weights, bias and activation parameters for BF16/FP32 inner product.

This function must be called before Forward(). If constB was SimdTrue during initialization, weight provides matrix B in FP32 form and the implementation converts it to BF16 and may reorder it into internal storage. Bias is copied to an internal FP32 array; when bias is NULL, zeros are used. Activation parameters are copied or expanded to the internal FP32 array according to SimdConvolutionActivationType.

Note

This function is a C++ wrapper for function SimdSynetInnerProduct16bSetParams.

Parameters

[in]	weight	- a pointer to FP32 matrix B weights. Can be NULL only when B is not constant.
[in]	bias	- a pointer to FP32 bias array with N elements. Can be NULL.
[in]	params	- a pointer to FP32 parameters of activation function (see SimdConvolutionActivationType). Can be NULL when activation does not require parameters.

Forward()

SIMD_INLINE void Forward	(	const uint8_t *	A,
		const uint8_t *	B,
		uint8_t *	buf,
		uint8_t *	C
	)

Performs BF16/FP32 inner-product forward propagation.

The function converts FP32 A or B inputs to BF16 when requested by the context, uses BF16 inputs directly otherwise, accumulates the matrix product in FP32, adds bias, applies activation and writes FP32 or BF16 output according to typeC. If B is constant, it can be NULL when it was set by SetParams(). The buf argument can be NULL (it causes usage of internal buffer).

Note

This function is a C++ wrapper for function SimdSynetInnerProduct16bForward.

Parameters

[in]	A	- a pointer to A matrix. Actual element type is defined by typeA in initialization.
[in]	B	- a pointer to B matrix. Can be NULL if B is constant.
[out]	buf	- a pointer to external temporary byte buffer. Can be NULL.
[out]	C	- a pointer to output matrix. Actual element type is defined by typeC in initialization.

Clear()

SIMD_INLINE void Clear

(

)

Releases internal context and clears stored inner-product parameters.