Functions to accelerate GatherElements algorithm in Synet Framework. More...

Functions
SIMD_API void *	SimdSynetGatherElementsInit (SimdTensorDataType dataType, SimdTensorDataType indexType, SimdBool indexConst, size_t indexUsers, const size_t *outer, size_t outerSize, size_t srcCount, size_t inner, size_t idxCount)
	Initializes a gather-elements context. More...

SIMD_API void	SimdSynetGatherElementsSetIndex (const void context, const uint8_t idx)
	Sets and analyzes constant gather-elements indexes. More...

SIMD_API size_t	SimdSynetGatherElementsInternalBufferSize (const void *context)
	Gets the size in bytes of internal storage used by a gather-elements context. More...

SIMD_API void	SimdSynetGatherElementsForward (void context, const uint8_t src, const uint8_t idx, uint8_t dst)
	Performs gather-elements forward propagation. More...

Detailed Description

Functions to accelerate GatherElements algorithm in Synet Framework.

Function Documentation

◆ SimdSynetGatherElementsInit()

void * SimdSynetGatherElementsInit	(	SimdTensorDataType	dataType,
		SimdTensorDataType	indexType,
		SimdBool	indexConst,
		size_t	indexUsers,
		const size_t *	outer,
		size_t	outerSize,
		size_t	srcCount,
		size_t	inner,
		size_t	idxCount
	)

Initializes a gather-elements context.

The function creates a context for ONNX-style GatherElements along the dimension of length srcCount. It supports FP32, BF16 and UINT8 data tensors and INT32 or INT64 index tensors. The input tensor shape is:

outer[0] * ... * outer[outerSize - 1] * srcCount * inner

The index and output tensor shape is:

outer[0] * ... * outer[outerSize - 1] * idxCount * inner

Algorithm's details:

for(b = 0; b < outer[0]*...*outer[outerSize - 1]; ++b)
    for(c = 0; c < idxCount; ++c)
        for(i = 0; i < inner; ++i)
        {
            ic = idx[b, c, i];
            if (ic < 0)
                ic += srcCount;
            dst[b, c, i] = src[b, ic, i];
        }

If indexConst is SimdTrue, constant indexes can be analyzed by SimdSynetGatherElementsSetIndex to avoid repeated negative-index checks and to reduce repeated outer index processing when possible.

Parameters

[in]	dataType	- a type of input and output tensor. It can be FP32, BF16 or UINT8.
[in]	indexType	- a type of index tensor. It can be INT32 or INT64.
[in]	indexConst	- a flag indicating that index tensor is constant and can be set once.
[in]	indexUsers	- a number of consumers sharing the same constant index tensor. The current implementation stores this value but does not otherwise use it.
[in]	outer	- a pointer to outer shape dimensions before the gathered dimension.
[in]	outerSize	- a number of dimensions in outer.
[in]	srcCount	- a length of the gathered dimension in the input tensor.
[in]	inner	- a product of dimensions after the gathered dimension.
[in]	idxCount	- a length of the gathered dimension in the index and output tensors.

Returns: a pointer to gather elements context. On error it returns NULL. It must be released with using of function SimdRelease. This pointer is used in functions SimdSynetGatherElementsSetIndex, SimdSynetGatherElementsInternalBufferSize and SimdSynetGatherElementsForward.

◆ SimdSynetGatherElementsSetIndex()

void SimdSynetGatherElementsSetIndex	(	const void *	context,
		const uint8_t *	idx
	)

Sets and analyzes constant gather-elements indexes.

The function has an effect only when the context was created with indexConst equal to SimdTrue. It detects whether negative-index correction is needed and may collapse repeated outer batches of identical indexes. The current implementation still expects the index pointer to be passed to SimdSynetGatherElementsForward.

Parameters

[in]	context	- a pointer to gather elements context. It must be created by function SimdSynetGatherElementsInit and released by function SimdRelease.
[in]	idx	- a pointer to INT32 or INT64 index tensor. Its shape is outer[0] * ... * outer[outerSize - 1] * idxCount * inner.

◆ SimdSynetGatherElementsInternalBufferSize()

size_t SimdSynetGatherElementsInternalBufferSize ( const void * context )

Gets the size in bytes of internal storage used by a gather-elements context.

The returned value reports implementation-specific buffers used by the context.

Parameters

[in] context - a pointer to gather elements context. It must be created by function SimdSynetGatherElementsInit and released by function SimdRelease.

Returns: size of internal buffer in bytes used inside gather elements algorithm.

◆ SimdSynetGatherElementsForward()

void SimdSynetGatherElementsForward	(	void *	context,
		const uint8_t *	src,
		const uint8_t *	idx,
		uint8_t *	dst
	)

Performs gather-elements forward propagation.

The function gathers elements from src according to idx. If SimdSynetGatherElementsSetIndex was called, the context can use the analysis results, but idx must still point to the index tensor in the current implementation. Negative indexes are interpreted relative to srcCount.

Parameters

[in]	context	- a pointer to gather elements context. It must be created by function SimdSynetGatherElementsInit and released by function SimdRelease.
[in]	src	- a pointer to input tensor. Its shape is outer[0] * ... * outer[outerSize - 1] * srcCount * inner.
[in]	idx	- a pointer to INT32 or INT64 index tensor. Its shape is outer[0] * ... * outer[outerSize - 1] * idxCount * inner.
[out]	dst	- a pointer to output tensor. Its shape is outer[0] * ... * outer[outerSize - 1] * idxCount * inner.

Simd Library Documentation.

Functions

Detailed Description

Function Documentation

◆ SimdSynetGatherElementsInit()

◆ SimdSynetGatherElementsSetIndex()

◆ SimdSynetGatherElementsInternalBufferSize()

◆ SimdSynetGatherElementsForward()