Simd Library Documentation.

Home | Release Notes | Download | Documentation | Issues | GitHub
Functions
NormalizeLayer functions
Functions » Synet Framework

Functions to acceleratŠµ NormalizeLayer in Synet Framework. More...

Functions

SIMD_API void SimdSynetNormalizeLayerForward (const float *src, size_t batch, size_t channels, size_t spatial, const float *scale, const float *eps, SimdBool acrossSpatial, SimdTensorFormatType format, float *buf, float *dst)
 Performs forward propagation of NormalizeLayer. More...
 
SIMD_API void SimdSynetNormalizeLayerForwardV2 (const float *src, size_t batch, size_t channels, size_t spatial, const float *scale, const float *shift, const float *eps, SimdTensorFormatType format, float *buf, float *dst)
 Performs forward propagation of NormalizeLayer (Version 2). More...
 
SIMD_API void SimdSynetNormalizeLayerForwardV3 (const float *src, size_t batch, size_t channels, size_t spatial, const float *scale, const float *shift, const float *eps, SimdTensorFormatType format, float *buf, float *dst)
 Performs forward propagation of NormalizeLayer (Version 3). More...
 
SIMD_API void SimdSynetNormalizeLayerForwardV4 (const float *src, size_t batch, size_t channels, size_t spatial, const float *scale, const float *shift, const float *eps, SimdTensorFormatType format, float *buf, float *dst)
 Performs forward propagation of NormalizeLayer (Version 4). More...
 

Detailed Description

Functions to acceleratŠµ NormalizeLayer in Synet Framework.

Function Documentation

◆ SimdSynetNormalizeLayerForward()

void SimdSynetNormalizeLayerForward ( const float *  src,
size_t  batch,
size_t  channels,
size_t  spatial,
const float *  scale,
const float *  eps,
SimdBool  acrossSpatial,
SimdTensorFormatType  format,
float *  buf,
float *  dst 
)

Performs forward propagation of NormalizeLayer.

Algorithm's details (NHWC format, acrossSpatial is false): 

for(b = 0; b < batch; ++b)
    for(s = 0; s < spatial; ++s)
    {
        sum = 0;
        for(c = 0; c < channels; ++c)
            sum += Square(src[b, s, c]);
        for(c = 0; c < channels; ++c)
            dst[b, s, c] = src[b, s, c] * scale[c] / Sqrt(sum + eps);
    }
Note
This function is used in Synet Framework.
Parameters
[in]src- a pointer to the input 32-bit float tensor.
[in]batch- a batch size of input and output tensor.
[in]channels- a number of channels in input and output tensor.
[in]spatial- a spatial size (height*width) of input and output tensor.
[in]scale- an array with scale parameters. The size of the array is equal to channels.
[in]eps- a pointer to epsilon parameter. It is used to prevent division by zero.
[in]acrossSpatial- a flag of whole image normalization. Otherwise the normalization is performed across channels.
[in]format- a format of input and output tensor. It can be SimdTensorFormatNchw, SimdTensorFormatNhwc.
[out]buf- a pointer to external temporary buffer. The size of the buffer must be equal to spatial. Can be NULL (it causes usage of internal buffer).
[out]dst- a pointer to the output 32-bit float tensor.

◆ SimdSynetNormalizeLayerForwardV2()

void SimdSynetNormalizeLayerForwardV2 ( const float *  src,
size_t  batch,
size_t  channels,
size_t  spatial,
const float *  scale,
const float *  shift,
const float *  eps,
SimdTensorFormatType  format,
float *  buf,
float *  dst 
)

Performs forward propagation of NormalizeLayer (Version 2).

Algorithm's details: 

for(b = 0; b < batch; ++b)
    for(s = 0; s < spatial; ++s)
    {
        sum = 0;
        for c = 0; c < channels; ++c)
            sum += src[b, s, c];
        mean = sum / channels;
        for (c = 0; c < channels; ++c)
            dst[b, s, c] = src[b, s, c] - mean;

        sqsum = 0;
        for (c = 0; c < channels; ++c)
            sqsum += Square(dst[b, s, c]);
        norm = 1 / Sqrt(sqsum / channels + eps);
        for (c = 0; c < channels; ++c)
            dst[b, s, c] = dst[b, s, c] * norm * scale[c] + shift[c];
    }
Note
This function is used in Synet Framework.
Parameters
[in]src- a pointer to the input 32-bit float tensor.
[in]batch- a batch size of input and output tensor.
[in]channels- a number of channels in input and output tensor.
[in]spatial- a spatial size (height*width) of input and output tensor.
[in]scale- an array with scale parameters. The size of the array is equal to channels.
[in]shift- an array with shift parameters. The size of the array is equal to channels.
[in]eps- a pointer to epsilon parameter. It is used to prevent division by zero.
[in]format- a format of input and output tensor. It can be SimdTensorFormatNchw, SimdTensorFormatNhwc.
[out]buf- a pointer to external temporary buffer. The size of the buffer must be equal to spatial. Can be NULL (it causes usage of internal buffer).
[out]dst- a pointer to the output 32-bit float tensor.

◆ SimdSynetNormalizeLayerForwardV3()

void SimdSynetNormalizeLayerForwardV3 ( const float *  src,
size_t  batch,
size_t  channels,
size_t  spatial,
const float *  scale,
const float *  shift,
const float *  eps,
SimdTensorFormatType  format,
float *  buf,
float *  dst 
)

Performs forward propagation of NormalizeLayer (Version 3).

Algorithm's details: 

for(b = 0; b < batch; ++b)
    for(c = 0; c < channels; ++c)
    {
        sum = 0;
        for (s = 0; s < spatial; ++s)
            sum += src[b, c, s];
        mean = sum / spatial;
        for (s = 0; s < spatial; ++s)
            dst[b, c, s] = src[b, c, s] - mean;

        sqsum = 0;
        for (s = 0; s < spatial; ++s)
            sqsum += Square(dst[b, c, s]);
        norm = 1 / Sqrt(sqsum / spatial + eps);
        for (s = 0; s < spatial; ++s)
            dst[b, c, s] = dst[b, c, s] * norm * scale[c] + shift[c];
    }
Note
This function is used in Synet Framework.
Parameters
[in]src- a pointer to the input 32-bit float tensor.
[in]batch- a batch size of input and output tensor.
[in]channels- a number of channels in input and output tensor.
[in]spatial- a spatial size (height*width) of input and output tensor.
[in]scale- an array with scale parameters. The size of the array is equal to channels.
[in]shift- an array with shift parameters. The size of the array is equal to channels.
[in]eps- a pointer to epsilon parameter. It is used to prevent division by zero.
[in]format- a format of input and output tensor. It can be SimdTensorFormatNchw, SimdTensorFormatNhwc.
[out]buf- a pointer to external temporary buffer. The size of the buffer must be equal to channels. Can be NULL (it causes usage of internal buffer).
[out]dst- a pointer to the output 32-bit float tensor.

◆ SimdSynetNormalizeLayerForwardV4()

void SimdSynetNormalizeLayerForwardV4 ( const float *  src,
size_t  batch,
size_t  channels,
size_t  spatial,
const float *  scale,
const float *  shift,
const float *  eps,
SimdTensorFormatType  format,
float *  buf,
float *  dst 
)

Performs forward propagation of NormalizeLayer (Version 4).

Algorithm's details: 

for(b = 0; b < batch; ++b)
{
    sum = 0;
    for (c = 0; c < channels; ++c)
    {
        sqsum = 0;
        for (s = 0; s < spatial; ++s)
            sqsum += Square(src[b, c, s]);
        buf[c] = sqrt(sqsum);
        sum += buf[c];
    }
    for (c = 0; c < channels; ++c)
    {
        buf[c] = 1 + scale[c] * buf[c] / (sum / channels + eps);
        for (s = 0; s < spatial; ++s)
            dst[b, c, s] = src[b, c, s] * buf[c] + shift[c];
    }
}
Note
This function is used in Synet Framework.
Parameters
[in]src- a pointer to the input 32-bit float tensor.
[in]batch- a batch size of input and output tensor.
[in]channels- a number of channels in input and output tensor.
[in]spatial- a spatial size (height*width) of input and output tensor.
[in]scale- an array with scale parameters. The size of the array is equal to channels.
[in]shift- an array with shift parameters. The size of the array is equal to channels.
[in]eps- a pointer to epsilon parameter. It is used to prevent division by zero.
[in]format- a format of input and output tensor. It can be SimdTensorFormatNchw, SimdTensorFormatNhwc.
[out]buf- a pointer to external temporary buffer. The size of the buffer must be equal to channels. Can be NULL (it causes usage of internal buffer).
[out]dst- a pointer to the output 32-bit float tensor.