pycmtensor.models.layers#

Model layers

Module Contents#

class pycmtensor.models.layers.Layer[source]#

Default class type

class pycmtensor.models.layers.DenseLayer(w, bias, activation=None)[source]#

Bases: Layer

Default class type

Class object for dense layer

Parameters:

  • w (TensorSharedVariable) – layer weights with ndim=2

  • bias (TensorSharedVariable) – layer bias with ndim=1

  • activation – the activation function, possible options are tanh, relu, sigm, None

Note

Layer activation function is set based on the type of weight initialization. If weight init is “he”, the activation is relu, if “glorot”, the activation is tanh, otherwise the activation defaults to sigm. Setting activation to other than None overrides this.

property updates[source]#

Returns a list of update tuple pairs

property output[source]#

Returns the output of this layer

apply(input)[source]#

Function to apply the input to the computational graph

class pycmtensor.models.layers.BatchNormLayer(gamma, beta, batch_size, factor=0.05, epsilon=1e-06)[source]#

Bases: Layer

Default class type

Class object for Batch Normalization layer

Parameters:

  • gamma (TensorSharedVariable) – gamma variable for variance

  • beta (TensorSharedVariable) – beta variable for mean

  • batch_size (int) – batch size indicator

  • factor (float, optional) – exponential moving average factor

  • epsilon (float, optional) – small value to prevent floating point error

Notes

The ema factor controls how fast/slow the running average is changed. Higher factor value discounts older values faster.

property mv_mean[source]#

Returns the stored running mean

property mv_var[source]#

Return the stored running variance

property updates[source]#

Returns a list of update tuple pairs

property output[source]#

Returns the output of this layer

Note

Returns the full normalized layer using the running mean if the input length is not equivalent to the batch size

apply(input)[source]#

Function to apply the input to the computational graph

class pycmtensor.models.layers.ResidualLayer(layers: list)[source]#

Definition of the Residual layer block

Parameters:

layers (list) – a list of layers that defines the residual block

Example

res_layer = ResidualLayer(layers=[
    DenseLayer(w_1, b_1, activation=relu),
    DenseLayer(w_2, b_2, activation=relu)
])
property updates[source]#

Returns a list of update tuple pairs

apply(input)[source]#

Function to apply the input to the computational graph

output()[source]#

Returns the output of this layer