Package com.jml.neural_network.layers

Class Linear

java.lang.Object
com.jml.neural_network.layers.Linear
All Implemented Interfaces:
BaseLayer, TrainableLayer
public class Linear extends Object implements TrainableLayer
Simple fully-connected linear layer. Applies the linear transform f(x)=Wx+b for an input vector x. This is equivalent to a Dense layer with a linear activation.

- The default weightInitializer is the GlorotNormal initializer.
- The default biasInitializer is the Zeros initializer.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    String
    LAYER_TYPE
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    Linear​(int outDim)
    Creates a Linear layer with specified input and output dimensions.
    NOTE: this constructor infers the input dimension from the previous layer in the network.
    Linear​(int inDim, int outDim)
    Creates a Linear layer with specified input and output dimensions.
    Linear​(int inDim, int outDim, Initializer weightInitializer)
    Creates a Linear layer with specified input and output dimensions.
    Linear​(int inDim, int outDim, Initializer weightInitializer, Initializer biasInitializer)
    Creates a Linear layer with specified input and output dimensions.
    Linear​(int outDim, Initializer weightInitializer)
    Creates a Linear layer with specified input and output dimensions.
    NOTE: this constructor infers the input dimension from the previous layer in the network.
    Linear​(int outDim, Initializer weightInitializer, Initializer biasInitializer)
    Creates a Linear layer with specified input and output dimensions.
    NOTE: this constructor infers the input dimension from the previous layer in the network.

  • Method Summary

    Modifier and Type
    Method
    Description
    linalg.Matrix
    back​(linalg.Matrix upstreamGrad)
    Computes backward pass for layer.
    linalg.Matrix
    forward​(linalg.Matrix input)
    Computes forward pass for layer.
    String
    getDetails()
    Constructs a string containing this all details of the model pertinent for saving the model to a file.
    int
    getInDim()
    Gets the input dimension of this layer.
    int
    getOutDim()
    Gets the output dimension of this layer.
    linalg.Matrix[]
    getParams()
    Gets the trainable parameters for this layer as an array of matrices.
    linalg.Matrix[]
    getUpdates()
    Gets the update matrices for parameters of this layer.
    String
    inspect()
    Gets and formats details of this layer in a human-readable String.
    void
    resetGradients()
    Resists the accumulation of gradients for this layer.
    void
    setParams​(linalg.Matrix... params)
    Sets the parameters for this layer.
    String
    toString()
    Gets the name of the layer type as a string.
    void
    updateInDim​(int newInDim)
    Updates this layers input dimension.

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Field Details

  • Constructor Details

    • Linear

      public Linear​(int inDim, int outDim)
      Creates a Linear layer with specified input and output dimensions.
      Parameters:
      inDim - Input dimension for this layer. (i.e. the number of nodes in the previous layer)
      outDim - Output dimension for this layer. (i.e. the number of nodes in this layer.)

    • Linear

      public Linear​(int inDim, int outDim, Initializer weightInitializer)
      Creates a Linear layer with specified input and output dimensions.
      Parameters:
      inDim - Input dimension for this layer. (i.e. the number of nodes in the previous layer)
      outDim - Output dimension for this layer. (i.e. the number of nodes in this layer.)
      weightInitializer - Initializer for the weights of this layer.

    • Linear

      public Linear​(int inDim, int outDim, Initializer weightInitializer, Initializer biasInitializer)
      Creates a Linear layer with specified input and output dimensions.
      Parameters:
      inDim - Input dimension for this layer. (i.e. the number of nodes in the previous layer)
      outDim - Output dimension for this layer. (i.e. the number of nodes in this layer.)
      weightInitializer - Initializer for the weights of this layer.
      biasInitializer - Initializer for the bias terms of this layer.

    • Linear

      public Linear​(int outDim)
      Creates a Linear layer with specified input and output dimensions.
      NOTE: this constructor infers the input dimension from the previous layer in the network. Thus, it cannot be used as the first layer of the neural network. For the first layer use Linear(int, int).
      Parameters:
      outDim - Output dimension for this layer (i.e. the number of nodes in this layer).

    • Linear

      public Linear​(int outDim, Initializer weightInitializer)
      Creates a Linear layer with specified input and output dimensions.
      NOTE: this constructor infers the input dimension from the previous layer in the network. Thus, it cannot be used as the first layer of the neural network. For the first layer use Linear(int, int, Initializer).
      Parameters:
      outDim - Output dimension for this layer (i.e. the number of nodes in this layer).
      weightInitializer - Initializer for the weights of this layer.

    • Linear

      public Linear​(int outDim, Initializer weightInitializer, Initializer biasInitializer)
      Creates a Linear layer with specified input and output dimensions.
      NOTE: this constructor infers the input dimension from the previous layer in the network. Thus, it cannot be used as the first layer of the neural network. For the first layer use Linear(int, int, Initializer, Initializer).
      Parameters:
      outDim - Output dimension for this layer (i.e. the number of nodes in this layer).
      weightInitializer - Initializer for the weights of this layer.
      biasInitializer - Initializer for the bias terms of this layer.

  • Method Details

    • forward

      public linalg.Matrix forward​(linalg.Matrix input)
      Computes forward pass for layer.
      Specified by:
      forward in interface BaseLayer
      Parameters:
      input - Input to this Layer. Must be a column vector.
      Returns:
      Result of the forward pass of a layer as a matrix.

    • back

      public linalg.Matrix back​(linalg.Matrix upstreamGrad)
      Computes backward pass for layer.
      Specified by:
      back in interface BaseLayer
      Parameters:
      upstreamGrad - Upstream gradient of the network.
      Returns:
      Result of the backwards pass of the layer as a Matrix. If this layer has weights, this matrix will have the same shape as the weight matrix for the layer.

    • resetGradients

      public void resetGradients()
      Resists the accumulation of gradients for this layer.
      Specified by:
      resetGradients in interface BaseLayer
      Specified by:
      resetGradients in interface TrainableLayer

    • getInDim

      public int getInDim()
      Gets the input dimension of this layer.
      Specified by:
      getInDim in interface BaseLayer
      Returns:
      The input dimension of this layer.

    • getOutDim

      public int getOutDim()
      Gets the output dimension of this layer.
      Specified by:
      getOutDim in interface BaseLayer
      Returns:
      The output dimension of this layer.

    • updateInDim

      public void updateInDim​(int newInDim)
      Updates this layers input dimension. This is useful for creating a layer with an unknown input dimension and inferring it from the previous layer in the network.
      Specified by:
      updateInDim in interface BaseLayer
      Parameters:
      newInDim - New input dimension for layer.

    • getParams

      public linalg.Matrix[] getParams()
      Gets the trainable parameters for this layer as an array of matrices.
      Specified by:
      getParams in interface BaseLayer
      Specified by:
      getParams in interface TrainableLayer
      Returns:
      The trainable parameters for this layer as an array of matrices {weights, bias}.

    • getUpdates

      public linalg.Matrix[] getUpdates()
      Gets the update matrices for parameters of this layer.
      Specified by:
      getUpdates in interface BaseLayer
      Specified by:
      getUpdates in interface TrainableLayer
      Returns:
      The parameter update matrices.

    • setParams

      public void setParams​(linalg.Matrix... params)
      Sets the parameters for this layer.
      Specified by:
      setParams in interface BaseLayer
      Specified by:
      setParams in interface TrainableLayer
      Parameters:
      params - Parameter Matrices for this layer.

    • inspect

      public String inspect()
      Gets and formats details of this layer in a human-readable String.
      Specified by:
      inspect in interface BaseLayer
      Returns:
      The details of this layer in a human-reusable String.

    • getDetails

      public String getDetails()
      Constructs a string containing this all details of the model pertinent for saving the model to a file.
      Specified by:
      getDetails in interface BaseLayer
      Returns:
      A string containing all information, including trainable parameters, needed to recreate the layer.

    • toString

      public String toString()
      Gets the name of the layer type as a string.
      Overrides:
      toString in class Object
      Returns:
      The type of this layer as a String.