Package com.jml.neural_network.layers

Class Dense

java.lang.Object
com.jml.neural_network.layers.Dense
All Implemented Interfaces:
BaseLayer, TrainableLayer
public class Dense extends Object implements TrainableLayer
A fully connected layer with an activation function. For an activation function g(x), this layer applies the transform f(x) = g(Wx+b) for an input vector x.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    String
    LAYER_TYPE
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    Dense​(int inDim, int outDim, ActivationFunction activation)
    Creates a Linear layer with specified input and output dimensions.
    Dense​(int inDim, int outDim, ActivationFunction activation, Initializer weightInitializer)
    Creates a Linear layer with specified input and output dimensions.
    Dense​(int inDim, int outDim, ActivationFunction activation, Initializer weightInitializer, Initializer biasInitializer)
    Creates a Linear layer with specified input and output dimensions.
    Dense​(int outDim, ActivationFunction activation)
    Creates a Linear layer with specified input and output dimensions.
    NOTE: this constructor infers the input dimension from the previous layer in the network.
    Dense​(int outDim, ActivationFunction activation, Initializer weightInitializer)
    Creates a Linear layer with specified input and output dimensions.
    NOTE: this constructor infers the input dimension from the previous layer in the network.
    Dense​(int outDim, ActivationFunction activation, Initializer weightInitializer, Initializer biasInitializer)
    Creates a Linear layer with specified input and output dimensions.
    NOTE: this constructor infers the input dimension from the previous layer in the network.

  • Method Summary

    Modifier and Type
    Method
    Description
    linalg.Matrix
    back​(linalg.Matrix upstreamGrad)
    Computes the backward pass for this layer.
    linalg.Matrix
    forward​(linalg.Matrix input)
    Computes the forward pass of this layer.
    String
    getDetails()
    Constructs a string containing this all details of the model pertinent for saving the model to a file.
    int
    getInDim()
    Gets the input dimension of this layer.
    int
    getOutDim()
    Gets the output dimension of this layer.
    linalg.Matrix[]
    getParams()
    Gets the trainable parameters for this layer as an array of matrices.
    linalg.Matrix[]
    getUpdates()
    Gets the update matrices for parameters of this layer.
    String
    inspect()
    Gets and formats details of this layer in a human-readable String.
    void
    resetGradients()
    Resists the accumulation of gradients for this layer.
    void
    setParams​(linalg.Matrix... params)
    Sets the parameters for this layer.
    String
    toString()
    Gets the name of the layer type as a string.
    void
    updateInDim​(int newInDim)
    Updates this layers input dimension.

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Field Details

  • Constructor Details

    • Dense

      public Dense​(int inDim, int outDim, ActivationFunction activation)
      Creates a Linear layer with specified input and output dimensions.
      Parameters:
      inDim - Input dimension for this layer. (i.e. the number of nodes in the previous layer)
      outDim - Output dimension for this layer. (i.e. the number of nodes in this layer.)
      activation - Activation function for this layer.

    • Dense

      public Dense​(int inDim, int outDim, ActivationFunction activation, Initializer weightInitializer)
      Creates a Linear layer with specified input and output dimensions.
      Parameters:
      inDim - Input dimension for this layer. (i.e. the number of nodes in the previous layer)
      outDim - Output dimension for this layer. (i.e. the number of nodes in this layer.)
      activation - Activation function for this layer.
      weightInitializer - Initializer for the weights of this layer.

    • Dense

      public Dense​(int inDim, int outDim, ActivationFunction activation, Initializer weightInitializer, Initializer biasInitializer)
      Creates a Linear layer with specified input and output dimensions.
      Parameters:
      inDim - Input dimension for this layer. (i.e. the number of nodes in the previous layer)
      outDim - Output dimension for this layer. (i.e. the number of nodes in this layer.)
      activation - Activation function for this layer.
      weightInitializer - Initializer for the weights of this layer.
      biasInitializer - Initializer for the bias terms of this layer.

    • Dense

      public Dense​(int outDim, ActivationFunction activation)
      Creates a Linear layer with specified input and output dimensions.
      NOTE: this constructor infers the input dimension from the previous layer in the network. Thus, it cannot be used as the first layer of the neural network. For the first layer use todo...
      Parameters:
      outDim - Output dimension for this layer (i.e. the number of nodes in this layer).
      activation - Activation function for this layer.

    • Dense

      public Dense​(int outDim, ActivationFunction activation, Initializer weightInitializer)
      Creates a Linear layer with specified input and output dimensions.
      NOTE: this constructor infers the input dimension from the previous layer in the network. Thus, it cannot be used as the first layer of the neural network. For the first layer use todo...
      Parameters:
      outDim - Output dimension for this layer (i.e. the number of nodes in this layer).
      activation - Activation function for this layer.
      weightInitializer - Initializer for the weights of this layer.

    • Dense

      public Dense​(int outDim, ActivationFunction activation, Initializer weightInitializer, Initializer biasInitializer)
      Creates a Linear layer with specified input and output dimensions.
      NOTE: this constructor infers the input dimension from the previous layer in the network. Thus, it cannot be used as the first layer of the neural network. For the first layer use todo...
      Parameters:
      outDim - Output dimension for this layer (i.e. the number of nodes in this layer).
      activation - Activation function for this layer.
      weightInitializer - Initializer for the weights of this layer.
      biasInitializer - Initializer for the bias terms of this layer.

  • Method Details

    • forward

      public linalg.Matrix forward​(linalg.Matrix input)
      Computes the forward pass of this layer.
      Specified by:
      forward in interface BaseLayer
      Parameters:
      input - Input to this Layer. Must be a column vector.
      Returns:
      Result of the forward pass of this layer as a Matrix.

    • back

      public linalg.Matrix back​(linalg.Matrix upstreamGrad)
      Computes the backward pass for this layer.
      Specified by:
      back in interface BaseLayer
      Parameters:
      upstreamGrad - Upstream gradient of the network.
      Returns:
      Result of the backwards pass of the layer as a Matrix. If this layer has weights, this matrix will have the same shape as the weight matrix for the layer.

    • getInDim

      public int getInDim()
      Gets the input dimension of this layer.
      Specified by:
      getInDim in interface BaseLayer
      Returns:
      The input dimension of this layer.

    • getOutDim

      public int getOutDim()
      Gets the output dimension of this layer.
      Specified by:
      getOutDim in interface BaseLayer
      Returns:
      The output dimension of this layer.

    • updateInDim

      public void updateInDim​(int newInDim)
      Updates this layers input dimension. This is useful for creating a layer with an unknown input dimension and inferring it from the previous layer in the network.
      Specified by:
      updateInDim in interface BaseLayer
      Parameters:
      newInDim - New input dimension for layer.

    • getParams

      public linalg.Matrix[] getParams()
      Gets the trainable parameters for this layer as an array of matrices.
      Specified by:
      getParams in interface BaseLayer
      Specified by:
      getParams in interface TrainableLayer
      Returns:
      The trainable parameters for this layer as an array of matrices {weights, bias}.

    • setParams

      public void setParams​(linalg.Matrix... params)
      Sets the parameters for this layer.
      Specified by:
      setParams in interface BaseLayer
      Specified by:
      setParams in interface TrainableLayer
      Parameters:
      params - Parameter Matrices for this layer.

    • getUpdates

      public linalg.Matrix[] getUpdates()
      Gets the update matrices for parameters of this layer.
      Specified by:
      getUpdates in interface BaseLayer
      Specified by:
      getUpdates in interface TrainableLayer
      Returns:
      The parameter update matrices.

    • inspect

      public String inspect()
      Gets and formats details of this layer in a human-readable String.
      Specified by:
      inspect in interface BaseLayer
      Returns:
      The details of this layer in a human-reusable String.

    • resetGradients

      public void resetGradients()
      Resists the accumulation of gradients for this layer.
      Specified by:
      resetGradients in interface BaseLayer
      Specified by:
      resetGradients in interface TrainableLayer

    • getDetails

      public String getDetails()
      Constructs a string containing this all details of the model pertinent for saving the model to a file.
      Specified by:
      getDetails in interface BaseLayer
      Returns:
      A string containing all information, including trainable parameters, needed to recreate the layer.

    • toString

      public String toString()
      Gets the name of the layer type as a string.
      Overrides:
      toString in class Object
      Returns:
      The type of this layer as a String.