Which of the following statements is true about layers of a deep neural network?

- Single layer NN
- 2-layer NN
- N-layer NN

Match the hidden layers to the neural networks.

A feedforward network has 4 hidden layers between the input and output. How many layers does it have in total?

The layer count of a feedforward neural network excludes the input layer but includes every hidden layer and the output layer. If the network has $$H$$ hidden layers, its total layer count is $$H+1$$. For example, a network with one hidden layer and one output layer is a two-layer network.

University of Michigan - Ann Arbor

Google

In a feedforward neural network represented as a chain structure, such as $$f(x) = f^{(3)}(f^{(2)}(f^{(1)}(x)))$$, $$f^{(1)}$$ is called the first layer, $$f^{(2)}$$ is the second layer, and so on. The final layer is called the output layer, whose behavior is explicitly specified by the training data to match a label $$y \approx f^*(x)$$ for each input example $$x$$. The intermediate layers are called hidden layers because their desired outputs are not explicitly given by the training data. Instead, the learning algorithm must determine how these hidden layers should behave to best implement an approximation of the target function $$f^*$$.

Layers of a Feed Forward Neural Network

In the chain structure of a feedforward network, Let's assume the input example is $x \in \mathbb{R}^d$, then the first layer is given by
$h^{(1)} = g^{(1)} (W^{(1)T}x + b^{(1)})$
the second layer is given by 
$h^{(2)} = g^{(2)} (W^{(2)T} h^{(1)}+ b^{(2)})$
and so on.
For the output layer,
$\hat{y} = g^{(n)}(W^{(n)T} h^{(n-1)}+ b^{(n)})$

where the matrix $W^{(k)}\in \mathbb{R}^{d\times h_k}$ is the weight parameter for layer $k$ and $g^{k}$ is the activation function for layer $k$. Whether transposing $W^{(k)}$ depends on the shape of the vector.

Connection between the Layers of Neural Network

There is a lot of research going on activation functions, new units/activation functions which vividly exhibit great performance  will get published in research papers. The common hidden units found in advanced deep learning models are. 

- Linear Hidden Units
- Softmax units
- Radial basis functions
- Softplus
- Hard tanh

Learn Before

Related

Learn After