Subject
Computer Science
Topic
Artificial Intelligence
Which activation function, defined as max(0, x), is the most common choice for hidden layers in deep networks?
Which activation function, defined as max(0, x), is the most common choice for hidden layers in deep networks?
a)
Sigmoidb)
Step functionc)
Softmaxd)
ReLUExplanation
ReLU is cheap to compute and suffers less from vanishing gradients than sigmoid or tanh.Related questions
In the figure, a perceptron has weights 1,1, input 0,0, bias -1.5, and threshold 0. What output does it produce?In the figure, a perceptron has weights 1,1, input 1,1, bias -1.5, and threshold 0. What output does it produce?In the figure, a perceptron has weights 1,-1, input 2,1, bias 0, and threshold 0. What output does it produce?In the figure, what is sigmoid(-2) rounded to three decimals?In the figure, what is sigmoid(2) rounded to three decimals?In the figure, a fully connected layer has 4 inputs and 3 neurons, with one bias per neuron. How many trainable parameters does it have?What output does a ReLU unit produce for input -3?A weight is 2, learning rate 0.1, and loss gradient 0.5. What is the weight after one gradient-descent step?