Initialize the activation function for each neuron

model.setActivationFunction(ftype)
model.setActivationFunction(ftype, 'OptionName', optionValue, ...)

Input

• ftype The type of activation function. default 'Sigmoid'. Possible activation functions:
  • Identity Identity function: f(x) = x
  • Sigmoid Symmetrical sigmoid: f(x) = beta * (1-exp(-alpha*x))/(1+exp(-alpha*x)). See note below.
  • Gaussian Gaussian function: f(x) = beta * exp(-alpha^2*x*x)
  • ReLU ReLU function: f(x) = max(0,x)
  • LeakyReLU Leaky ReLU function: f(x) = x, for x>0 and f(x) = alpha*x, for x<=0

Options

• Param1 The first parameter of the activation function, alpha. default 0
• Param2 The second parameter of the activation function, beta. default 0

Currently the default and the only fully supported activation function is 'Sigmoid'.

Note

If you are using the default Sigmoid activation function with the default parameter values Param1=0 and Param2=0 then the function used is y = 1.7159*tanh(2/3 * x), so the output will range from [-1.7159, 1.7159], instead of [0,1]. Recall that by definition tanh(x) = (1 - exp(-2*x)) / (1 + exp(-2*x)).