Help understanding unipolar transfer function

Question

There is a question I am stuck on using the following formula for the unipolar transfer function:

f(net)=     1
         __________
               -net
         1 + e 

The example has the following:

out =        1
        ____________  = 0.977
               -3.75
        1 + e

How do we arrive at 0.977?

What is e?

Answer 1

e is Euler's number == 2.718281828....

If you raise e to the -3.75 power, add one to it, and take the inverse, you'll get precisely 0.977022630....

Answer 2

'e' is the base for the natural logarithm function, the value of which is equivalent to the sum of the infinite series 1/n! for n from 0 to infinity. It is available in the C standard library or the java Math package as the exp() function.

If you evaluate 1/(1+exp(-3.75)) you will get 0.977

Answer 3

While factually correct the other responses merely provide the value of e and confirm the underlying computation. This type of sigmoid functions is so ubiquitous to neural networks that some additional insight may be welcome.

Essentially the exponential function (e to the x power), has a very characteristic curve:

• Mostly flat at zero (very slightly above zero, actually), from - infinity to about -2
• incrementally sharp turn towards the vertical, between about -2 and +4
• quasi "vertical", with values in excess of 150 and increasingly huge, from +5 to infinity

As a result exponential curves are very useful for producing "S-shaped" functions; BTW, "S" is Sigma in Greek which supplied the etymology for "sigmoid". Such functions are often patterned on the formula shown in the question:

 1/(1 + e^-x)

where x is the variable. Typically such functions also include constants aimed at stretching the range (the input zone where changes in x are significant) and/or at modifying the curve in this middle zone.
The result of such functions is that up to a particular value of the input, the function is quasi constant, then, for a particular range of inputs, the function provides a increasing output, and finally past the upper value of the range, the function is quasi constant. Also looking in more details, such Sigmoids have a point of inflection which correspond to a reversing of the rate of change of the ouptut and which also marks an area of the curve, on either side, where the changes are the slowest, relatively.

In turn, such S-shaped curves (1) are very useful to normalize the output of neural network neurons, or more generally, to normalize various numeric values during processes of various nature. Intuitively these correspond to a "sweet spot" or a "sweet range" of the underlying neuron or device.

(1) Or also, possibly, "step-down" shaped curves, i.e. curves with a mostly constant high value, a decreasing value within the mid-range, and a low mostly constant value thereafter.