How to find the value of a neuron in a neural network

Question

Since a lot of these sites found on google use mathematical notation and I have no idea what any of it means I want to make a feedforward neural network like this:

    n1  
i1      n3
    n2      o1
i2      n4
    n3

Now can someone explain to me how to find the value of o1? How is it possible to make a neuron active when none of its inputs are active?

Answer 1

If none of the inputs are live, then you won't get anything out of the output.

It's been a long time since I spent some time on this, but back in the day, we'd add noise to the equation. This can be in the form of inputs that are always on or by adding a small random amount to each input before shoving it at the neural network.

Interestingly, the use of noise in neural networks has been shown to have a biological analog. If you're trying to hear something, and you add in a bit of white noise, it makes it easier to hear. same goes for seeing.

As for your initial question - How to find out the value of o1 depends on ...

• The formula used throughout the neural network.
• The values of n1 to n4.
• The inputs.

http://www.cheshireeng.com/Neuralyst/nnbg.htm

Has some basic info on the maths.

Answer 2

Since the question isn't really clear to me... I'll say this in case it's what you're looking for:

Often times a bias neuron is added to the input and hidden layers to allow for the case you're mentioning. This extra neuron is always active and is used to handle the case when all other neurons on the layer are inactive.

Answer 3

This question is a good example of why "neural networks" do such an amazingly poor job of emulating the behavior of real-world neurons. Most real neurons have an intrinsic (or "natural") rate at which they fire action potentials, with no input from pre-synaptic neurons. The effect of pre-synaptic neurons is almost always to speed up or slow down this intrinsic firing rate, not to produce a single action potential in the post-synaptic neuron.

Why don't "neural networks" typically model this phenomenon? I don't know - you'd have to ask the people for whom "the approach inspired by biology has more or less been abandoned for a more practical approach based on statistics and signal processing".