Neural networks - input values

Question

I have a question that may be trivial but it's not described anywhere i've looked. I'm studying neural networks and everywhere i look there's some theory and some trivial example with some 0s and 1s as an input. I'm wondering: do i have to put only one value as an input value for one neuron, or can it be a vector of, let's say, 3 values (RGB colour for example)?

Answer 1

It can be whatever you want, as long as you write your inner function accordingly.

The examples you mention use [0;1] as their domain, but you can use R, R², or whatever you want, as long as the function you use in your neurons is defined on this domain.

In your case, you can define your functions on R3 to allow for RGB values to be handled

A trivial example : use (x1, y1, z1),(x2,y2,z2)->(ax1+x2,by1+y2,cz1+z2) as your function to transform two colors into one, a b and c being your learning coefs, which you will determine during the learning phase.

Very detailed information (including the answer to your question) is available on Wikipedia.

Answer 2

When dealing with multi-dimensional data, I believe a two layer neural network is said to give better result.

In your case:

R[0..1] => (N1)----\
                    \
G[0..1] => (N2)-----(N4) => Result[0..1]
                    /
B[0..1] => (N3)----/

As you can see, the N4 neurone can handle 3 entries.

The [0..1] interval is a convention but a good one imo. That way, you can easily code a set of generic neuron classes that can take an arbitrary number of entries (I had template C++ classes with the number of entries as template parameter personally). So you code the logic of your neurons once, then you toy with the structure of the network and/or combinations of functions within your neurons.

Answer 3

Normally a single neuron takes as its input multiple real numbers and outputs a real number, which typically is calculated as applying the sigmoid function to the sum of the real numbers (scaled, and then plus or minus a constant offset).

If you want to put in, say, two RGB vectors (2 x 3 reals), you need to decide how you want to combine the values. If you add all the elements together and apply the sigmoid function, it is equivalent to getting in six reals "flat". On the other hand, if you process the R elements, then the G elements, and the B elements, all individually (e.g. sum or subtract the pairs), you have in practice three independent neurons.

So in short, no, a single neuron does not take in vector values.