The color for point E will be the combination of Red / Green / Blue. Calculate each color axis as the average of the same color axis for A,B,C,D, ponderating by distance.
distance_a = sqrt((xa-xe)^2+(ya-ye)^2)
distance_b = ....
sum_distances = distance_a + distance_b ...
red = (red_a*distance_a + red_b*distance_b ... ) / sum_distances
color_E = ColorFromARgb(red,green,blue)
The best solution when a gradient is required between two colors, is to use the HSV representation (Hue Saturation Value).
If you have the HSV values for your two colors, you just make linear interpolation for H, S and V, and you have nice colors (interpolation in RGB space always lead to "bad" results).
You also find here the formulae to go from RGB to HSV and from HSV to RGB, respectively.
Now, for your problem with the four corner, you can make a linear combination of the four H/S/V values, weighted by the distance from E to that four points A,B,C and D.
EDIT: same method than tekBlues, but in HSV space (it is quite easy to test it in RGB and in HSV spaces. And you will see the differences. In HSV, you just turn around the chromatic cylinder, and this is why it gives nice result)
EDIT2: if you prefer "speed and simplicity", you may use a L1-norm, instead of a L2-norm (euclidian norm)
So, if a is the size of your square and the coordinate of your points are A(0,0), B(0,a), C(a,0), D(a,a), then the Hue of a point E(x,y) can be computed with:
Hue(E) = ( Hue(B)*y/a + Hue(A)*(1-y/a) ) * (x/a) + ( Hue(D)*y/a + Hue(C)*(1-y/a) ) * (1-x/a)
where Hue(A) is the Hue of point A, Hue(B) the Hue of B, etc...
You apply the same formulae for the Saturation and Value.
Once you have the Hue/Saturation/Value for your point E, you can transform it in RGB space.