I'm going to develop a comprehensive educational software which runs on the browser and has many visualization and simulation works (electrostatic and electromagnetic visualization, 2D and 3D).
Which language(Processing, javascript or something else) is the best toward my purpose?
You really have two choices ActionScript in Flash or VB.NET/C#/other in Silverlight.
So first you need to decide which of these platforms you will target.
The question is indeed broad but I will answer from the experience I've had.
Javascript is not really meant to do mathematical calculations, which is what might be necessary to calculate a lot of E&M phenomenon quickly (Especially if they are not represented as a closed form solution). It really goes into how much detail you want in your graphs as well (More steps = more calculations). You may find yourself needing to do more optimizations to make up for the performance difference.
I did some visualizations of antenna arrays (They had closed form solutions, only simple arrays) in Flash and it worked out ok. Javascript will definitely not be up to par with any 3D simulations you might want to do.
I wonder if Silverlight might be a better solution, because you may find more mathematics libraries for .NET than for Actionscript, that could save you a lot of work of writing the math out yourself (But you might end up doing this anyways because of the performance issues).
Flash may be the more common application stack right now for what you are looking for, but Silverlight is looking primed to take the title from them based on the powerful features that it contains.
You may be able to split the problem into two parts, the user-interaction and display part, and the heavy calculations part.
If you can move the heavy calculations to a server then you can still show everything in javascript.
One difficulty with javascript is that it is interpreted and you will need to write more of the equations yourself, so there is a performance hit and development time, but it will work without any plugins, unless you don't want to do 3D in the canvas tag.
Flash and Silverlight may have better options, but then you are learning new languages and requiring plugins, depending on what version of Flash you want to use.
As others have suggested javascript is not that strong of a language when it comes to visualization.
Processing is a really good language for what you're trying to do, it's easy to learn and is Java based. Data visualization is built directly into the language, as well as temporal space (ie advance "1 tick" in time and have the visualization react to that.)
Also if you're interested in going that route I'd suggest picking up Visualizing Data which is pretty much a Processing primer.
I would go Flex or Silverlight myself
I second LFSR Consulting's opinion: Processing is used a lot for educational purposes, it is free, and fast (Java is faster than Flash in general) and easy to learn, so you have faster results. It supports 3D, you can tackle Java libraries for simulation and computing, etc. And it has a great community! :-)
JavaScript is a bit light for such usage. JavaFX is hype, but it hasn't really 3D (although one used Java3D with it) and it is still a bit young.
Flash and Silverlight: no comment, not much experience in the field. OpenLazlo can be an alternative...
I'm a little late to the show, but what you want, has been implemented in JavaScript, and you'll find this incredibly useful. I recommend running it under Chrome as the JS processing engine is extremely fast. (You may even want to try Chrome 2 which is even faster)
See also: http://www.chromeexperiments.com/