what popular advance mathematics libraries for c++ are present out there, so that they can be used as a 1 stop solution and avoiding reinventing the wheel ?
Intel's MKL (Math Kernel Library) is to be looked at especially if doing large scale matrix operations; it's C based, but should not really be an issue IMO.
Other than that, maybe the boost math library could be interesting as it is free. (but I have no experience with it, so YMMV).
Max.
Check out GNU Scientific Library -- it's in C, but I use it all the time to avoid re-writing the Numerical Recipes code.
For commercial libraries, both NAG (Numerical Algorithms Group, http://www.nag.co.uk/) and IMSL ( http://www.vni.com/products/imsl/ ) are standards and provide industrial-strength numerical analysis algorithms.
look through the list and mix-and-match. You want very many things, unlikely any single package is going to do them all.
octave is the only one that is going to be more or less comprehensive (functionality comparable/clone to Matlab)
Like others have said, you will probably not find a single library to handle all of the areas you listed. For matrix algebra, I've heard good things about the Eigen C++ library from coworkers who are using it.