I have just started delving into the world of functional programming.
A lot of OOP (Object Oriented Programming) concepts such as inheritance and polymorphism apply to most modern OO languages like C#, Java and VB.NET.
But how about concepts such as Map, Reduce, Tuples and Sets, do they apply to all FP (Functional Programming) languages?
I have just started with F#. But do aforementioned concepts apply to other FP like Haskell, Nemerle, Lisp, etc.?
You bet. The desirable thing about function programming is that the mathematical concepts you describe are more naturally expressed in an FP.
It's a bit of tough going, but John Backus' Turing Award paper in which he described functional (or "applicative") programming is a good read. The Wikipedia article is good too.
Yes; higher-order functions, algebraic data types, folds/catamorphisms, etc are common to almost all functional languages (though they sometimes go by slightly different names in each language).
Functional tools apply to all programming, not just languages that handle that explicitly. For example, python has map and reduce builtin functions that do exactly what you expect, besides out of order evaluation. you'll need something like the multiprocessing module to get really clever.
Even if the language doesn't provide the exact primitives, most modern languages still make it possible to get the desired effect with a bit more work. This is similar to the way a class-like concept can be coded in pure C.
They apply to all languages that contain data types that can be "mapped" and "reduced", i.e maps, arrays/vectors, or lists.
In a "pure lambda calculus" language, where every data structure is defined via function application, you can of course apply functions in parallel (i.e., in a call fn(expr1, expr2), you can evaluate expr1 and expr2 in parallel), but that isn't really what map/reduce is about.
If you really want to jump into the deep end and understand why these concepts are not just conventional but, ahem, foundational, check out the paper "Functional programming with bananas, lenses, envelopes and barbed wire".