views:

210

answers:

6

What do functional programmers mean when they say a certain thing is composable or not composable?

Some of the statements of this sort that I've read are:

  • Control structures are not composable.
  • Threads do not compose.
  • Monadic operations are composable.
+7  A: 
Marcelo Cantos
+1, but your point about locks not composing can be made even stronger. Even if `lock(x)` *always* eventually succeeds when `x` is the only lock in the world, and `lock(y)` *always* eventually succeeds when `y` is the only lock in the world, a function that calls `lock(x); lock(y);` can deadlock if another thread/process tries to grab the locks in the opposite order. That is, failure is possible *even when the individual operations are guaranteed to succeed when performed in isolation*.
j_random_hacker
Good point, @j_random_hacker. In addition, a canonical lock-acquisition-order policy can help, but it isn't always possible to observe it. For instance, you may not know which lock to acquire on a destination hash table until you actually suck an element out of the source hash table, so you are forced to acquire locks in source-destination order, which might be the other way around on another thread.
Marcelo Cantos
+1  A: 

Another example: consider async programming in .NET. If you use a language like C# and need to make a series of asynchronous (non-blocking) I/O calls via Begin/End APIs, then in order to call two operations, Foo and Bar, in sequence, you have to expose two methods (BeginFooAndBar, EndFooAndBar), where BeginFooAndBar calls BeginFoo and passes a callback to Intermediate, and Intermediate then calls BeginBar, and you have to thread the intermediate values and IAsyncResult state information through, and if you want a try-catch block around the whole thing, then good luck, you need to duplicate the exception handling code in three places, yikes, and it's an awful mess.

But then with F#, there's the async type, built atop functional continuations that are composable, and so you can write e.g.

let AsyncFooAndBar() = async {
    let! x = Async.FromBeginEnd(BeginFoo, EndFoo)
    let! y = Async.FromBeginEnd(BeginBar, EndBar, x)
    return y * 2 }

or what have you, and it's simple, and if you want to put a try-catch around it, great, the code is all in one method, rather than spread across three, you just put a try-catch around it and it works.

Brian
+1  A: 

Here's a real-world example. The names of all of the people that live in your house is the list of names of all the males in your house combined with the list of all females in your house.

This is composable, as each of those two subproblems can be solved independently and without interfering with solving the other one.

On the other hand, many recipes are not composable, as the steps must be done in a particular order and rely upon the results of other steps. You have to break the eggs before you whisk them!

kyoryu
+1  A: 

I agree with Marcelo Cantos's answer, but I think it may assume more background than some readers have, which is also related to why composition in functional programming is special. Functional programming function composition is essentially identical to function composition in mathematics. In math, you may have a function f(x) = x^2, and a function g(x) = x + 1. Composing the functions means creating a new function, in which the function arguments are given to the "inner" function, and the "inner" function's output serves as input to the "outer" function. Composing f outer with g inner could be written f(g(x)). If you provide a value of 1 for x, then g(1) == 1 + 1 == 2, so f(g(1)) == f(2) == 2^2 == 4. More generally, f(g(x)) == f(x + 1) == (x+1)^2. I described composition using the f(g(x)) syntax, but mathematicians often prefer a different syntax, (f . g)(x). I think this is because it makes clearer that f composed with g is a function in its own right, which takes a single argument.

Functional programs are built entirely using compositional primitives. A program in Haskell is, to possibly oversimplify, a function that takes as an argument a run-time environment, and returns the result of some manipulation of that environment. (Grokking this statement will require some understanding of monads.) Everything else is done with composition in the mathematical sense.

Aidan Cully
+7  A: 

A simple example of composability is the Linux commandline, where the pipe character lets you combine simple commands (ls, grep, cat, more etc.) in a virtually unlimited number of ways, thereby "composing" a large number of complex behaviors from a small number of simpler primitives.

There are several benefits to composability:

  • More uniform behavior: As an example, by having a single command that implements "show results one page at a time" (more) you get a degree of paging uniformity that would not be possible if every command were to implement their own mechanisms (and command line flags) to do paging.
  • Less repeated implementation work (DRY): Instead of having umpteen different implementations of paging, there is just one that is used everywhere.
  • More functionality for a given amount of implementation effort: The existing primitives can be combined to solve a much larger range of tasks than what would be the case if the same effort went into implementing monolithic, non-composable commands.

As the Linux command line example shows, composability is not necessarily restricted to functional programming, but the concept is the same: Have smallish pieces of code that do restricted tasks, and build more complex functionality by routing the outputs and inputs appropriately.

The point is that functional programming is well suited to this: With immutable variables and restrictions on side effects you can compose more easily as you need not worry about what happens under the hood in the function being called - like updating a shared variable so the result will be invalid for certain sequences of operations, or accessing a shared lock so certain call sequences will give a deadlock.

That's functional programming composability - any function depends only on its input parameters, and the output can be passed to any function that can handle the type of the return value.

By extension, having fewer data types gives more composability. Rich Hickey of Clojure said something along the lines of

every new object type is inherently incompatible with all the code ever written

which is certainly a point well made.

Practical composability also depends on a standardizing on a small set of data types, like the Unix commands do with their "tab-delimited line-based text" standard.

Postscript

Eric Raymond wrote a book about the Unix Philosophy, two of the design principles he listed were

  • Rule of Modularity: Write simple parts connected by clean interfaces.
  • Rule of Composition: Design programs to be connected with other programs.

From http://catb.org/~esr/writings/taoup/html/ch01s06.html#id2877537

Composability in functional programming can be said to bring those principles down to the level of individual functions.

j-g-faustus
+1, Excellent explanation!
missingfaktor
+5  A: 

Marcelo Cantos gave a pretty good explanation, but I think it can be made slightly more precise.

A type of thing is composable when several instances can be combined in a certain way to produce the same type of thing.

Control structure composability. Languages like C make a distinction between expressions, which can be composed using operators to produce new expressions, and statements, which can be composed using control structures like if, for and the "sequence control structure" that simply performs statements in order. The thing about this arrangement is that these two categories are not on an equal footing -- many control structures make use of expressions (e.g. the expression evaluated by if to choose which branch to execute), but expressions cannot make use of control structures (e.g. you can't return a for loop). Although it might seem crazy or meaningless to want to "return a for loop", in fact the general idea of treating control structures as first-class objects that can be stored and passed around is not only possible but useful. In lazy functional languages like Haskell, control structures like if and for can be represented as ordinary functions, which can be manipulated in expressions just like any other term, enabling such things as functions that "build" other functions according to the parameters they are passed, and return them to the caller. So while C (for example) divides "the things that a programmer might want to do" into two categories and limits the ways in which objects from these categories can be combined, Haskell (for example) has just one category and imposes no such limits, so in this sense it provides more composability.

Thread composability. I'll assume as Marcelo Cantos did that you are really talking about the composability of threads that use locks/mutexes. This is a slightly trickier case because on the face of it we can have threads that use multiple locks; but the important point is that we can't have threads that use multiple locks with the guarantees that they are intended to have.

We can define a lock as a type of thing that has certain operations that can be performed on it, which come with certain guarantees. One guarantee is: suppose there is a lock object x, then provided that every process that calls lock(x) eventually calls unlock(x), any call to lock(x) will eventually return successfully with x locked by the current thread/process. This guarantee vastly simplifies reasoning about program behaviour.

Unfortunately, if there is more than one lock in the world, it is no longer true. If thread A calls lock(x); lock(y); and thread B calls lock(y); lock(x); then it's possible that A grabs lock x and B grabs lock y and they will both wait indefinitely for the other thread to release the other lock: deadlock. So, locks are not composable, because when you use more than one, you cannot simply claim that that important guarantee still holds -- not without analysing the code in detail to see how it manages locks. In other words, you can no longer afford to treat functions as "black boxes".

Things that are composable are good because they enable abstractions, meaning that they enable us to reason about code without having to care about all the details, and that reduces the cognitive burden on the programmer.

j_random_hacker