Why is there a limit of max 20 parameters to a clojure function

Question

Hi, there seems to be a limit to the number of parameters a clojure function can take.

When defining a function with more than 20 parameters I receive the following:

#<CompilerException java.lang.RuntimeException: java.lang.RuntimeException: java.lang.Exception: Can't specify more than 20 params (NO_SOURCE_FILE:0) (NO_SOURCE_FILE:0)>

Obviously this can be avoided, but I was hitting this limit porting the execution model of an existing DSL to clojure, and I have constructs in my DSL like the following, which by macro expansion can be mapped to functions quite easily except for this limit:

(defAlias nn1 ((element ?e1) (element ?e2)) number
"@doc features of the elements are calculated for entry into
      the first neural network, the result is the score computed by the latter"
(nn1-recall (nn1-feature00 ?e1 ?e2) (nn1-feature01 ?e1 ?e2) ... (nn1-feature89 ?e1 ?e2)))

which is a DSL statement to call a neural network with 90 input nodes. Can work around it of course, but was wondering where the limit comes from. Thanks.

Answer 1

First of all, the limit only applies to required positional arguments; you can always use the variable arity case (& more-args in the function's signature) to handle as many arguments as you want to:

(defn foo [& args]
  (count args))

(foo 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25)
;; returns 25

In fact, at first glance, & args is likely to be exactly the right solution to your problem. (E.g. you'll be able to map a function over your input nodes collected into a sequence, loop / recur over said sequence etc. -- tends to make more sense with a large number of similar items than assigning separate names to each one of them.

(Note that I don't pretend to know the nature of the specific DSL you're transcribing into Clojure or the kind of problems you're dealing with, just suggesting points which might be of interest to you. If you've got a really funky situation where this doesn't seem to apply, maybe you can provide more details and we'll see if someone here can offer some useful hints for dealing with it in Clojure.)

For the sake of completeness, you can add the & args bit to a function which takes its first 19 arguments as required positional args:

(defn bar [a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15 a16 a17 a18 a19 & as]
  (+ 19 (count as)))

(bar 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25)
;; returns 25

Note that if you specify 20 positional args and the & arg on top of that, apparently weirdness will ensue.

As for the rationale, I believe this has to do with the fact that the JVM can dispatch to a method by arity very efficiently, so the clojure.lang.Fn class has the invoke method overloaded for arities up to 20. I'm not entirely sure if it could go higher than that, but I suppose this isn't something people require that often... I mean, I certainly find any API specifying over 20 positional arguments to a function a bit suspect.

Answer 2

The answer of Michal Marczyk tells you very well how to get around the limit. If you're interrested in the reason of this limitation, you might want to take a glance at this bit of clojure source : IFn

invoke being a java overloaded method implemented by the Ifn interface, Rich overloaded it to support up to 20 arguments. When you call a function in clojure, the underlying implementation calls invoke on the function object, a method that only supports up to 20 args.

You could overload it to support more, but i doubt of the utility of such a thing. If you have a bunch of input sources, they can probably be treated as an array anyway.