Your generate_buffer()
NIF is creating a thread to call generate_binary()
but the calling NIF doesn't wait for the newly-created thread to finish. The thread just gets created and likely is still running by the time the NIF returns, though this will be nondeterministic, as threads are in general. You're probably crashing the Erlang BEAM emulator because generate_binary()
is off trying to call into the Erlang run-time system after generate_buffer()
has returned, confusing the poor thing horribly.
Now, even assuming you fix this to make it do what you wanted, I don't think you should be using threads here at all.
First, Erlang NIFs are supposed to look like regular Erlang functions, differing only in that they happen to be written in a different language. Erlang functions don't spawn separate threads of execution, then return, leaving that thread running. Excepting those that deal with I/O and persistent data storage, Erlang functions are deterministic and referentially transparent. Your NIF is neither. So, even if it worked, it's still "wrong" in the sense that it violates an experienced Erlang programmer's expectations.
Second, if you need multiprocessing, Erlang already provides the idea of processes. If your NIF will really do so much work that it can benefit from multiprocessing, why not rework your NIF so it can work on a subrange of the data, then call it multiple times, once each from a number of Erlang processes? Then you don't need native threads.
Third, that thread creation overhead is going to kill you if the lifetime of the thread only extends over the course of a single Erlang NIF call, as it seems you actually intended. Again, Erlang processes will be more efficient here.