How can I save shell output to a variable in LaTeX?

Question

I need to be able to execute a shell command through LaTeX [*] and have the resulting output available for use later on in the form of a LaTeX 'variable' (e.g., via a subsequent \newcommand command). If necessary a temporary file could be used to hold the output, although I would like to avoid this. I have been able to use

\immediate\write18

to invoke a bash command before, but this was only used to write to a temporary file.

[*] I realise that the -shell-escape flag must be set for this to be allowed

Answer 1

I don't think you can have LaTeX run a shell command as part of its typesetting process, but I'm completely wrong it turns out, LaTeX does support that. You could (and I would still prefer, despite having this new information) have your build system run the shell command and store it's output in a file, and use

\input{filename}

Answer 2

If you are using LuaLaTeX, you can do something like this:

\documentclass{article}
\begin{document}
\edef\out{\directlua{
  j = io.popen(" ls -l")
  tex.write(j:read("*all"))
  j:close()
}}
\out
\end{document}

Answer 3

You can then read the contents of the \write18d temporary file by opening the file (\openin) and then reading the file into a cs a line at a time using \read. Note that \read more-or-less asummes its input takes Tex syntax: in particular it will read from an open brace to the matching close brace. If this is a problem, you might fix it by messing about with catcodes.

Answer 4

TeX has support for file IO, and you can take advantage of this. To create a new input filehandle, you execute \newread\readFH; at this point, \readFH is a number representing a channel on which you can read or write (you've already seen one of these, the special channel 18). To open the file, you run \immediate\openin\readFH=filename.ext; now, reading from channel \readFH will read lines from filename.ext. To actually read from the file, you run \immediate\read\readFH to \nextline; this reads one line from \readFH and puts it in \nextline. Closing the file is then done with \immediate\closein\readFH.

Note that this treats newlines as spaces; if you have a file containing e.g.

foo
bar

and read a line into \nextline, it will be as though you wrote \def\nextline{foo }. To avoid this, you set \endlinechar to -1.

Overall, then, your example would look like this:

\newread\myinput
% We use '\jobname.temp' to create a uniquely-named temporary file
\immediate\write18{some command > '\jobname.temp'}
\immediate\openin\myinput=\jobname.temp
% The group localizes the change to \endlinechar
\bgroup
  \endlinechar=-1
  \immediate\read\myinput to \localline
  % Since everything in the group is local, we have to explicitly make the
  % assignment global
  \global\let\myresult\localline
\egroup
\immediate\closein\myinput
% Clean up after ourselves
\immediate\write18{rm -f -- '\jobname.temp'}
\dosomething{\myresult}

You could probably abstract this into a macro, but precisely what parts ought to be parametrized probably depends on your specific use case.

Also, just for future reference: creating filehandles to write to is done with \newwrite, you open them with \immediate\openout\writeFH=filename.ext, you write to them with \immediate\write\writeFH{some text}, and you close them with \immediate\closeout\writeFH.

Answer 5

I'm not very happy with the special mechanism of write18 (perhaps I haven't explored it enough). Instead I write my tex as noweb files, which can contain code chunks and documentation chunks. Then I have relatively simple Makefile (gnu) which interact the following way:

1. All code chunks get split out by notangle (into a special 'code' directory), and get their exec bit chmod'ed.

2. With a different target in the Makefile the required programs get compiled (if it is C) or just run if it is shell or awk...and produce tex output.

3. Then this tex output, tables, calculations, figures, whatever gets included back to the latex document.

4. pdflatex is run a couple of times (on the noweave'd document) to resolve forwards refs and whatever.

The end result is that when I work with some data set, producing statistical calculations, pictures, I can just say make and I get all the data, all the calculations, all the pictures refreshed if necessary, sometimes after hours(!).

The approach is not perfect yet, I don't know a good way to communicate from the latex file to the Makefile and back things like where in the filesystem does you data reside, and certain data must be duplicated which is error prone, but I am pretty happy that I can stuff in absolutely all code which processes my data into the document, expliain the necessary part in sufficient detail, refer to the document where I describe why the issue arose (separate chapter), with all parameters for the runs, and just type make. Hope this helps.