TeX: string parsing and whitespace stripping

Question

Hi,

In T_EX, how is it possible to split a string such as \mana{X1 2W/UB R /G} into parts and to feed them to another macro (in this case, to replace the macro invocation by something like \m{X}\m{12}\m{W/U}\m{B}\m{R/G}), grouping by very simple rules, namely: a) consecutive digits form a group, b) a slash creates a group of two nearby characters, c) spaces are to be stripped completely?

I tried the substr package but it wasn’t too helpful, only allowing one to find certain substrings. Hand-written loops such as

\def\lcpass#1.{}
\def\lcloop#1#2.{%
    \ifempty{#2}%
        #1%
        \let\continue=\lcpass%
    \else%
        \let\continue=\lcloop%
    \fi%
    \continue#2.}

\def\lastchar#1{\lcloop#1.} % returns the last character of a string

fail to work when the string ends in whitespace, neither was I successful with \futurelet.

In general, how does one approach the task of parsing strings in T_EX? For example, the texmate package allows one to write things like |1 e4 e5 Nf3 Nc6| and automatically draws corresponding chess positions. How does it do it? What can I read about looping over characters in a string and other T_EX hacks like this?

Answer 1

As it’s often the case, a well-posed question is half the solution. Here’s what I found after writing the above post:

http://www.tug.org/TUGboat/Articles/tb28-1/tb88glister.pdf

“One unfortunate property of \allchars is that it discards all spaces in the original string,” it says. Ha ha.

Answer 2

\def\m#1{\par$m$({\tt #1})}% Any macros
\def\removespaces{\catcode`\ =9 }% Ignore all spaces`

\let\manaNext\relax % aux def
\let\manaLastChar\relax % aux def
\newtoks\manaToks % aux toks
\newif\ifDigitProcessing

\def\mana#{\afterassignment \manaA \let\next= }% always next = {
\def\manaA{\bgroup \removespaces \let\manaNext\manaB \manaNext}% algorithm init: ignore spaces
\def\manaB{\futurelet\next\manaC}% algorithm start
\def\manaC{\ifx\next\egroup \def\nnext{\manaFlush\aftergroup\manaNext}\else\let\nnext\manaD\fi\nnext}% check for \egroup
\def\manaD{\ifx\next/\let\nnext\manaSlash\else \ifcat\next 1\let\nnext\manaDigit \else \let\nnext\manaE \fi\fi \nnext}% cases
\def\manaE#1{\manaFlush\DigitProcessingfalse\let\manaLastChar\next\manaNext}% Letters A-Z and a-z case
\def\manaFlush{\ifx\manaLastChar\relax\else\m{\manaLastChar}\fi\let\manaLastChar\relax
               \ifDigitProcessing\expandafter\m\expandafter{\the\manaToks}\fi\manaToks{}}% transform to \m{...}
\def\manaSlash#1#2{\m{\manaLastChar/#2}\let\manaLastChar\relax\manaNext}%#1=/, #2=next letter
\def\manaDigit#1{\ifDigitProcessing\else\manaFlush\fi
       \manaToks=\expandafter{\the\manaToks#1}\DigitProcessingtrue\manaNext}% 0-9 case

\hrule\medskip
\mana{X1 2W/UB R /G}
\medskip\hrule\medskip

\mana{X1 2W/UB s/SS 14 1 R /G XZ}
\medskip\hrule\medskip

Answer 3

Here’s what I eventually did to enter MTG mana in TeX/LaTeX:

% http://www.tug.org/TUGboat/Articles/tb28-1/tb88glister.pdf

\newif\if@mtg@digit

\def\@mtg@testdigit#1{% lame, I know, but nothing else worked
    \@mtg@digitfalse%
    \def\@mtg@tmp{0}\ifx\@mtg@tmp#1\@mtg@digittrue\fi%
    \def\@mtg@tmp{1}\ifx\@mtg@tmp#1\@mtg@digittrue\fi%
    \def\@mtg@tmp{2}\ifx\@mtg@tmp#1\@mtg@digittrue\fi%
    \def\@mtg@tmp{3}\ifx\@mtg@tmp#1\@mtg@digittrue\fi%
    \def\@mtg@tmp{4}\ifx\@mtg@tmp#1\@mtg@digittrue\fi%
    \def\@mtg@tmp{5}\ifx\@mtg@tmp#1\@mtg@digittrue\fi%
    \def\@mtg@tmp{6}\ifx\@mtg@tmp#1\@mtg@digittrue\fi%
    \def\@mtg@tmp{7}\ifx\@mtg@tmp#1\@mtg@digittrue\fi%
    \def\@mtg@tmp{8}\ifx\@mtg@tmp#1\@mtg@digittrue\fi%
    \def\@mtg@tmp{9}\ifx\@mtg@tmp#1\@mtg@digittrue\fi%
    }


\catcode`\^^G=12

\newcommand*{\@mtg@loop}[2]{%
    \let\@mtg@callback=#1%
    \def\@mtg@string{#2}%
    \ifx\@mtg@string\@empty%
    \else%
        \@@mtg@loop#2^^G%
    \fi%
}

\def\@@mtg@loop#1#2^^G{%
    \def\@mtg@car{#1}%
    \def\@mtg@cdr{#2}%
    \ifx\@mtg@car\@empty%
        \let\@mtg@next=\@gobble%
    \else%
        \@mtg@callback{#1}%
        \ifx\@mtg@cdr\@empty%
            \let\@mtg@next=\@gobble%
        \else%
            \let\@mtg@next=\@@mtg@loop%
        \fi%
    \fi%
    \@mtg@next#2^^G%
}

\catcode`\^^G=15

% these are meant to be redefined
\def\mtg@mana#1{(\MakeUppercase{#1})}
\def\mtg@mana@colored#1{\mtg@mana{#1}}
\def\mtg@mana@colorless#1#2{\mtg@mana{#1#2}}
\def\mtg@mana@hybrid#1#2{\mtg@mana{#1/#2}}

\gdef\@mtg@terminator{.}
\gdef\@mtg@slash{/}
\let\@mtg@a=\@mtg@terminator
\let\@mtg@b=\@mtg@terminator

\def\@mtg@doChar#1{%
    \gdef\@mtg@c{#1}%
    \ifx\@mtg@a\@mtg@terminator%
    \else%
        \ifx\@mtg@b\@mtg@slash%
            \mtg@mana@hybrid{\@mtg@a}{\@mtg@c}%
            \let\@mtg@b=\@mtg@terminator%
            \let\@mtg@c=\@mtg@terminator%
        \else%
            \@mtg@testdigit\@mtg@a\if@mtg@digit%
                \@mtg@testdigit\@mtg@b\if@mtg@digit%
                    \ifx\@mtg@c\@mtg@slash%
                        \mtg@mana@colorless{}{\@mtg@a}%
                    \else%
                        \mtg@mana@colorless{\@mtg@a}{\@mtg@b}%
                        \let\@mtg@b=\@mtg@terminator%
                    \fi%
                \else%
                    \mtg@mana@colorless{}{\@mtg@a}%
                \fi%
            \else%
                \mtg@mana@colored{\@mtg@a}%
            \fi%
        \fi%
    \fi%
    \let\@mtg@a=\@mtg@b%
    \let\@mtg@b=\@mtg@c%
}

\newcommand{\mana}[1]{%
\let\@mtg@a=\@mtg@terminator%
\let\@mtg@b=\@mtg@terminator%
\@mtg@loop{\@mtg@doChar}{#1\@mtg@terminator\@mtg@terminator}}