modify a datastream on the fly

Question

I need to hijack and modify a datastream. The stream consists of fixed-width commands. Each command is a new line, and the documentation says that each command starts and ends with an STX/ETX pair (start and end of text)

The sending system is using serial, but is attacked to an iPocket device that communicates over IP to our PBX. From what I can tell it's just converting the serial to telnet, which should keep things pretty simple hopefully.

The string I need to look for starts with "NAM". The structure is something like this:

STX NAM EXT# LASTNAME,FIRSTNAME ETX

I need to replace ",FIRSTNAME" with whitespace so it doesn't change the length of the command.

I've been attempting the following, and while it passes the data end to end in both directions as required, and removes the needed data, it isn't maintaining the length of the command.

ipocket <-> nc -kl 1100 | sed 's/,[^,]*/ /g' | nc target_ip target_port <-> PBX

I'll be using a linux box for this task. I'm reasonably certain that this could be done quite simply in perl or python but I haven't got a clue where to start. Any assistance would be greatly appreciated!

Answer 1

This should do it, its a one liner

perl -pe '/^STX\s\w+\s\d+#\s\w+,(\w+)\sETX$/;$len=length($1);s/$1/" " x $len/e'

I tested using.

echo "STX NAM 100# LASTNAME,FIRSTNAME ETX" | perl -pe '/^STX\s\w+\s\d+#\s\w+,(\w+)\sETX$/;$len=length($1);s/$1/" " x $len/e'

and it returns

STX NAM 100# LASTNAME,          ETX

To ensure the strings are the same length i tested.

echo "STX NAM 100# LASTNAME,FIRSTNAME ETX" | perl -pe '/^STX\s\w+\s\d+#\s\w+,(\w+)\sETX$/;$len=length($1);s/$1/" " x $len/e'| perl -pe 'print length($_);'

Gives 36.

echo "STX NAM 100# LASTNAME,FIRSTNAME ETX"|perl -pe 'print length($_);'

Gives 36.

Answer 2

In the Perl, with added check that the string starts with NAM and replacing really the ,FIRSTNAME:

nc -kl 1100 | \
perl -pe '/^STX NAM / && do { s/(,FIRSTNAME)/" " x length("$1")/ge }' | \
nc <...>

In Perl s///e as expected does substitution, but evaluates the replacement string as an expression. Operator x makes a new string, duplicating string on the left number of times given on the right. Non-matching strings are obviously not modified.

Answer 3

Instead of the sed command, you can use this:

perl -pe 's/(,\w+)/" " x length $1/ge'

The e option on the substitution means that the right side of the s/// is evaluated as a Perl expression. In this case, the expression returns the correct number of spaces, based on the length of the captured match.

Answer 4

Here's a sed version:

sed 'h;s/[^,]*,\([^ ]*\) ETX/\1/;s/./ /g;x;s/,.*/,/;G;s/\n//;s/$/ ETX/'

I'd be willing to bet that not only is the command fixed width, but that the fields are also fixed width. If that's the case then something like this would probably work:

sed 's/\(.\{22\}\).\{9\}\(.\{4\}\)/\1         \2/'

or

sed -r 's/(.{22}).{9}(.{4})/\1         \2/'

or

sed -r 's/STX (.{18}).{9} ETX/STX \1          ETX/'