Using a pre-parsed protocol definition in a script and keeping it up-to-date

Question

For my work, I sometimes have to deal with logfiles from a binary protocol (the logfiles contain hexdumps of the messages). I want to write a Perl script that can interpret the binary data for me and print the contents in a more friendly format.

I have a (machine readable) description of the protocol messages in a proprietary format and I have (mostly) figured out how to parse that format (the parts I can"t fully understand are not related to my goal, so I can just ignore them), so I can convert the description into a data structure for use in my script.

Because the protocol description only rarely changes, it seems a waste to re-parse the protocol description each time I want to analyse a logfile, but on the other hand, if the description does change or if I accidentally throw away my pre-parsed form of the description, then I would like my script to automatically trigger a re-parsing of the description.
What is the best way to realise this?

Answer 1

You can use a Makefile for this. Make the data structure you use a Makefile target that depends on the protocol description. When Make notices that the protocol was updated more recently than the script, it will run the commands you specify to recreate your data.

Answer 2

Assuming that the protocol description lives in a file accessible to the script, have a function to read in the parsed data which caches the parsed results in intermediate file. The logic is very very simple but the steps look very verbose since I tried to write out the full spec - in reality it should take <10 lines of Perl code.

1. Check if intermediate file exists. If it does not (or can not be read), skip to proprietary parsing step (#4)

2. If you can read in the intermediate cache file, read in the "protocol description timestamp" field (described below). Then find out modification time of "protocol description" file via stat() and compare. If modification time of "protocol description" file is >= cache file's stored timestamp, skip to proprietary parsing step (#4)

3. Else (e.g. the time of "protocol description" file is < cache file's stored timestamp), read the intermediary cache file data via Data::Dumper or Storable. End.

4. If you need to re-parse because of logic in #1 or #2, read in "protocol description" file, parse it into your data structure.

5. Then create a hash with 2 keys: "protocol_description_timestamp" (with the value being the modification time of protocol description file derived from stat call) and second key "data", with the value being a reference to the data structure you just produced as a result of parsing.

6. Then save that data structure into the intermediate cache file using Storable or Data::Dumper or any other method of your choice for storing Perl data structires.