ansaurus

Question

Are there any Java Frameworks for binary file parsing?

Answer 1

+7 A:

I have used DataInputStream for reading binary files and I write the rules in Java. ;) Binary files can have just about any format so there is no general rule for how to read them.

Frameworks don't always make things simpler. In your case, the description file is longer than the code to just read the data using a DataInputStream.

public static void parse(DataInput in) throws IOException {
//        file:
//          header: FIXED("MAGIC")
    String header = readAsString(in, 5);
    assert header.equals("MAGIC");
//          body: content(10)
// ?? not sure what this means
//        content:
    for(int i=0;i<10;i++) {
//          value1: BYTE
        byte value1 = in.readByte();
//          value2: LONG
        long value2 = in.readLong();
//          value3: STRING(10)
        String value3 = readAsString(in, 10);
    }
}

public static String readAsString(DataInput in, int len) throws IOException {
    byte[] bytes = new byte[len];
    in.readFully(bytes);
    return new String(bytes);
}

If you want to have a configuration file you could use a Java Configuration File. http://www.google.co.uk/search?q=java+configuration+file

Peter Lawrey 2009-03-13 21:52:59

+1: No framework. No easy parsing.

S.Lott 2009-03-13 21:58:30

I know that binary files can have any format, but a PNG-file has always the same structure, a BMP has always the same structure...What I want is: create a "description" for a eg. BMP file, feed it to the framework, feed a BMP and then have easy access to each single element of the parsed content.

Kosi2801 2009-03-13 22:10:15

Kosi2801: If you can write specification or just a serie of "how it really works" steps for such framework, then it's doable. If you can't, it's not.

Esko 2009-03-13 22:14:52

@Perter Lawrey: In that simple case you're right, the config is longer. But my goal is to be able to deliver something to the client where I can provide additional configs later to support additional file formats. Think of parsing image files where you cannot update the binary for the customer...

Kosi2801 2009-03-15 00:15:06

@Kosi2801 I am not sure why you feel providing a text file is any easier than providing a class or jar file. If you need to provide the update as text you can use BeanShell, apache-jci or the builtin Compiler API to compile/load java code on the fly.

Peter Lawrey 2009-03-15 06:52:49

Nice example, but I'd strongly discourage calling that String-constructor! Always define a character set, or it will come back to bite you in the behind ...

Joachim Sauer 2009-03-15 19:54:14

With your update you've lined out a solution where the structure of the file to read is hardcoded in the source. What I need is, that the config-file describes the structure of the file and the parser reads it in in an easy-accessible way. Changes only happen in the config file...

Kosi2801 2009-03-17 08:20:13

You can deploy a java file as a config file. There is no difference except the format you use. What is the difference between a config file written in text and a java file written in text. One is neither more hard coded that the other.

Peter Lawrey 2009-03-17 21:56:32

Tried to keep business out as long as possible, but: With the software the customer has to be able to provide a description of the file-format to the app without the need to learn Java and have to deal with all of our (to be done then) interfaces. Just let him "describe" his file-format and done...

Kosi2801 2009-03-18 16:59:13

So instead of learning a sub-set of java (which they might know or be able to get help off the web for) they have to learn a new language which is unlikely to describe all the situations they will need which they definitely won't know and won't have much incentive to invest much time in.

Peter Lawrey 2009-03-18 21:01:33

Exactly. :) If the customer orders it and doesn't want to be convinced, we try to deliver. I also think it's safer and easier to provide him with 2 pages of description and a restricted language instead of allowing him to fiddle him around in the guts of the application and doing all sorts of stuff.

Kosi2801 2009-03-19 18:30:04

This reason is valid. In this case I would suggest something which has a simple mapping to what you would do in Java. Note: you may have to support loops and conditional logic which is not simple. You may find that you introduce more bugs than you solve.

Peter Lawrey 2009-03-19 20:37:55

Kosi2801 2009-03-19 21:28:02

Answer 2

A:

Have you looking into the world of parsers. A good parser is yacc, and there may be a port of it for java.

Milhous 2009-03-14 03:14:20

Answer 3

A:

You can parse binary files with parsers like JavaCC. Here you can find a simple example. Probably it's a bit more difficult than parsing text files.

asalamon74 2009-03-14 09:19:23

Answer 4

+1 A:

Parser combinator library is an option. JParsec works fine, however it could be slow.

stepancheg 2009-03-14 21:51:40

Answer 5

+2 A:

Google's Protocol Buffers

Ben Reeves 2009-03-15 20:02:50

Had a short look at it but it seems not to provide bit-level access to the data stream content but encapsulates it somehow (to provide optionality etc.). My assumption is also backed because it seems to be necessary to use the Protocol Buffers on both ends of the comm channel.

Kosi2801 2009-03-17 08:23:41

Answer 6

+3 A:

give a try to preon

dfa 2009-06-12 09:01:39

That's it. Thank you!

Kosi2801 2009-06-13 22:42:01

Answer 7

+3 A:

In Preon, it would be like this:

public class File {

  @BoundString(match="MAGIC")
  private String header;

  @BoundList(size="10", type=Body.class)
  private List<Body> body;

  private static class Body {

    @Bound
    byte value1;

    @Bound
    long value2;

    @BoundString(size="10")
    String value3;

  }


}

And you would decode data like this:

Codec<File> codec = Codecs.create(File.class);
File file = codecs.decode(codec, buffer);

Let me know if you are running into problems.

Wilfred Springer 2009-07-10 19:05:06

ansaurus

tags:

views:

answers:

Are there any Java Frameworks for binary file parsing?

related questions