Is there a way for reading/writing serializable objects to a RandomAccesFile?

Question

How can I read/write serializable object instances to a RandomAccessFile in Java? I want to be able to do this the same way you do it in c++ through structs. In java only ObjectInputStreams/ObjectOutputStreamscan can read/Write objects. I am amazed Java does not have something already implemented.

Answer 1

There is a difference between serialized Java objects, and C structs you write out with fwrite: serialized java objects may not have a fixed size in bytes.

I am amazed Java does not have something already implemented.

Sure it does. Newer versions of Java ship with Java DB, an embedded database, which does what you want, and much much more. (Note that this type of databases are fundamentally random access files which work on data types like strings and numbers instead of bytes). This does have the downside that you have to write some JDBC boilerplate.

And if you don't mind including external dependencies, object databases (like db4o) are quite interesting and require minimal code.

Answer 2

Write the data to a ByteArrayOutputStream, via a ObjectOutputStream and then you can put the byte[] into the random access file. You can do the reverse much the same way.

However, why are you trying to do this? I suspect there are products which do this for you such as caches which can be persisted to disk. In this cause you can have just a Map which you put objects into and the library takes care of the rest.

Answer 3

I can see why you would want to do this to read legacy file formats. In that case, default Java serialization mechanisms are a hindrance, not a help. To a degree, you can read/write struct-like classes using reflection.

Sample code:

public static class MyStruct {
 public int foo;
 public boolean bar = true;
 public final byte[] byteArray = new byte[3];
}

public static void main(String[] args) throws IOException {
 LegacyFileHandler handler = new LegacyFileHandler();
 MyStruct struct = new MyStruct();

 RandomAccessFile file = new RandomAccessFile("foo", "rw");
 try {
  for (int i = 0; i < 4; i++) {
   struct.foo = i;
   handler.write(file, struct);
  }

  struct = readRecord(file, handler, 2);
  System.out.println(struct.foo);
 } finally {
  file.close();
 }
}

private static MyStruct readRecord(RandomAccessFile file,
  LegacyFileHandler handler, int n) throws IOException {
 MyStruct struct = new MyStruct();

 long pos = n * handler.sizeOf(struct);
 file.seek(pos);

 handler.read(file, struct);

 return struct;
}

The handler class; can handle primitive types and byte arrays, but nothing else:

public class LegacyFileHandler {

    private final Map<Class<?>, Method> readMethods = createReadMethodMap();

    private final Map<Class<?>, Method> writeMethods = createWriteMethodMap();

    private Map<Class<?>, Method> createReadMethodMap() {
     Class<DataInput> clazz = DataInput.class;
     Class<?>[] noparams = {};
     try {
      Map<Class<?>, Method> map = new HashMap<Class<?>, Method>();
      map.put(Boolean.TYPE, clazz.getMethod("readBoolean", noparams));
      map.put(Byte.TYPE, clazz.getMethod("readByte", noparams));
      map.put(Character.TYPE, clazz.getMethod("readChar", noparams));
      map.put(Double.TYPE, clazz.getMethod("readDouble", noparams));
      map.put(Float.TYPE, clazz.getMethod("readFloat", noparams));
      map.put(Integer.TYPE, clazz.getMethod("readInt", noparams));
      map.put(Long.TYPE, clazz.getMethod("readLong", noparams));
      map.put(Short.TYPE, clazz.getMethod("readShort", noparams));
      return map;
     } catch (NoSuchMethodException e) {
      throw new IllegalStateException(e);
     }
    }

    private Map<Class<?>, Method> createWriteMethodMap() {
     Class<DataOutput> clazz = DataOutput.class;
     try {
      Map<Class<?>, Method> map = new HashMap<Class<?>, Method>();
      map.put(Boolean.TYPE, clazz.getMethod("writeBoolean",
        new Class[] { Boolean.TYPE }));
      map.put(Byte.TYPE, clazz.getMethod("writeByte",
        new Class[] { Integer.TYPE }));
      map.put(Character.TYPE, clazz.getMethod("writeChar",
        new Class[] { Integer.TYPE }));
      map.put(Double.TYPE, clazz.getMethod("writeDouble",
        new Class[] { Double.TYPE }));
      map.put(Float.TYPE, clazz.getMethod("writeFloat",
        new Class[] { Float.TYPE }));
      map.put(Integer.TYPE, clazz.getMethod("writeInt",
        new Class[] { Integer.TYPE }));
      map.put(Long.TYPE, clazz.getMethod("writeLong",
        new Class[] { Long.TYPE }));
      map.put(Short.TYPE, clazz.getMethod("writeShort",
        new Class[] { Integer.TYPE }));
      return map;
     } catch (NoSuchMethodException e) {
      throw new IllegalStateException(e);
     }
    }

    public int sizeOf(Object struct) throws IOException {
     class ByteCounter extends OutputStream {
      int count = 0;

      @Override
      public void write(int b) throws IOException {
       count++;
      }
     }

     ByteCounter counter = new ByteCounter();
     DataOutputStream dos = new DataOutputStream(counter);
     write(dos, struct);
     dos.close();
     counter.close();
     return counter.count;
    }

    public void write(DataOutput dataOutput, Object struct) throws IOException {
     try {
      Class<?> clazz = struct.getClass();
      for (Field field : clazz.getFields()) {
       Class<?> type = field.getType();

       if (type == byte[].class) {
        byte[] barray = (byte[]) field.get(struct);
        dataOutput.write(barray);
        continue;
       }

       Method method = writeMethods.get(type);
       if (method != null) {
        method.invoke(dataOutput, field.get(struct));
        continue;
       }

       throw new IllegalArgumentException("Type "
         + struct.getClass().getName()
         + " contains unsupported field type " + type.getName()
         + " (" + field.getName() + ")");
      }
     } catch (IllegalAccessException e) {
      throw new IllegalArgumentException(e);
     } catch (InvocationTargetException e) {
      throw new IllegalStateException(e);
     }
    }

    public void read(DataInput dataInput, Object struct) throws IOException {
     try {
      Class<?> clazz = struct.getClass();
      Object[] noargs = {};
      for (Field field : clazz.getFields()) {
       Class<?> type = field.getType();

       if (type == byte[].class) {
        byte[] barray = (byte[]) field.get(struct);
        dataInput.readFully(barray);
        continue;
       }

       Method method = readMethods.get(type);
       if (method != null) {
        Object value = method.invoke(dataInput, noargs);
        field.set(struct, value);
        continue;
       }

       throw new IllegalArgumentException("Type "
         + struct.getClass().getName()
         + " contains unsupported field type " + type.getName()
         + " (" + field.getName() + ")");
      }
     } catch (IllegalAccessException e) {
      throw new IllegalArgumentException(e);
     } catch (InvocationTargetException e) {
      throw new IllegalStateException(e);
     }
    }

}

This code has only undergone cursory testing.

There are problems with trying to generalise reading/writing data like this.

• Dealing with any object type is tricky as you have to convert them to byte arrays yourself and ensure they remain a fixed size.
• The biggest problem will probably be keeping all your strings the correct length - you'll need to know a bit about unicode to understand the relationship between character count and encoded byte size.
• Java uses network byte order and doesn't really have unsigned types. If you're reading legacy data, you might need to do some bit manipulation.

If I needed to handle a binary format, I would probably encapsulate the functionality in specialized classes which would be able to write/read their state to/from an I/O source. You'll still have the problems in the above list, but it is a more object-oriented approach.

If I had the freedom to choose the file format, I would go with what waqas suggested and use a database.