Sending compressed data via socket

Question

Hi, I have to make a log server in java, and one task is to send the data compressed. Now I am sending it line by line in plain text, but I must compress it. The server handle "HTTP like" request. For example, I can get a log sending "GET xxx.log". This will entablish a TCP connection to the server, the server response with a header and the log, and close the connection. The client, reads line by line and analyzes each LOG entry. I tried some ways without success. My main problem is that I don't know where each line ends(in the client size). Any idea?

Answer 1

The thing that would make the most sense is something like this

1. Client sends GET foo.log
2. Server sends "Size: 15042\n"
3. Server sends the compressed log file, whole
4. Client parses the Size: header (reading until a newline) and then knows how many bytes to expect
5. Client then reads all those bytes from the server (15042 in the example)
6. Client decompresses the received data
7. Client then processes the log (decompressed you'll find the newlines)

Answer 2

You might take a look at the HTTP Range header. The client can request a range of bytes, and the server will return only those bytes. Use GZIP compression.

You could have an action that you call from the client that will provide an index, with byte ranges, prior to requesting part of the log.

There is a lot of overhead with HTTP headers, so you might consider sending 100 lines at a time or more.

Finally, look at chunked transfer encoding, which will allow you to stream the data to the client.

Answer 3

Use a GZIPOutputStream at the server and a GZIPInputStream at the client, with an InputStreamReader around that and a BufferedReader around that. Then just read lines.