Why doesn't Java properly re-create this image from an InputStream?

Question

I've looked at this every way I can think... The problem is that I end up writing the PERFECT number of bytes, and the files are VERY similar - but some bytes are different. I opened the Java generated file in Scite as well as the original, and even though they are close, they are not the same. Is there any way to fix this? I've tried doing everything possible - I've used different wrappers, readers, writers and different methods of taking the byte array (or taking it as chars - tried both) and making it into a file.

The image in question, for the test, is at http://www.google.com/images/srpr/nav_logo13.png. Here's the code:

import java.awt.Image;
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;

import javax.imageio.ImageIO;

public class ImgExample
{

    private String address = "http://www.google.com";
    /**
     * Returns a 3 dimensional array that holds the RGB values of each pixel at the position of the current
     * webcam picture. For example, getPicture()[1][2][3] is the pixel at (2,1) and the BLUE value.
     * [row][col][0] is alpha
     * [row][col][1] is red
     * [row][col][2] is green
     * [row][col][3] is blue
     */
    public int[][][] getPicture()
    {
        Image camera = null;
        try {
            int maxChars = 35000;
            //The image in question is 28,736 bytes, but I want to make sure it's bigger
            //for testing purposes as in my case, it's an image stream so it's unpredictable
            byte[] buffer = new byte[maxChars];
            //create the connection
            HttpURLConnection conn = (HttpURLConnection)(new URL(this.address+"/images/srpr/nav_logo13.png")).openConnection();
            conn.setUseCaches(false);
            //wrap a buffer around our input stream
            BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
            int bytesRead = 0;
            while ( bytesRead < maxChars && reader.ready() )
            {
                //reader.read returns an int - I'm assuming this is okay?
                buffer[bytesRead] = (byte)reader.read();
                bytesRead++;
                if ( !reader.ready() )
                {
                    //This is here to make sure the stream has time to download the next segment
                    Thread.sleep(10);
                }
            }
            reader.close();

            //Great, write out the file for viewing
            File writeOutFile = new File("testgoog.png");
            if ( writeOutFile.exists() )
            {
                writeOutFile.delete();
                writeOutFile.createNewFile();
            }
            FileOutputStream fout = new FileOutputStream(writeOutFile, false);
            //FileWriter fout = new FileWriter(writeOutFile, false);
            //needed to make sure I was actually reading 100% of the file in question
            System.out.println("Bytes read = "+bytesRead);
            //write out the byte buffer from the first byte to the end of all the chars read
            fout.write(buffer, 0, bytesRead);
            fout.flush();
            fout.close();

            //Finally use a byte stream to create an image
            ByteArrayInputStream byteImgStream = new ByteArrayInputStream(buffer);
            camera = ImageIO.read(byteImgStream);
            byteImgStream.close();
        } catch ( Exception e ) { e.printStackTrace(); }
        return ImgExample.imageToPixels(camera);
    }

    public static int[][][] imageToPixels (Image image)
    {
        //there's a bunch of code here that works in the real program, no worries
        //it creates a 3d arr that goes [x][y][alpha, r, g, b val]
        //e.g. imageToPixels(camera)[1][2][3] gives the pixel's blue value for row 1 col 2
        return new int[][][]{{{-1,-1,-1}}};
    }

    public static void main(String[] args)
    {
        ImgExample ex = new ImgExample();
        ex.getPicture();
    }
}

Answer 1

The problem as I see it is that you're using Readers. In Java, Readers are for processing character streams, not binary streams, and the character conversions that it does are most likely what's changing your bytes on you.

Instead, you should read() from the InputStream directly. InputStream's read() will block until data is available, but returns -1 when the end of the stream is reached.

Edit: You can also wrap the InputStream in a BufferedInputStream.

Answer 2

BufferedReader is intended for reading character streams, not byte/binary streams.

BufferedReader.read() returns The character read, as an integer in the range 0 to 65535. This will likely truncate any binary data where the byte value is greater than 65535.

I think you want to use InputStream.read() directly, not wrapped in a BufferedReader/InputStreamReader.

And finally, not related to the problem, but if you open a FileOutputStream to append=false, there isn't really any point to deleting any file that already exists - append=false does the same thing.

Answer 3

I think your problem is you are using an InputStreamReader. From the javadocs

An InputStreamReader is a bridge from byte streams to character streams: It reads bytes and decodes them into characters using a specified charset. The charset that it uses may be specified by name or may be given explicitly, or the platform's default charset may be accepted.

You dont want the conversion to character streams.

Answer 4

And you also shouldn't be using ready() like that. You are just wasting time with that and the sleeps. read() will block until data arrives anyway, and it will block the correct length of time, not an arbitrary guess. The canonical copy loop in Java goes like this:

int count;
byte[] buffer; // whatever size you like
while ((count = in.read(buffer)) > 0)
{
  out.write(buffer, 0, count);
}