tags:

views:

258

answers:

1

Hi, I am using a slightly modified version of the DotZlib which is part of the contrib directory with the zlib source code to inflate a realtime data stream.

Instead of the regular inflateInit I need to use InflateInit2 - but this the only difference to the provided library.

Netherlesse after a couple of reads I receive error code 1 from the zlib and do not manage to recover when adding bytes.

The original code from the zlib contrib directory is:

    public override void Add(byte[] data, int offset, int count)
        {
            if (data == null) throw new ArgumentNullException();
            if (offset < 0 || count < 0) throw new ArgumentOutOfRangeException()
;
            if ((offset+count) > data.Length) throw new ArgumentException();

            int total = count;
            int inputIndex = offset;
            int err = 0;

            while (err >= 0 && inputIndex < total)
            {
                copyInput(data, inputIndex, Math.Min(total - inputIndex, kBufferSize));
                err = inflate(ref _ztream, (int)FlushTypes.None);
                if (err == 0)
                    while (_ztream.avail_out == 0)
                    {
                        OnDataAvailable();
                        err = inflate(ref _ztream, (int)FlushTypes.None);
                    }

                inputIndex += (int)_ztream.total_in;
            }
            setChecksum( _ztream.adler );
        }

BTW Does anyone know how to contribute improved code? The implementation is nicely designed, but incomplete from my point of view.

+1  A: 

I think that

            err = inflate(ref _ztream, (int)FlushTypes.None);
            if (err == 0)
                while (_ztream.avail_out == 0)
                {
                    OnDataAvailable();
                    err = inflate(ref _ztream, (int)FlushTypes.None);
                }

should be

            while (_ztream.avail_in > 0)
            {
                err = inflate(ref _ztream, (int)FlushTypes.None);
                if (err!=0)
                    break;
                OnDataAvailable();
            }

There are two problems I see with the first version of the code:

  1. if inflate() produces data but does not produce enough data to make avail_out be 0, you won't call OnDataAvailable, even though data is available.
  2. you might call inflate() even though avail_in is 0, which I could easily imagine might produce a stream-end error.

(NB: I suspect that you know me in a professional capacity. This answer is given in a private capacity and is not related to my work for my employer.)

Jon Bright
I think that this might be the case. I will test it - it fits to what I have seen at http://www.koders.com/csharp/fidB377599DB0D658DEA5BD81F67CA407C4FE12E93B.aspx?s=login.
weismat
This case is handled fine, but how do I recover from a Memory error correctly - should I free and allocate the buffer? The documentation on inflateReset is not so clear.
weismat
What sort of memory error are we talking about? inflate returns Z_MEM_ERROR? Or something else?
Jon Bright
yes - error code -5 - I am not sure if I it some kind of overrun - the .NET zlib allocates 16 kByte for in- and output and I think that there is another issue hidden.
weismat
Main question is how to handle it - the zlib documentation says that it is not fatal and maybe an additinal malloc will help - but I am not too sure if this is good enough.
weismat
-5 is Z_BUF_ERROR, not Z_MEM_ERROR. My first suggestion would be to increase kBufferSize in CodecBase.cs from 16384 to, say, 131072. Assuming no data that arrives expands to more than 128k, you then won't get the error in the first place and don't need to worry about how to recover from it. In fact, if memory usage isn't a concern, you could make it 1048576. The chances of needing more than a 1MB buffer strike me as slim.
Jon Bright
if you need to support many streams concurrently you'd probably be better off allocating a new buffer as suggested in the zlib howto page. I'm not sure about the code posted - the first point you made isn't critical because that's what Finish is for. and a Z_STREAM_END isn't a problem for the same reason.PS: the DotZlib code has some bugs in it, for instance on Inflater.cs ~line 59 it should be int total = count + offset; and not int total = count;also the CodecBase CleanUp(bool isDisposing) doesn't use isDisposing in any way. do you know where/how to contribute to the project?
Yonatan Karni