On Azure Development Storage I get "the specified blob already exists" when uploading block blob in parallel

Question

Hi,

Wondering if you clever people can help me with a right doozy of a bug.

I'm uploading blob blocks in parallel and this works perfectly fine when running against live storage however against dev storage I get the error "The specified blob already exists" for the occasional block. The weird/ironic thing is that if the blob does already exist then I never see the error.

Here's my code:

        var container = _cloudBlobClient.GetContainerReference(containerName);
        container.CreateIfNotExist();

        CloudBlockBlob blob = container.GetBlockBlobReference(blobname);

        // calc number of blocks. Add 1 for remainder
        var blockCount = ((int)Math.Floor((double)(length / mainBlockSize))) + 1;
        var blockIds = new List<string>();

        Parallel.For(0, blockCount, j =>
        {
            int blockSize = mainBlockSize;


            // if the last block then calculate the remaining block size
            if (j == blockCount - 1)
                blockSize = (int)length - (mainBlockSize * (blockCount - 1));

            var bytes = new byte[blockSize];

            string blockId = Convert.ToBase64String(Encoding.UTF8.GetBytes(j.ToString("00000")));

            lock (_mutex)
            {
                // these operations need to co-exist so the correct block order is maintained
                blockIds.Add(blockId);
                stream.Read(bytes, 0, blockSize);
            }

            blob.PutBlock(blockId, new MemoryStream(bytes), null);
        });

        // commit the blob with the list of blocks
        blob.PutBlockList(blockIds);

I've tried setting

_cloudBlobClient.ParallelOperationThreadCount = 1;

as suggested by this thread What is the Behaviour of UploadFile-CloudBlockBlob? to see if it would make any difference but it still errored.

Arg! First time I've had a "works in live but not on my machine bug"! :)

I've checked all the storage requests with Fiddler and there's no difference between Live and Dev so I'm assuming this is a bug with the development storage. Any ideas on how best to deal with this?

Thanks!

Answer 1

Have you posted this on the Windows Azure MSDN forum (http://tinyurl.com/wazforum)?

If you do, someone from the WA storage team should jump in, investigate, and confirm it's a bug with dev storage.