How to write only regularly spaced items from a char buffer to disk in C++

Question

How can I write only every third item in a char buffer to file quickly in C++?

I get a three-channel image from my camera, but each channel contains the same info (the image is grayscale). I'd like to write only one channel to disk to save space and make the writes faster, since this is part of a real-time, data collection system.

C++'s ofstream::write command seems to only write contiguous blocks of binary data, so my current code writes all three channels and runs too slowly:

char * data = getDataFromCamera();
int dataSize = imageWidth * imageHeight * imageChannels;
std::ofstream output;
output.open( fileName, std::ios::out | std::ios::binary );
output.write( data, dataSize );

I'd love to be able to replace the last line with a call like:

int skipSize = imageChannels;
output.write( data, dataSize, skipSize );

where skipSize would cause write to put only every third into the output file. However, I haven't been able to find any function that does this.

I'd love to hear any ideas for getting a single channel written to disk quickly. Thanks.

Answer 1

You'll probably have to copy every third element into a buffer, then write that buffer out to disk.

Answer 2

First, I'd mention that to maximize writing speed, you should write buffers that are multiples of the sector size (eg. 64KB or 256KB)

To answer your question, you're going to have to copy every 3rd element from your source data into another buffer, and then write that to the stream.

If I recall correctly Intel Performance Primitives has functions for copying buffers, skipping a certain number of elements. Using IPP will probably have faster results than your own copy routine.

Answer 3

There is no such a functionality in the standardlibrary afaik. Jerry Coffin's solution will work best. I wrote a simple snippet which should do the trick:

const char * data = getDataFromCamera();
const int channelNum = 0;
const int channelSize = imageWidth * imageHeight;
const int dataSize    = channelSize * imageChannels;
char * singleChannelData = new char[channelSize];
for(int i=0; i<channelSize ++i)
    singleChannelData[i] = data[i*imageChannels];
try {
    std::ofstream output;
    output.open( fileName, std::ios::out | std::ios::binary );
    output.write( singleChannelData, channelSize );
}
catch(const std::ios_base::failure& output_error) {
    delete [] channelSize;
    throw;
}
delete [] singleChannelData;

EDIT: i added try..catch. Of course you could aswell use a std::vector for nicer code, but it might be a tiny bit slower.

Answer 4

I'm tempted to say that you should read your data into a struct and then overload the insertion operator.

ostream& operator<< (ostream& out, struct data * s) {
    out.write(s->first);
}

Answer 5

Let's say your buffer is 24-bit RGB, and you're using a 32-bit processor (so that operations on 32-bit entities are the most efficient).

For the most speed, let's work with a 12-byte chunk at a time. In twelve bytes, we'll have 4 pixels, like so:

AAABBBCCCDDD

Which is 3 32-bit values:

AAAB
BBCC
CDDD

We want to turn this into ABCD (a single 32-bit value).

We can create ABCD by applying a mask to each input and ORing.

ABCD = A000 | 0BC0 | 000D

In C++, with a little-endian processor, I think it would be:

unsigned int turn12grayBytesInto4ColorBytes( unsigned int buf[3] )
{
   return (buf[0]&0x000000FF) // mask seems reversed because of little-endianness
        | (buf[1]&0x00FFFF00)
        | (buf[2]&0xFF000000);
}

It's probably fastest to do this another conversion to another buffer and THEN dump to disk, instead of going directly to disk.

Answer 6

You can use a codecvt facet on a local to filter out part of the output.
Once created you can imbue any stream with the appropraite local and it will only see every third character on the input.

#include <locale>
#include <fstream>
#include <iostream>

class Filter: public std::codecvt<char,char,mbstate_t>
{
    public:
   typedef std::codecvt<char,char,mbstate_t> MyType;
   typedef MyType::state_type          state_type;
   typedef MyType::result              result;

    // This indicates that we are converting the input.
    // Thus forcing a call to do_out()
    virtual bool do_always_noconv() const throw()   {return false;}

    // Reads   from -> from_end
    // Writes  to   -> to_end
    virtual result do_out(state_type &state,
             const char *from, const char *from_end, const char* &from_next,
             char       *to,   char       *to_limit, char*       &to_next) const
   {
       // Notice the increment of from
       for(;(from < from_end) && (to < to_limit);from += 3,to += 1)
       {
            (*to) = (*from);
       }
       from_next   = from;
       to_next     = to;

       return((to > to_limit)?partial:ok);
   }
};

Once you have the facet all you need is to know how to use it:

int main(int argc,char* argv[])
{
   // construct a custom filter locale and add it to a local.
   const std::locale filterLocale(std::cout.getloc(), new Filter());

   // Create a stream and imbue it with the locale
   std::ofstream   saveFile;
   saveFile.imbue(filterLocale);


   // Now the stream is imbued we can open it.
   // NB If you open the file stream first. 
   // Any attempt to imbue it with a local will silently fail.
   saveFile.open("Test");
   saveFile << "123123123123123123123123123123123123123123123123123123";

   std::vector<char>   data[1000];
   saveFile.write( &data[0], data.length() /* The filter implements the skipSize */ );
                                           // With a tinay amount of extra work
                                           // You can make filter take a filter size
                                           // parameter.

   return(0);
}