buffer

Why doesn't Ruby have a real StringBuffer or StringIO?

I recently read a nice post on using StringIO in Ruby. What the author doesn't mention, though, is that StringIO is just an "I." There's no "O." You can't do this, for example: s = StringIO.new s << 'foo' s << 'bar' s.to_s # => should be "foo\nbar" # => really is ''` Ruby really needs a StringBuffer just like the one Java has. Stri...

Binary buffer in Python

In Python you can use StringIO for a file-like buffer for character data. Memory-mapped file basically does similar thing for binary data, but it requires a file that is used as the basis. Does Python have a file object that is intended for binary data and is memory only, equivalent to Java's ByteArrayOutputStream? The use-case I have i...

Is there a reason to use BufferedReader over InputStreamReader when reading all characters?

I currently use the following function to do a simple HTTP GET. public static String download(String url) throws java.io.IOException { java.io.InputStream s = null; java.io.InputStreamReader r = null; //java.io.BufferedReader b = null; StringBuilder content = new StringBuilder(); try { s = (java.io.InputStrea...

screen scraping a command window using .net managed code

I am writing a program in dot net that will execute scripts and command line programs using the framework 2.0's Process object. I want to be able to access the screen buffers of the process in my program. I've investigated this and it appears that I need to access console stdout and stderr buffers. Anyone know how this is accomplished us...

Buffer a large file; BufferedInputStream limited to 2gb; Arrays limited to 2^31 bytes

I am sequentially processing a large file and I'd like to keep a large chunk of it in memory, 16gb ram available on a 64 bit system. A quick and dirty way is to do this, is simply wrap the input stream into a buffered input stream, unfortunately, this only gives me a 2gb buffer. I'd like to have more of it in memory, what alternatives d...

How do you write the end of a file opened with FILE_FLAG_NO_BUFFERING?

I am using VB6 and the Win32 API to write data to a file, this functionality is for the export of data, therefore write performance to the disk is the key factor in my considerations. As such I am using the FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH options when opening the file with a call to CreateFile. FILE_FLAG_NO_BUFFERING ...

What is the buffer size to create a .zip archive using Java?

Hello, I use this code to create a .zip with a list of files: ZipOutputStream zos = new ZipOutputStream(new FileOutputStream(zipFile)); for (int i=0;i<srcFiles.length;i++){ String fileName=srcFiles[i].getName(); ZipEntry zipEntry = new ZipEntry(fileName); zos.putNextEntry(zipEntry); InputStream fis = new FileInputStream...

How do you determine the ideal buffer size when using FileInputStream?

I have a method that creates a MessageDigest (a hash) from a file, and I need to do this to a lot of files (>= 100,000). How big should I make the buffer used to read from the files to maximize performance? Most everyone is familiar with the basic code (which I'll repeat here just in case): MessageDigest md = MessageDigest.getInstance(...

What's the point of a cache on the swap space?

Isn't the point of caching things in main memory to avoid the expensive disk i/o? If you're caching things in the swap space of a hard drive, how does that avoid disk i/o? ...

Java Video Player

I need a java api, that cannot use JMF, to play video interpreted by the SO codecs but i want to retrieve the each frames in java code. Somebody know some? ...

When to build your own buffer system for I/O (C++)?

I have to deal with very large text files (2 GBs), it is mandatory to read/write them line by line. To write 23 millions of lines using ofstream is really slow so, at the beginning, I tried to speed up the process writing large chunks of lines in a memory buffer (for example 256 MB or 512 MB) and then write the buffer into the file. This...

Buffer pool management using C#

We need to develop some kind of buffer management for an application we are developing using C#. Essentially, the application receives messages from devices as and when they come in (there could be many in a short space of time). We need to queue them up in some kind of buffer pool so that we can process them in a managed fashion. We w...

Buffering data for delimiter separated blocks

Hi There is a question I have been wondering about for ages and I was hoping someone could give me an answer to rest my mind. Let's assume that I have an input stream (like a file/socket/pipe) and want to parse the incoming data. Let's assume that each block of incoming data is split by a newline, like most common internet protocols. T...

Best Collection To Use?

Hi There, I am reading log files but not all lines want to be processed straight away. I am using a queue / buffer to store the lines while they wait to be processed. This queue is regularly scanned for particular lines - when they are found, they are removed from the queue (they can be anywhere in it). When there isn't a particular li...

Java: CharBuffer vs. char[]

Is there any reason to prefer a CharBuffer to a char[] in the following: CharBuffer buf = CharBuffer.allocate(DEFAULT_BUFFER_SIZE); while( in.read(buf) >= 0 ) { out.append( buf.flip() ); buf.clear(); } vs. char[] buf = new char[DEFAULT_BUFFER_SIZE]; int n; while( (n = in.read(buf)) >= 0 ) { out.write( buf, 0, n ); } (where in...

What is the best resizable circular byte buffer available in Java?

I need a byte buffer class in Java for single-threaded use. I should be able to insert data at the back of the buffer and read data at the front, with an amortized cost of O(1). The buffer should resize when it's full, rather than throw an exception or something. I could write one myself, but I'd be very surprised if this didn't exist y...

How do I run an emacs hook when a buffer is modified?

Building on http://stackoverflow.com/questions/318553/getting-emacs-to-untabify-when-saving-files , I'd like to run a hook to untabify my C++ files when I start modifying the buffer. I tried adding hooks to untabify the buffer on load, but then it untabifies all my writable files that are autoloaded when emacs starts. (For those that wo...

MemoryStream.Read doesn't copy bytes to buffer - c#

I don't really get it and it's driving me nuts. i've these 4 lines: Image img = Image.FromFile("F:\\Pulpit\\soa.bmp"); MemoryStream imageStream = new MemoryStream(); img.Save(imageStream, ImageFormat.Bmp); byte[] contentBuffer = new byte[imageStream.Length]; imageStream.Read(contentBuffer, 0, contentBuffer.Length); when debugging i ca...

Python: flush a buffer before program termination via a finalizer

I keep a cache of transactions to flush (to persistent storage) on the event of a watermark or object finalization. Since __del__ is no longer guaranteed to be called on every object, is the appropriate approach to hook a similar function (or __del__ itself) into atexit.register (during initialization)? If I'm not mistaken, this will ca...

How to disable Response.Buffer

this seems so silly - i must be missing something obvious. I have the following code (just as a test): <%@ Page Language="C#" %> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"&gt; <script runat="server"> void page_load(object o, EventArgs e) { Re...