views:

69

answers:

3

What are the underlying constructs of the concept known as a "stream"? How do they work internally?

How does a memory stream work?

How does a file stream work?

Edit - From the answers I'm guessing that each type of stream is implemented differently, but the core concept is a sequence of bytes controlled and modelled by a class or a set of functions. Is that correct?

A: 

The concept of a "stream" is simply a sequence of bytes encapsulated by a class that tracks the length, current position, number of "objects" or bytes on the stream etc... and provides for seeking to and fro in the stream of objects (or bytes, etc...)

So the underlying "construct" as you put it, is simply an array of bytes (or a file on disk or pointers to classes, structures or even integral types) which "serves" up the objects on it according to the streams encapsulation.

Imagine that I have 10 rubber duckies and am standing next to an actual real world stream of water. I place a duck in the stream and it starts to float down-stream. I now have 1 duck on the stream. I place another duck in the stream and it starts to float down-stream. I now have 2 ducks on the stream. This continues on until... you run out of memory. Now I run downstream and grab the 2nd ducks from the water (seeking). There are now 9 ducks in the stream and I'm at position 2...

So streaming an object differs from placing it in an array or a list in that it can be "operated on" by the encapsulating stream class.

What's neat about streaming is that you can create different streams on top of other streams allowing for a certain level of abstraction. For example in .NET I can create a StreamWriter on top of a FileStream which allows me to write strings to the file stream without having to manually convert them to bytes arrays.

Hopefully that helps. Feel free to correct or add whatever is necessary. I don't claim to be an expert.

(BTW, there is no need to be obtuse and claim "too general", "i don't understand" etc... Let's be a little more understanding and courteous, get down off our pulpits and explain things in a reasonable fashion.)

You are describing a very narrow meaning of streams here. Streams need not be of bytes only, they may not have a definite length (even .NET streams - think `NetworkStream`!), they may not be seekable, etc. Most definitely, the underlying concept is not an array of bytes, either - show me one in `FileStream` or, again, `NetworkStream`.
Pavel Minaev
@Pavel - Since you seem to be the expert, why don't you answer the question with your expertise instead of just commenting.
@Pavel - BTW, you come across as very snide and rude.
As I pointed out in the comment, the question as asked doesn't have a good answer. Yours isn't one because it is factually incorrect, and I do not want factually incorrect information to propagate because someone sees it without knowing better, and relies on it (or, worse still, repeats it) because it sounds right. Which is why I feel obliged to point out things that are incorrect in your reply, so that readers do not mistakingly take them for truth. If you consider it offensive, then it implies that you consider your answer perfect - so which one of us is snide here, then?
Pavel Minaev
@Pavel - That's not my point. My point is you keep criticizing without providing an answer. That's what makes you snide.
I do not have to provide an answer to criticize an obviously incorrect one. Nonetheless, the question was edited to be more specific, so the answer is here.
Pavel Minaev
+1  A: 

How about this definition from Wikipedia: http://en.wikipedia.org/wiki/Stream_(computing)

Colin Mackay
Particularly of interest are the first definition, which discusses a stream as a series of bytes (or other objects), and the third one, from Scheme, which defines a stream as being a logical sequence with no defined length. (Think about that last, it's a very nice abstraction, and the highest level definition I can think of.)
quark
A good response to question as it originally stood, but it's been edited to be a tad more specific now.
Pavel Minaev
A: 

I'll assume that you are interested in C# FileStream and MemoryStream (or their analogs in Java, and other similar languages/frameworks).

A memory stream exposes a contiguous block of memory via stream interface. This is typically implemented as an array and an index (or pointer) indicating current byte. Memory streams can be non-growing, in which case size of array is defined in advance and cannot change, and growing, in which case new array is allocated, and memory is copied, if existing array does not fit the data. As an optimization, for growing streams array is typically allocated of size larger than what is needed to hold all data presently in the stream - this is so that when you write more data in the end, that unused space can be quickly used, without need for array reallocation on every write. With this optimization, stream will need to keep track of how much space in the array is unused, in addition to its size.

File streams are usually much simpler, because the OS itself already exposes files as streams (i.e. you get OS APIs to read/write/seek within files). So file streams typically just wrap an OS file handle, and provide methods that delegate to those OS APIs. Sometimes they provide additional buffering for performance reasons - i.e., data that is given to FileStream.Write isn't passed on to OS "write block to file" API immediately, but is cached in a buffer (usually, array). When enough writes happen that the buffer is full, the OS API is called.

How the OS itself implements those stream primitive operations on its files is highly OS-specific.

Pavel Minaev