views:

119

answers:

2

Is opening a file stream a costly operation

I'd like to provide a lazy loading functionality in a class that reads a structured file. Every element of the file has a header and a payload.

the idea is to load only the headers from the file and access the payload data only when the relevent field is accessed

a prototype of the class would look like this in C#

public class OneElement
{
     public Header TheHeader {get; private set;}

     private string _FileName;

     private long _StreamPosition; // this value is initialized when the header is read

     private Payload _ThePayload;


     public Payload ThePayload
     {
         get{
                    if (_ThePayload == null)
                        using (var stream = File.OpenRead(_FileName) )
                        {
                             stream.seek(_StreamPosition,SeekOrigin.Begin); // seek to the relevent position
                             _ThePayload =  ReadPayload(stream); // this method return the paylod read reads from the current position
                        }
                    return _ThePayload;

               }
     }
}

Will the operation of opening the file and getting the payload be costly especially in a context where the payload will represent audio or video data

+4  A: 

If you're reading audio/video then I'd expect you to be reading rather a lot of data. That would dwarf the cost of just opening the file. Reading large amounts of data from disk is generally a costly operation though.

On the other hand, if you were just reading a few bytes at a time then repeatedly opening/closing the file wouldn't be a good idea - it would be better to read large chunks and cache them appropriately.

Do you have a benchmark for this? How does it perform at the moment? If it's working okay, do you have any reason to try to think of a more complicated system?

Jon Skeet
i haven't had the time to tried it yet (and we actually don't really need it right now), I was thinking of implementing this like that case to allow the user to filter in a large file, for example getting all the keyframe from the file would look like var keyframes = ListElements.Where(e => e.IsKeyframe);and that would not cause the loading of all frame in totality
Johnny Blaze
So that would mean loading each frame separately, opening and closing the file many thousands of times? That could be a significant cost.
Jon Skeet
+1  A: 

Opening a file does cost some resources. But if your payload is audio or video it will be far more resource intensive to read those than opening the file.

So in case you are trying to cache the contents to save you a single file open forget about that idea.

Foxfire