Hi All,
I have a file that is in structured storage format. I was wondering if this format be accessed concurrently by threads.
Meaning have multiple threads read the different streams process it at once. The objective is to load the file faster.
When i refer to a file i refer one that represents CAD information.
Thank you.
The question with threads always boils down to sharing resources. Define what resources will be shared, and then figure out if they can be shared in such a way that big chunks of processing can happen in parallel.
If you are building a lot of small independent data structures from a table of contents at the beginning of the file, it could benefit from threads. Each thread would know, independent of the other threads, what part of the file to read and what to do with the data it read, without interacting with other threads.
If you are building a large tree or hierarchy of related data structures, or if the location of the next bit of data depends on the previous bit of data, then more than a few threads could get bound up waiting for each other. You still might benefit from one thread that reads and queues up chunks of data for another thread to process.
Most likely you will have some balance between the above examples. You might have one file reader thread and a small pool of processing threads. The processing threads might queue up results for a single assembling thread to organize into whatever hierarchy or relationships there are.
If you have threads waiting for each other, then you did not need a thread. If you have independent chunks of data that can be atomically added to queues for processing, then threads get to shine.
Depending on externalities, threads can interfere with each other in subtle ways. For example, multiple read threads can cause more seek time for hard drives, resulting in a net performance loss.
The simple answer is yes.
The longer answer is Structured Storage is part of COM and the COM threading model (sometimes called the Apartment model) is different to the Win32 threading model and has its own rules and API for marshalling data across apartments.
Before you embark on this, ask yourself if your storage mechanism really has to be structured storage...IMO it does not give you much over standard IO except a world of hurt (threading wise)
The actual structured part is pretty basic and any XML file is far more structured.