how to test io performance on linux
how to test io performance on linux ...
file-io
How can I create a file with StreamWriter using a relative path?
When I run the following code, an XML file is correctly created in c:\temp: XmlSerializer xs = new XmlSerializer(typeof(ObservableCollection<Models.Customer>)); using (StreamWriter wr = new StreamWriter("C:/temp/CustomerMock2.xml")) { xs.Serialize(wr, CustomerList); } However, I actually want it to be created in a sub-directory un...
DirectoryInfo.GetFiles slow when using SearchOption.AllDirectories
I am searching a moderate number (~500) of folders for a large number (~200,000) of files from a .NET application. I hoped to use DirectoryInfo.GetFiles, passing in SearchOption.AllDirectories. However this approach seems to be a lot slower than writing my own code to iterate through the directories and do GetFiles just passing in a sea...
Non-threadsafe file I/O in C/C++
While troubleshooting some performance problems in our apps, I found out that C's stdio.h functions (and, at least for our vendor, C++'s fstream classes) are threadsafe. As a result, every time I do something as simple as fgetc, the RTL has to acquire a lock, read a byte, and release the lock. This is not good for performance. What's ...
What is the Fastest Method for High Performance Sequential File I/O in C++ ?
Assuming the following for... Output: The file is opened... Data is 'streamed' to disk. The data in memory is in a large contiguous buffer. It is written to disk in its raw form directly from that buffer. The size of the buffer is configurable, but fixed for the duration of the stream. Buffers are written to the file, one after another. ...
When doing a parallel search, when will memory bandwidth become the limiting factor?
I have some large files (from several gigabytes to hundreds of gigabytes) that I'm searching and trying to find every occurrence of a given string. I've been looking into making this operate in parallel and have some questions. How should I be doing this? I can't copy the entire file into memory since its too big. Will multiple FILE* ...
Unable to overwrite uploaded images in C# using the Image object
I am trying to create a utility in C# using the MVC framework where a user uploads a picture which is used to crop pieces out of to be used as thumbnail icons (only one user at a time will be doing this at any given time). The user can then upload a different picture and continue cropping. I have a controller action that handles the fi...
PHP Simultaneous File Writes
I have two different PHP files that both write to the same file. Each PHP script is called by a user action of two different HTML pages. I know it will be possible for the two PHP files to be called, but will both PHP files attempt to write to the file at the same time? If yes, what will happen? Also, it is possible to make one of the PH...
SAS printing to file
I have a SAS file handle created as such: filename filehandle "report.htm"; I have several files (packed Javascript files) which have very long lines (>32K in length). I would like a way to append their contents to the file above. I know that if I do: data _null_; file filehandle; put "very long string here"; run; Sometime...
How might I remove duplicate lines from a file?
I have a file with one column. How to delete repeated lines in a file? ...
Read Something After a Word in C++
Hello, I'm building a simple interpreter of a language that i'm developing, but how i can do a cout of something that is after a word and in rounded by "", like this: #include <iostream> #include <fstream> #include <string> #include <cstdlib> using namespace std; int main( int argc, char* argv[] ) { if(argc != 2) { cout << "Err...
Copy files with widestring path in C++
I'm having some trouble using wchar_t* strings for copying a file, how do I open them in C/C++ I need to use wide chars because the filenames are in unicode with different foreign languages. Thanks in advance. ...
Load a System.IO.FileStream using a WildCard
I am trying to load a file that I know part of the name (and know that it will be uniquely identified by the part that I know.) Here is the jist of it: string fileName = ID + " - " + Env + " - "; byte[] buffer; using (FileStream fileStream = new FileStream(Server.MapPath("~") + fileName + "*", FileMode.Open)) { using (BinaryRead...
Saving a file in Java
I have a String that holds a path to a file. I want the users to be able to select a path and filename with a file chooser and have the program save the file given in the String variable to a location of their choice. What is the best way to do this? ...
File open mode inconsistency between C# and C++
Hi, I am exporting a function from vc++ DLL to write to a binary file. In the C++ code the file is opened using FILE* fp = ::_tfopen (FilePath, _T("a+b")); I a using the "a+b" mode to append the file later on and b is for the binary mode. Now I am importing this function in my C# application. When I make a call to this function fro...
std::getline and eol vs eof
I've got a program that is tailing a growing file. I'm trying to avoid grabbing a partial line from the file (e.g. reading before the line is completely written by the other process.) I know it's happening in my code, so I'm trying to catch it specifically. Is there a sane way to do this? Here's what I'm trying: if (getline (stream,...
What does CancelIo() do with bytes that have already been read?
What happens if I ReadFile() 10 bytes (in overlapped mode without a timeout) but invoke CancelIo() after 5 bytes have been read? The documentation for CancelIo() says that it cancels any pending I/O, but what happens to the 5 bytes already read? Are they lost? Are they re-enqueued so the next time I ReadFile() I'll get them again? I'm l...
Store data series in file or database if I want to do row level math operations?
I'm developing an app that handle sets of financial series data (input as csv or open document), one set could be say 10's x 1000's up to double precision numbers (Simplifying, but thats what matters). I plan to do operations on that data (eg. sum, difference, averages etc.) as well including generation of say another column based on co...
Read local file in Flash
Is it possible to read a local file in a Flash? My application uploads files to server using the flash.net.FileReference class, and I would like to peek inside before I upload them. There are two indications as to why it could be possible: I'm going to see the content of the files on the server anyway. Silverlight can do that. The Syst...
Howto cancel a delayed FileInputStream file-open
Is there a way to cancel a long running file open operation? Long delays due anti-virus scans, mapped network drives etc. new FileInputStream("a_file_that_the_antivirus_" + "will_process_for_minutes_before_returning.jar") // process the contents... In the above example, the first line can block, but I don't know any way to asynch...