What would be the fastest way to do it? I remember someone telling me using ifstream was bad because it worked on a small number of bytes, and it would be better to just read the file into memory first. Can anyone confirm this? Thanks Edit: I am running on windows, and the file format is for a point cloud that is stored in rows like ...
I am creating a method in C# which generates a text file for a Google Product Feed. The feed will contain upwards of 30,000 records and the text file currently weighs in at ~7Mb. Here's the code I am currently using (some lines removed for brevity's sake). public static void GenerateTextFile(string filePath) { var sb = new StringBui...
I built an app that performs work on thousands of files, then writes modified copies of these files to the disk. I am using a ThreadPool but it was spawning so many threads the pc was becoming unresponsive 260 total), so i changed the max from the default of 250 down to 50, this solved that issue (app only spawns about 60 threads total)...
Having recently worked on a project which required some more IO interaction than I'm used to, I felt like I wanted to look past the regular libraries (Commons IO, in particular) and tackle some more in depth IO issues. As an academic test, I decided to implement a basic, multi-threaded HTTP downloader. The idea is simple: provide a URL ...
The Python 2.7 update note says: A new version of the io library, rewritten in C for performance. I've played with Python 2.7 a bit, but I don't see any performance gain: >>> from timeit import Timer >>> t = Timer('f = open("E:\\db.txt", "r"); f.read(); f.close()') >>> t.timeit(10000) And the result: Python 2.6.5 -- 12....
I know how to use numpy.savetxt to write an array to a file. How can I write multiple arrays to the same file? Essentially I want to do math to a column of numbers, and then replace the old column with the modified numbers. I read the easiest way to do this is to write a new file completely, put the modified numbers in, and just 'copy...
I have a VPS in cloud environment based on XEN Virtualization platform. /etc/fstab looks like that: LABEL=PRGMRDISK1 / ext3 errors=remount-ro 0 0 As far as I know IO can pretty easily become a bottleneck if some other VPS neighbor will use it intensively(IO bandwidth is a shared resource for multiple guest VPS). ...
i am using the following code to write an array to the file: FileWriter fstream1=new FileWriter("outx.txt"); BufferedWriter out1= new BufferedWriter(fstream1); FileWriter fstream2=new FileWriter("outy.txt"); BufferedWriter out2= new BufferedWriter(fstream2); for(int i=0;i<320*240;i++) { out1.write(0+System.getProperty("line.se...
I'm creating a console application where I would like to have two outputs and one input. The reason would be so that one output is always visible. This is the first output Text flows upwards just like a regular console application, however... --------- This is a second output This is placed at the bottom of the console // also input go...
Here is a sample program that uses stringstream. The goal is to accept lines from the user(standard input) and print each word in a separate line. int main() { std::istringstream currentline; std::string eachword; std::string line; // Accept line from the standard input till EOF is reached while ( std::getline(std:...
I'm confused as to what happens when I issue a write from user space in Linux. What is the full flow, down to the storage device? Supposing I use CFQ and a kernel that still uses pdflush. CFQ is said to place requests into sync and a-sync queues. Writes are a-sync, for example, so they go into an a-sync queue based on the io priority. T...
How can I perform restricted instructions such as IN and OUT from protected mode? I've found out that It would require privilege level (CPL) high enough to perform the IO instruction. How can I run in kernel mode, have IO permission or anything other that may help me? - I would like to have direct access to hardware, without anything bl...
Hi, I'm using Spring's Resource abstraction to work with resources (files) in the filesystem. One of the resources is a file inside a JAR file. According to the following code, it appears the reference is valid ResourcePatternResolver resourceResolver = new PathMatchingResourcePatternResolver(); // The path to the resource from the ro...
I have written an object to file but when i read the object from the file is not getting the correct value. if 1 object is considered as 1 record if only 1 record is there in the file its getting the value. if i write more record (many object of same type) then its not getting the value. plz help me what to do. ...
Good applications should expect short reads/write almost everywhere. But actually, for example, when reading small number of bytes from filesystem or from pipe short reads just [almost] never occur and defects in it's handling can remain undetected. How to test group of applications interconnected by pipes and sockets and reading and w...
I need to write multiple blocks of the following chunk into a file +----------+--------------+---------------------+ | FileName | Size of File | Binary Dump Of File | +----------+--------------+---------------------+ There are two variants of file writers I could see. 1. FileWriter - allows writing [string,char[],one byte] 2. BufferF...
launchd has option to run process with low priority I/O. How does it work exactly? (how low is low, does it affect all operations?) Is there an API that enables low priority I/O in applications not launched via launchd? I need to scan watched (FSEvents) directories in background application, and I'd like this to be as gentle as possibl...
Hi, I'm writing a Maven plugin that deletes and renames various files using the File.delete() and File.renameTo(File) JDK methods. Roughly every second time I run the plugin, one of these operations fails, and each time it fails it's a different file that cannot be deleted or renamed. An obvious explanation for why a file cannot be del...
I have a dynamic thumbnail script I found laying around the web and tweaked a bit. One of the things I added was a caching mechanism. Whenever a new thumb is generated, it is saved to disk, and the disk copy will be used if the same thumbnail (with all the same options) is requested again. A snippet: // name of cached file $thumb_...
I am trying to understand concurrency in Haskell more deeply. I have the following code: import Control.Concurrent main :: IO () main = do arr <- return $ [1..9] t <- newMVar 1 forkIO (takeMVar t >> (print.show) arr >> putMVar t 1) forkIO (takeMVar t >> (print.show) arr >> putMVar t 1) forkIO (takeMVar t >...