file-io

Searching for strings in big binary files

I have a big binary file (1 MB < size < 50 MB). I need to search for a string and extract the subsequent four bytes (which is the {size,offset} of actual data in another file). What is the most efficient way to do it so that the search would be fastest? EDIT: The strings in the index files are in sorted order. ...

How to insert / retrieve a file stored as a BLOB in a MySQL db using python

I want to write a python script that populates a database with some information. One of the columns in my table is a BLOB that I would like to save a file to for each entry. How can I read the file (binary) and insert it into the DB using python? Likewise, how can I retrieve it and write that file back to some arbitrary location on th...

File size restriction or limitation in C#

I want to generate XML file from one object (Contains nested collection) with large amount of data. but there is a limitation with XML that it can't exceed 50MB. Are there any good way to do this? Update : speed is not important, the main thing is split into 50MB for each file ...

Reading binary files without buffering the whole file into memory in C++

Hi, In order to make a binary comparer I'm trying to read in the binary contents of two files using the CreateFileW function. However, that causes the whole file to be bufferred into memory, and that becomes a problem for large (500MB) files. I've looked around for other functions that'll let me just buffer part of the file instead,...

SQL Server File Operations?

Using SQL Server 2005: How can I read a file into a SPROC using T-SQL? So, imagine I have a CSV file like so: ID,OtherUselessData<br/> 1,asdf<br/> 2,asdf<br/> 3,asdf<br/> etc... I basically want to do this: Select * from mytable where id in (select id from txtFile) ...

setting option in config file using SafeConfigParser

I'm trying to set an option (xdebug.profiler_enable) in my php.ini file using python's ConfigParser object. here is the code: section in php.ini file im trying to modify [xdebug] ;XDEBUG SETTINGS ;turn on the profiler? xdebug.profiler_enable=0 xdebug.profiler_append=1 xdebug.profiler_enable_trigger=0 xdebug.trace_output_name="%R" xd...

How to write data into text file in blackberry?

Dear Friends, I want to know how to write content in file. example:I am having one basic edit field and one save button if click the save button then it should write the content of basiceditfield into one text file and i can able to retrive that file in later.Can any one tell me how to do ? If u can provide me some code snippet...

Encoding.Default is not the same as no encoding in File.ReadAllText?

(Sorry if this is a dupe) I've just spent a long time trying to read a text file correctly. Having started with File.ReadAllText(path) and getting screwed-up characters, I tried several variants of File.ReadAlltext(path, Encoding) after which I got bogged down trying to analyse my input files to work out which byte was the problem, etc...

Slowdown of Microsoft Visual Studio due to different Virus scanner

What is the least slow Virus scanner to use with Microsoft Visual Studio I have just had Microsoft Visual Studio “go slow” on me again due to my Virus Checker… We all know that development tools do a lot of file access so are badly effect by Virus Scanner. Most of us have to run a Virus scanner due to do many reasons. So has anyone...

Unable to upload a file from web page in ASP.NET 3.5 web application running in Medium Trust

Possible Duplicate File I/O in medium trust? I get the following error when I try to upload a file from a web page: Request for the permission of type 'System.Security.Permissions.FileIOPermission, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed The web application runs on ASP.NET Framewor...

StreamReader.EndOfStream missing last line?

I am trying to read a text file using the code (pasted below), but the last line of the file does not get read. Is my logic correct? using (StreamReader reader = new StreamReader(stream)) { try { string line = reader.ReadLine(); string[] data = BreakLine(line); ...

Reading binary data with seekg fails.

I'm trying to read binary data from a specific offset. I write the data in the following way: long RecordIO::writeRecord(Data *record) { this->openWrite(); fstream::pos_type offset = file->tellp(); file->write(reinterpret_cast<char*>(record), sizeof(Data)); return (long)offset; } The offset returned is stored, and re...

How to take advantage of Disk IO queueing

I need to read small sequences of data from a 3.7 GB file. The positions I need to read are not adjacent, but I can order the IO so that the file is read from beginning to end. The file is stored on a iSCSI SAN which should be capable of handling/optimizing queued IO. The question is, how can I make a one shot request of all the data/p...

[Java] How to open file without extension

I would like to try open file without extension. When I try to open file without extension, then system show me "Open with" form. But when I am trying to open that file in side my application using method: private static void openFile(String fileName) throws IOException { if(Desktop.isDesktopSupported()) { Desktop desktop...

Best place to save program config file on Windows and OS X using Java?

I have a SWT Java app that runs on Windows XP / Vista / 7 and Mac OS X. I'm currently saving a config file to: System.getProperty("user.home") + filename With the changes in security in Windows Vista and Windows 7 this doesn't seem to be the best place to save it anymore. This file saves information such as registration key info and i...

FileSystemWatcher - how to determine when file is closed?

Hi, I'm downloading a file from a server and opening it using Process.Start() and attaching a file watcher to the file to catch any changes and re-upload them to the server. Is there anyway to determine when the file has closed using the FileWatcher or any other method? The problem being I can't decide how to stop watching the file an...

FileSystemWatcher - only the change event once firing once?

Hi, I'm using the below code to listen for change events of a file i download off a server and open. however the change event only gets fired the first time the file is saved, then on subsequent saves the file watcher does not fire the change events? Can anyone see whats going on? private FileSystemWatcher StartWatchingFile() { fw = ...

System.IO Versus VisualBasic.FileIO

I typically develop in C# and am quite accustom to using System.IO. I recently ran across the FileIO library and have found that some of the advantages (such as sending an item to the recycle bin) are quite nice to have. Especially in the program I am currently working on. My question is, Does System.IO out perform the VisualBasic.FileIO...

moving file pointer up one line (php)?

I am attempting to automate the sitemap.xml file on my site since the content is constantly changing. I currently open the file for appending: fopen($file_name, 'a'); so that I can add the new set of tags. However, I just noticed that the entire sitemap file has to be ended with a tag which means that every time I open the file, I ne...

Comparing files to each other

Hi I have written some piece of code that copies chunks of big files to a remote machine. Now I would like to figure out if I really copied all the bits into the right place by comparing those two files on a byte-per-byte basis. Is there some tool for windows that allows me to do that? I guess Linux has such a tool already built in, doe...