Hi, I need to parse a bunch of delimited flat files and import them into a database. I could build my own simple metadata-driven parser and importer but figure there must be something like this out there already? Thanks, Dave. ...
Hi all, I have a 10^7 lines file, in which I want to choose 1/100 of lines randomly from the file. This is the AWK code I have, but it slurps all the file content before hand. My PC memory cannot handle such slurps. Is there other approach to do it? awk 'BEGIN{srand()} !/^$/{ a[c++]=$0} END { for ( i=1;i<=c ;i++ ) { num=int(r...
I have the following data that looks like this for example: 34 foo 34 bar 34 qux 62 foo1 62 qux 78 qux These are sorted based on the first column. What I want to do is to process lines that starts with 34, but I also want the file iteration to quit after it finds no more 34s, without having have to scan through whole f...
Following up on this question, I need to get exactly n lines at random out of a file (or stdin). This would be similar to head or tail, except I want some from the middle. Now, other than looping over the file with the solutions to the linked question, what's the best way to get exactly n lines in one run? For reference, I tried this:...
Hi, I'd need to read and process somewhat large file with Java and I'd like to know, if there is some sensible way to protect the file that it wouldn't be overwritten by other processes while I'm reading & processing it? That is, some way to make it read-only, keep it "open" or something... This would be done in Windows environment. ...
I am trying to append to a log file, using HTML tables format for each incident. However, there is one difficulty which I am not sure how to solve. Here's a static output of the HTML <table> <tr><td> .... </td> </tr> <!-- new log entry should enter here --> </table> </html> The question is how I can quickly insert log entries...
I have been tasked with building a new workflow system to handle our service orders. I have been investigating Windows Workflow Foundation as the engine for managing the workflow steps, and like what I see up until file processing. We have a step in our workflow where we are waiting for a file to be returned from a vendor. The file con...
Is there a way to append or remove a line of text from a file using PHP. I am in the process of writing a hosting control panel for my specific web hosting stack and would like to be able to make changes to the files with minimal requirements to touch the file system, and as such would like not to have to rewrite the whole file to add o...
I need to take a file and find the first occurrence of a literal string pattern as a complete line of the file: Acknowledgments: And then I wish to create a new file from the line of match all the way to the end of the file. I expect perl is a good way to do this, but I'm not much of a perl person, alternatively maybe sed is a good w...
I've created a small tool which is used to process binary files. All the functions involved in reading the files and processing them are templated and are similar to this: template <class T> void processFile( const char* fileName ); The template parameter T is used to determine the size of data which will be read and treated as one it...