Hi, I need to write a text file viewer (not the directory tree, but the actual file contents) for use in a browser. It will be used to view large files. I want to give the user the ability to actually ummm, browse the file, ie prev page & next page buttons, while each page will show only a portion of the file. Two question: Is there a...
I have read that boost iostreams supposedly supports 64 bit access to large files semi-portable way. Their FAQ mentions 64 bit offset functions, but there is no examples on how to use them. Has anyone used this library for handling large files? A simple example of opening two files, seeking to their middles, and copying one to the oth...
I have a few relatively big files (Flash movies) which I want to serve. Where should I put them inside my project? I would like not to export them each time I export the .war file, since they take up a lot of space. ...
I wrote a simple PHP application to create salary cheques for the company employees. The interaction between db & app is nice, the data is cached & the performance is high enough. So, what's the problem? I need to provide the ability to print the whole cheques by just one click for the end-user. printing such a large HTML file is not ...
Why is the size of files capped at 4 GB when outputting to a file using print? I would expect that with streaming output it should be possible to generate files of arbitrary size. Update: ijw and Chas. Owens were correct. I thought the F: drive was NTFS formatted, but in fact it used the FAT32 filesystem. I tried it on another drive a...
I'd built an WSSv3 application which upload files in small chunks; when every data piece arrives, I temporarly keep it into a SQL 2005 image data type field for performance reasons**. Problem come when upload ends; I need to move data from my SQL Server to Sharepoint Document Library through WSSv3 object model. Right now, I can think t...
I'm a bioinformatician currently extracting normal-sized sequences from genomic files. Some genomic files are large enough that I don't want to put them into the main git repository, whereas I'm putting the extracted sequences into git. Is it possible to tell git "Here's a large file - don't store the whole file, just take its checksum,...
I've found loads of examples on to to replace text in files using regex. However it all boils down to two versions: 1. Iterate over all lines in the file and apply regex to each single line 2. Load the whole file. No. 2 Is not feasible using "my" files - they're about 2GiB... As to No. 1: Currently this is my approach, however I was won...
How to Send Large File From Client To Server Using WCF in C#? Below the configuration code. <system.serviceModel> <bindings> <basicHttpBinding> <binding name="HttpStreaming_IStreamingSample" maxReceivedMessageSize="67108864" transferMode="Streamed"> </bind...
Hi all, I need to work with large files and must find differences between two. And I don't need the different bits, but the number of differences. For the differ rows I come up with diff --suppress-common-lines --speed-large-files -y File1 File2 | wc -l And it works, but is there a better way to do it? And how to count the exact ...
Iostream, when all of the files it includes, the files that those include, and so on and so forth, adds up to about 3000 lines. Consider the hello world program, which needs no more functionality than to print something to the screen: #include <iostream> //+3000 lines right there. int main() { std::cout << "Hello, World!"; retu...
Hello, I have a huge (~2GB) file that I need to navigate around. I don't actually need to edit it, just jump around efficiently. I tried vim but it choked. Any recommendations for working with huge files on Linux? Thanks ...
So I've got an algorithm that reads from a (very large, ~155+ MB) binary file, parses it according to a spec and writes out the necessary info (to a CSV, flat text). It works flawlessly for the first 15.5 million lines of output, which produces a CSV file of ~0.99-1.03 GB. This gets through hardly over 20% of the binary file. After this ...
I'm interesting in experience of others developers with Zlib, and compressing large files, and how it will depends on application processes, is that going to make application slower, or not, or will decrease any other application performance. If there is any other better solution shoot. ...
Hello There, For a new project I need to load big XML files (200MB+) to a mySQL database. There are +- 20 feeds i need to match with that (not all fields are the same). Now when i want to catch the XML I get this error: Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 171296569 bytes) in E:\UsbWebserve...
I'm working on a project involving some large XML files (from 50MB to over 1GB) and it would be nice if I could view them in eclipse (simple text view is fine) without Java running out of heap space. I've tried tweaking the amount of memory available to the jvm in eclipse.ini but haven't had much success. Any ideas? ...
I have a large (~50Mb) file containing poorly formatted XML describing documents and properties between <item> </item> tags, and I want to extract the text from all English documents. Python's standard XML parsing utilities (dom, sax, expat) choke on the bad formatting, and more forgiving libraries (sgmllib, BeautifulSoup) parse the ent...
I am about to make the transition from using standard FILE pointers from some older code to using C++ streams but I need to have LARGEFILE seeking support (the compiler flags that activate this support are: *-D_FILE_OFFSET_BITS=64* et al) which I am able to obtain by using the *off64_t* datatype. My original question was answered regar...
I would put under version control a big amount of data, i.e. a directory structure (with depth<=5) with hundreds files with size about 500Mb). The things I need is a system that help me: - to detect if an files has been changed - to detect if files were added/removed - to clone the entire repository in another location - to store a ...
I have the need to compress a one Big file (~450 Mbyte) through the Java class ZipOutputStream. This big dimension causes a problem of "OutOfMemory" error of my JVM Heap Space. This happens because the "zos.write(...)" method stores ALL the file content to compress in an internal byte array before compressing it. origin = new Buff...