csv

How can i export records of more than 1,00,000 from sql server database to CSV formats in LINQ?

My team working asp.net File manage project In our project we need to export database contents to csv formats The Database table contains 6 fields We are using LINQ. ...

Data extraction from source with lots of white space

I'm trying to extract data from : http://www.phillysheriff.com/old_site/properties.html Ideally I'd be able to get a CSV file with the address, ward, price, and square feet? Is there an easy way to do this? ...

Upload and parse csv file with google app engine

Hi, I'm wondering if anyone with a better understanding of python and gae can help me with this. I am uploading a csv file from a form to the gae datastore. class CSVImport(webapp.RequestHandler): def post(self): csv_file = self.request.get('csv_import') fileReader = csv.reader(csv_file) for row in fileReader: ...

Unexpected Blank lines in python output

I have a bit of code that runs through a dictionary and outputs the values from it in a CSV format. Strangely I'm getting a couple of blank lines where all the output of all of the dictionary entries is blank. I've read the code and can't understand has anything except lines with commas can be output. The blank line should have values...

Capturing Field Name Metadata from a CSV File in Altova MapForce

I've been asked to prototype a replacement "file transformation process" (that currently is a mess of SQL) using Altova's MapForce. My input is a CSV file with headers. My problem is that I need to capture both the data AND the column name to use in downstream processing. I need to have MapForce feed a C# method (imported as that takes ...

How do i ignore rows with unique string while parsing a csv file in ruby

i want to ignore ALL TopLevel, Result,,,,,,,, AddHost,10.1.3.1,,,,,,,, and Flush,,,,,,,,, AddHost,10.1.3.4,,,,,,,, from TopLevel, Result,,,,,,,, AddHost,10.1.3.1,,,,,,,, Add,10.1.3.1,43172 Add,10.1.3.1,44172 Add,10.1.3.1,4172 Add,10.1.3.1,432 Add,10.1.3.1,435472 Flush,,,,,,,,, AddHost,10.1.3.4,,,,,,,, ...

CSV to MySQL conversion and import

Hi all, Im working on a large project, havent had to do what I need help with before. I have a csv file, it contains a large amount of data, namely all of the cities, towns, suburbs in Australia. I need to convert the csv file to sql for mysql, and then import it into the database. What would be the best way to achieve this? Thanks al...

Question about DBD::CSB Statement-Functions

From the SQL::Statement::Functions documentation: Function syntax When using SQL::Statement/SQL::Parser directly to parse SQL, functions (either built-in or user-defined) may occur anywhere in a SQL statement that values, column names, table names, or predicates may occur. When using the modules through a DBD or in any other con...

Efficient way to access a mapping of identifiers in Python

I am writing an app to do a file conversion and part of that is replacing old account numbers with a new account numbers. Right now I have a CSV file mapping the old and new account numbers with around 30K records. I read this in and store it as dict and when writing the new file grab the new account from the dict by key. My question i...

Definitive way to parse alphanumeric CSVs in Python with scipy/numpy

I've been trying to find a good and flexible way to parse CSV files in Python but none of the standard options seem to fit the bill. I am tempted to write my own but I think that some combination of what exists in numpy/scipy and the csv module can do what I need, and so I don't want to reinvent the wheel. I'd like the standard feature...

speeding up parsing of files

the following function parses a CSV file into a list of dictionaries, where each element in the list is a dictionary where the values are indexed by the header of the file (assumed to be the first line.) this function is very very slow, taking ~6 seconds for a file that's relatively small (less than 30,000 lines.) how can I speed it up...

writing header in csv python with DictWriter

assume I have a csv.DictReader object and I want to write it out as a csv file. How can I do this? I thought of the following: dr = csv.DictReader(open(f), delimiter='\t') # process my dr object # ... # write out object output = csv.DictWriter(open(f2, 'w'), delimiter='\t') for item in dr: output.writerow(item) Is that the best wa...

Specifying formatting for csv.writer in Python

I am using csv.DictWriter to output csv files from a set of dictionaries. I use the following function: def dictlist2file(dictrows, filename, fieldnames, delimiter='\t', lineterminator='\n'): out_f = open(filename, 'w') # Write out header header = delimiter.join(fieldnames) + lineterminator out_f.write(header...

Java: CSV file read & write.

Im reading in 2 csv file: store_inventory & new_acquisitions... I want to be able to compare the store_inventory csv file with new_acquisitions. 1) If the item names match just update the quantity in store_inventory. 2) If new_acquisitions has a new item that does not exist in store_inventory, then add it to the store_inventory. Heres ...

Java: Use a downloaded java library with NetBeans

I am trying to use opencsv library http://opencsv.sourceforge.net/ How to i import it so that i can start using it with NetBeans? I have no idea what im doing so start from the very beggining after downloading and unzipping the file. Thanks. ...

Check if file has a CSV format with Python

Could someone provide an effective way to check if a file has CSV format using Python ? ...

Using excel import wizard through code?

I am creating a Generate Report button for some data in my application. The report will just generate a CSV file, which I then automatically launch in excel. Problem is with one of the data fields, is text and has leading zeroes (which I need to keep). When the file is automatically opened in excel, excel automatically trims these. Is ...

Culture independent CSV

Hi all. I wonder if there is any way to generate culture neutral CSV file or at least specify data format of certian columns present in file. For example I generated CSV file that contains numbers with decimal separator (.), and after pass it to the client which is in the country where decimal separator is (,), client opens it with Ex...

Python: split files using mutliple split delimiters

Hi, I have multiple CSV files which I need to parse in a loop to gather information. The problem is that while they are the same format, some are delimited by '\t' and others by ','. After this, I want to remove the double-quote from around the string. Can python split via multiple possible delimiters? At the minute, I can split the l...

.NET how to output csv from enumeration of anonymous type?

Using FileHelpers, I decorated a class with [DelimitedRecord(",")] and was going to output an enumeration of objects of that type as CSV. But, it didn't work because my class inherits from ActiveRecordLinqBase<T>, which caused some problems. So, I was wondering if I could just select an enumeration of anonymous types and somehow have f...