csv

Reading large csv files with strings containing commas as one field

Hi, I have a large .csv file (~26000 rows). I want to be able to read it into matlab. Another problem is that it contains a collection of strings delimited by commas in one of the fields. I'm having trouble reading it. I tried stuff like tdfread, which won't work here. Any tricks with textscan i should be aware about? Is there any oth...

What's the best way to read a huge CSV file using Perl?

Requirements: a) I have a very large CSV file to read (about 3Gb). b) I won't need all records, I mean, there are some conditionals that we can use, for example, if the 3rd CSV column content has 'XXXX' and 4th column has '999'. Can I use these conditionals to improve the read process? If so, how can I do that using Perl? Please you s...

Need Dynamic/ default width for CSV file when EXPORTING through SERVLET

Hi all, I am exporting my List to CSV file through Servlet. Everything is working fine. but i want to set default/ dynamic width for cell/column? Here is my coding. Your swift reply will be helpful.. Thanks in advance. package com.uson.stat.action; import javax.servlet.ServletException; import javax.servlet.http.HttpServlet; import ...

Why when I try to output a query result to CSV in SSIS instead of decimal numbers I get ones and zeros?

I have created a very simple Data Flow in SSIS that is run inside a loop. I have a simple OLE DB Source control which is connecting to a SQL Server and running quite a complex query to split daily data by 30 minute intervals as shown below. I then have a Flat File Destination control which is taking the output from the OLE DB Sourc...

django csv import threading

Is it possible to use threading when importing data from csv files to django. ...

Import huge 550000+ row CSV file into Access

Hi All, I have a CSV file with 550000+ rows, I need to import this data into Access, but when I try it throws an error that the file is too large (1.7GB), can you recommend a way to get this file into Access? Thanks, Darryl ...

Authorize.net Integration

I have integrated authorize.net into my site. I heard there was a way for authorize.net to send a merchant weekly CSV files that document payment statuses of customers. The other option I've noticed is the Silent Post, which I prefer not to use. How do I get the weekly CSV files if it's even possible? I believe it doesn't work in the tes...

.csv file issue with split lines in unix, not windows

Hi - got a problem I'm sure someone somewhere has encountered before. We'd been FTPing a customers .csv files down to our laptops, then SQLLoading them to our Oracle DB's, but network made it a slow process.. I set up a shell script to LFTP those files down to the Solaris DB box, and sqlload them - much faster. There were some character ...

Python regex for reading CSV-like rows

I want to parse incoming CSV-like rows of data. Values are separated with commas (and there could be leading and trailing whitespaces around commas), and can be quoted either with ' or with ". For example - this is a valid row: data1, data2 ,"data3'''", 'data4""',,,data5, but this one is malformed: data1, data2, da"ta3", 'd...

writing to data to excel

Hello, I have data that I need to export to excel, I just don't know how to go about it, here's the view I'm using, I've commented out my attempts.A push to the right direction will be greatly appreciated. def month_end(request): """ A simple view that will generate a month end report as a PDF response. """ current_dat...

How do I get csv file to download on IE? Works on firefox...

I'm struggling with an odd error. I have a simple web app that grabs stuff from a DB then outputs it as a downloadable csv file. It works on firefox and chrome, but IE fails to recognize it as a csv file (thinking it is a html fle) and when I click save I get the error, "Unable to download {name of file} from {name of site}. Unable to op...

Converting HTML table data to CSV and XML

Hi, I'm building a table script and one of the functionalities is to convert XML, CSV data to HTML and vice-versa. I found some similar scripts on the web, that converts CSV to HTML and vice-versa but didn't find for XML. I'm writing the script from scratch, but I thought I'd better read few scripts before I write my own, to avoid mis...

[PHP] fgetcsv() ignores special characters when they are at the beginning of line!

I have a simple script that accepts a CSV file and reads every row into an array. I then cycle through each column of the first row (in my case it holds the questions of a survey) and I print them out. The survey is in french and whenever the first character of a question is a special character (é,ê,ç, etc) fgetcsv simply omits it. Spec...

Microsoft.Jet.OLEDB.4.0 provider turns string into a date

I am using Microsoft.Jet.OLEDB.4.0 from .NET to read a CSV file. Here is a sample input data row: 102A Avenue,97 Street,99 Street,2 Lanes Closed,2007-04-13,2009-12-31 When I read the last two valuee they come out as DateTime rather than strings and that is neither what I want nor what I expect. It seems that the provider performs type...

Regarding Java Split Command Parsing Csv File.

I have a csv file in the below format. H,"TestItems_20100107.csv",07/01/2010,20:00:00,"TT1198","MOBb","AMD",NEW,, I require the split command to ignore the commas inside the double quotes . So i used the below split command from an earlier post. Pasted the URL that i took this command String items[] = line.split(",(?=([^\"]*\"[^\"]*...

reading csv file without for

Hi All I need to read a CSV file in python. Since for last row I receive a 'NULL byte' error I would like to avoid using for keyword but the while. Do you know how to do that? reader = csv.reader( file ) for row in reader # I have an error at this line # do whatever with row I want to substitute the for-loop wit...

Changelog file: YAML vs JSON vs CSV

Hello everybody. I am creating a simple Changelog lib in CodeIgniter that will basically log a message everytime someone adds, deletes, changes or publish a blog post. I will log messages in files by batches of 300. So every 301st message will go in a new file. At first I wanted to write the logs to simple .log files but then I got the ...

parsing textarea input with PHP as if its a csv?

hey all i'm trying to make a site where people paste a CSV file directly into a textarea (say its called "original-text") and parse it, with my script then generating something from that. i know how to parse a csv file on the server (using fgetcsv) but not how to get the input from a POST variable INTO that fgetcsv. so far i've tried som...

.Net DataTable column export

Im looking to get a column and all its row data from a DataTable object then create a new column in another data table and append it with the new column and its rows. The issue I keep encountering is the row data will come out, as will the column names, but all of the rows are appended to the same column, I'm sure im missing something ob...

How to convert XML to text file using XSLT

I'm trying convert an XML file to text using XSLT, but I don't have any experience using XSLT. Here's a sample XML file: <DOC xsi:noNamespaceSchemaLocation="" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"&gt; <DOC_REQUISITES DOC_DATE="2009-04-23" DOC_NO="99999999"/> <DOCID TradeDate="2009-04-23" Weekday="Monday" MainFirmId="Z...