csv

Matching lines across multiple csv files and merging a particular field

I have about 20 CSV's that all look like this: "[email]","[fname]","[lname]","[prefix]","[suffix]","[fax]","[phone]","[business]","[address1]","[address2]","[city]","[state]","[zip]","[setdate]","[email_type]","[start_code]" What I've been told I need to produce is the exact same thing, but with each file now containing the start_code...

perl how to regex parts of data instead of entire string and then print out a csv file

I have a working perl script that grabs the data I need and displays them to STDOUT, but now I need to change it to generate a data file (csv, tab dellimited, any delimiter file). The regular expression is filtering the data that I need, but I don't want the entire string, just snippets of the output. I'm assuming I would need to store ...

How to get multiple column data in a comma separated string?

I am getting data like Result ------ 10 23 21 But i want to get data in the following format. Result ------ 10, 23, 21 How to get that in a Query? Thanks in advance for any help :) ...

How do I remove comment lines from an Oracle CSV export?

Hi, I am exporting data from an Oracle database to csv file using the spool command as follows: set heading off set linesize 1000 set long 1000 set pagesize 0 set echo off set verify off spool D:\OVERNIGHT\TEMP_FILES\SE_NEW_PFRA_CRYSTAL_OUTPUT.txt SELECT TRIM(FU_BAN) ||'|'|| TRIM(CASE_ID) ||'|'|| TRIM(case when fu_type is null then ...

Powershell import-csv with empty headers

I'm using PowerShell To import a TAB separated file with headers. The generated file has a few empty strings "" on the end of first line of headers. PowerShell fails with an error: "Cannot process argument because the value of argument "name" is invalid. Change the value of the "name" argument and run the operation again" bec...

Reading and Uploading files in java

How do I iterate through a directory to read *.csv files and upload them in my system in Java? Ignore the upload part of post. ...

How do I add a new column of data to a csv file.

I am reading a csv file into a variable data like this: def get_file(start_file): #opens original file, reads it to array with open(start_file,'rb') as f: data=list(csv.reader(f)) data is the entire csv file. How do I add an extra column to this data? ...

make a change to an element within a list within a list

i am reading a csv file into a data: def get_file(start_file): #opens original file, reads it to array with open(start_file,'rb') as f: data=list(csv.reader(f)) i need to go through every row and add a value to row[1] like this. initially row[1] = 'Peanut', i need to add 'Butter' so the result would be row[1]='PeanutButter' ...

How to break a large CSV data file into individual data files using R?

I have a CSV file the first row of which contains the variables names and the rest of the rows contains the data. What's a good way to break it up into files each containing just one variable in R? Is this solution going to be robust? E.g. what if the input file is 100G in size? The input files looks like var1,var2,var3 1,2,hello 2,5,...

Validating Uploaded Files Server Side

I have some CSV files that I need uploaded to a site I'm writing in CodeIgniter. I need to validate the CSV to make sure they contain various info, column counts match up and stuff like that. Does CI have any sort of plugin to make this easy? ...

regional problem with fastercsv

Hi, what is the right way to check csv separator character, either using ";" or "," in a different computer regional setting ? I want my fastercsv able to parse both of them without complaining at all, what is the right way to do it ? ...

Django with huge mysql database

What would be the best way to import multi-million record csv files into django. Currently using python csv module, it takes 2-4 days for it process 1 million record file. It does some checking if the record already exists, and few others. Can this process be achieved to execute in few hours. Can memcache be used somehow. Update: Th...

PHP csv functions, Microsoft safe?

Hi there, I have been looking into the php fputcsv function, and have heard some comments that the files created using this method will not work on microsoft excel, is this correct? Comment on php.net since PHP is generally *nix-based, it makes sense that the line endings are always \n instead of \r\n. However, certain Micros...

how to skip lines of a csv file while using LOAD DATA command?

hi, i'm using sql command load data to insert data in a csv file to mysql database. the problem is that at the end of the file there's a few line like ",,,,,,,,,,,,,,,,,," (the csv file is a conversion of an excel file). so when sql get to those lines he send me : #1366 - Incorrect integer value: '' for column 'Bug_ID' at row 661. the ...

Python csv without header

With header information in csv file, city can be grabbed as: city = row['city'] Now how to assume that csv file does not have headers, there is only 1 column, and column is city. ...

Ruby on Rails coding efficiency - map CSV data to model

I have a question relating to minimizing some code I'm working with. I am using the map-fields plugin/gem to allow my user to select what fields in a CSV file will map to attributes in a model. map-fields with upload The map-fields plugin uses the name of a select object in the plugin to determine where the CSV columns match up too....

How to get a .csv file into R?

Hello.... I have this .csv file: ID,GRADES,GPA,Teacher,State 3,"C",2,"Teacher3","MA" 1,"A",4,"Teacher1","California" And what I want to do is read in the file using the R statistical software and read in the Header into some kind of list or array (I'm new to R and have been looking for how to do this, but so far have had no luck). ...

python: how do i always start from the second row in csv?

b holds the contents of a csv file i need to go through every row of b; however, since it has a header, i dont want to pay attention to the header. how do i start from the second row? for row in b (starting from the second row!!): ...

python: append a list of values into a list

c1=[] for row in c: c1.append(row[0:13]) c is a variable containing a csv file i am going through every row in it and i want only the first 14 elements to be in the c1 what am i doing wrong? ...

python: list index of out of range

for row in c: c1.append(row[0:13]) for row in c1: row.append(float(row[13])/100) row.append(float(row[12])/float(row[13])/100) row.append(math.log10(float(row[12]))) c contains a csv file with many rows and columns c1 is a subset of c containing only the first 14 elements i am getting IndexError: list index out of...