tags:

views:

34

answers:

2

Hello Everyone!

I am importing a 2.5gb csv file to a mysql table. My storage engine is innodb. Here is the script:

use xxx;
DROP TABLE IF EXISTS `xxx`.`xxx`;
CREATE TABLE  `xxx`.`xxx` (
  `xxx_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `name` varchar(128) NOT NULL,
  `yy` varchar(128) NOT NULL,
  `yyy` varchar(64) NOT NULL,
  `yyyy` varchar(2) NOT NULL,
  `yyyyy` varchar(10) NOT NULL,
  `url` varchar(64) NOT NULL,
  `p` varchar(10) NOT NULL,
  `pp` varchar(10) NOT NULL,
  `category` varchar(256) NOT NULL,
  `flag` varchar(4) NOT NULL,
  PRIMARY KEY (`xxx_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
set autocommit = 0;
load data local infile '/home/xxx/raw.csv' 
into table company
fields terminated by ',' optionally enclosed by '"'
lines terminated by '\r\n'
(
  name,
  yy,
  yyy,
  yyyy,
  yyyyy,
  url,
  p,
  pp,
  category,
  flag
);
commit;

Why does my PC (core i7 920 with 6gb ram) only consume 9% cpu power and 60% ram when running these queries?

+1  A: 

Because it's not really a CPU or memory operation, it's mainly a disk I/O operation?

Cade Roux
ok, thanks for all the replies, it's actually an SSD drive. But still, maybe my ram and CPU is faster then the SSD. Everything makes sense. And here's something interesting: I run the same script on a PC loaded with Ubuntu and it runs twice as fast as my Win7 computer when my Win7 has way better specs.:)thanks!
vick
@vick Disk is just about the slowest thing in any computer, SSD or otherwise (optical drives being by far the worst). As far as disk performance between OSes, that can also be anything - drives, controllers, I/O subsystem configuration, OS filesystem configuration including auditing - you'd have to look at performance counters - that all would belong on serverfault.com...
Cade Roux
+7  A: 

Probably because it's an I/O bound process. That means that the slowest part of the process is reading the CSV file from the disk and writing the MySQL database back to the disk. Disk access is much slower than pretty much anything else that goes on inside the computer, so your CPU probably only needs a few percent usage to keep up with it.

Also, MySQL is copying the data in a piece at a time, not all at once, so it only needs to store a little bit at a time in RAM. That's why it might not be using a whole lot of memory.

David Zaslavsky