views:

1582

answers:

4

If I start copying a huge file tree from one position to another or if some other process starts doing lots of disk activity, the foreground app (GUI) slows way down. For example, take a 2gb file tree with 100k files in it. Open a console and do cp -r bigtree bigtree2. Then go to firefox and start browsing. Firefox is almost unusable. Even if I set firefox's nice level to really high priority (-20), it's still super slow with huge delays.

I remember some years ago when I worked on a Solaris box, the system behaved much better in similar circumstances.

My HD is using DMA, not PIO. It's SATA. Not mounted with the the atime flag.

+5  A: 

Try ionice-ing or nice-ing the copy process. The issue is due to the fact that IO gets the same priority as the GUI, which for a desktop, affects perceived responsiveness.

There's an Ubuntu brainstorm about this currently.

codelogic
I will try that. But that's not exactly what I want. I want all background disk activity to automatically ionice itself or whatever. I want the system to be smart enough to say "hey, there's a human trying to do stuff. Stop pissing him off."
Bill Ataras
I'm not sure if there's currently any way to configure the priority class on a global level. However you can configure the IO scheduler that is used for each block device (echo deadline > /sys/block/<device>/queue/scheduler). Read http://donami.com/118 for more details.
codelogic
There's no way for the system to know automatically. You typed the cp command, for all that the system knows you are anxiously waiting for it to finish.
Zan Lynx
The system knows which window is in the foreground and what keystrokes/mouse activity is happening in which window. Therefore it knows what I'm anxiously waiting for
Bill Ataras
+11  A: 

Linux has long had a problem with programs that hog all the system's "dirty" cache memory. What is happening is that the copy process is filling the write cache with the file data it is copying and it is doing it very quickly. So when Firefox comes along and needs to write it must first wait for dirty buffer space or an available disk queue write slot. While waiting it is competing with the copy process and the kernel's pdflush thread, which moves data from dirty buffers to the disk write queue.

Firefox has yet another problem in this scenario. It uses SQLite to store its bookmarks, history and other things. SQLite is a ACID compliant database and it uses a transaction system with its disk writes flushed to disk. So not only does it have to wait for buffer space, it must wait for the disk queue, which is full of copied file, to clear out before it can acknowledge a successful write.

There has been a lot of tweaking done to the Linux disk queuing and buffering system. There are changes in almost every kernel release. Try one of the newer releases. You can also try tweaking the sysctl values. I sort of like these:

vm.dirty_writeback_centisecs = 100
vm.dirty_expire_centisecs = 9000
vm.dirty_background_ratio = 4
vm.dirty_ratio = 80

You can also try tweaking the number of slots in the disk queue. This value is in /sys/block/sda/queue/nr_requests. You need to substitute sda with whatever your drive really is. More slots means more chances to merge IO requests and the CFQ IO scheduler can do a better job with priorities. Fewer slots usually means a shorter wait to get written to disk for synchronous IO like SQLite's transactions. Fewer slots also means a shorter wait to get read IO into the disk queue if a write-heavy process completely stuffs the queue with write IO.

Zan Lynx
+1 for very good and detailed answer. Maybe he could use another browser, too. Firefox is lame and bloated.
BeowulfOF
A: 

You're not the first to notice this problem. Former kernel developer [Con Kolivas] (http://en.wikipedia.org/wiki/Con_Kolivas) found that a lot of companies are paying to improve linux server performance at the expense of desktop performance. Con had an impressive set of patches for making the desktop more responsive. Unfortunately there was some sort of code war and eventually Con dropped out.

I would love to know how to petition the Linux kernel developers for better desktop performance. In the meantime, if you are willing to run kernel 2.6.22, you can run with the -ck patch set.

Norman Ramsey
A: 

Make sure that DMA is enabled on all your drives that support it. Depending on your distribution this may not be the default. Read man hdparm, and look into your systems init mechanism.

dmckee