So we've decided on an EMC NX4 for our storage setup.
Database workload
Write: ~220 IOPS
Read: ~6 IOPS
Web workload
Write: ~10 IOPS
Read: ~55 IOPS
I had planned to go for 5 x 15k SAS disks in RAID 10 + hot spare for our databases, and then a 7 x 7,2k SATA RAID 6 + hot spare for our web data. However, EMC conveniently forgot to mention that the first 5 disks had to be setup in a RAID 5 for the system volumes. We haven't signed the agreement yet, but I obviously need to come up with a new disk layout for this to work out.
They suggest we put the 15k SAS disks in a RAID 5 + hot spare, but won't this rather quickly give me write performance issues? Besides the RAID 5, we'd then be left with a 6 disk RAID 6 leaving us 4/3TB usable data depending on hot spare/not.
Alternatively we should stick with the RAID 10 for our DB's and put our 1TB 7,2k SATA disks in RAID5 - Though I'm not fond of the rebuild time and insecurity of running these in RAID5. The raid level suits web data much better however.
So we basically have three choices as I see it:
6 x 15k SAS RAID5 for System Volumes + DB data+log + hot spare
6 x 7,2k SATA RAID6 for file data + hot spare
OR
6 x 7,2k SATA RAID5 for System Volumes + file data + hot spare
5 x 15k SAS RAID10 for DB data+log + hot spare
OR
5 x 15k SAS RAID5 for System Volumes + DB Data
2 x 15k SAS RAID1 for DB Log
1 x 15k SAS hot spare
4 x 7,2k SATA RAID5 for file data + hot spare
Option 1 may have write performance issues. Option 2 can become dangerous during rebuilds. Option 3 is good for DB performance but leaves us only 2TB usable file storage, meaning we'll soon have to expand with an extra shelf - and at that point convert our file data to a RAID6.