Tag Archives: command line

iSCSI performance in the cloud

This is one of the more entertaining things I’ve done at work and according to the emails i received from co-workers one of the most controversial. Someone asked if this could be done, and i figured i’d give it a shot. This will be in two parts, the first being FileIO since I was having problems trying to get partition re-sizes to work in the cloud. Part two will be BlockIO once I get around to fixing the kernels.

I used CentOS 6 for all of the servers involved in this setup and all network traffic is over the internal service network.

[root@vsan1 ~]# dd if=/dev/zero of=/root/block.img bs=4M count=1250

The other iSCSI drive is a RAID 0 configuration using a much smaller block size for comparison.

[root@vsan2 ~]# dd if=/dev/zero of=/root/block.img bs=512K count=30720 
[root@vsan3 ~]# dd if=/dev/zero of=/root/block.img bs=512K count=30720
[root@master ~]# mdadm -Cv /dev/md0 -l0 -n2 -c128 /dev/sdb1 /dev/sdc1

All of the targets are setup the same way:

Target iqn.2012-03.local.node:vsan2 
Lun 0 Path=/root/block.img,Type=fileio,IOMode=wb 
Alias vsan2 
ImmediateData Yes 
MaxConnections 2 
InitialR2T No

I’m just using hdparm for basic tests. I wasn’t really inspired to test further. For comparison here is the actual block device offered by xenserver:

[root@master ~]# hdparm -tT /dev/xvda 
/dev/xvda: 
Timing cached reads: 1948 MB in 1.99 seconds = 977.19 MB/sec 
Timing buffered disk reads: 82 MB in 3.10 seconds = 26.45 MB/sec

That is actually lower than what I received before during my tests, but it’s what I got when writing this response. Now to test the single drive, 4M block size device:

[root@master ~]# hdparm -tT /dev/sda 
/dev/sda: 
Timing cached reads: 1696 MB in 1.99 seconds = 850.29 MB/sec 
Timing buffered disk reads: 16 MB in 3.40 seconds = 4.71 MB/sec

And finally for the RAID0 performance results:

[root@master ~]# hdparm -tT /dev/md0 
/dev/md0: 
Timing cached reads: 1968 MB in 1.99 seconds = 987.01 MB/sec 
Timing buffered disk reads: 28 MB in 3.04 seconds = 9.21 MB/sec

Kinda meh results, but that’s what you can expect from fileio. I’m really looking forward to the blockio tests, but the kernels on the gentoo images are 3.0+ and I need 2.6+ for iscsi to work. When I get around to it i’ll finish setting that up.