Home | Uncategorized | How Many IOPS?

How Many IOPS?

0 Flares Twitter 0 Facebook 0 Google+ 0 StumbleUpon 0 Buffer 0 LinkedIn 0 0 Flares ×

A question I get asked occasionally is; “How many IOPS can my RAID group sustain?” in relation to Enterprise class arrays.

Obviously the first question is to determine what the data profile is, however if it isn’t known, then assume the I/O will be 100% random. If all the I/O is random, then each I/O request will require a seek (move the head to the right cylinder on the disk) and the disk to rotate to the start of the area to read (latency) which for 15K drives is 2ms. Taking the latest Seagate Cheetah 15K fibre channel drives, each drive has an identical seek time of 3.4ms for reads. This is a total time of 5.4ms, or 185 IOPS (1000/5.4). The same calculation for a Seagate SATA drive gives a worst case throughput of 104 IOPS, approximately half the capacity of the fibre channel drive.

For a RAID group of RAID-5 3+1 fibre channel drives, data will be spread across all 4 drives, so this RAID group has a potential worst case I/O throughput of 740 IOPS.

Clearly this is a “rule of thumb” as in practical terms, not every I/O will be completely random and incur the seek/latency penalties. Also, enterprise arrays have cache (the drives themselves have cache) and plenty of clever algorithms to mask the issues of the moving technology.

There are also plenty of other points of contention within the host->array stack which makes this whole subject more complicated, however, when comparing different drive speeds, calculating a worst case scenario gives a good indication of how differing drives will perform.

Incidentally, as I just mentioned, the latest Seagate 15K drives (146GB, 300GB and 460GB) all have the same performance characteristics, so tiering based on drive size isn’t that useful. The only exception to this is when a high I/O throughput is required. With smaller drives, data has to be spread across more spindles, increasing the available bandwidth. That’s why I think tiering should be done on drive speed not size…

About Chris M Evans

Chris M Evans has worked in the technology industry since 1987, starting as a systems programmer on the IBM mainframe platform, while retaining an interest in storage. After working abroad, he co-founded an Internet-based music distribution company during the .com era, returning to consultancy in the new millennium. In 2009 Chris co-founded Langton Blue Ltd (www.langtonblue.com), a boutique consultancy firm focused on delivering business benefit through efficient technology deployments. Chris writes a popular blog at http://blog.architecting.it, attends many conferences and invitation-only events and can be found providing regular industry contributions through Twitter (@chrismevans) and other social media outlets.
  • BarryWhyte

    Chris, totally agreed. Its not a capacity statement, but a performance statement. 15K, 10K, 7.2K, and then SATA!

    It does however make difference now that Samsung are offering 32MB cache models – ok SATA desktop drives, but the difference with a ‘Spinpoint’ with the larger cache over even 10K RPM SATA Raptors is amazing!

    Provisioning needs to take into account, drive speed, RAID type (and in a SVC type model) controller type. The differences between one vendors RAID-x and anothers is quite amazing. I’d be in trouble if I quotes names… but think about who was last to market to bring pseudo RAID-5 …

    Thats where a ‘pooling’ strategy helps. Define your gold, silver and bronze (maybe platinum too these days with flash) and then provision from a pool. The key thing here being a ‘price to your internal customers’ so maybe a SATA drive only makes a minimal difference to your overall costs if you put it in a big monolith – say DMX. But if you buy a SATA based controller, like DS4200, then the overall cost for 1TB is *way* lower. Thus being able to pass on the saving to your customers has a benefit, not only for the bronze, but the platinum users.

    A system that can provide just such a varied cost point would be nice wouldn’t it… guess what, thats where SVC comes in :)

  • rwhiffen

    Although the drives may have the same performance characteristics, the larger drives hold more data, and there for have more tickets in the random I/O lottery and can have it’s numbered called to the I/O dance floor more often than a smaller drive right next to it. I usually put it as a secondary consideration when making storage decisions because, as you said, cache and OS components can mask things. The drive size probably won’t change ‘if’ the performance is going to hockey stick, but it might change ‘when’. If I’m in a situation where 146gb vs 300gb is going to make or break my performance, I’ve done something seriously wrong.

    Great post… keep it up.

  • James

    “That’s why I think tiering should be done on drive speed not size…”

    Errrr…. not exactly. In my less than humble opinion, the data itself should determine the tiering based on the performance required.

    -James Orlean
    StorageMonkeys.com

  • Chris M Evans

    Guys

    Thanks for the comments. Barry, I agree with your comments on service. I don't like the terms tier 1/2/3; your medal comparison suits better – and what a great lead in to advertise SVC! :-)

    James, I guess what I was meaning was that I don't see enough of a differentiator between 146 & 300GB 15K drives to call them separate tiers. I think performance is the usual tier 1/2 consideration, so drives of the same performance aren't going to meet that requirement.

    rwhiffen, thanks for the words of support!

  • Blaese

    Enjoyed reading your little review of IOps performance and it sparked a couple of thoughts that you might find interesting – posted on my blog at http://www.infrageeks.com/groups/infrageeks/weblog/bab55/How_Many_IOPS_.html

    I do a lot of work in storage sizing for VMware deployments and it remains one of the fuzziest parts of the project. Anything we can do to normalize this is a step in the right direction.

    Cheers!

  • Rob

    “How many IOPS can your [Enterprise] RAID group sustain?”

    More than you post depending on workload and other things. But in general more.

    A quick google reveals things like this:

    270 IOPs per FC drive

    tagged command queuing allows you to push them that hard. Of course response time tails off and that would be punishing to drive them that hard if interactive use was the norm.

    Secondly, as mentioned, what about usage? What if the DBAs did a normal morning import of data and the array is getting hammered with large IOs for a time? Writes and transfer time for the larger IOs come into play so maybe that 185 IOPS you are quoting is too generous.

    “With smaller drives, data has to be spread across more spindles, increasing the available bandwidth.”

    Not sure drive size comes into play unless your “Enterprise array” is feature-poor. Barry suggests you can front-end with SVC. You can carve and migrate to meta-LUNs touching more arrays in a clariion. You could carve your LUNs out of a 100 disk pool in an EVA. Or (duck) try an XIV and furgetaboutit. Point not to be overlooked is LUN management can overcome IO issues and future-bound a number of vendors will be working with 1MB “partitions” and side-stepping LUN management pain.

  • Adriaan

    I see the question as asked above often.
    Problem is the client usually means “how many IOPS can I push before performance drops”.
    This answer is very different as your use case changes.
    In an environment supporting many servers all competing for IOPS this may occur once queues start to build and this may occur well before the ideally balanced paradise of 185 IOPS (per spindle).
    In these environments I find it very beneficial to have QoS management to prioritise latency critical IOPS and allow less urgent processes to mop up remaining capacity.

  • @zaxstor

    I am late to the party, however we need more discussions on correct sizing of storage systems for various workloads.

    “For a RAID group of RAID-5 3+1 fibre channel drives, data will be spread across all 4 drives, so this RAID group has a potential worst case I/O throughput of 740 IOPS.”

    This does not take into account the overhead of RAID (For instance RAID 05 has a 3-1 penalty on writes). If a customer is only looking at their workload and the ability of the disks in their disk system, they could be gravely mistaken in their design.

    For example, the table below shows what HP calls safe IOPS in an EVA system based on drive types:

    Found at : http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=c01671044

    Drive Type Vraid type Per Drive Host IOPS
    15K Fiber Channel VRAID-1 115 VRAID-5 73
    10K Fiber Channel VRAID-1 83 VRAID-5 53
    250GB FATA VRAID-1 62 VRAID-5 39
    400/500GB FATA VRAID-1 42 VRAID-5 26
    1000GB FATA VRAID-1 54 VRAID-5 34

    This table states that in a raided config drives perform at half or even a third of their actual capability. I have tested other arrays in the same range as an EVA which perform 2X-3X as many IOPS as an EVA in various raided configurations. IMO understanding the capabilities of the array you are using is more important than understanding the RAW IOPS a disk in a non-raided configuration can produce by itself.

    • IgalK

      It’s a good thing you pulled out the numbers because the document was removed from HP site…

  • http://www.brookend.com Chris Evans

    Zaxstor

    I agree that perhaps I was generalising a little; I was looking at worst case from the pure disk level. You’re right that the array architecture needs to be taken into consideration too, including RAID calculations. I’ll add this as a subject for a future post.

    Chris

  • Chris Evans

    Adriaan

    Agreed, there’s an issue of workload mix which can cause problems. What your comments also demonstrate is the requirement to be continually monitoring performance and adjusting the mix of tiers and data location. This can be costly and time consuming, but is necessary in large mixed multi-host environments,

    Chris

  • Josh

    Hi Chris,

    I’v very new to storage architecting and I’m very thankful for people like youself who share the knowledge. Anyhow, I have a quick question on the equation to find out the total I/O on the 15k drive. Where did you get 1000, in the equation 1000/5.4 = 185iops?

    Thanks,
    morjo

  • Mat

    Hi Morjo,
    5,4 … time for one IO (in milisecond)
    1/5,4 … number of IO per milisecond
    1*1000/5.4 .. number of IO per second

0 Flares Twitter 0 Facebook 0 Google+ 0 StumbleUpon 0 Buffer 0 LinkedIn 0 0 Flares ×