Man Bites Dog – Why IBM (and everyone else) should fear EMC’s acquisition of Isilon Systems

Firstly, apologies for the length of time since my last post.  A combination of family, work and late vacation hasn’t left much time for blogging in the last few months.

The recent fun and games with Dell and HP nearly brought me back to the keyboard – okay, so the technology was wrong (3PAR, really?), but I did say HP needed to do SOMETHING about the EVA, so half points for trying, maybe? 🙂

I have to say though, I was at a Dell partner event recently, and the Dell guys seemed pretty relaxed about the whole thing – consensus seemed to be that they dodged a bullet (and a repeat of EMC’s black eye when they bought RSA for what the market thought was “too high” a price a few years ago) by not staying in to the bitter end.  The market seems to agree with them (Dell shares have been rising steadily since they stepped aside), which I suppose in the end is the important thing really.

But while I was away, I had the chance to evaluate a number of up and coming technologies, one of the best of which was by a small company named Isilon Systems.  These guys have an excellent story but seem to have been ignored by large sectors of the market (up until this week that is).

So I saw this product, went away with my mind full of the potential and started talking to my sales people, getting things lined up, and had half the draft of an article lined up which would have made me look like some kind of prophet if I’d published it last week (or maybe arrested on suspicion of espionage – but hey, them’s the breaks :-))

Because now, from being a complete unknown, Isilon are now in the spotlight as EMC’s next acquisition target, with a number of tech journals and media outlets twittering on about the “perfect fit” with EMC’s product set and how “complimentary” it would be.

So let me politely respond:

I don’t think so.

Isilon is in no way complementary to EMC’s current product line.  It’s a competitor to many of EMC’s core products, and it has the capability to do to EMC’s lineup, what XIV has done inside IBM (think hot knife through butter).

To back up this assertion, let’s consider a few things:

  1. EMC’s mid-range technology is based on variations of the Clariion platform.  This is the clunky dual-processor architecture from last century, being replaced in other forward-looking vendors by scalable grid architectures – spookily similar to the Isilon architecture.
  2. The Celerra NAS platform, the closest EMC currently has to Isilon, has spent 10 years developing from a “dire box of misery” (actual engineer quote) to its’ current, actually pretty usable form, but let’s be honest, it’s never going to move NetApp from the top spot.  EMC need a revolution in this space, and the Celerra/Clariion combo is never going to be that revolution, no matter how integrated it gets.  Isilon is built from the ground up for NetApp competitive takeout and forms a far more credible proposition.
  3. Data Domain, EMC’s other recent acquisition, uses a variation on the same Xyratex-manufactured storage server as its basic hardware.  Rather than moving Data Domain onto the Clariion platform, it would surely be less painful to integrate the Data Domain architecture with the Isilon stack (this would also plug one of the more painful holes in the Isilon array itself – more on this later).
  4. Centera, EMC’s archiving product, is being left further an further behind, with software vendors providing better resilience and compression/deduplication and is looking old and tired compared to products such as HDS’ HCP product family.  Replacing this with a product with the compliance capabilities of Centera and big disks would get EMC back in this game (similar to what NetApp have done with SnapLock – provide Tier 4 storage on an existing array, rather than force a new architecture).

If you strip away the Isilon File System, what is left looks suspiciously like a better version of my favourite product of the last year, the IBM XIV.  Based on the same Xyratex hardware platform, Isilon has the same scaling of processor, cache and disk capacity that makes XIV such fun to design with, but at the same time, Isilon has some excellent unique points:

  • 8Gb/s Infiniband back-end network
  • Scale to 144 modules (nodes)  in a single manageable environment – slightly more than XIV’s current 15 modules per manageable system limit – this is something XIV has promised since the original rollout presentation, but has yet to deliver.
  • Virtualised equivalent to RAID 5 (N+1 protection level)
  • Virtualised equivalent to RAID 6 (N+2 protection level)
  • Virtualised equivalent to protection levels the market doesn’t have a name for yet – N+3 and N+4.
  • Seriously, if there’s a snappy marketing term for these out there (other than “holy sh*t that’s a lot of protection”) then I want to know about it.
  • Choice of disks and enclosure types – chose from SSD (SSD!!!!) SAS (SAS!!) and SATA (oh well) to give a choice of performance and capacity
  • Diskless enclosure gives a performance boost without having to increase capacity – one of my big issues with XIV (#4 I think) was a need to scale capacity to get performance – if you wanted 50,000 IOPS, but only 5TB of capacity (actual customer of mine had this) then you bought 79TB and liked it (or else).
  • Starting point of 10TB (3 modules) – technically you could start with less, but you don’t get parity protection till you have 3 modules, so this is the best place to start – still less than XIV’s 6 module 27TB starting point.

Of course, it’s not perfect yet:

  • One of my more backup-focused colleagues poked a few holes in Isilon’s NDMP-based backup solution.  Okay, not so much poked holes as picked up the argument-based equivalent of a cannon and started blowing it away.  End result is – NDMP is unlikely to be fast enough to backup a moderate-sized Isilon.  The fix? A conversion appliance that takes the IP traffic and converts it to FC (for backup only).
  • This is where Data Domain comes in – combining the DD backup architecture with Isilon would remove backup issues.
  • Isilon is currently file only – EMC would have to develop a block-level equivalent which can run in parallel with Isilon’s ONEFS file system, unless they wanted to do as NetApp and keep everything in the file space.
  • To an extent this is feasible – with more enterprises moving to virtualised environments (courtesy of EMC again) the block vs file argument is becoming a little less important.
  • Still, at least we know it would be possible to develop a grid-based block system to run on the Isilon hardware (thanks for the pointers, Mr Yanai!!)
  • Isilon is not Fibre Channel capable for host traffic, but being based on Xyratex, FC capable HBAs can be fitted)
  • Of course, by the time Isilon is integrated into EMC’s portfolio, FCoE may be the standard (he says hopefully) – again, the Xyratex servers can be retro-fitted with CNAs as required.
  • Most important – please please please EMC, please hire someone to design a proper user interface?  Maybe the guys who did the XIV interface could do something tasteful for you in blue?

A couple of years ago, Moshe Yanai told me that EMC would never develop something like XIV, as they were too wedded to the Symmetrix and Clariion architectures – too much development has gone into these platforms for EMC to contemplate spend money developing a competing architecture.

As this was based on Yanai’s belief that nothing like XIV existed to be acquired, I’d argue that a successful acquisition/integration of Isilon by EMC would blow that position away completely.  Without spending time and money on the basic architecture, EMC will be in a position to develop a grid-based replacement, first for the Celerra then the Clariion, Centera and beyond (Isilon-MAX anyone?  Just kidding, like the DS8000 range, Symmetrix has a number of features that Isilon can’t and probably will never copy)

In fact, if I were an EMC employee working in anything starting with a “C” I’d be looking for a quick change of career if this goes through.

The Competition

Of course, as Dell proved, it doesn’t matter who makes the first bid, so much as who makes the last.  I’ve listed potential counter-bidders in order of who I think is most likely to try and take this away from EMC:

  1. IBM – at the conclusion of the original presentation from Isilon, my first comment was “you guys are so getting bought by IBM” (seriously, this was my actual comment).  I hadn’t actually considered EMC as a buyer (for reasons, see above) so IBM appeared the perfect choice – I don’t have a lot of faith in the SONAS stack yet, and Isilon would give IBM a properly developed scale-out file platform, with some of the features they’d like to have in iteration 3 of the XIV platform (Infiniband, parity protection, multi-rack scalability, choices).  Plus it keeps EMC stuck with last-generation architectures.
  2. Dell – flush with cash from the 3PAR try, Dell could make a try for Isilon.  Let’s face it, they’ve already hacked EMC off once this year by showing they want their own storage product, so why not go the whole hog just to see the look on Joe Tucci’s face?  Some synergies with their existing storage lines – the Equalogic iSCSI array is based on a similar Xyratex storage server (seriously, is there anything out there that isn’t made by Xyratex these days?).  At the very least they could do what they achieved with HP/3PAR and skyrocket the price, at least damaging EMC a little.
  3. NetApp – can’t really see the Synergies here, NetApp are unlikely to abandon WAFL, and could probably develop their own Xyratex-based or similar grid platform for an expanded WAFL file system in the future without hitting ideological barriers (of the kind that protect the Symmetrix architecture at EMC).  All the same, they can’t be pleased at the thought of EMC acquiring a “NetApp killer” and may be looking for pay-back after the Data Domain debacle.  As with Dell, NetApp may enter the fight just to push the price up and make things hot for EMC.
  4. Cisco – perfect chance to acquire a storage product to complement their UCS servers.  Unlikely as they have a lot invested in the different storage collaborations, but possible.

or Oracle – who from the rumours, may wait until EMC acquire Isilon, and then acquire EMC.  Not my favourite vision of the future from an independence point of view, but possible.


I really hope EMC goes through with this (the Isilon acquisition that is, not the Oracle one).  I’ve always enjoyed working with EMC (products and as an organisation), but it’s become harder to justify in the last year, when you know there are better products on the market – better service only takes you so far against better/cheaper/faster products such as XIV.  Assuming EMC make a proper job of integration (and they’ve done reasonably well with some of their other acquisitions) this may be the thing to get them back on track.

Here’s hoping.


DataDirect Networks – the “other XIV” ?

A year or two ago, I got a call from one of our sales guys, who was looking for a solution to compete with a technology I’d never heard of, for a pretty unusual requirement.  The customer was doing (very) high-performance video effects work, and the sales guy was looking at potential replacements for an existing solution, the customer having mentioned in passing that while the equipment worked extremely well, she had some concerns around support for what was a non-mainstream product.

Sales guy, having jumped on this opening like a starving weasel, wanted to understand which of the vendors would be the best solution to compete with the incumbent, a company called DataDirect Networks.

Having sat down and looked at the set of requirements provided and done a bit of research into DDN, my response was that nothing currently on the market would do what the customer wanted, at the price they were prepared to pay.  Sure, a DMX or USP-V could be configured to provide what were, quite frankly, outlandish performance requirements, but even preliminary configurations were pricing at around double what the customer claimed to have paid for the DDN system they already had.

“Are you sure they’re actually getting this level of performance?”

The above became my catchphrase for the next few days until the sales guy, disheartened by conversations around solid state disks, huge cache requirements and the need to mortgage a kidney to pay for it all, went off to look for some slightly lower hanging fruit and I was left with the feeling that I should find out more about this DDN bunch.


So who are DDN?

DataDirect Networks (DDN) can be characterised as the most successful unknown niche storage vendor on the market.  A privately held company founded in 1988 and based in Chatsworth California, DDN are the manufacturers of an expanding product range, based around a parallel processing architecture.

Basically, imagine if we took the XIV capabilities and rather than using SATA disk, we attach higher-speed SAS disks.  At this point, we get something which can power “40 of the 100 fastest computing environments in the world”.  At the time I looked at DDN, their architecture was capable of a sustained 2.8GB/s (bytes, not bits) – this has since been expanded to 6GB/s.

That’s.  21.  Terabytes.  Per.  Hour.


Put simply, this is about what I’d expect to get practically out of an enterprise system costing twice as much and smothered in solid state technology.  DDN achieve this with spinning disks and a Clariion-style footprint. And, as I keep coming back to, a lower price-tag.  With the implementation of new architectures and solid state disks, DDN are claiming that 10GB/s is now possible.


So what’s the secret?

The DDN product range is based around the Silicon Storage Appliance (S2A) and Storage Fusion architectures.  As with the IBM XIV, these arrays use a massively parallel processing architecture to ensure that sequential data is processed at effectively wire speeds.  Unlike mid-range arrays such as the EMC Clariion or Hitachi AMS, data is not funnelled to a single cache to speed up disk access – all error checking is done by the individual S2A processing complexes and passed directly to disk.

So what can DDN do for me?

Basically, DDN have opted for more flexibility than IBM have with XIV.  Rather than a single disk type, DDN provide a range of disk types and shelf configurations.  The performance option is similar to XIV’s front-loaded, 12 disks per shelf design, providing a small capacity footprint, but achieving the promised high-performance.  This is the configuration which is used in the supercomputing environments which make up DDN’s more interesting case studies.

The high-capacity solution on the other hand, uses a top-loading, densely packed shelf, allowing 60 disks per enclosure – with 2TB SATA drives, this would be capable of around 2 petabytes in a single rack.  This is marketed as a high-performance VTL and archiving product, rather than as a compute platform.  More recently, the company has moved from building block storage arrays, to also providing scalable file-system storage, based on their own NAS designs.

So where can I buy one?

In the UK at least, DDN arrays tend to turn up as part of larger vendors’ solutions.  IBM have long sold the S2A arrays as part of their supercomputing portfolio and have integrated the capacity versions into their SONAS product. 

Update – I’m told on good authority that the DDN products are also resold outside of the suporcomputing area as the DCS9900.  I’d be interested to hear from any IBM customers (or anyone else) who are using DDN, but particularly in this area, as I’ve not seen a case-study or use-case for non BlueGene implementations of DDN.

HP have recently announced that they will sell DDN as their top-end NAS product.  It appears from the announcements that, rather than bolt the storage array onto their own NAS offerings as IBM have done, HP will sell DDN’s own NAS offering.


Conclusion – so is this a competitor to XIV?

Strangely no, not really.  At least not yet.

At least at the present time, DDN tend to sell to pretty specific use-cases.  While XIV is now sold as a general computing platform, the DDN arrays tend to be sold for big number-crunching and media-serving environments.  The head of the company was quoted as saying that, in non-media serving environments, the S2A architecture would be “horrible” compared to competitors such as the EMC Clariion. 

But then, as IBM showed by marketing the XIV in the early days as a web serving platform only, things can change (P45 for the head of marketing, hey?).  It may be that DDN forms the upper and lower tiers of a storage environment, with the middle taken up by a general computing platform such as Clariion or XIV. 

The diagram below shows a possible solution of this type:

My own feeling is that the only thing stopping DataDirect Networks from becoming the next generation of EMC or HP storage is the fact that it is still a private company.  If DDN ever take the IPO route, I predict they will last about 5 minutes before the first takeover attempts (hostile or not) begin. 

This might not be a bad thing.  XIV is currently gaining IBM market share based on a combination of usability, cost and performance which I’ve not seen traditional storage architectures begin to touch.  DDN for their part, would get the same kind of boost that XIV received from IBM – a properly global reach, greater R&D spend and far greater brand awareness, at the cost of identity.

Depressed by 10 years of reselling LSI seconds, IBM went for a complete makeover, and stole a march on the competition by purchasing the technology most likely to shake up the market.  Other storage vendors will have difficulty developing an answer to the XIV architecture in a reasonable time – purchasing the DDN architectures would give an immediate route to replace existing dual-processor architectures (and an existing user-base of weird and wonderful customers). 

The attempts may already have begun.  Commentators have pointed out that the HP reseller deal fills no gaps in the HP product range – they already have large scale NAS products so DDN is effectively redundant within their portfolio.  I’d suggest that this is an attempt by HP to get an understanding of the practical reality of the DDN products, with a view to understanding whether it’s a credible product set for HP to buy up – possibly as a replacement for the rebadged HDS arrays currently sold in the enterprise space. 

Either way, I’m keeping an eye on DDN – in any case, reading their customer case studies is always going to be entertaining.  A selection below gives a sample of customers:

  • TimeWarner Cable – Video on Demand
  • Pacific Title & Art – The post-production effects house for Batman – The Dark Knight movie
  • Microsoft Studios – Xbox Live Media & Data Serving
  • CCTV – Chinese State Television, Media Serving for Coverage of the Beijing Olympics
  • Shutterfly – Photo sharing site serving up to 1 billion photos at present (growth from 100 million in 2 years)
  • Kodak Easyshare
  • National Center for Data Mining (UIC)
  • Northwestern University & John Hopkins University – Winner, 7th Annual Bandwidth Challenge (2006)
  • CalTech, CERN, University of Florida and the University of Michigan – 2nd, 7th Annual Bandwidth Challenge
  • Indiana University – 3rd place, Bandwidth Challenge at Supercomputing 2006
  • Lawrence Livermore National Laboratories,
  • Sandia National Laboratories,
  • NASA and NASA Ames
  • Argonne National Laboratory