Nexenta Blog

Global Leader in Software Defined Storage

Nexenta

A Bridge Over Troubled (v)Storage

By

Every great technology shift requires the means to move from the old way of doing things to a new and vastly improved approach. A bridge, if you will.

We found that analogy very fitting while reading some interesting and thought-provoking articles on the storage industry. These are A Major Shift in the Data Storage Market is on the Horizon by Kalen Kimm of TweakTown, and Understanding Storage Architectures by Chad Sakac at VirtualGeek.

Both articles are ambitious (nearly 6,000 words combined!). Of the many observations, however, these lines from Kalen Kimm led us to comment:

“The shared visibility between compute, application, and storage is a large step forward to a true software-defined data center. Instead of having to pre-configure LUNs and then presenting them to applications to be consumed, applications will be able to consume storage on an as-needed basis.”

At Nexenta, we are obviously big believers in the software defined data center (SDDC), and the importance of software defined storage (SDS). The SDDC makes too much sense not to take hold; the only variables are around timing and adoption speed. We see VMware’s release of their Virtual SAN SDS solution as an important catalyst to address these variables. A larger player such as VMware can have significant impact on the way that enterprises run their datacenter.

A clear example of the impact is with networking. When VMware first released the hypervisor, all network switching was contained within the host and you were wholly dependent upon the physical switching layer from traditional companies, then the distributed virtual switch (DVS) was released. This allowed network segments to traverse hosts and spread throughout the virtualization cluster. This took switching to a software layer but then they added the Cisco 1000v as an option. The 1000v allowed management to be consistent throughout the data center, physical or virtual but still software based.

In our opinion, VSAN is the necessary bridge between legacy storage and a fully efficient SDS model. VSAN allows customers to utilize their existing internal storage and spread across hosts, similar to the DVS. What is missing is the next layer that allows software defined storage to traverse the entire data center. This is where VSAN bridges internal storage, then Nexenta extends the bridge across the data center. The ability to not only utilize the internal storage but also third party arrays and commodity hardware all presented to both physical and virtual machines.

Nexenta takes this one step further with NexentaConnect, our solution that simplifies the process of deploying a VDI solution. It is a combined all-in-one VDI automation, storage auto-deployment and storage acceleration solution. NexentaConnect can either be used in conjunction with or as an alternative to VSAN. Think of NexentaConnect this way, if you have VSAN in place then deploying your virtual desktops using local storage is a process of creating the VSAN, then using it much like any other traditional storage array. Using NexentaConnect, you deploy the storage only after looking at the desktop needs. This gives you end to end SDS.

The technology industry is famous for forcing customers into either/or decisions. But while vendors want customers to choose one product over another, customers very often need and want both. VSAN strikes us as a great example of a savvy vendor realizing that customers want both the comfort of their existing legacy storage system, and the gateway to SDS. The combination of VSAN and NexentaStor gives the combination that users are looking for.

CCOW-Negotiating Rendezvous Groups

By

CCOWTM Replicast is a storage protocol that uses multicast communications to determine which storage servers will store content and then retrieve it for a consumer. It also allows content to be accepted/delivered/replicated using multicast communications. Content can be placed once and received concurrently on multiple storage servers. Replicast can also scale to very large clusters and can support multiple sites, and each site can be as large as the networking elements will allow.

In order to understand how Replicast works, you must first understand how it uses Multicast addressing. Specifically, how the role of the Negotiating Group and Rendezvous Group differs from Consistent Hashing algorithms which are the normal solution for distributed storage systems.

How Conventional Object Storage Systems use Consistent Hashing

The Object name, sometimes referred to as the payload of a chunk, is used to calculate a Hash ID. This ID is then mapped to an aggregate container for multiple objects/chunks (for OpenStack Swift these are called “Swift Partitions” and for CEPH they are called “Placement Groups”). Although the quality of the hashing algorithm can vary, the content of a chunk has to map to a set of storage servers that is based on an Object Name in order to achieve a consistent hash algorithm. If you start with the same set of storage servers, the same content will always map to the same storage servers.

Promoters of Consistent Hashing make the point that Consistent Hashing limits the amount of content that must be moved when a set of storage servers change. If there is a 1% change in the cluster membership then 1% of the content must be relocated. In the long run, you actually want 1% of the content to move to the new servers. Should 1% of the content be lost, you will want to create new replicas of the lost 1% on other servers anyway.

Where CCOW Replicast differs is that it can be far more flexible about when that replication occurs and more selective as to which data is replicated. Replicast has a different method of assigning locations. These more efficiently deal with evolving cluster membership to achieve far higher utilization of cluster resources when the membership isn’t changing.

Negotiating Group

CCOW Replicast uses a “Negotiating Group” to effectively support the chunks “location”. An object name still yields a Name Hash ID (using the Name of the Named Manifest) but that Hash ID maps to a Negotiating Group. When a Manifest references a Chunk, it is found by mapping its Chunk ID (which is the Content Hash ID of the Chunk) to a Negotiating Group.

The Negotiating Group will be larger than the set of servers that would have been assigned by Consistent Hashing. Typically ten to twenty members of the Negotiating Group is preferred. The key is that the client, or more typically the Putget Broker on the client’s behalf, uses multicast messaging to communicate with the entire Negotiating Group at the same time. Effectively the Putget Broker asks “Hey you guys in Group X, I need three of you to store this Chunk”. A Negotiation then occurs amongst the members of the Negotiating Group to determine which three (or more) of members will accept the Chunk, when and at what bandwidth.

“Negotiating” sounds complex but the required number of message exchanges is actually the same as any TCP/IP connection setup. So the Negotiating Group can determine where the Chunk will be stored and with the same number of network interactions as Swift requires for the first TCP/IP connection. For the default replication count of three, Swift requires three connections to be setup.

More importantly, a consistent hashing algorithm (such as Swift uses) will always pick the same storage servers. This is independent of the workload of these servers. Many consider this as the price of eliminating the need for a central metadata server.

With Consistent Hashing, the 3 servers with the lightest workload are selected out of 3 storage servers (assuming the replication count is 3). Of course that also means you are also selecting the 3 busiest servers. With CCOW Replicast you select the 3 servers with the least workload from all the available servers.

Implications of Dynamically Selection

With dynamic load-sensitive selection, CCOW Replicast enables you to a) run your cluster at higher performance levels than Consistent Hashing would allow, and b) still have lower latency.

A well balanced storage cluster will at peak usage want individual storage servers to be loaded only 50% of the time. If they are heavily loaded less than 50%  of the time then the cluster could accommodate heavier peak traffic and you have overspent on your cluster. If they are loaded more than 50% of the time then some requests will be much delayed and your users could start complaining. Should the chance of a randomly selected storage server being busy is 50%, what are the chances that all 3 randomly selected storage servers will not already be working on at least one request.

When it is time to retrieve a Chunk, the client/putget broker does not need to know what servers were selected. It merely sends a request to the Negotiating Group. The negotiating group picks one of its members with the desired chunk and the rendezvous is scheduled to transfer the data.

Rendezvous Group

While the Negotiating Group plans a transfer, it is executed by the Rendezvous Group. The Rendezvous Group implements Replicast’s most obvious feature: Send Once, Receive Many times. Transfers sent via the Rendezvous Group are efficient not only because they only need to be sent once, but also because all Rendezvous Transfers are using reserved bandwidth which they can start at full speed. There is no need for a TCP/IP ramp up.

An important aspect of Rendezvous Groups is that they are easily understood and verified with a known relationship to the membership in the Negotiating Group:

  • Put transactions – every member of a Rendezvous Group is a member of the Negotiating Group with a planned Rendezvous Transfer or a slave-drive under active control of a member
  • Get transactions – the principle member of the Rendezvous Group is the client or Putget Broker that initiated the get transaction. Additional storage servers could have also been part of a put transaction. These additional targets are piggy-backing creation of extra replicas on the delivery that the client required anyway.

Cheers – It’s been a blast! – From Evan Powell

By

After six years I’m leaving Nexenta.

I could not be prouder of what we’ve built at Nexenta.  We took an idea that at the time was radical – let’s bring openness right to the foundation of IT, to the storage itself.  And we pulled it off.

Along the way I learned a lot including:

  • Team, team, team – the best team beats the brilliant individual every time
  • ZFS is great – and not perfect; thank you Sun for ZFS and Solaris (now Illumos).  It says a lot about the requirements of storage that even Solaris, arguably the 2nd most deployed OS for mission critical enterprise environments was not fully mature for storage. We’ve added a lot of fixes to Illumos over the last few years.
  • And much more that I’ll just call “experience.” Much of what you learn when you start and build companies ends up sounding like common sense; for example, what kind of executive is right for what stage of a company or how do you make money on open source?

So, what am I up to now?  A few answers:

  • Continuing to spend time as an EIR with xSeed.  Over the last several months I’ve beenblessed to meet many start-ups while an EIR at xSeed.  xSeed is an unusual seed fund.  They are deep in enterprise and have arguably the best ties of any early stage enterprise focused fund into Stanford and Berkeley.  They are old school in that they are definitely NOT spray and pray (which can be a great strategy actually).  Instead, they seek to go deep and to understand a domain and to find within that domain the very best entrepreneurs; they then surround that entrepreneur with coaches and advisors and much more than just capital.  In addition to meeting many companies I’ve been privy to many mind expanding discussions at xSeed about companies, and domains – about everything from the chess of strategy to the tough soft stuff of finding, retaining and sometimes transforming teams.  So I’ll be spending a bit more time with my colleagues at xSeed.
  • Finally, the rumors are true – I’m helping found another company.  We are in stealth mode.  I can tell you it is NOT in storage (although storage infects everything in IT).  And that it IS still very much on the side of openness and open source.  As you might imagine, I did a huge amount of diligence and founder dating before deciding to bet the next X years of my life on the opportunity.  So it is out of the frying pan and into the fire for me.

Here’s what I’d like you to do:

  • Stay in touch.  Maybe I’m getting to be sentimental in my middle age, but the people I’ve met as Nexenta has grown up are truly extraordinary.  You know who you are. You are the early channel partners who never shied away from offering “constructive feedback.”  You complain too much, but I love you.  And likewise you are the team members that parked your personal lives for weeks, months and years at a time as we fought the good fight.  Despite many missteps and the joy of starting a company during the worst financial collapse since the 1930s, you stayed true to a vision of more open storage and of what we now call software defined storage.  And maybe most importantly – you are the thousands of customers that bet your company’s data on Nexenta and sometimes bet your jobs and careers as well.  Follow me and, yes, complain to me at @epowell101.
  • Insist on excellence.  Like many entrepreneurs I think I start companies because half assed solutions to important problems make me sick (plus I probably have a complicated relationship with authority).  At a fundamental level – at the level of a life’s mission – I am dedicated to finding waste and calcification and cracking through it somehow.  How many of the world’s problems would be less serious if we all talked less and fixed stuff more?
  • Insist on openness and transparency.   Go open, it is the future, and it will make your company a smarter, more competitive business; don’t cop out and defer the decision onto the next guy in your job.  Eventually people will be fired for buying vendor X’s product; don’t be the last buyer to abandon ship when a legacy vendor goes aground.  Also, please be open within your organization as well.  Collectively we are all smarter than any one of us; by remaining open, we’ll get better technology, better team work, and happier lives.

I guess I did get sentimental.

Thanks for tolerating that sentimentality and for reading this blog and thanks everybody for helping a crazy vision come true.  Stay tuned here or via @epowell101 on twitter.

-Evan Powell

How Does Software Defined Data Storage Equate to Savings?

By

Ask any CIO what their greatest concern is, and they’ll invariably come up with some variation of concern over the budget. It’s a Catch-22 for many businesses when it comes to technology: There is always a faster, more reliable option, but it always comes at a cost. So how did the big guys get to the top? How have they learned to strike a balance between cost and effectiveness without compromising either entirely? When it comes to data, more and more have chosen to look at software defined data centers. Here’s why:

There’s no question that data is growing exponentially as we frequently use technology for every aspect of our lives – from shopping and paying bills to reconnecting with friends on social media. All of these things produce data – a lot of data in fact. Studies have estimated that we create 2.5 quintillion bytes of data each day. All of this data needs to be stored, and it can be quite costly. Some organizations are spending as must as 40% of their IT budget on storage solutions.

Therefore, government agencies, credit card companies, health care facilities, social media sites, retailors and many other entities are constantly looking for ways to store this data efficiently and affordably. Software Defined Data Centers (SDDC) address all of these concerns and more because they take the virtues of virtualization and apply them to data storage.  In fact, our customers have reported that they have saved as much as 75% in storage costs.

Here are just a few of the ways businesses can save using software defined data storage:

Scalability: Did you know that 90% of the data out there has been created in the last two years? Just think about what this could mean in terms of data storage ten years from now. As data grows, more and more hardware must be purchased to keep up with demand. Software defined data storage solutions are more easily scalable, with greater savings.

Operational Cost Savings: We talked about scalability, but what about the problems that traditional storage methods present in regard to operational costs? Energy costs to cool the storage system, labor costs to monitor and troubleshoot the system, and maintenance costs are just some things that are negated through a software defined storage system.

Open Source Opportunities: Hardware storage systems are closed systems. You are bound by specific vendors, which could limit your flexibility and opportunities to adapt when needed. Because software defined data storage solutions are often open-source, you can take advantage of the latest technologies and adapt your storage solution for the lowest cost.

For more information about the benefits of software defined data storage solutions, contact us today.

Nexenta Systems-Powered Storage Solution Achieves 1.6 Million IOPS

By

Nexenta has achieved 1.6 million IOPS (Input/Output Operations per Second) and high-availability with no single point of failure. Comparable solutions from proprietary vendors cost significantly more than the Nexenta and Area Data Systems solution and cannot guarantee high-availability. With the combination of Nexenta’s Software-defined Storage, NexentaStor™, and high-performance, all-flash hardware, there is now a clear enterprise-class alternative to meet the scalability demands of big data.

“Our customers can now reach well over one million IOPS and capitalize on big data opportunities without breaking the bank on proprietary storage technologies that cost hundreds of thousands of dollars,” said Bridget Warwick, Chief Marketing Officer, Nexenta Systems. “This is further proof that Nexenta’s Software-defined Storage is changing the economics of the enterprise storage market.”

Nexenta is demonstrating the 1.6 million IOPS storage configuration at Intel® Solutions Summit 2013 from March 19-21, 2013 in Los Angeles, Calif. Nexenta is a Silver Sponsor and will be at its booth in the storage zone to discuss the enormous opportunity for Intel channel partners to drive ideal storage solutions, powered by Nexenta, to their customers.Architecture recipes using Nexenta and Intel products are listed on Intel’s website at: http://www.esaa-members.com/recipes/advSearchList/182.