10+ Lessons from my Software-Defined (SDx) Life

After an exciting week in Amsterdam, Paris and London, where we had Nexenta’s quarterly sales meeting, the first of our global OpenSDx Summits, our French launch in Paris, and my first TV interview for Nexenta in London (@cloudchantv) a number of key themes are bubbling up.  Multiple industries, organizations, people and technologies are energized by the promise of Open, SoftwareDefined “everything” – from storage, servers, and networks, to data centers, infrastructure and ultimately enterprises.  Few vendors are delivering on this promise, and few organizations understand that Software Defined Storage (#softwaredefinedstorage) (SDS) is the first critical step on this journey.  Here’s what’s top of mind from a week with the movers, shakers and influencers of OpenSDx:

  1. Hip or Hype? the Future of OpenSDx
  2. Software-Based and Software-Defined are different
  3. “Open” is also for Organizations
  4. CIOs have new imperatives – and new opportunities 
  5. Evolution is the name of the game
  6. Scale changes everything
  7. Agility + Simplicity = Happy
  8. Macroeconomics affect storage economics
  9. CIOs need a better technology cost basis
  10. Think twice about the cost of Cloud
  11. Civil rights mean data rights
  12. UK, France & Spain lead SDS pack


1. Hip or Hype? the Future of OpenSDx

There’s a lot of buzz in the industry right now around Software Defined “everything”– even traditional hardware vendors suddenly have “software-defined”, yet inexplicably hardware-based, solutions.   The resulting market fog means end users can’t get a clear view of OpenSDx.  What to do?  Get the facts straight.  Look to industry leaders like VMware who invite you to master the new reality of the Software Defined Enterprise.  Look to analysts like IDC pointing to the starring role that software increasingly plays in IT infrastructures, and survey data that show over 50% of companies in leading countries considering software defined solutions.  There’s a lot of hype, but at the end of the day, your enterprise will be software defined, and SDS is the first platform on which to achieve competitive advantage.

2. Software-Based and Software-Defined are different

To improve the view of SDx, we need standards and definitions that we can all agree on.  Analysts differentiate between Software Based and Software Defined solutions.  Why shouldn’t you?  If you want to tell the difference, ask, “How many hardware platforms does your Software Defined Solution run on?”  One is not the right answer … Software Based means you should expect hardware dependency, Software Defined means hardware, application, and protocol agnostic, enabling a Software Defined Infrastructure, for your Software Defined Enterprise.

3.  “Open” is also for Organizations

Open Source started a fundamental shift in how people think about technology – collaborative, flexible, cross-functional, team oriented.  OpenSDx is the next generation of Open Source for enterprise technology, be it compute, storage, or networking  -  like Nexenta – 100% Software.  Total Freedom.  All love. :)  Development of such solutions means breaking down barriers to create integration opportunities, increasing communication between both people and technologies.  As the gravitational pull of OpenSDx gets stronger, organizational movements are beginning to reflect this desire for open, collaborative environments, from the creation and empowerment of cross-functional Dev Ops teams, to industry collaborations like the Open Compute framework.

4. CIOs have new imperatives – and new opportunities 

The best CIOs have never been just about technology – they have a holistic view of the business and IT, and now they’ll be rewarded as they use this special insight to qualify OpenSDx solutions and understand where and why it best fits in their organization.  OpenSDx holds incredible power for CIOs – it helps transform them from IT service providers to strategic partners, capable of improving the speed of business and delivering not just technology but innovation.   OpenSDx = Efficiency = Innovation = Competitive Advantage.

5.  Evolution is the name of the game

Revolution is a scary thought for most IT leaders – few organizations want to be on the bleeding edge of technology innovation for their mission critical systems – but rapid, low risk evolution is oh so attractive, especially when your initial steps build a foundation giving a competitive advantage.  Organizations taking steps now to implement Software Defined solutions will find not only near term business benefits, but also longer term competitive ones.  This is why so many analysts and industry leaders are highlighting SDS as one of today’s big trends.  Storage is the bedrock of the data center; if you can evolve this expensive, growing component of your data center, all the change layered on top will be easier.

6.  Scale changes everything

The scale of IT has exponentially increased – environments are built with hundreds and thousands of devices and systems, proof of concepts on 100 units no longer suffices.  The complexity associated with such environments is immense, and resources and knowledge must scale more efficiently.  Architectures, people and processes will all need to evolve, and simple, manageable tools are needed to do so effectively.  At Nexenta, our typical installation used to be on the order of tens of terabytes; now, our largest customer will grow to half an Exabyte in the next eighteen months.  We expect to see more customers and organizations moving that direction.

  7.  Agility + Simplicity = Happy

Much like taking part in a revolution, end users cheer for flexibility and choice – but with boundaries.  While Software Defined (SDx) solutions mean you can make more choices, the potential permutations can be overwhelming – analysis stops, paralysis ensues, innovation stalls.  It’s incumbent upon OpenSDx solution providers to develop simple, manageable solutions, so that agility is delivered with simplicity.  Choice is wonderful, but introduces complexity, and at the end of the day, IT managers need to be sure they can still run their own house. (#agilesimplehappy).  When we engaged customers in release planning for NexentaStor 4, the loudest chants were for simplicity and improved manageability, and delivery of those characteristics is critical to customer satisfaction.

8.  Macroeconomics affect storage economics

According to the IDC surveys shared by Donna Taylor (@Donna_IDC), macroeconomic trends have a trickle down effect on storage buying behaviors.  Storage is the fastest growing, and often largest, line item in an organization’s IT budget.  How do you stave off additional CAPEX and OPEX costs?   Keep your storage longer, keep it off warranty, do more with less.  Customers are also willing to pay a little extra for flexibility and choice, so that longer-term options exist that extend the life of their storage assets.   You can also just buy NexentaConnect to get simple, better performance and density for your VDI environment. (Yes, shameless plug!)

9.  CIOs need a better cost baseline

Organizations have two related problems when trying to address their return on investment.  First, many IT organizations spend over 70% of their budget to keep the lights on – this inhibits innovation, because resources focus on maintenance instead of value add.  Second, most IT organizations lack true IT cost transparency.  Budgets are based on past behaviors and high-level estimates vs. on fact-based usage of IT services.  The highest benefit of the Software Defined Enterprise is that it balances Business and Technology, empowering Technology to deliver, price and project IT services against a business strategy – and make recommendations on the right course of action.  How to solve for these challenges and achieve SDE benefits?  Deliver simple, flexible, manageable solutions that free up time, and enable a better IT operating model with intelligence from cost data.  CIOs looking to improve their infrastructure economics via SDDC / SDI will quickly need to examine their costs.

10.  Think twice about the cost of Cloud

Like the Hotel California, when it comes to your data and the cloud, “you can check-out any time you like, but you can never leave.”  (#ThomasCornely).  Many organizations choose public cloud services as an easy way to quickly add capacity, or ramp up new products; however cost needs to be examined carefully and holistically.  Remember that you’re not only paying for storage, but also for use and access.  It’s not a one-time cost, but a year over year, growing, expenditure.   Nexenta’s CIO-validated cost analysis, based on list prices, reveals that our solutions are 70% cheaper than legacy system solutions over a 3 year time horizon, and 15% less than cloud providers.  Do you know the true cost of your storage?

And a few bonus items for our friends in Europe!

11.  Civil rights mean data rights

With Edward Snowden on the television screen, frequent discussion of the US Patriot Act and concerns about European data being on American soil, it was clear that data security and privacy are top of mind for our EMEA friends.  European organizations must enable end user preferences on how their is data used, understand what constitutes consent, how long it lasts, and also permit the “right to be forgotten”.  While data security itself is generally handled in the application layer, storage solutions like Nexenta’s with self-healing properties like those of ZFS help reduce data corruption and ensure data integrity, thus making sure the right data is available to the right people at the right time.

12.  UK, France & Spain lead SDS pack

The localization of SDS solution adoption is evident in Europe not only by industry but also by geography; as I am finding, it’s incredibly important to understand not only the culture and expectations of the countries where we work, but also where they are in their SDS journey, and what’s needed to help them take the next step.  According to survey data presented by Donna Taylor, IDC’s EMEA storage analyst, there is a continuum of adoption in Europe, with the UK, France and Spain leading the pack in terms of interest and adoption around SDS solutions, and the Nordics at the other end, exhibiting some interest.

So, what’s the upshot?  OpenSDx is real.  It’s here.  Everyone’s talking about it.  In my interview with CloudTV (@CloudTV), I was asked what makes me passionate about Nexenta and Software Defined Storage.  My answer?  We are at the forefront of a fundamental shift in how business and technology operate today – one that’s going to make all industries more efficient, more innovative, more competitive, and better.  What better place to be than leading that revolution?

Join the conversation. Let us know what you think via @Nexenta and stay tuned for videos of Nexenta’s OpenSDx Summit EMEA sessions on our YouTube channel.


Jill Orhun
Marketing & Chief of Staff
Nexenta Systems

Posted in Software-defined data center, Software-defined storage, Uncategorized

A Bridge Over Troubled (v)Storage

Every great technology shift requires the means to move from the old way of doing things to a new and vastly improved approach. A bridge, if you will.

We found that analogy very fitting while reading some interesting and thought-provoking articles on the storage industry. These are A Major Shift in the Data Storage Market is on the Horizon by Kalen Kimm of TweakTown, and Understanding Storage Architectures by Chad Sakac at VirtualGeek.

Both articles are ambitious (nearly 6,000 words combined!). Of the many observations, however, these lines from Kalen Kimm led us to comment:

“The shared visibility between compute, application, and storage is a large step forward to a true software-defined data center. Instead of having to pre-configure LUNs and then presenting them to applications to be consumed, applications will be able to consume storage on an as-needed basis.”

At Nexenta, we are obviously big believers in the software defined data center (SDDC), and the importance of software defined storage (SDS). The SDDC makes too much sense not to take hold; the only variables are around timing and adoption speed. We see VMware’s release of their Virtual SAN SDS solution as an important catalyst to address these variables. A larger player such as VMware can have significant impact on the way that enterprises run their datacenter.

A clear example of the impact is with networking. When VMware first released the hypervisor, all network switching was contained within the host and you were wholly dependent upon the physical switching layer from traditional companies, then the distributed virtual switch (DVS) was released. This allowed network segments to traverse hosts and spread throughout the virtualization cluster. This took switching to a software layer but then they added the Cisco 1000v as an option. The 1000v allowed management to be consistent throughout the data center, physical or virtual but still software based.

In our opinion, VSAN is the necessary bridge between legacy storage and a fully efficient SDS model. VSAN allows customers to utilize their existing internal storage and spread across hosts, similar to the DVS. What is missing is the next layer that allows software defined storage to traverse the entire data center. This is where VSAN bridges internal storage, then Nexenta extends the bridge across the data center. The ability to not only utilize the internal storage but also third party arrays and commodity hardware all presented to both physical and virtual machines.

Nexenta takes this one step further with NexentaConnect, our solution that simplifies the process of deploying a VDI solution. It is a combined all-in-one VDI automation, storage auto-deployment and storage acceleration solution. NexentaConnect can either be used in conjunction with or as an alternative to VSAN. Think of NexentaConnect this way, if you have VSAN in place then deploying your virtual desktops using local storage is a process of creating the VSAN, then using it much like any other traditional storage array. Using NexentaConnect, you deploy the storage only after looking at the desktop needs. This gives you end to end SDS.

The technology industry is famous for forcing customers into either/or decisions. But while vendors want customers to choose one product over another, customers very often need and want both. VSAN strikes us as a great example of a savvy vendor realizing that customers want both the comfort of their existing legacy storage system, and the gateway to SDS. The combination of VSAN and NexentaStor gives the combination that users are looking for.

Posted in Software-defined storage, Virtualization

CCOW-Negotiating Rendezvous Groups

CCOWTM Replicast is a storage protocol that uses multicast communications to determine which storage servers will store content and then retrieve it for a consumer. It also allows content to be accepted/delivered/replicated using multicast communications. Content can be placed once and received concurrently on multiple storage servers. Replicast can also scale to very large clusters and can support multiple sites, and each site can be as large as the networking elements will allow.

In order to understand how Replicast works, you must first understand how it uses Multicast addressing. Specifically, how the role of the Negotiating Group and Rendezvous Group differs from Consistent Hashing algorithms which are the normal solution for distributed storage systems.

How Conventional Object Storage Systems use Consistent Hashing

The Object name, sometimes referred to as the payload of a chunk, is used to calculate a Hash ID. This ID is then mapped to an aggregate container for multiple objects/chunks (for OpenStack Swift these are called “Swift Partitions” and for CEPH they are called “Placement Groups”). Although the quality of the hashing algorithm can vary, the content of a chunk has to map to a set of storage servers that is based on an Object Name in order to achieve a consistent hash algorithm. If you start with the same set of storage servers, the same content will always map to the same storage servers.

Promoters of Consistent Hashing make the point that Consistent Hashing limits the amount of content that must be moved when a set of storage servers change. If there is a 1% change in the cluster membership then 1% of the content must be relocated. In the long run, you actually want 1% of the content to move to the new servers. Should 1% of the content be lost, you will want to create new replicas of the lost 1% on other servers anyway.

Where CCOW Replicast differs is that it can be far more flexible about when that replication occurs and more selective as to which data is replicated. Replicast has a different method of assigning locations. These more efficiently deal with evolving cluster membership to achieve far higher utilization of cluster resources when the membership isn’t changing.

Negotiating Group

CCOW Replicast uses a “Negotiating Group” to effectively support the chunks “location”. An object name still yields a Name Hash ID (using the Name of the Named Manifest) but that Hash ID maps to a Negotiating Group. When a Manifest references a Chunk, it is found by mapping its Chunk ID (which is the Content Hash ID of the Chunk) to a Negotiating Group.

The Negotiating Group will be larger than the set of servers that would have been assigned by Consistent Hashing. Typically ten to twenty members of the Negotiating Group is preferred. The key is that the client, or more typically the Putget Broker on the client’s behalf, uses multicast messaging to communicate with the entire Negotiating Group at the same time. Effectively the Putget Broker asks “Hey you guys in Group X, I need three of you to store this Chunk”. A Negotiation then occurs amongst the members of the Negotiating Group to determine which three (or more) of members will accept the Chunk, when and at what bandwidth.

“Negotiating” sounds complex but the required number of message exchanges is actually the same as any TCP/IP connection setup. So the Negotiating Group can determine where the Chunk will be stored and with the same number of network interactions as Swift requires for the first TCP/IP connection. For the default replication count of three, Swift requires three connections to be setup.

More importantly, a consistent hashing algorithm (such as Swift uses) will always pick the same storage servers. This is independent of the workload of these servers. Many consider this as the price of eliminating the need for a central metadata server.

With Consistent Hashing, the 3 servers with the lightest workload are selected out of 3 storage servers (assuming the replication count is 3). Of course that also means you are also selecting the 3 busiest servers. With CCOW Replicast you select the 3 servers with the least workload from all the available servers.

Implications of Dynamically Selection

With dynamic load-sensitive selection, CCOW Replicast enables you to a) run your cluster at higher performance levels than Consistent Hashing would allow, and b) still have lower latency.

A well balanced storage cluster will at peak usage want individual storage servers to be loaded only 50% of the time. If they are heavily loaded less than 50%  of the time then the cluster could accommodate heavier peak traffic and you have overspent on your cluster. If they are loaded more than 50% of the time then some requests will be much delayed and your users could start complaining. Should the chance of a randomly selected storage server being busy is 50%, what are the chances that all 3 randomly selected storage servers will not already be working on at least one request.

When it is time to retrieve a Chunk, the client/putget broker does not need to know what servers were selected. It merely sends a request to the Negotiating Group. The negotiating group picks one of its members with the desired chunk and the rendezvous is scheduled to transfer the data.

Rendezvous Group

While the Negotiating Group plans a transfer, it is executed by the Rendezvous Group. The Rendezvous Group implements Replicast’s most obvious feature: Send Once, Receive Many times. Transfers sent via the Rendezvous Group are efficient not only because they only need to be sent once, but also because all Rendezvous Transfers are using reserved bandwidth which they can start at full speed. There is no need for a TCP/IP ramp up.

An important aspect of Rendezvous Groups is that they are easily understood and verified with a known relationship to the membership in the Negotiating Group:

  • Put transactions – every member of a Rendezvous Group is a member of the Negotiating Group with a planned Rendezvous Transfer or a slave-drive under active control of a member
  • Get transactions – the principle member of the Rendezvous Group is the client or Putget Broker that initiated the get transaction. Additional storage servers could have also been part of a put transaction. These additional targets are piggy-backing creation of extra replicas on the delivery that the client required anyway.
Posted in Uncategorized

Cheers – It’s been a blast! – From Evan Powell

After six years I’m leaving Nexenta.

I could not be prouder of what we’ve built at Nexenta.  We took an idea that at the time was radical – let’s bring openness right to the foundation of IT, to the storage itself.  And we pulled it off.

Along the way I learned a lot including:

  • Team, team, team – the best team beats the brilliant individual every time
  • ZFS is great – and not perfect; thank you Sun for ZFS and Solaris (now Illumos).  It says a lot about the requirements of storage that even Solaris, arguably the 2nd most deployed OS for mission critical enterprise environments was not fully mature for storage. We’ve added a lot of fixes to Illumos over the last few years.
  • And much more that I’ll just call “experience.” Much of what you learn when you start and build companies ends up sounding like common sense; for example, what kind of executive is right for what stage of a company or how do you make money on open source?

So, what am I up to now?  A few answers:

  • Continuing to spend time as an EIR with xSeed.  Over the last several months I’ve beenblessed to meet many start-ups while an EIR at xSeed.  xSeed is an unusual seed fund.  They are deep in enterprise and have arguably the best ties of any early stage enterprise focused fund into Stanford and Berkeley.  They are old school in that they are definitely NOT spray and pray (which can be a great strategy actually).  Instead, they seek to go deep and to understand a domain and to find within that domain the very best entrepreneurs; they then surround that entrepreneur with coaches and advisors and much more than just capital.  In addition to meeting many companies I’ve been privy to many mind expanding discussions at xSeed about companies, and domains – about everything from the chess of strategy to the tough soft stuff of finding, retaining and sometimes transforming teams.  So I’ll be spending a bit more time with my colleagues at xSeed.
  • Finally, the rumors are true – I’m helping found another company.  We are in stealth mode.  I can tell you it is NOT in storage (although storage infects everything in IT).  And that it IS still very much on the side of openness and open source.  As you might imagine, I did a huge amount of diligence and founder dating before deciding to bet the next X years of my life on the opportunity.  So it is out of the frying pan and into the fire for me.

Here’s what I’d like you to do:

  • Stay in touch.  Maybe I’m getting to be sentimental in my middle age, but the people I’ve met as Nexenta has grown up are truly extraordinary.  You know who you are. You are the early channel partners who never shied away from offering “constructive feedback.”  You complain too much, but I love you.  And likewise you are the team members that parked your personal lives for weeks, months and years at a time as we fought the good fight.  Despite many missteps and the joy of starting a company during the worst financial collapse since the 1930s, you stayed true to a vision of more open storage and of what we now call software defined storage.  And maybe most importantly – you are the thousands of customers that bet your company’s data on Nexenta and sometimes bet your jobs and careers as well.  Follow me and, yes, complain to me at @epowell101.
  • Insist on excellence.  Like many entrepreneurs I think I start companies because half assed solutions to important problems make me sick (plus I probably have a complicated relationship with authority).  At a fundamental level – at the level of a life’s mission – I am dedicated to finding waste and calcification and cracking through it somehow.  How many of the world’s problems would be less serious if we all talked less and fixed stuff more?
  • Insist on openness and transparency.   Go open, it is the future, and it will make your company a smarter, more competitive business; don’t cop out and defer the decision onto the next guy in your job.  Eventually people will be fired for buying vendor X’s product; don’t be the last buyer to abandon ship when a legacy vendor goes aground.  Also, please be open within your organization as well.  Collectively we are all smarter than any one of us; by remaining open, we’ll get better technology, better team work, and happier lives.

I guess I did get sentimental.

Thanks for tolerating that sentimentality and for reading this blog and thanks everybody for helping a crazy vision come true.  Stay tuned here or via @epowell101 on twitter.

-Evan Powell

Tagged with: ,
Posted in Corporate, Software-defined storage

How Does Software Defined Data Storage Equate to Savings?

Ask any CIO what their greatest concern is, and they’ll invariably come up with some variation of concern over the budget. It’s a Catch-22 for many businesses when it comes to technology: There is always a faster, more reliable option, but it always comes at a cost. So how did the big guys get to the top? How have they learned to strike a balance between cost and effectiveness without compromising either entirely? When it comes to data, more and more have chosen to look at software defined data centers. Here’s why:

There’s no question that data is growing exponentially as we frequently use technology for every aspect of our lives – from shopping and paying bills to reconnecting with friends on social media. All of these things produce data – a lot of data in fact. Studies have estimated that we create 2.5 quintillion bytes of data each day. All of this data needs to be stored, and it can be quite costly. Some organizations are spending as must as 40% of their IT budget on storage solutions.

Therefore, government agencies, credit card companies, health care facilities, social media sites, retailors and many other entities are constantly looking for ways to store this data efficiently and affordably. Software Defined Data Centers (SDDC) address all of these concerns and more because they take the virtues of virtualization and apply them to data storage.  In fact, our customers have reported that they have saved as much as 75% in storage costs.

Here are just a few of the ways businesses can save using software defined data storage:

Scalability: Did you know that 90% of the data out there has been created in the last two years? Just think about what this could mean in terms of data storage ten years from now. As data grows, more and more hardware must be purchased to keep up with demand. Software defined data storage solutions are more easily scalable, with greater savings.

Operational Cost Savings: We talked about scalability, but what about the problems that traditional storage methods present in regard to operational costs? Energy costs to cool the storage system, labor costs to monitor and troubleshoot the system, and maintenance costs are just some things that are negated through a software defined storage system.

Open Source Opportunities: Hardware storage systems are closed systems. You are bound by specific vendors, which could limit your flexibility and opportunities to adapt when needed. Because software defined data storage solutions are often open-source, you can take advantage of the latest technologies and adapt your storage solution for the lowest cost.

For more information about the benefits of software defined data storage solutions, contact us today.

Tagged with: , ,
Posted in Corporate, Software-defined storage

Get every new post delivered to your Inbox.

Join 3,827 other followers