Open source and cloud computing

Open Source And Cloud Computing A Position Paper by Krishnan Subramanian


Before the proliferation of cloud computing, open source software entered the enterprise IT helping them cut costs, avoid vendor lock-in and innovate on top of them. It democratized the enterprise software eventually leading to a newer operational model called cloud computing. As enterprises moved into a services based world, the natural question in everyone’s mind was about the usefulness of source code when everything from infrastructure to applications are offered as services. This report will critically examine this question and discuss the importance of open source in the cloud based world while also pointing out to potential problems.


Introduction

The first decade of 21st century saw large-‐scale adoption of open source software in the enterprise IT helping them break free of expensive proprietary software wherever applicable. This also lead to the democratization of enterprise software market, helping organizations cut costs, mitigate risks and invest valuable resources for innovation. Even though open source adoption increased dramatically inside enterprises, it is not the only option for enterprises. As cloud computing rides similar path to mainstream adoption, there is a lingering question on whether open source is still relevant in the services based world.

Recent trends in the cloud infrastructure and platforms market segments seems to suggest the growing importance of open source in the world of cloud computing. Cloud infrastructure platforms like Eucalyptus, OpenStack and Cloud Stack along with VMware’s CloudFoundry and Red Hat’s OpenShift platforms are making waves altering the shape and dynamics of early cloud marketplace. As cloud computing services mature and as more and more enterprises adopt cloud computing, what will be the role of open source inside and outside the enterprises? Is it still relevant? If yes, how it is going to help the enterprises, in particular, and the cloud computing market, in general. What are the pitfalls of trusting open source software? In this report, we will analyze answers to some of these questions.

Open Source: Building Blocks Of Cloud

If we look back at the history of cloud computing, it is pretty straightforward to see that open source is the reason why cloud computing exists today. Let us start with SaaS that evolved from the web services idea from the Web 2.0 era. Even though open source is not the reason for the existence of web services today, it played a critical role in the evolution and eventual adoption of web services. The availability of LAMP stack at a much affordable cost (free in many cases) helped accelerate the proliferation of web services and the Web 2.0 economy of the late 90s. Even though ASPs (Application Service Providers) were ahead of historical timeline compared to SaaS, they faltered for various reasons including lack of modern architecture and delivery mechanisms. The idea of web services and proliferation of many open source tools for enabling web scale architecture (including the world wide web itself) helped SaaS providers gain traction in the market.

Infrastructure as a Service (IaaS) took a similar path with tremendous help from open source licensing. Imagine if Amazon had to rely on proprietary software when they wanted to offer Infrastructure as a Service? Without going into the hardware costs associated to meet the requirements of many proprietary software, think about the difficulty associated with convincing a proprietary vendor to alter their licensing terms so that Amazon could offer infrastructure services. In short, it is next to impossible as software vendors will clearly resist innovation that could potentially disrupt their business. Clearly, Amazon depended on the open source software to innovate a new model of offering IT services which we call as cloud computing today. It is a similar story with Google too. There are many cloud services built on top of proprietary software today but they are more of a market reaction than any innovation.


Open Source in a Services Market?

No, it is irrelevant

When cloud computing was taking off in 2008 and AWS was synonymous with cloud, Tim O’ Reilly, a well known technology pundit and an advocate of open source, published a blog post arguing that in a services based world, open architecture (in terms of open protocols and formats) triumphs licensing. he essentially argued that when compute resources are consumed as services, having source code doesn’t make any sense to users. Instead, he argued, it is important to focus on open protocols and open formats so that users are not locked into any vendor. Based on this school of thought, a group of advocates launched Open Cloud Initiative which puts focus on open protocols and formats than the source code.

Yes, it is critical

There is another school of thought, to which I belong, that argues that even though open protocols and open formats are important to avoid vendor lock-‐in, open source is not just relevant but also critical for the cloud market. Open source cloud infrastructure software like Eucalyptus, OpenStack and CloudStack (along with other smaller initiatives) are highlighting the importance on the Infrastructure as a Service space while CloudFoundry, WSO2, Cloudify, Ironfoundry and OpenShift are gaining traction on the Platform as a Service (PaaS) space. One of the criticisms against these open source projects is their lack of traction when compared to Amazon Web Services. On the infrastructure side, AWS has a head start against these open source projects, especially OpenStack, and it will take some time before these projects gain traction. More importantly, if we consider the entire infrastructure market which IaaS is expected to disrupt, AWS itself has a very small marketshare and there are plenty of opportunities for these open source platforms to gain their marketshare. As enterprises embrace infrastructure services in large numbers, we will see this market reshaping differently even if AWS continues to extend their lead. As far as criticism against CloudFoundry and OpenShift, the entire PaaS market is in its nascent stage and it will be anything from 3-‐5 years before PaaS becomes mainstream.

Importance of Open Source in Cloud Computing

Even though the proponents of Open Cloud Initiative make a case for open protocols and formats over open source, it should be noted that it is possible to dodge open protocols with open source software. As we will explain in the section below, even though open standards can help users avoid vendor lock-‐in to a certain degree, the usual lock-‐in happens at the architectural level. Open source based infrastructure services can help mitigate these lock-‐in risks even though it doesn’t always eliminate it. There are many advantages of open source both from the cloud service provider perspective and the enterprise perspective. We will highlight some of the important ones below.

Service Providers

Even though open protocols are important in reducing the vendor lock-‐in risks, the natural tendency of the market system towards consolidation means that we will end up with a handful of cloud providers


serving the world’s computing needs. It is akin to wireless services industry in US where end customers are affected by the monopoly of handful of providers. Handful of cloud providers will lead to lack of competition in the market leading to increased risks of higher costs, lack of innovation, etc..

Moreover, a market served by handful of cloud providers like AWS, Microsoft, Google, etc. cannot meet the diverse compute needs of the world including regulatory requirements. As world gets more and more globalized and as more and more countries participate in the global economy due to newer technologies like cloud computing, social, mobile, etc., the computing needs are going to be even more diverse. A standardized set of services from a handful of cloud providers implies users reprioritizing their needs to take advantage of IT services available in the market rather than the market meeting the needs of the end users. This is clearly not going to work. Similarly, the handful of service providers cannot have datacenters in countries around the world and they cannot meet the regulatory requirements of users in many of these countries.

A federated ecosystem of cloud providers is needed to meet these diverse compute and regulatory needs. Open source cloud infrastructure platforms offer an opportunity for service providers around the world, including hundreds of datacenters built around the world and even smaller regional hosters, to offer cloud services as open source software lowers the barriers to entry considerably. Service providers can take advantage of software like OpenStack or CloudStack and build cloud services that can meet the needs of wide variety of users around the world, from small businesses to large enterprises. Problems like network latency, local support requirements, etc. will make regional cloud providers an attractive option for many cloud customers.

The future of cloud services is going to be federated and open source software is going to help in the proliferation of such an ecosystem by lowering the barrier to entry for smaller service providers. A good example to highlight this possibility is the proliferation of shared hosting providers to meet the varying needs even when the hosting market was heavily commoditized. Thus, open source will ensure competition in the cloud service provider market helping drive innovation at a rapid pace.

Enterprises

As enterprises start using cloud services, they understand that a standardized set of services offered by handful of cloud providers cannot help them. Today’s modern enterprise is a good mixture of modern web applications along with legacy applications. The first generation of cloud services like AWS, based on commodity servers, alone cannot meet their needs. Enterprises have varying needs on performance, reliability, security and compliance. Hybrid clouds are going to solve the enterprise needs along with a federated ecosystem of cloud providers. Open source cloud platforms are important as they give enterprises a seamless hybrid cloud experience without exorbitant costs and lock-‐in risks.

At a recent Enterprise PaaS conference called DeployCon 2012, the CTO of Warner Music Group (WMG) was talking about how their organization is taking advantage of PaaS to meet the needs of their customers who access media from many different devices over the internet. During his keynote, he talked about WMG’s expectations from cloud services. One of the key points he highlighted was how he wanted the service to be available on private fabric as well as a hosted offering. He then talked about


how they used CloudFoundry to build a platform for their organization. Even though it is not a necessary condition, it is evident that lack of restrictions offered by open source license of CloudFoundry and its modular architecture made it easy for them to have a platform suitable for their needs. This requirement is not specific to WMG alone but most of my conversations with CTOs and CIOs of modern enterprises highlight this point. Open source platforms accelerate enterprise innovation as the lower costs and flexibility helps them focus their valuable resources on technical innovations that has direct impact on their business.

Potential Pitfalls With Open Source Cloud Software

Even though open source offers numerous advantages, it also has some pitfalls that could have an impact in large-‐scale adoption. In this section, we will highlight some of these potential pitfalls.

• Even though open source significantly reduces vendor lock-‐in and interoperability issues, there is no guarantee that open source can eliminate them. For example, even within the OpenStack service provider ecosystem, there are many service providers like HP Cloud services who add value added services on top of the core OpenStack platform or certain customizations to it but the code may not be contributed back to the original project. This increases the compatibility risks but these risks are significantly less than those associated with proprietary platforms.

• Some of the open source platforms are still not mature enough for production services. Over time, as the code matures, this will change and these platforms will be used by both service providers and enterprises.

• Some of the open source cloud distributions, like some of the OpenStack distributions, require some specialized hardware or vendor specific hardware, ruling out the use of certain commodity servers or existing hardware. This is not specific to these open source distribution alone as most of the proprietary solutions has rigid hardware requirements.

• Except for vendor centric open source software like Eucalyptus, OpenShift, WSO2, CloudFoundry, etc., some of the community driven Open Source projects are plagued with lack of solid vendor support. It is important to take this into account while planning the cloud strategy.

• Some of these open source cloud platforms have Apache licenses. There is always a risk of the project or one of its forks moving to proprietary license in the future.

Conclusion

In spite of some of the pitfalls described in the previous section, open source cloud platforms and the federated ecosystem of cloud providers enabled by these platforms stand as a good alternative to proprietary services. Even though Amazon Web Services has a runaway lead over other cloud services and platforms, it is still a very small piece of the infrastructure market pie. As enterprise adoption of cloud services increase, the market will reshape with open source as a compelling and competitive alternative to proprietary services and products. Even though the federated ecosystem of cloud


providers may appear to be a pipe dream today, we are still at the beginning stages of this evolution. As products like OpenStack and CloudStack mature, we will be seeing more and more service providers evolve themselves into cloud service providers. After all, the hundreds of datacenters around the world are not going to convert themselves into football fields. They will take advantage of open source software and compete in the cloud service provider space. We are going to see innovation in many areas on top of these federated offerings including search and discovery, workload automation, etc.. Open source not only empowers users in terms of reduced costs and much reduced risks, it also serves as a platform for innovation.

Technology

Open source and cloud computing