Monday, July 11, 2016

Open Source in the Enterprise - Part 2 of a 2-Part Series

By: Paul Marcinkiewicz, Solution Architect for Slalom Consulting

How Your Company Can Become An Open Source Organization

In Part 1, I talked about the history and the many advantages of utilizing open source. The advantages outlined are numerous and the more advanced companies are not only utilizing open source, but creating it as well. Part 2 will address how your company can not just leverage open source, but become an open source organization and an active user and contributor.   

Hurdles and Challenges

Overcoming the Legacy Wall

The foremost challenges surrounding a shift to open source are related to platform legacy and existing company culture. Most organizations are using or have built software applications and platforms that are tied up with licenses from big corporations like Microsoft, IBM, and Oracle. This is often a large annual cost burden. These applications and platforms cannot be torn down and replaced overnight. Software professionals are trained and often have lengthy backgrounds in legacy technologies, so introducing them to an entirely new stack and development approach can be challenging. Operations are also in place to monitor and deploy these older technology stacks. So, you may ask, then how do we get started?  

Enterprise Processes and Regulations

Netflix is a sterling example of a company that has not only grown into a large enterprise but has created and adopted a strategy to overcome enterprise processes and regulations. Netflix relies on a model that lowers process and regulation, which will increase innovation and speed of delivery. Rather than creating more process to prevent problems, Netflix relies on individual and team responsibility. This same philosophy holds true for open source adoption. Many enterprises are reluctant to use open source because of the lack of enterprise support. However, if each team is responsible for their development, testing, deployments and service level agreements (SLAs), they are free to explore and build as long as they deliver quality. 

Your Company’s Open Source Strategy

Think Like a Startup

Startups have traditionally been early adopters of open source for a number of reasons. Startups are often cash-strapped and do not have the resources to acquire software licenses, so using open source is often a decision made out of necessity. As a result, startups usually lead the way in terms of innovation and time-to-market. How do they do it? It’s done through constant experimentation and tremendously fast iteration cycles achieved by quickly leveraging and adopting these tools ad hoc. Unlike large companies that are often limited by vendor software, startups are able to choose the right tool for the job at the right time. This is especially important in the technology industry, where software release cycles are accelerating as quickly as ever. In an ecosystem where frameworks, tools, and technologies are falling in and out of favor virtually every day, the advantage of being a startup is not necessarily in the tools they choose, but in their ability to rapidly prototype, iterate, and swap technologies as they see fit. This is in stark contrast to many large companies, where adding layers of complexity to combat legacy software is the status quo.

While it’s unrealistic to advocate for the same level of flexibility often achieved at startups, it is imperative that companies approach open source in a similar spirit. We’ve seen numerous examples in recent years of large, seemingly invincible companies suffocating under the weight of a legacy tech stack. Organizations looking to join the open source movement must not expect to find a “golden hammer,” but rather build their systems to be as flexible and loosely-coupled as possible.

Find or Hire Your Own Open Source Mavericks

These open source mavericks are often hidden gems within your company. These are the developers who work nights and weekends at home and tinker. They often have their own GitHub pages showing what great work they’ve done.  A lot of them have an entrepreneurial spirit and many are working on their own side projects.  

Sometimes, these open source mavericks are professionals who have been working with the same technology for a while and want the opportunity to work on something new and challenging. Some people in your organization are happy with working on the legacy systems, while others want to be challenged. Recognizing these mavericks and giving them the opportunity to work on what’s new and shiny can change a stale culture into a fresh and exciting one and can transform your company into a destination that is desirable for the best and brightest.  

Offer Incentives

Companies such as Yahoo will offer incentives for developing technology that will either make or save the company money. In some cases, these will be in the form of offers that will give employees a direct percentage of revenue or savings generated from the initiative.  Innovating will take a lot of blood, sweat, and tears. Employees will often ask “what’s in it for me?”, other than building a good resume to allow them to move on to a better opportunity somewhere else.


Hackathons are another great method to market and find talent for a specific technology. Open source skills are some of the most sought after in the marketplace both from the employee and the employer perspective, so this is a great way of matching potential prospects with an opening. This is also a great way to market to the masses that your company is using these technologies.

Don’t Be Afraid To Fail

More often than not, teams and organizations are under constant pressure to deliver. This can cause enterprises to fall back to what we know and often take us to the path of least resistance. However, this approach will never spur innovation. If we keep patching the same legacy systems, we can never move forward and will only build the legacy wall even higher. Well-run organizations understand that consistent investment in their software is paramount to their success as opposed to constant band-aid solutions.  It’s OK to fail as long as you learn and iterate and do not spend an undue amount of time. This is what R&D is all about.  

Organize Into Small Autonomous Groups

It is often the case that not everyone will have the same opinion within an organization on how to best solve a problem. While many of your teams may be trying to solve similar problems, the backgrounds and skillsets may vary greatly. Many companies have found success with a Microservices architecture, providing their engineers with the autonomy to create the best solutions for themselves in the context of the support network that large companies benefit from. Yelp and Spotify are two examples of companies that have found success with and written about this approach.

Pick a Pilot Project

The best way to get started is to find a project that is either net-new without legacy dependencies or a project that can be retrofitted into your existing architecture. This will allow you to do "bake-offs" and "A/B performance tests" against different technologies.  To start, try something small to experiment with the new technology stack. Don't be afraid to try a number of competing technologies such as MongoDB vs Couchbase or NodeJS vs Tomcat.  Do this for every layer of your stack, from client tools through the database.  This will allow you to find the lines of demarcation within your organization.  

Create a Layer of Abstraction

Since each technology has its own tooling and modules, the last thing you want to do is be tied to a particular product because it's all wired together too tightly or has too many dependencies.  To account for this, design an API for each tier of your application to develop layers of abstraction. This approach is consistent with Microservices architecture patterns and best practices.

As an example, Node.js provides excellent asynchronous API orchestration. A NodeJS layer can stitch together compound requests into a single asynchronous request. These requests can be written in any blend of technologies and languages. For instance, high computational requests can be written in something like C++ or Java and be fronted by a Tomcat/Spring server.   

MVP (Get Your Win)

Projects often suffer from overdesign and extensive  timelines. Think long and hard about determining what your project's minimum viable product (MVP) is. The principles of Agile will support iterations before and after the initial MVP release. Getting your open source-based project released will allow your team to put a “stake in the ground” and begin to establish an open source culture within your organization. If you can eliminate the need to renew licenses or replace them with less expensive licensing deals, then that’s a big win.    

Cloud Agnostic Approach

Building an application or platform that will only run on a specific cloud provider’s infrastructure like AWS or Azure can lead to the same legacy wall issues. Your organization will become dependent on a provider’s proprietary tools.  Certainly leveraging these tools can provide benefit, but a design around a specific cloud provider’s tooling can also lock you into that specific vendor. Avoid this pitfall by leveraging tools that work on a number of cloud providers. Good examples of this are Docker for deployment containers and Node.js.  Some organizations will require certain assets to remain within their walls, so there is usually a need to think agnostically. Doing so will also allow you to burst out onto the public cloud in the future.

Use Licensing Where It Makes Sense for our Organization

There are situations where it may make sense to license certain software. If your company’s lifeline depends on a highly-available application, then buying an open source support contract may be the right decision. This license agreement can be temporary as your organization ramps up its internal skill set.  This is why it’s best to build software that does not have a strong dependency on versions that are only available in a licensed product.   

The Benefits to Contributing and the Power of Community

As you become proficient with open source, consider contributing back. This can be achieved in several ways. For one, you can submit pull requests directly to packages’ repositories on GitHub. NPM (Node Package Manager) is a great example of this. The NPM community has grown exponentially over the years and new modules are uploaded constantly.  Of course, this can also create a bit of a minefield, so proper QA and due diligence is a must.  Contributing back to open source can also provide a great way to attract talent and advertise your organization as a destination of open source innovation.

Get Bootstrapped By Getting Help

If you truly feel your company just doesn’t have enough experience with open source in-house to get started, then you can always scaffold the effort with a consulting firm.  This can provide the benefits of rapid building and allow your in-house talent to pair up with consultants to learn and sharpen their skills.  Again, this can be a transitional step once your company feels comfortable with your level of background, skill, and proficiency.

Final Thoughts

Open source provides many good options for your organization and should be seriously considered as an option to utilize. My suggestion is that you proceed in incremental steps and try to create a technological and cultural shift that will make your organization more innovative and adaptable in the fast changing marketplace. 

Paul Marcinkiewicz is a Solution Architect for Slalom Consulting - Boston and is an AWS Certified Solutions Architect. Paul has worked as an architect ranging from startups to Fortune 500 firms and has experience designing, building and migrating onto platforms utilizing open source technologies. Connect with Paul on Twitter or LinkedIn.

No comments: