Sanjeev Sharma is the author of ‘The DevOps Adoption Playbook’ and ‘DevOps for Dummies’ and an analyst at Accelerated Strategies Group. I recently spoke with Sanjeev about the importance of security in DevOps efforts.
Sanjeev: One of the main principles of DevSecOps is to ensure that security is addressed early in the application delivery lifecycle. From a data perspective, this informs us of the need to secure all data through the application delivery lifecycle, not just data in production. Data in all non-prod environments needs to be secured. In fact, I would argue it needs to be secured with more care than data in production as non-prod environments tend to have lesser ‘hardening’ as they are not exposed to the outside world.
Sanjeev: I have personally been a proponent of not calling it DevSecOps. While I fully agree with the goals of DevSecOps, creating a new term implies that those doing just plain old vanilla ‘DevOps’ are not including security, or their product is not secure. That is a red herring. In my opinion, DevOps was always about bringing in all stakeholders into a cultural model that fostered collaboration and communication across organizational and functional silos. This has, or should have, always included the security teams.
DevOps is about shifting capabilities ‘left’ in the application delivery pipeline. Stakeholders who owned and delivered a certain capability, now deliver the same capability as a service to practitioners earlier (left of them) in the application delivery lifecycle. Dev(Sec)Ops drives this thinking to security. Security teams, now instead of just enforcing security policies and capabilities, need to focus on enabling practitioners to include security capabilities as they develop, test and deliver applications. For example, instead of testing code for vulnerabilities after all the code is developed and built, security teams now provide tools to developers to test their code for such vulnerabilities as they develop code.
Sanjeev: Certificates or secrets in general are becoming more and more prevalent in today's API-driven, Cloud hosted machine/server world. Developers now have to deal with the overhead of managing multiple keys and certificates all through the application delivery lifecycle. They now need to own managing and securing them, managing expirations, and updating them as they move from non-prod to non-prod environments and vice-versa. This is overhead that is becoming unmanageable for developers to handle using manual processes and tools. Governance processes for managing certificates and keys in most organizations are also still archaic and have not been brought into the current era, further adding to developer’s burden.
Sanjeev: There are two vectors here. The first is the obvious—the explosion of the number of machines or the virtual instances of compute as we move towards true elastic compute—whether with traditional VMs in the cloud, or containers. The second is the explosion of devices connected to the system, as the ‘edge’ evolves and compute moves closer to the end user. Both these vectors are increasing the need to manage and govern machine identity to ensure that the right machine is in the right security group, has the right privileges and access controls, and the they are provisioned, configured and de-provisioned correctly, etc. In addition, there is also the risk of rogue machines being able to insert themselves into the system. Protecting machine identity via secure, verifiable certificates becomes imperative to ensure this does not happen.
Helen: How has the DevOps world evolved since you published your books and do you plan to update them?
Sanjeev: I wish I had the time to update them. The pace of change is exponential! It is virtually impossible to keep up across the spectrum. The advent of Kubernetes is totally changing the landscape. It is driving new approaches to DevOps like GitOps, which are ideal for a Kubernetes-centric application delivery lifecycle. It is going to be easier to just write a new book.
Sanjeev: They address two dimensions of the same problem. Secure by Design focuses on securing the application and the environments on which it will be deployed. DevSecOps secures the process of delivering the application and the environments themselves.
Sanjeev: Have complete visibility—observability, if I may—of their software supply chain. No component of the supply chain should be a ‘black box’, even if it is owned by a vendor or outsourced. Supply chains are made up of multiple intertwined value streams. There are development and delivery dependencies. There are architectural dependencies. Different ‘suppliers’, internal or external, may be using different technology stacks. To ensure there is proper hygiene, there needs to be standardization in some key areas. Namely: planning and metrics, architecture, deployment automation, integration and system testing, release management, and operational readiness. These are all areas where I can spend a day speaking on each, and I do when I am working with clients. The main takeaway I would give is that at some level in the organization there needs to be visibility across the supply chains. While we may have different methodologies and tech stacks being used in different parts of the supply chain, knowing what is going on where and understanding what the impact of any change in the supply chain may be is imperative.
Sanjeev: Bringing security into the fold. There is push back from both sides—for dev teams who do not want to be responsible for security, and for security teams who do not want to give up control. This is a cultural challenge that needs to be overcome. The security teams need to work on elevating their roles from being enforcers to enablers. How can they use policy-based automation to put up guardrails for the developers and ‘shift-left’ security validation? Developers need to integrate automation tools into their workflows, that allow them to validate their code and environments for security compliance. They need to learn to collaborate better with the security teams, and vice versa.
Sanjeev: Very badly, unfortunately. I did a whole chapter in my book on building the right business case for DevOps because I saw a failure to invest in DevOps, a failure to make the right organizational changes, a failure to quantify the ROI.
I have helped several organizations build business cases. The first thing that needs to be understood is who is building the business case, for whom and for what purpose. Are IT and the business building the business case together to determine the right investment and expected return on that investment? Is IT leadership building this case to ‘sell’ the business on the need and ROI of the transformation that needs to be undertaken, and convince them to make the necessary investments? Or is IT building the business case for their own leadership solely as a budgetary exercise. This will determine how the business case is to be built and how to ensure it is consumable by the end audience, which needs to make investment decisions.
DevOps/DevSecOps transformations need both bottom-up and top-down buy-in. They need executive sponsorship and their ‘skin in the game’ to make the hard decisions— in terms of investments and in terms of making the necessary organizational changes. Building the right business case is a prerequisite to getting executive buy-in.
Sanjeev: Product teams have to think of business value first. Releasing product to production is not the endgame. Providing business value to clients is. And everyone across the team is responsible for this. This needs to be internalized by each and every practitioner, from developer to tester to deployment engineer to the coffee-machine maintenance guy. That is the mechanism, the mindset to adopt. Remember, getting code to production does not bring value to the business. Providing business value to the customer, the end-user, does. When Obamacare’s website healthcare,gov was initially launched in the US, all the code was in production, but no one could even register. No one was getting any value.
Sanjeev: Of course. Until the next thing comes about.