The US Department of Defense (DoD) say in their reference design for their enterprise DevSecOps initiative that:
“Legacy software acquisition and development practices in the DoD do not provide the agility to deploy new software at the “speed of operations”. In addition, security is often an afterthought, not built in from the beginning of the lifecycle of the application and underlying infrastructure. DevSecOps is the industry best practice for rapid, secure software development.”
This sentiment that security is an afterthought is one of the drivers of the DevSecOps movement—along with the recognition that security expertise is a constraint in virtually every organization. This constraint needs to be broken through a combination of automation and human capability, shifting security left in the software delivery pipeline and sharing security knowledge. Even in ‘The DevOps Handbook’, security is mentioned last in the subtitle ‘How to Create World-Class Agility, Reliability & Security in Technology Organizations’ and features mainly in the final two chapters: 22: ‘Information Security as Everyone’s Job, Every Day’ and 23: ‘Protecting the Deployment Pipeline and Integrating into Change Management and Other Security and Compliance Controls’.
Knowing what DevSecOps is, and whether an organization is good at it, is tough as many organizations still consider this an emerging area of practice—even though the US DoD state it’s “an established mature capability in industry”. Taking a look at how people describe and share their experiences can help. This, after all, is what drives DevOps: a community movement driven by practitioners, or a ‘scenius’.
Here are six examples of what expert DevSecOps practitioners are doing today:
DevSecOps Evangelist and Security Architect, DJ Schleen, now at Sonatype, but previously at healthcare provider, Aetna, said the following in his talk at AllDayDevOps 2018, titled ‘Automating Security in DevOps—Security in the Pipeline.’:
“DevOps is an unprecedented opportunity for security. DevOps breaks the chain of waterfalls. With DevOps you can get fixes out quickly and easily. No one has to come in on Saturday.”
You can hear from Schleen’s about his experiences at Aetna in his talk ‘Implementing DevOps in Regulated Environment’ given at the DevSecOps RSA Conference in 2017.
Aetna’s previous CISO, Jim Routh, has also shared his experiences adopting DevSecOps practices at the organization:
"Our whole software security program is predicated on the objective of reducing the total cost of ownership of the application portfolio. We try to prevent as many defects as we can, and we try to fix as many bugs as well as we can, and that ends up creating a productivity gain of 20 to 50 percent."
Routh also makes important points about how our traditional approach to performing security tests after code deployment doesn’t suit the new model:
“We are not a big fan of running industrial-strength software security testing tools after the code is written, taking a day or two to compile the results, and then sending it back to the development teams. If the development teams are doing a DevOps model, they are continuously doing builds.”
Larry Maccherone is the DevSecOps Transformation Director at media giant, Comcast. Maccherone’s Trust Algorithm helps organizations work through the steps they need to take to implement effective DevSecOps. The underlying trust formula says that trust equals credibility plus reliability plus empathy all divided by self-interest. Maccherone is not afraid to be controversial in his quest for truth and improvement, using statements such as:
“A typical security specialist will not have the necessary credibility to accomplish mindset shift and behavior change in developers because they are not developers themselves.”
However, I think these types of statements are important—and essential for organisations to accept in order to realise how important it is for the development and security professionals to join forces to protect their mutual interests. Maccherone is also responsible for the DevSecOps Manifesto, and its core values:
- Build security in more than bolt it on
- Rely on empowered development teams more than security specialists
- Implement features securely more than security features
- Use tools as feedback for learning more than end-of-phase stage gates
- Build on culture change more than policy enforcement
3: ING Bank
At the DevOps Enterprise Summit in London in 2019, Jan-Joost Bowman and Leon Janson spoke about their work at ING Bank in a talk titled ‘Cybersecurity Starts with Risk Aware Engineers’. They explained that:
“In waterfall there was no time to consider risk and governance until the end—so we took the same approach to testing and shifted it left. Hackers go after the easiest options so we need to close these options down early. We call this ‘shift left’ Security by Design; it’s also known as DevSecOps. Automation is essential: starting with testing. New starters go to a bootcamp where they learn risk by design and security by design.”
The team recognise that software is more like milk than wine: i.e. it doesn’t age well and there is constant pressure on security professionals to have their fingers on the pulse of new vulnerabilities as they appear. They recognised how this accountability has to extend further than the security teams in a world where cyber attacks are only ever on the increase:
“It all starts with a risk aware mindset. We didn’t think we could measure this in the culture but we could measure it in the effort. We started risk awareness days. One day dedicated to risk for all engineers with the CIO as a role model. He said that people would not progress in their careers if they could not demonstrate that they incorporated risk into their daily work.”
“Anything we've done in public cloud with our new apps—anything that falls into our DevSecOps model—my team now has complete visibility as every change happens. They know configuration changes, and we're able to be part of security testing of code, penetration testing, and we've developed these iterative cycles with the development and the ops teams where we're just included in every step along the way”.
It’s great to hear clear joy from the mouths of the security teams themselves because of their involvement and visibility through the value delivery cycle.
“Regulatory pressures and increased exposure are driving more complex requirements for managing security risks. We’ve gained the ability to view and act upon security risk as it pertains to our organisation's assets. In addition, being able to report on remediation and response plans has helped us meet strict financial compliance requirements.”
The US Department of Defense describe, in their DevSecOps reference design, a characteristic of DevSecOps as achieving fully automated risk characterization, monitoring and mitigation across the application lifecycle. They recognise that there is no “one size fits all” approach to DevSecOps and advise that each part of the organization needs to “tailor its culture and its DevSecOps practices to its own unique processes, products, security requirements, and operational procedures”. They say that the evolution starts with commitment from leadership, whilst also recognising that a dual top down and bottom up approach is needed, that leads to cultural change and automation. Consideration of all four of the DevSecOps pillars they have identified (organization, process, technology and governance) is essential.
They emphasise the importance of nurturing safety culture where both success and failure are seen as learning opportunities to be shared across the entire organization. Operating in an incremental manner is also seen as foundational to DevSecOps success. Nicolas M, Chaillan, the US Air Force Chief Software Officer, who shared the reference design on LinkedIn, has invited feedback and recognises that this is a living artifact that will continue to evolve as we learn and discover new practices and new technologies are developed.
Global organizations from a wide range of industries are practicing DevSecOps today and sharing their experiences for the greater good, contributing to the DevOps Scenius. Do you have stories to share too?