In my last conversation with Larry Maccherone, the DevSecOps Transformation Senior Director at entertainment giant, Comcast, we spoke about his DevSecOps manifesto. In this second conversation, we’ll dive into his vision for DevSecOps in 2020 and beyond.
What is your vision for cybersecurity in 2020 in your organization?
Larry: People think I have certain expertise because they get exposed to me in a certain way. You think I'm a security expert or a DevSecOps expert, but my real secret superpower is data and analytics. I published the largest ever study correlating agile practices with performance. The performance dimensions were productivity, predictability, quality including security, and responsiveness. And then the x-axis dimensions, the behaviors, the things that we think were predictive of those outcomes were agile practices; team size and iteration length and some non-agile things, like gender diversity on the team. Did you know that teams with even a single female outperform all-male teams?
I'm launching another research initiative correlating practices with security outcome performance. So, for instance, we have a bunch of recommendations, do X, do Y, do Z, that we tell people. But we don't know how much X is really going to impact the outcome, or Y, or Z. So quantifying that is really the key.
"Cross-team dependency is the killer of predictability"
In some cases, some of the recommendations might actually be counterproductive. We've found that in the agile world. The recommendations around team size, for instance, are not universally true. We are told that seven plus or minus two is sort of the ideal agile team size. That means nine is the maximum team size for a development team. Well, we've found teams larger than nine that had better quality and predictability metrics.
What happens when you have smaller teams is you have more cross-team dependencies. So by having larger teams, you have fewer cross-team dependencies. Cross-team dependencies are the killer of predictability. You can't count on the other team to finish doing what they need to do before you need to consume it, and if you make assumptions about the way they're going to implement that thing, the assumptions are invariably wrong and will cause you extra rework. So larger teams are actually much more predictable, and they also have more resources to dedicate to things like design and design review and code review and dedicated QA resources on the team.
How large can a team go, do you think, before it becomes unproductive or it sinks again?
Larry: Below fifteen. Fifteen was the place where performance started to fall off dramatically. The Productivity dimension of Performance was actually worse with the nine to 15 size teams as well, so if you're just cranking out code, then smaller teams are going to be better there. But it's all about the cost of coordination and the cost of coordination is exponentially proportional to the number of things you're coordinating. So if you think of individuals on the team as the only thing you're coordinating, then smaller is better almost always. But when you're building products with multiple teams, that creates cross-team dependencies. Coordinating across teams has even higher unit cost of coordination per team you're coordinating. The larger your teams are the fewer teams you'll need for the same number of people. So, it's a big tradeoff between inter-team cost of coordination and intra-team cost of coordination. The optimal tradeoff point for you is a function of your context. The two biggest drivers are the effective use of continuous integration (CI) with robust tests and the congruence between your team structure and your product architecture. This is why adopting CI with tests is so valuable. This is why folks are adopting microservices architectures. Even if they don't articulate it this way, it's all about lowering inter-team or intra-team cost of coordination.
Product Owners: are they business, are they technical, are they a mixture of both? Where do we get them from? What is their actual role, are they just a funnel for requirements? What sort of challenges do you see around product ownership?
Larry: There is a natural tendency for product ownership to drift into project management too easily, and if you aren't really an owner of the product decisions then you're not a product owner.
To me a true product owner is somebody who is an expert in that domain. If you're building a product that is an API, then you better be an API user. You better be able to write code to consume that API to be a product owner, or you're never going to make smart decisions about what the clients of that product are going to find delightful. And so the question of technical/non-technical is really a function of who the users of that product are. If the users are other developers like an API product is, then you have to be an expert developer in order to actually do a good job being a product owner in that circumstance. Now, if the product is medical records keeping, then you have to be an expert in medical record keeping more than you have to be an expert in the code.
"Be an expert in your user's field"
So the key is to be an expert in your user's field, and when I say expert, I mean seeing around corners expert; the kind of expert that can predict where it's going so that you can get the organization there first.
That's an idealistic view of a product owner, but the reality is very few development teams are run by product owners like that. And I think that it's impossible to really staff that way, so other models have evolved, and having more project manager-ish product owners that are led by a more senior visionary head product owner is not necessarily a bad model, although most agile coaches would shoot me for saying that.
In part 3 of this interview series, Larry Maccherone speaks on how zero trust impacts DevSecOps