Continuous Delivery…Continuous Integration…Continuous Deployment…How About Continuous Measurement?

I spend a lot of my free time these days mentoring startups in the Washington, DC and Baltimore, Maryland markets. I mentor a few CEO’s, who are building software for the first time, as well as a few folks in the roles of VP of Engineering or Director of Development. It’s fun and exciting in so many ways. I feel connected to a lot of these startups and personally feel a lot of satisfaction mentoring some really great people, who are willing to put it all out there for the sake of fulfilling an entrepreneurial spirit.

I’m not just partial to startups. I enjoy collaborating with peers and colleagues that work at more tenured companies. I think it’s important to get alternative perspectives and different outlooks on various subjects such as engineering, organizational management, leadership, quality, etc…

For about four years or so there’s been a common theme amongst many of my peers and the folks I mentor. Everyone wants to be agile. They also want to be lean. There’s a common misconception that agile = lean. Yikes! I’ve also noticed that a lot of them want to follow the principals of Continuous Delivery. Many assume that Continuous Delivery also means Continuous Deployment. The two are related, but they are not one and one the same. Many of them miss that Continuous Integration is development oriented while Continuous Delivery focuses on bridging the gap between Development and Operations (aka…the DevOps Movement). Note: DevOps is a movement people…not a person or job.

The missing piece…and I say this with the most sincere tongue by the way…is that there still remains this *HUGE* gap with regards to “What Happens to Software In Production?”. My observation is that the DevOps movement and the desire for being Continuous prompted a lot of developers and operations folks to automate their code for deployment. The deployments themselves have become more sophisticated in terms of the packaging and bundling of code, auto-scaling, self-destructing resiliency tools, route-based monitoring, graphing systems galore, automated paging systems that make you extra-strong cappuccinos, etc…Snarky Comment: Developers and Operation Engineers can’t be satisfied with deploying an APM and/or RUM tool and calling it a day.

gadget  monkey

Continuous Measurement is really what I’m getting at. It’s not just the 250k metrics that Etsy collects, although that’s impressive or maybe a little obsessive to say the least. I would define Continuous Measurement as the process of collecting, analyzing, costing, quantifying and creating action from measurable data points. The consumers of this data have to be more than just Operations folks. Developers, architects, designers and product managers all need to consume this data. They *need* to do something with it. The data needs to be actionable and the consumer needs to be responsive to the data and thoughtful going forward for next generation or future designs.

In the state of Web Operations today, companies like Etsy or Netflix make a tremendous amount of meaning from the data they collect. The data drives their infrastructure and operations. Their environments are more elastic…more resilient and most of all scalable. I would ask some follow-up questions. For example, how efficient is the code? Do they measure the Cost of Compute? (aka: the cost to operate and manage executing code)

Most companies don’t think about the Cost of Compute. With the rise of metered computing, it’s amazing to abstract the lost economic potential and the implied costs because of inefficient code. Continuous Measurement should strive to balance that lost economic opportunity (aka…less profit). Compute should be measured as best as can be from a service, feature and even at a patch set level.

A lot of software companies measure the Cost to Build.  Some companies measure the Cost to Maintain. Even less measure the Cost to Compute. Every now and again you see emphasis placed on Cost to Recover. Wouldn’t it be a more complete story with regards to Profit if one was able to combine the Cost to Build with the Cost to Maintain and the Cost to Compute?

Maybe the software community worries about the wrong things. Rather than being focused on speed/delivery of code and features, maybe there should be greater emphasis placed on efficiency of code and effectiveness of features. Companies like Tesla restrict their volume so that each part and component can be guaranteed. Companies like Nordstrom’s and the Four Seasons are very focused on profit margins, but at the same time they value brand loyalty in favor. I used to think that of Apple, but it’s painfully obvious that market domination and profitability have gotten in the way of reliable craftsmanship. I love my Mac and IPhone, but I wish they didn’t have so many issues.

I have no magic beans or a formula for success per se. I would argue that if additional emphasis was placed on Continuous Measurement, many software organizations would have completely different outcomes in their never-ending quest to achieve Continuos Delivery, Continuous Integration and Continuous Deployment. It just takes a little bit of foresight to consider the notion that Continuous Measurement is equally important.


One thought on “Continuous Delivery…Continuous Integration…Continuous Deployment…How About Continuous Measurement?

  1. sneezypb

    A vendor I work with hosts the software for most of the customers. A few like us host it ourselves. Where it gets weird is even in that model the vendor deploys the code. And not very well. So their recent focus on Continuous Delivery, like you said, means they will have to get really good at automating their deployments. They do appear to be getting better, but we have also gotten better at evaluating their work for problems so our customers do not see the vendor’s mistakes.

    It would be nice to be able to depend on them to self-measure, but the reality is we have to keep them accountable. So we have to measure their work product and hold them to SLAs.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s