As an engineer I’ve always felt critical of the various applications I’ve purchased or downloaded for free. If I’m going to take the time to use an application, I really want it to work. I want it to work all of the time. I don’t want to have to be super technical to figure some ridiculous workaround. I certainly don’t want a kludgy experience. I simply want the software to work.
So I have these expectations about the software developers who build the products that I use. I expect them to build the best products. I just assume their code is tested. I assume it scalable and responsive. I maybe take for granted that it’s secure, but with a blind eye I assume it is secure. When I report issues, I guess I expect them to drop everything they are doing and fix my issue. Why do I have those expectations? My attitude is that if your core product doesn’t work correctly, why are you building more product? Shouldn’t your core product work first before you introduce new product?
Motivation for this Blog
For weeks I’ve been reaching out to various contacts we have at the company and then through my own personal network to find customers who are great at giving us feedback. Feedback is essential for software companies looking to expand and grow. When you are lucky to find someone who will tell you that your product has issues, but they will continue to use it…well those kinds of customers are like the equivalent of gold.
What Features Are Considered Most Important That We Stop Developing New Features?
When I joined Blackboard in 2003, we had a really feature rich product. We had about 30 sub-systems (web apps). Each of those sub-systems had dozens to some cases hundreds of use cases. We had bugs….lots of bugs. Customers reported a lot of those bugs. We had so many bugs that in my first year, we had more support engineers than we had real engineers because we couldn’t keep up with the in-bound calls from customers. Some customers would log 10+ issues a day.
We knew we had a huge problem on our hands as an engineering team at Blackboard because we wanted to build more features. We didn’t want to really touch old code. We were focused on the new features that would help us get newer, cooler, better paying customers. On the one-hand that’s great to want to get those bigger fish. On the other hand, you have a pool of dedicated customers who quite frankly were your early adopters. They took a risk on us. Some took a small risk financially and some took a bigger risk financially. All have taken the risk of their time in using our product. We need to treat time like money.
I bring all of this up because when we attempted to filter through the minutia of issues, the team came to a collective agreement about two things. We agreed that we would be responsive to customer escalations when issues tied to a set of core features and use cases were sent to support. These core features were our bread and butter features. They were features that absolutely had to be fixed. We had to be responsive, because we knew we simply couldn’t spend the time improving our test coverage. In the early days we followed 3 core principals:
Step 1: Identify our Most Important Features and Capabilities
As a team (Engineering, Product Management and Support) we sat down and categorized every feature and use case in the product. We missed a few here and there, but met again and re-prioritized the list. We defined escalation paths for reported issues that we provided to our Support team to leverage when customer issues were reported.
Step 2: When an Issue Tied to these Most Important Features were Reported…We Agreed to Re-Prioritize Our Work and Fix Them
Since Product Management was in the discussions from the beginning, they understood that new features would sometimes be delayed due to buggy legacy features. They would review the issues coming in as well and would more often then not give the thumbs up to delay a feature in favor of making a core feature more robust and reliable.
Step 3: Always Make the Product Better
As we were forming this 3-step model, every single person chimed in that we should stop development in lieu of test automation. Let me just say this once and only once…”That my friends will never happen…”. Software has to be fluid. We can’t stop producing software and releasing software because it makes our software stale. Customers want small variations of change (not huge changes) on a reliable and consistent cadence. If the train was to stop, getting it back on the tracks is really…really hard.
We need carve out time in three ways:
a) When a customer issue is reported and we fix it…we need to add automated test coverage to minimize re-introducing issues.
b) When a customer reports one issue…we should dig into shared parts of the code and make sure nothing else is broken.
c) When we build new features, we need to incorporate the time to include better automated test coverage.