Monthly Archives: February 2015

Why It’s Important to Build a Better Product

As an engineer I’ve always felt critical of the various applications I’ve purchased or downloaded for free. If I’m going to take the time to use an application, I really want it to work. I want it to work all of the time. I don’t want to have to be super technical to figure some ridiculous workaround. I certainly don’t want a kludgy experience. I simply want the software to work.

So I have these expectations about the software developers who build the products that I use. I expect them to build the best products. I just assume their code is tested. I assume it scalable and responsive. I maybe take for granted that it’s secure, but with a blind eye I assume it is secure. When I report issues, I guess I expect them to drop everything they are doing and fix my issue. Why do I have those expectations? My attitude is that if your core product doesn’t work correctly, why are you building more product? Shouldn’t your core product work first before you introduce new product?

buggysoftware

Motivation for this Blog

For weeks I’ve been reaching out to various contacts we have at the company and then through my own personal network to find customers who are great at giving us feedback. Feedback is essential for software companies looking to expand and grow. When you are lucky to find someone who will tell you that your product has issues, but they will continue to use it…well those kinds of customers are like the equivalent of gold.

What Features Are Considered Most Important That We Stop Developing New Features?

When I joined Blackboard in 2003, we had a really feature rich product. We had about 30 sub-systems (web apps). Each of those sub-systems had dozens to some cases hundreds of use cases. We had bugs….lots of bugs. Customers reported a lot of those bugs. We had so many bugs that in my first year, we had more support engineers than we had real engineers because we couldn’t keep up with the in-bound calls from customers. Some customers would log 10+ issues a day.

stronbadtechsupport

We knew we had a huge problem on our hands as an engineering team at Blackboard because we wanted to build more features. We didn’t want to really touch old code. We were focused on the new features that would help us get newer, cooler, better paying customers. On the one-hand that’s great to want to get those bigger fish. On the other hand, you have a pool of dedicated customers who quite frankly were your early adopters. They took a risk on us. Some took a small risk financially and some took a bigger risk financially. All have taken the risk of their time in using our product. We need to treat time like money.

I bring all of this up because when we attempted to filter through the minutia of issues, the team came to a collective agreement about two things. We agreed that we would be responsive to customer escalations when issues tied to a set of core features and use cases were sent to support. These core features were our bread and butter features. They were features that absolutely had to be fixed. We had to be responsive, because we knew we simply couldn’t spend the time improving our test coverage. In the early days we followed 3 core principals:

Step 1: Identify our Most Important Features and Capabilities

As a team (Engineering, Product Management and Support) we sat down and categorized every feature and use case in the product. We missed a few here and there, but met again and re-prioritized the list. We defined escalation paths for reported issues that we provided to our Support team to leverage when customer issues were reported.

Step 2: When an Issue Tied to these Most Important Features were Reported…We Agreed to Re-Prioritize Our Work and Fix Them

Since Product Management was in the discussions from the beginning, they understood that new features would sometimes be delayed due to buggy legacy features. They would review the issues coming in as well and would more often then not give the thumbs up to delay a feature in favor of making a core feature more robust and reliable.

Step 3: Always Make the Product Better

As we were forming this 3-step model, every single person chimed in that we should stop development in lieu of test automation. Let me just say this once and only once…”That my friends will never happen…”. Software has to be fluid. We can’t stop producing software and releasing software because it makes our software stale. Customers want small variations of change (not huge changes) on a reliable and consistent cadence. If the train was to stop, getting it back on the tracks is really…really hard.

We need carve out time in three ways:

a) When a customer issue is reported and we fix it…we need to add automated test coverage to minimize re-introducing issues.

b) When a customer reports one issue…we should dig into shared parts of the code and make sure nothing else is broken.

c) When we build new features, we need to incorporate the time to include better automated test coverage.

Advertisements

Bees With Machine Guns

bees

So apparently I’m late to the party. A really cool tool is making the rounds this week on one of my favorite podcasts (Reply All) called Bees with Machine Guns. The kind folks at the Chicago Tribune have created this little app to generate tons of EC2 micro-instances for load-testing. Who needs JMeter, Apache AB, Grinder or even commercial tools like LoadRunner or Soasta when you could get away with a nice little tool like this. The tool is a few years old based on this blog. It’s gone through 2 releases now. It might be a worthy project to make a pull request against.

Take a look at this blog here for a full breakdown of setup and running of the tool.

Time to Start Blogging Again…Hopefully More to Come

I have been meaning to post a blog for quite some time. I’ve been busy growing my new team, building new product and establishing a new identify for me in the AppSec space. It’s been more than rewarding to say the least. So I figured I would put a quick blog for those few readers out there who have wondered where I’ve been all of these months.

Let’s Start with Velocity 2013

I got a call around 6:30am PST on June 21, 2013 from my former boss at Blackboard. She had been my boss for the better part of a decade and she knew I was west coast, so I was surprised to get a call so early. It was one of those quick calls. It lasted 2 minutes. Basically, she said she was leaving Blackboard to move onto another job and that some changes were coming and that I would be meeting my new boss in a couple of days.

Gene Kim and DevCon 2013

A few weeks after Velocity was our own conference in July. I had been planning it for months with a few of my colleagues at Blackboard. I was to co-emcee the Developer Conference with Mike McGarr with special guest Gene Kim as our keynote. The conference was my defining moment at Blackboard. I had given dozens of talks over the years at DevCon and BbWorld, but this was a conference I had been dreaming of putting on for years. It went without a hitch. Mike and I nailed it…our customers were psyched…Gene Kim rocked the house!

Carbon Based Units and Lifestyle Jobs

When we all got back from Vegas, many of us who were part of Bb’s core LMS division realized we were not in Kansas anymore. There was new leadership in town. The company was going to change whether we wanted it to change or not.

I don’t want to go into too many details. From BbWorld 2013 to when I left before BbWorld 2014, I saw some of the best engineers in the world…many who had spent the better half of the decade working with me, decide to pick-up and leave. Most if not all of these engineers worked long days and delivered good work. In the eyes of some, they were compared to CBUs (Carbon Based Units) and their work was considered a lifestyle job because they worked from home or in remote offices.

For those of us who were close to the ground and had an ear on the pulse, we knew that this was a talented bunch of engineers, passionate about their work and loyal to the brand. I saw more engineers walk out the door than walk in. I tried to bring in some new engineers and for a short period, I was successful. In the end, I too succumbed to being miserable and had to pack-up 11 years worth of books and memories.

Summer of Steve

I took the whole month of July off before I would start work again. My wife was kind and patient enough to let me do my own thing. It was the best 3 weeks a guy could ask for. I would start my days taking my kids to swim practice. I would go to the gym or hangout with the other parents. On Monday’s I would volunteer with my daughter’s golf practice. Tuesdays through Thursdays I would volunteer at tennis. Everyday I played golf. I got in 19 rounds over the course of 3 weeks.

The best part is that every day I was excited to wake-up and do it all over again…

6 Weeks of Consulting

I took a 6-week tour for a gig that I will look back upon and say I was a really good consultant. I wasn’t a consultant though…I went to work with one of my longtime colleagues from Blackboard. Sadly, I feel bad that I let him down. The job just wasn’t for me. My expectations were different than theirs. They wanted me to do one thing…I thought I was brought on to do something else…In the end, when Contrast came calling, I realized it was better to cut bait at 6-weeks before it was too late.

Starting Over Has It’s Perks

The “new normal” as my wife called it was all about re-establishing the passion in my career. For 9.5 of my 11 years of Blackboard, I woke-up every day excited to take on the world. Those last 18 months were quite possibly the most painful 18 months I’ve ever gone through. I won’t allow that to happen again…Mark my word 😉

It’s Feels Like I’m 28 All Over Again

The best part of my day starts around 6am when my alarm clock goes off. I love to get up in the morning. I love to look over my left shoulder and see my beautiful wife. I love waking my kids up to get them ready for school. As it relates to me…well, I absolutely love to get going on work. Every new day at Contrast brings a different challenge and an excitement that reminds me of when I was 28 and started to build the PerfEng team at Blackboard.

Growing Our Team…Building Some Cool Products

I was fortunate enough to inherit a good size engineering organization (9 engineers) considering the size of our company. Most of our employees are engineering. We have a rich, engineering culture. I’ve added to it in my short-time. I’ve brought on 4 engineers so far. I have 2 more starting in a couple days to get the count to 6 new engineers. It’s the right size for where we are now. We are lean…nimble…We can do almost anything we want.

A Promise…Of Sorts

It’s been entirely too long since I last blogged. It may be an empty promise, but I am going to do my best to keep blogging. This blog was more for me than anyone…If anyone got something out of it, I’m glad I could help.