For those of you who know me, you probably wonder why I read so many books that have nothing to do with Performance Engineering and mostly have to do with management, organizational behavior and sociology. At any given time, I am typically reading 3 or 4 books in which the majority are always a business book and then of course I read at least one technical book. There is one book that I read quite some time ago from Steven Covey called the Seven Habits of Highly Effective People that I enjoyed most of all. While driving this evening, some folks on NPR brought up the book and it got me thinking. There are some characteristics that I believe define a good performance engineer from an average performance engineer. If you know me well, you know that I love what I do. It’s not just a job for me, but a career.
Learn SPE…Don’t Just Memorize It
As a young performance engineer I lacked any formal guidance. The team I worked on had little process or approach. It wasn’t a bad team. In fact at our old organization, it was one of the most respected teams in the company…almost to the point that my old manager was god-like in the eyes of so many employees at the company. Before I left, I joined a professional group called CMG and quickly learned about an interesting methodology called SPE (Software Performance Engineering). Below is a definition of SPE from Williams and Smith:
Software Performance Engineering (SPE) is a systematic, quantitative approach to the cost-effective development of software systems to meet performance requirements. SPE is a software-oriented approach that focuses on architecture, design, and implementation choices.
SPE gives you the information you need to build software that meets performance requirements on time and within budget.
The SPE process begins early in the software life cycle and uses quantitative methods to identify satisfactory designs and to eliminate those that are likely to have unacceptable performance before developers invest significant time in implementation. SPE continues through the detailed design, coding and testing phases to predict and manage the performance of the evolving software as well as monitor and report actual performance versus specifications and predictions. SPE methods encompass: performance data collection; quantitative analysis techniques; prediction strategies; management of uncertainties; data presentation and tracking; model verification and validation; critical success factors; and performance design principles, patterns and antipatterns.
When I came to Blackboard in the fall of 2003, the first thing I set-out to do was build an SPE practice. I’ve been moderately successful, which got me thinking as recently as 1 month ago that maybe the methodology is just too cumbersome for the team. I’ve been watching the team quite closely and I realized, I don’t think that the methodology is hard to understand. It’s quite practical and can be easily streamlined. Where I think the problem resides is that learning the methodology for most of the team has been viewed as a heavy academic exercise. Heck, I hand every new employee on the team a copy of Performance Solutions: A Practical Guide to Creating Responsive and Scalable Software and I think the book is so big and heavy (it’s a text book used by a lot of graduate school programs) that the content get’s lost. You tell me when the last time you snuggled up in bed to read a text book? Most likely it was college and I bet within 10 minutes you used to fall asleep while reading a book like this. The thing about this book is that the workflow and process for executing SPE is so simple and so straightforward that getting through the book shouldn’t be such a struggle. I’ve read the book 30 times in 5+ years. That’s 6 times a year or once every 2 months…
Research Daily
Another tidbit about me…did you know that I am a list guy? That’s right…every day I write a list of everything that I am to work on. I start the list in the car at stop light, at my desk and sometimes even at meetings. I work through the list each and every day. I don’t necessarily finish my list each day, but I get real close. Before having lists my days were quite chaotic and I wasn’t quite sure what I accomplished. One of the items that makes my list each day is a line item to read through Google Reader. I use Google Reader as my RSS reader. As of today, I have 186 subscriptions and the list is growing each day. I recommend that you download my OPML file (google-reader-subscriptions_110308) and use it as a starting point. Every day, usually around 12:30pm I scan through the blogs from the previous day. Some blogs are bogus and I skip over them…others are pretty cool and I tag them. Lately I’ve been starring them in Google Reader, posting them to Scholar and sending it out to the team. Every day we need to learn. Blogs are one way. Books are another…remember that the library is free. So when you think that you are having a bad day stuck in meetings or whatever…take 30 minutes to do some reading about our discipline of performance engineering.
Share Your Experiences
Each and every day we run into problems or learn something new (ie: habit #2). We need to be more vocal about our experiences so that everyone on the team can learn. Our blogs are meant to be our space for sharing these experiences. I’m really starting to believe that if we don’t have an entry a day, it’s a wasted day. That might sound odd, but I really mean it. Let me throw in a caveat. Some days you simply don’t have time to write. That happens to me all of the time. In those case, I save my writings for the days that I can contribute a posting. Often, I rail off 3 or 4 blog posts in a day as a way of catching up for those days that I’m too busy to post. What does it mean to share your experiences? It really depends. I tend to talk as though my reading audience is really a listening audience. My blogs are somewhat conversational and impersonal. I like to put lots of links in so that if my point(s) aren’t as obvious as they should be, the reader can navigate to one of those links for additional context. I also like to put pictures in my postings as often as I can. Sadly, some of the visuals I want to post are too time consuming to put together, so you might get a low budget visual. As of most recent, I’ve been doing video blogs so that I can review a topic or tool set. This is something that I want to make more a part of my weekly routine. Watching a video can be extremely powerful…especially with embedded audio.
Show Your Work
This is really an extension of #3 Share Your Experience. For this point, I want to share a quick story. In high school, I had a Math teacher named Captain McIsaac. My high school was originally a feeder school for the Naval Academy, Coast Guard Academy and the Merchant Marine. So we had a lot of older teachers who used to be former Navy. Well anyways…Old Cap McIsaac was an interesting guy. He looked like Ted Kennedy’s twin and probably scored the same on most of his breathalizer tests. He was a terrible Math teacher. Most of us thought he was awesome because he would give us the answers to the questions on our exams during an exam. We never had to show our work. That’s great for kids who cheat off each other. I have to admit…looking back the guy was terrible. He didn’t hold us accountable for our work. It showed in all of my Math classes after Cap’s class. I did well because I love Math, but it takes an awfully long time to break bad habits. You can pick-up a bad habit in seconds, but it takes weeks…sometimes years to break a bad habit.
There’s an important reason for showing your work…actually there are multiple. The number one reason is so that you personally can spend the time reviewing what you did and explaining it to your peers in a visual manner. Don’t worry if you change your ideas…you just write new blogs. The second reason is that we are a global team. Everyone on the team should get the opportunity to learn from other members of the team. It’s a good way to get feedback and share work. The third reason, which is sadly a bit lame is that our days become so busy, that sometimes we need to be able to comment on a blog rather then having a conversation or email thread.
Become a Statistics Guru
I love statistics. I think about them night and day. You too should think about them every opportunity you have. We use statistics for everything we do as performance engineers. Think about it:
- Response time sampling and analysis
- Probability models for our performance simulations
- Profiling and analysis
- Data modeling
- SQL tuning
The list goes on and on. If you haven’t taken a statistics class, I strongly recommend you do and do it fast. Then once you finished getting a good primer on statistics, we need you to start making recommendations on more meaningful statistics that will help the team.
The Need to Be a Generalist
Systems engineers are generalists. They need to understand hardware, software, integrations, functionality, business process, etc…For all intensive purposes, Performance Engineering is really just another name for a systems engineer. To be a great performance engineer, you need to know a lot about lot. That’s a lot to digest, but it’s true. I recommend that with each project, you try to learn as much as you can about one thing. As of late, I’ve taken this approach with the team. In Sprint 1 of 9.0 cyclical testing, the team focused on database performance. There were a lot themes such as query optimization and cost execution plans. The gist is that the team focused on just the database. In sprint 2, the team is focusing on ehCache (learning as much as they can to calibrate) and JVM instrumentation. With both projects, I can guarantee that each member of the team will learn something about their area of focus during a sprint. This way they won’t be too over loaded with data. This gets back to habit #2 (Researching). Use these sprint opportunities to focus on a given component in the stack or a particular technology as an opportunity to do external research. If you are learning about ehCache, one of the first things you should do is search for blogs that talk about ehCache on a regular or irregular basis. Chances are if someone is writing about it, they are likely to share links or references to others who are writing about the component or technology. One hint I give to you would be to look at the blogroll when you identify a good blog worth reading. I tend to build my blog OPL file based on evaluating other blogs referenced in the blog roll.
Question…Hypothesize and Prove It Out
The seventh habit is my favorite. We work in a lab. It might not be pretty and we don’t wear white overcoats. No matter how we spin it, the work we do is lab-like. That means we need to able to do 3 things really well (aside from researching). First we need to question. I don’t mean it in a negative way. I mean in it in an interrogative way. For example as a kid I always wondered how a radio worked. It’s a fairly easy thing to learn, but if you don’t start by asking questions such as: * How does a radio work when it’s not stationary? * How does it work inside buildings? From there, we go back to habit #2 and partake in research. From our research we develop hypothesis. A hypothesis is a calculated guess based on some form of empirical data. Then we must prove the hypothesis. We do that by testing. Where do we test? We test in the lab…So don’t ever forget…you work in a lab.
Seven Seconds 