Sometimes it’s hard for performance engineers to wrap their minds around when it’s time to study a transaction in isolation versus going through a full-scale design of experiment. I wanted to put some thoughts down about what drives a UI performance analysis project. What I mean by a UI performance analysis project is a profiling project in which we are studying the end-to-end response time characteristics across our certified browsers. We have been calling this work E2E, which it is.
We first start by measuring end-to-end round trip response time for all of our browsers. Subsequently, we profile each browser to understand the impact of latency caused by the browser platform. Next we profile at the application tier. Finally, we study query execution at the database tier. Our end goal is that we can take a transaction and decompose latency at each layer in the end-to-end pipeline.
I’ve decided to put some initial thoughts about factors that should be used by SPE’s for recommending a UI Performance Analysis project. I’ll briefly summarize below.
1. New Design Pattern: At the heart of SPE, we as performance engineers are to identify good design patterns and call-out poor anti-patterns. By design patterns I am talking not only API patterns, but also new approaches to workflow and client-side interaction from the UI. Whenever a new interface design pattern is introduced, we should study the behavioral characteristics of this interface across multiple browsers.
2. Variable Workload: Any time we allow the user to interact with a flexible or variable workload of data from a single page request, then we should without a doubt study how the workload affects page responsiveness.
3. Rich User Interface: Without a doubt, the richer the interface, the greater the need to study UI performance behavior.
4. Predictable Model of Concurrency: My argument about predictable models of concurrency is that use cases should be studied under non-concurrent scenarios in order to understand the service time of a single request. Once this is understood, a clearer picture can be had of the model under concurrent conditions.
5. Core State or Action: I am a firm believer that when we introduce a use case that will most likely change session behavior of a user, then it should be studied. If we essentially force users to perform a particular operation or traverse a particular page, then it should be studied.
6. Use Case Affecting User Adoption: This is a fairly broad statement. What I am getting at is that when a use case is going to increase adoption of the product or tool, then it’s a worthy candidate for studying. For example, a few weeks ago, 1-800-Flowers was the first to set-up a commerce site in Facebook. Their ultimate goal is to drive sales for their company. The underlying goal for a company like Facebook to enable such applications is to keep the application sticky so more users adopt and remain loyal to the application platform.
7. Resource and/or Interface Intensive Transaction Hypothesized: What I mean by this is that as SPE’s we hypothesize whether a transaction will be resource and/or interface intensive as part of our modeling efforts. If we have a shred of doubt that a transaction will have impact on the system execution model, then it should be an immediate candidate for analysis.
8. Transactions Affecting Cognition: We need to call-out transactions that affect how users perceive the transaction they interface with. Users have response time expectations. When those expectations are not achieved, users become impatient and/or abandon. Ultimately, poor responsiveness decreases adoption.