4. Performance Measurement
There is a lot of guidance around the measurement of performance and which performance tool to use.
The goal of this secton is to consolidate this guidance for Agencies and Marketers to help them understand how to think about performance and navigate performance tool offerings.
In order to understand how to accurately approach measuring mobile there are three key myths that need to be recognised and overcome.
Mythbusting
There are many common myths surrounding tools and their use
MYTH 1
User experience can be captured with a single metric.
Good user experience is not captured by a single point in time. It's composed of a series of key milestones in your users' journey.
Understand the different metrics and track the ones that are important to your users' experience.
MYTH 2
User experience can be captured with a single “representative user.”
Real-world performance is highly variable due to differences in users’ devices, network connections, and other factors.
Calibrate your lab and development environment to test a variety of such different conditions. Use field data to inform selection of test parameters for device type (i.e., mobile vs. desktop), network connections (i.e., 3G or 4G), and other key variables.
MYTH 3
My website loads fast for me, so it should load fast for my users.
The devices and networks that developers test load performance on are often much faster than what your users actually experience.
Use field data to understand what devices and networks your users are on and appropriately mirror those conditions when you test performance.
Data types
Lab data
Lab data is performance data collected within a controlled environment with predefined device and network settings.
This offers reproducible results and debugging capabilities to help identify, isolate, and fix performance issues.
Strengths
- Helpful for debugging performance issues
- End-to-end and deep visibility into the UX Reproducible testing and debugging environment
Limitations
- Might not capture real-world bottlenecks
- Cannot correlate against real-world page KPIs
Field data
Field data is performance data collected from real page loads your users are experiencing in the wild. (Also called Real User Monitoring or RUM)
Strengths
Captures true real-world user experience Enables correlation to business key performance indicators
Limitations
- Restricted set of metrics
- Limited debugging capabilities
