Session Length Metric Changes
Posted by Sean Byrnes on Wed, Oct 06, 2010 @ 12:13 PM
We spend a lot of time here at Flurry ensuring that the metrics we provide to you are as useful and meaninful as possible.
Since the Flurry Analytics service launched, we've offered a number of metrics around Session Length (the length of time a user uses the application) and one of them was Average Session Length. Viewing the Average Session Length over time can give you a good sense as to whether users are spending more or less time in the application as you add new features or make other changes. For all of Flurry Analytics we define the Average to be the arithmetic mean.
Unfortunately, we surveyed the distribution of Session Lengths and it turns out that for the vast majority of applications it is not a normal distribution. In statistics, a normal distribution is the typical bell-shaped distribution you expect to measure with a mean and a standard deviation. Distributions that are not normal don't necessarily follow this bell-shaped curve and hence the mean and standard deviation are not as meaningful. In fact, the mean will often be much larger than what you would otherwise consider a typical measurement.
To remedy this we removed the Average Session Length and replaced it with the Median Session Length. The Median Session Length is a session length value where half of the sessions were longer and half were shorter. The median is a much more flexible statistic and can more clearly represent the typical value of a non-normal distribution. You'll also notice that in most cases the Median Session Length is substantially lower than the Average Session Length.
We will continue to survey all of the analytics we offer for cases where we can improve and simplify. In the meantime if you have any questions on these or other metrics please let us know.