Most popular GPUs on Nightly and Release

We have a long-standing bug about Firefox not scrolling smoothly with Intel GPUs. It turns out that recently, thanks to a driver update, the performance has drastically improved for some of our users. While trying to confirm this hypothesis I found some interesting data about Firefox users on Nightly 33 and Release 30 that might come in handy in deciding on what hardware to run our benchmarks in the future.

First things first, let’s have a look at the popularity of the various GPU vendors:

That doesn’t really come as a surprise considering how ubiquitous Intel chips are nowadays. This also means we should concentrate our optimization efforts towards Intel GPUs since it’s there where we can have the biggest impact.

But how well does Firefox perform with GPUs from the above mentioned vendors? It turns out that one of our telemetry metrics, that measures the average time to render a frame during a tab animation, comes in pretty handy to answer this question:

Ideally we would like to reach 60 frames per second for any vendor. Considering that practically every second user has an Intel GPU, the fact that our performance on those chips is not splendid weights even more on our shoulders. Though, as one might suspect, Intel chips are usually not as performant as Nvidia’s or AMD’s ones. There is also a suspicious difference in performance between Nightly and Release; could this be related to Bug 1013262?

The next question is what specific models do our users possess. In order to answer it, let’s have a look at the most popular GPUs that account for 25% of our user base on Nightly:

nightly_pop

and on Release:

As you can notice there are only Intel GPUs in the top 25% of the population. The HD 4000 and 3000 are dominating on Nightly while the distribution is much more spread out on Release with older models, like the GMA 4500, being quite popular. Unsurprisingly though the most popular chips are mobile ones.

Let’s take now as reference the most popular chips on Release and see how their performance compares to Nightly:

We can immediately spot that newer chips perform generally better than older ones, unsurprisingly. But more importantly, there is a marked performance difference between the same models on Nightly and Release which will require further investigation. Also noteworthy is that older desktop models outperform newer mobile ones.

And finally we can try to answer the original question by correlating the tab animation performance with the various driver versions. In order to do that, let’s take the Intel HD 4000 and plot the performance for its most popular drivers on the release channel and compare it to the nightly one:


We can notice that there is a clear difference in performance between the older 9.X drivers and the newer 10.X which answers our original question. Unfortunately though only about 25% of our users on the release channel have updated their driver to a recent 10.X version. Also, the difference between the older and newer drivers is more marked on the nightly channel than on the release one.

We are currently working on an alerting system for Telemetry that will notify us when a relevant change appears in our metrics. This should allow us to catch pre-emptively regressions, like Bug 1032185, or improvements, like the one mentioned in this blog post, without accidentally stumbling on it.

9 thoughts on “Most popular GPUs on Nightly and Release

  1. Thank you for sharing this analysis.

    Grouping the marketshare by the chip architecture generation would be reasonable because newer ones have more models ( https://en.wikipedia.org/wiki/Intel_HD_Graphics )

    Do you know why sometimes older drivers outperform newer drivers (twice in your chart)? Is this due to different operating systems?

    Is it possible to compare the GPU performance on Nightly to other GPU performance data (maybe even for other browsers)?

    Start and end of the y-axis in the last chart can confuse the reader when having a glimpse on it.

    1. I don’t know why older drivers perform in some cases slightly better than newer ones. This is something I should mention to our graphics team which might want to follow up this analysis with a deeper investigation.

      I also adjusted the plot.

      Thanks!

    1. Or you could consider running Nightly yourself. It’s pretty stable and very worthy of being your daily browser. If you want to have more input and involvement in Fx development, that’s the way to go.

      1. canukistani is already a MoCo employee (PM for devtools). He is interested in more representative data about the user population than Nightly offers.

    2. I don’t think the population is too small, the confidence intervals for the medians are small enough that we can reasonably assume what we are seeing makes sense.

      I think the real issue is that the Nightly population is biased towards developers. Rerunning the analysis on Release data has shown that, among other things, some older Intel GPUs are quite popular.

      I have updated the post accordingly, thanks for the suggestion.

  2. Good post. Sadly, unlike AMD and Nvidia, Intel prevents 1st party drivers from being installed on some OEM machines, claiming the driver “isn’t certified.” This makes driver updates few and far between for many users.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s