Hardware acceleration and compositing

Let’s talk about some common misconceptions about hardware acceleration.

Hardware acceleration conveys the idea that some parts of the rendering use the GPU in order to speed things up.

In a web browser there are two main topics for hardware acceleration:

  • Hardware accelerated content rendering: Drawing content on the GPU, like rendering text, shadows, shapes, etc. I also place video decoding in that bag but it is a somewhat different topic and I am not going to talk about it in this post.
  • Hardware accelerated compositing: We render different elements of a web page in different layers (think of layers as photoshop layers) and compositing is the action of flattening all these layers into the final image that gets to the screen.

The former is generally very hard to achieve. Rendering content on the GPU is hard because GPUs are not always good at it. But it depends and there are things that we can do efficiently on the GPU and things that we can’t do efficiently on the GPU. For example graphics cards are not good at rendering text (text that looks good). Some canvas operations are very fast on the gpu (like blitting surfaces) but others are not (drawing bezier curves and shapes in general), so there is a trade-off and “hardware-accelerated canvas” is not always “accelerated” depending on what you are doing with your canvas.

The latter, GPU compositing, is a much more interesting optimization for browser implementors because it is a simple task that the GPU is very good at doing: blitting quads. Gecko performs compositing on the GPU except when the graphics card model or driver is black-listed and unfortunately, except on desktop Linux because we haven’t yet dedicated enough resources to support it efficiently (it can be activated manually but the results will not always be better). Note that Linux accelerated compositing is a very good area to contribute to.

When people around the internet speak of hardware acceleration in web browsers, they almost always refer to accelerated compositing.

Now let’s talk about a very bad misconception that has spread: I have read in various places on the internet that using 3D transforms enable hardware acceleration. This is very wrong. Gecko, Webkit, Blink, etc. use heuristic to determine when elements of a web page should be placed in a separate layer. Doing so let’s us do things like not invalidating (and re-drawing) an element that is overlapping with something that is animated, and some other optimizations. Applying a 3D transform to an element just happens to trigger this element to have its own layer. Layerizing an element is different from enabling hardware acceleration. when the browser starts up, it decides whether it will do compositing on the CPU or the GPU, and it won’t switch between the two because someone applied a transform on some element of a web page.

Layerizing an element is good if the element is going to be animated, but it is not something that will just make things faster. It can, and most likely will, use more memory because it will require us to allocate an extra texture to host the pixels of the element. For example, Gecko tends to layerize elements during a CSS transition, and merge back the element with the rest as soon as the transition is over, to avoid having too many layers at the same time.

Sometimes people blog about how applying a 3D transform can accelerate rendering, then other people read it and believe that applying transforms is the silver bullet to making web pages faster. Please, please, don’t blindly do things like that. I realize that we need to document more what layers acceleration in a web browser is, and how compositing works. Thankfully most browsers today have roughly the same notions of layers and compositing, so talking about layers in Gecko or Webkit should be equivalent from a web developer’s point of view. Layerizing for the sake of layerizing can degrade performances and memory usage. Heuristics evolve, and using hacks in websites today can prevent us from doing clever optimizations in the browser tomorrow.

So only use this kind of hacks when you understand what it actually does in the browser and when you can observe a real difference. Always remember that using a hack for something it is not meant to can probably speed things up in some browser today, and make things worse later as browsers evolves.

9 thoughts on “Hardware acceleration and compositing

  1. Thanks for this! I’ve never understood what everyone was talking about when they talk about HW acceleration. I mean, I thought I did, but then it’s referred to so casually, like doing all of your text rendering on the GPU is no big deal or something. From past experience, I knew that is just not the case.

    Is it too late to stop calling it “HW acceleration”? If it were “GPU compositing” or something, I would have understood it immediately. “GPU layers” might make more sense, I suppose, except that I’ve always been very fuzzy on what a “layer” is.

  2. It is hard to find a term that talks to everybody. GPU compositing would be a better name but the notion of compositing is abstract for web devs and users in general since it is a borwser implementaion detail.

  3. What are we going to call the next level of GPU-based acceleration, then? :-) “HW acceleration 2”? Hm, maybe “HW accelerated X”, like “HW accelerated layout”, “HW accelerated fonts”, etc.

    For that matter, what about HW video decoders, SIMD extensions, TCP offload engines, random number generators, FPGAs, HW audio mixers, etc.?

    Oh well. It’s just marketing being marketing, I suppose. We should just call it “yellow speedsauce” vs “spicy red speedsauce”.

  4. Very interesting article, it was very educational.

    So, according to what you say, only the browser can decide when (and on what) to activate GPU acceleration ? By “on what”, I mean for example Canvas 2D or WebGL.

    Besides, is that why we should create buffers for a “fillText” on a canvas ? Because GPU is bad as rendering text ?

    1. I am not sure about what you mean by “only the browser can decide” but at least in the case of Gecko, we do compositing on the GPU as much as possible (when we know the hardware and driver support is good enough) regardless of the content of the page. The content of the page can trigger some “layerization” heuristics which makes Gecko make a better use of the GPU. Doing things on the GPU is not a silver bullet, we have to do the right things with it.
      For content (canvas2d, etc) it’s trickier because some things will be faster and some will be slower on the GPU so there is a trade-off and the choice would probably be harder to make. We mostly do software canvas2D right now but there is some work and experimentations with GPU backends. I am not the most experienced person to provide with details in this area.

      1. I meant that I did thought first that Gecko/Webkit would choose to render divs with CPU and canvas with GPU (for example), but after reading your article again, it made things more clear to me.

      2. Yes, Gecko and Webkit can decide to paint canvas on the GPU and do the rest of the page in software. Choosing to do compositing on the GPU and painting on the GPU are different things. Although if you can do painting you can do compositing since compositing is very simple in comparison. So when one say “hardware acceleration is enabled” it’s a good thing to ask yourself “which acceleration ? compositing, canvas2D, video decoding, (etc.)?”

  5. There’s something inherently wrong with this: you (browser vendors) can’t tell people not to use hacks to achieve hardware acceleration if the only way you (browser vendors) allow them to achieve hardware acceleration is with hacks. This completely makes no sense! If you (browser vendors) don’t want people to rely on hacks for acceleration, then give them something official…

    1. My point is (was, this post is quite old!) that “achieving hardware acceleration” is the wrong goal. You should pursue performance and/or smoothness, and if these are achieved through something that is under the umbrella of what people tend to call hardware acceleration, then great.
      Note that we actually do have an official way to tell the browser that something will change or be animated to improve the rendering heuristics: https://developer.mozilla.org/en-US/docs/Web/CSS/will-change .
      That said we are in the process of rewriting our rendering architecture in order to better take advantage of modern graphics hardware. With this new architecture, all of the literature about using transforms and such to tweak compositor heuristics will be more irrelevant than ever but a lot more of the rendering will be performed on the GPU.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s