Technology Blog

On2 VP8 achieves high compression with a bitstream that is less compute intensive to decode than either its predecessor (VP7) or competing technologies like H.264. Here's how it works.  

Announced in the fall of 2008, On2 Technologies' 8^th generation video codec, On2 VP8 offers significant gains in compression performance in a bitstream that is less compute intensive to decode than either its predecessor (VP7) or other competing technologies such as H.264 Inheriting many great innovations from its predecessors (VP7 and VP6) such as golden frames, processor-adaptive real-time encoding and a low-complexity loop filter, VP8 adds more than fifty new techniques to achieve its goal of outstanding quality at low bitrates, with very low complexity.
On2 VP8 has been designed with a wide range of machines in mind, from 60 MHz ARM9 processors to today's highly parallel multi-core processors. It encodes in real-time on low-end machines, and takes fewer cycles to decode than other leading algorithms. This article explores some of the innovations that make VP8 work so well.
The Constructed Reference Frame
One of the most exciting innovations in VP8 is the constructed reference frame. A constructed reference frame is a frame of image data (a reference buffer) that's encoded into the bitstream but never displayed. It serves solely to improve the encoding of subsequent frames by providing an additional and hopefully better predictor than any previously transmitted "normal frames."
The creation of a constructed reference frame is not defined by the bitstream. Instead, creating the best possible constructed reference frame is a task left to the encoder. It could, for example, be created by compositing several past and future frames, with appropriate motion compensation and temporal filtering, or it could contain arbitary elements such as graphic overlays, that need to be inserted in some of the subsequent frames.
A New Take on Loop Filtering
Loop filtering is the process of removing blocking artifacts introduced by quantization of DCT coefficients from block transforms. VP8 brings several loop-filtering innovations (some of which have also now been reverse engineered into VP6 and VP7) that speed decoding by not applying any loop filter at all in some situations. VP8 also supports a method of implicit segmentation where it is possible to select different loop filter strengths for different parts of the image, according to the prediction modes or reference frames used to encode each macroblock. For example it would be possible to apply stronger filtering to intra coded blocks and at the same time specify that inter coded blocks that use the Golden Frame as a reference and are coded using a 0,0 motion vector should use a weaker filter. The choice of loop strengths in a variety of situations is fully programmable on a frame by frame basis, so the encoder can adapt the filtering strategy in order to get the best possible results.


The More Cores the Better
On2 VP8 is built with parallelism in mind and can exploit multiple processor cores like no other codec. Cross macroblock-row data dependencies that plague other codecs have been removed in VP8, so the encoder can make efficient use of multiple cores to encode several macro block rows at the same time. In theory, VP8 can make use of as many cores as there are macro-block rows in the image, so it could take advantage of up to 68 cores when encoding 1080p content, all without sacrificing compression efficiency. Indeed, for live encoding, because of VP8's adaptive algorithms, the more cores that are available the better the quality is likely to be.
Even the entropy encode and decode (a bottleneck in other codecs that is not readily amenable to parallelization), can be split across multiple cores by virtue of special VP8 options that allow for several independently coded data partitions.

        Simplicity in Mind
A major design goal in VP8 was to simplify the decoding process. To this end some VP7 features were reviewed and either modified to reduce computational complexity or even removed if the balance of compression benefit versus complexity was considered unfavorable. Decode performance on low powered DSPs and in particular on the world's most ubiquitous microprocessor, the ARM9, was also a significant design parameter from the outset.
To achieve these goals, a number of new techniques and options were added to VP8 and some existing techniques first used in VP6 or VP7 were extended. The following sections detail some of the areas that have received attention.
Making the Most of SIMD
A number of VP8's core algorithms have been modified such that they require only character (8 bit) arithmetic. This includes such things as the sub pixel filtering and the loop filter and it enables VP8 to make the most of SIMD (single instruction multiple data) instructions where available.
In addition, during certain arithmetic operations, such as the reconstruction of image data from a prediction block and a residual error signal, the VP8 bitstream also mandates that the result will never out range. For example if the allowed range for output values is 0-255 then you will never have a situation where adding an 8 bit residual error value to an 8 bit predictor value results in an output value that cannot also be expressed in 8 bits.
This may seem trivial at first sight and it imposes some additional overhead on the encoder, which is required to insure that outranging can never occur, but it has two key benefits.
Firstly, it means that the decoder does not need to perform expensive clamping operations (conditional checks on the results of each computation which force the output values back into the allowed range).
Secondly, it allows VP8 to be optimized using "pseudo-SIMD" operations on processors such as the ARM9 that don't support any dedicated SIMD instructions. For example, instead of adding 8 bit predictor and 8 bit residual values together one at a time, using 8 bit arithmetic, VP8 can grab four values at a time and add them together using a standard 32 bit add instruction. This is possible only because VP8 mandates that there will never be any 'carry' between any of the 8 bit sub elements. This use of a standard 32 bit instruction to perform four 8 bit adds in one go, can allow as much as a fourfold speed up of some speed critical sections of code.


Faster Sub Pixel Filtering
VP8 also supports the use of a single stage 2-tap sub-pixel filter instead of its normal 6-tap filter. Even the standard 6-tap filter used in VP8 is less complex than the two stage filtering process used in H.264 but the optional 2-tap filter further reduces the number of operations by as much as 50%.
A key point to note here, and one that also applies to some of the other features described in this document, is that this feature that can be switched on and off on the fly by the encoder. On2's aim here is to provide bitstream tools that can enable an encoder to make trade-offs between quality and decode complexity. In general the 2- tap filter will not match the standard 6-tap filter in terms of sharpness, but its use on a few particularly complex frames may be a small price to pay if it allows smooth playback on a low powered device.
Entropy Encoder
The entropy encoding strategy that On2 has developed for VP8 is similar to that used in VP7, in that it uses complex predictive modeling based on contexts that are updated at the frame level. An important point to make here is that unlike other context based arithmetic coders, it is not necessary to re-compute contexts after each token. On2 has also improved on previous codecs by re-balancing the token set and has succeeded in cutting down the average time spent on entropy decoding by up to 15%.
Faster Color Conversion
Most PC graphics cards offer direct support for a variety of standard YUV formats including the 12-bit planar YUV format that is used by default in VP8. However, this is not the case for many other devices such as mobile phones. On these devices it is necessary to do a costly RGB conversion before displaying each decoded frame. To address this issue, VP8 supports (by means of a bitstream flag) a second custom YUV format that can be converted to and from RGB much more cheaply.
More Enhancements
Lagged compression is a half way house between traditional one pass and two pass encoding where a VP8 encoder can look ahead a predetermined number of frames before compressing the current frame. This technique is not suitable for very low latency cases such as video conferencing but in many other encoding scenarios, where a delay of a few frames or perhaps a second is not a problem, it facilitates many of the benefits of two pass encoding in a live stream.
Small area inter-frame prediction (motion vectors for regions smaller than a 16x16 pixel macroblock), has also been improved in VP8, which greatly benefits video coding of low-resolution clips for devices such as mobile phones.
Additionally, VP8 adds a whole new level of temporal scalability to the On2 codec family.
Conclusion
Has VP8 met its design goals? Absolutely. Building on the success of VP7, VP8's innovations allow it to perform extremely well on a very wide range of material, and on platforms as simple as ARM9 or as complex as Intel's newest multi-core machines. Its versatility and economy ensures that VP8 will play an important role in video compression for years to come.