A crisis, the causes and a solution

A crisis, the causes and a solution

Why this post?

Because there are rumours spreading about a presumed “MPEG-Video collapse” and Brownian motion-like initiatives trying to remedy – in some cases by the very people who have contributed to creating the “crisis”.

Who is the author of this post?

Leonardo Chiariglione, the founder and chairman of MPEG, but I am writing in a personal capacity.

Why is MPEG important?

In its 30 years of operation MPEG has created digital media standards that have enabled the birth and continue promoting the growth of digital media products, services and applications. Here are a few, out of some 180 standards: MP3 for digital music (1992), MPEG-2 for digital television (1994), MPEG-4 Visual for video on internet (1998), MP4 file format for mobile handsets (2001), AVC for reduced bitrate video (2003), DASH for internet streaming (2013), MMT for IP broadcasting (2013) and more. In other words, MPEG standards have had and keep on having an impact on the lives of billions of people.

How could MPEG achieve this?

Thanks to its “business model” that can be simply described as: produce standards having the best performance as a goal, irrespective of the IPR involved. Because MPEG standards are the best in the market and have an international standard status, manufacturers/service providers get a global market of digital media products, services and applications, and end users can seamless communicate with billions of people and access millions of services. Patent holders who allow use of their patents get hefty royalties with which they can develop new technologies for the next generation of MPEG standards. A virtuous cycle everybody benefits from.

Why is there a “crisis”?

Good stories have an end, so the MPEG business model could not last forever. Over the years proprietary and “royalty free” products have emerged but have not been able to dent the success of MPEG standards. More importantly IP holders – often companies not interested in exploiting MPEG standards, so called Non Practicing Entities (NPE) – have become more and more aggressive in extracting value from their IP.

I saw the danger coming and designed a strategy for it. This would create two tracks in MPEG: one track producing royalty free standards (Option 1, in ISO language) and the other the traditional Fair Reasonable and Non Discriminatory (FRAND) standards (Option 2, in ISO language). Option 1 standards, obviously less performing than Option 2 ones, would counter the “proprietary” threat and provide an incentive to produce even more effective Option 2 standards while keeping at bay excessive claims by patent holders because of the incoming Option 1 standards “competition”.

The Internet Video Coding (IVC) standard was a successful implementation of the idea – kind of. Indeed, a few years after approving AVC, IVC was found to perform better than AVC. Unfortunately 3 companies made blank Option 2 statements (of the kind “I may have patents and I am willing to license them at FRAND terms”), a possibility that ISO allows. MPEG had no means to remove the claimed infringing technologies, if any, and IVC is practically dead.

In 2013 MPEG approved the HEVC standard which provides the same quality as AVC at half the bitrate. The licensing situation is depicted in the picture below (courtesy of Jonathan Samuelsson of Divideon): of ~45 patent holders ~1/3 published their licences  and 2/3 have joined one of the 3 patent pools, one of which has not published their licence.

 I saw the threat coming and one year ago I tried to bring the matter to the attention of the higher layers in ISO. My attempts were thwarted by a handful of NPEs.

Alliance for Open Media (AOM) has occupied the void created by MPEG’s outdated (but still largely used) video compression standard (AVC), absence of competitive Options 1 standards (IVC) and unaccissible modern standard (HEVC). AOM’s AV1 codec, due to be released soon, is claimed to perform better than HEVC and is said to be offered royalty free.

At long last everybody realises that the old MPEG business model is now broke, all the investments (collectively hundreds of millions USD) made by the industry for the new video codec will go up in smoke and AOM’s royalty free model will spread to other business segments as well.

Can something be done?

The situation can be described as tragic. This does not mean that there is nothing left to do. I personally doubt that something will be done, though, seeing how blindfolded the industry is. As I like to say, God blinds those He wants to lose.

The first action is to introduce what I call “fractional options”. As I said, ISO envisages two forms of licensing: Option 1, i.e. royalty free and Option 2, i.e. FRAND, which is taken to mean “with undetermined licence”. We could introduce fractional options in the sense that proposers could indicate that their technologies be assigned to specifically identified profiles with an “industry licence” (defined outside MPEG) that does not contain monetary values. For instance, one such licence could be “no charge” (i.e. Option 1), another could be targeted to the OTT market etc.

The second action, not meant to be alternative to the first, is to streamline the MPEG standard development process. Within this a first goal is to develop coding tools with “clear ownership”, unlike today’s tools which are often the result of contributions with possibly very different weights. A second goal is not to define profiles in MPEG. A third goal could be to embed in the standard the capability to switch coding tools on and off.

The work of patent pools would be greatly simplified because they could define profiles with technologies that are “available” because they would know who owns which tools. Users could switch on tools once they become usable, e.g. because the relevant owner has joined a patent pool.

These are just examples of how the MPEG standard development process can be adapted to better match the needs of entities developing licences and without becoming part – God (ISO) forbid – of a licence definition process.

Is this enough?

Even if industry will decide to get their acts together and accept to put a patch to MPEG’s business model, it is easy to anticipate that the next threat is just around the corner. But MPEG cannot have a future if it is going to pass from crisis to crisis, each of which has an inevitable “cost”.

MPEG’s problem – so far a blessing – is that it is a working group, the lowest organisational structure in ISO. MPEG’s governance is weak: if there is a need, like it happened recently, to bring problems to the attention of the decision making layers in ISO, it is necessary to cross several layers of other committees with completely different priorities and concerns, each time asking for their approval and each time getting a diluted message. Only by becoming a Technical Committee can MPEG, the forerunner of problems that other committees will experience in the years to come, stay competitive in the market.

End of the world as we know it?

The reader should not think that I am personal concerned by all this, if not intellectually. I have been running MPEG for the last 30 years serving the industry – and billions of users – and I have been blessed by professional satisfactions that few have had enjoying the collaboration of thousands of experts each driven by their own motivations, but united in their desire to make the best standards. If MPEG ends now it will be a pity, but if this is the decision of the stakeholders – the industry in MPEG – so be it.

My concerns are at a different level and have to do with the way industry at large will be able to access innovation. AOM will certainly give much needed stability to the video codec market but this will come at the cost of reduced if not entirely halted technical progress. There will simply be no incentive for companies to develop new video compression technologies, at very significant cost because of the sophistication of the field, knowing that their assets will be thankfully – and nothing more – accepted and used by AOM in their video codecs.

Companies will slash their video compression technology investments, thousands of jobs will go and millions of USD of funding to universities will be cut. A successful “access technology at no cost” model will spread to other fields.

So don’t expect that in the future you will see the progress in video compression technology that we have seen in the past 30 years.

StandardMPEG-1MPEG-2MPEG-4 VisualMPEG-4 AVCMPEG-H HEVC
Bitrate vs previous25% less25% less30% less60% less
Year of approval19921994199820032013

Posts in this thread (in bold this post)