Let’s put MPEG on trial

You are currently viewing Let’s put MPEG on trial

Let’s put MPEG on trial

Introduction

An Italian proverb says “those who do things make mistakes” to mean that only if you are inactive you don’t make mistakes. According to that proverb, then, MPEG, who has done a lot of things, must have done a lot of mistakes.

So, in this article I imagine there is a prosecutor making charges to MPEG and a defence attorney speaking on behalf of the defendant. Whoever wants to be part of the jury is welcome to join.

Charge #1: MPEG standards are losing market share

Prosecutor: In several markets the importance and adoption of MPEG standards is decreasing.

Attorney: MPEG standards has never been “catch all” solutions. Originally MPEG-1 Video went nowhere because interactive video went nowhere. MPEG-1 Audio layer 2 had a very slow adoption, but MP3 was a huge success. When some Far East markets needed disc-based video distribution without using DVD, MPEG-2 Video and Audio layer 2 became a huge success (1 billion Video CD players). MPEG-2 Video was universally adopted but Audio faced strong competition from proprietary solutions. The 4:2:2 profile of MPEG-2 Video faced strong competition from a market solution. Companies vying for video distribution on the nascent internet turned their back to MPEG-4 Visual and looked at proprietary solutions when they learned that they would have to pay for the amount of video streamed. A more market friendly licensing of MPEG-4 AVC gave a big boost to the adoption of the standards.

Conclusion? As the then candidate Bill Clinton said: “It’s the economy, stupid”.

That was a conclusion, but maybe too hasty. It is true that, with MPEG-2 Video and MPEG-4 AVC, MPEG standards have dominated the video distribution market. Things, however, are changing. The appearance of bigplayers on the internet distribution market has favoured the appearance of proprietary solutions. VP8, VP9 and AV1 had/have millions of users, while the penetration of MPEG-H HEVC is still low. Some, including myself, think that a confused HEVC licensing situation, not the usability and quality of HEVC, is the cause of this situation. MPEG has nothing to say or do about licensing or about the rules that originate the current state of affairs, but MPEG-5 EVC, a standard due to be approved in 4.5 months, can unlock the situation. EVC is expected to have significant more compression performance than HEVC (still less than VVC), but a more streamlined licencing landscape.

Obviously MPEG cannot be involved in the MPEG Future initiative, but one of its action points reads:

Enhance the value of intellectual property that make MPEG standards unique, while facilitating their use

Conclusion: MPEG has its hands tied but the market has not. The people of good will who are part of MPEG Future need not have their hands tied, either, when they operate within the initiative.

Charge #2: too many standards are less than successful

Prosecutor: in 30 years the defendant has produced some 180 standards, but the really successful ones out of these are maybe 1/6 of the total. This has been a waste of resources.

Attorney: it is a matter of where you stop. MPEG could have decided to place the bar of acceptance of new standard work higher and would thus have produced a smaller number of less successful standards. If it had done so, however, most likely MPEG could also not have produced some of its successful standards. Which is better: more attempts or lost opportunities?

One should not forget that, if making a market-ready product costs 1000, designing a product costs 100, making research for a product costs 10 and making a standard for that product costs 1. Companies should concentrate on the 1000s and 100s, not on the 1s.

This does not mean that MPEG has not been aware of the need to increase the number of successful standards. In 2014 MPEG did try to address the problem. It dreamed of a new structure where communication with the the world was not only at the end of the process (“look, this is the standard we’ve developed for you”) but also at the beginning (“look, we plan to develop this standard”).

Figure 1 – The MPEG organisation designed in 2014

The plan went nowhere. As it is perceived as a technical body, MPEG was unable to find the leader of the industry liaison subgroup, never mind the members.

The problem of raising the rate of successful standards, however, does not go away. The MPEG Future initiative has this as a key element of its proposal to make MPEG a subcommittee. Unlike the structure of Figure 1, however, the Industry Liaison and Requirements functions are no longer sequential. They operate in co-competition to provide proposals for action that take into account the unique positioning of MPEG as the body producing standards that are anticipatory but aim to address markets of million or billion products, services and applications. It does that by blending technology evolution trends (as assessed by Technical Requirements) against the expected market need (as assessed by Market Needs).

Figure 2 – The MPEG organisation proposed for MPEG as an SC

Conclusion: applying the English proverb “you cannot have your cake and eat it”, one can say that if you want to have many successful standards you must make many attempts. Nevertheless, injecting market awareness in the standards process will help.

Charge #3: the MPEG structure is too static

Prosecutor: In the last ten years, the organisation of MPEG has not changed appreciably.

Attorney: I am not sure that an organisation that changes its structure frequently is necessarily a healthy organisation. There are many companies out there who are in constant restructuring and their performance can hardly be described as excellent. In the last few years, however, MPEG has concentrated in making its organisation more flexible and suitable for the development of its standards.

Since its early days MPEG had Subgroups and, within it, Units addressing specific areas who are typically temporary, but can also be permanent. Today MPEG can claim to have a flat organisation whose elementary Units belong to specific Subgroups and carry out work for a particular standard interacting with other units under the supervision the Technical Coordination made up of Convenor and Subgroup Chairs. This is represented in Figure 3 where the letter before the Unit indicates the Subgroup the Unit belongs to.

Figure 3 – The flat MPEG organisation

This organisation is functional to the development of the highly integrated MPEG standards.

Conclusion: an organisational chart is typically viewed the sexy part of a company. In MPEG it is not necessarily the most important one.

Charge #4: MPEG does not collaborate with JPEG

Prosecutor: a sizeable part of MPEG deals with (moving) pictures and JPEG deals with (static) pictures. There are a lot of technical commonalities, still the two groups do not collaborate.

Attorney: Collaboration and brotherhood are nice words. Both, however, have to be put in a concrete setting to give them a practical meaning. Here I will only talk about the former.

The word collaboration can be easily applied to a group of researchers belonging to different organisations when they write a paper where each author brings different contributions.

The raison d’être of a standards committee is, well, … making standards. Therefore, “collaboration between two standards committees” can only be invoked when there is a need for a standard serving the constituencies of both committees. MPEG serves the industries engaged in the “distribution of moving pictures and audio on broadcast, broadband, mobile and physical media” while JPEG serves the industries engaged in “digital photography and distribution of still image sequences”. There is little intersection between the two as I am going to explain.

MPEG and JPEG may very well share some technologies, but the needs of their constituencies are different. For instance, JPEG is close to approving the FDIS of a standard for compression of light field images because its industries feel that they are ready to implement such a standard. On the other hand MPEG is just carrying out an exploration on dynamic light fields, because its industries are nowhere ready to do the same. If and when MPEG will develop a standard for dynamic light fields, it will most likely use a different capture format (JPEG has used plenoptic 1 in its standard and MPEG is using plenoptic 2 in its exploration). Therefore, little if anything of what has been developed by JPEG for its light field image compression standards, will be used by MPEG.

This is not a new story. In all 6 generations of MPEG moving picture coding standards no need or significant benefit has ever been found that justifies the adoption of a JPEG standard as the still picture coding mode of an MPEG standard. This assessment holds also for point clouds and, as said above, is expected to hold for other digital moving picture representation technologies such as light fields.

On the other hand MPEG has developed the ISO Base Media File Format (ISOBMFF), a standard used also by several non-ISO bodies such as 3GPP and AOM. Collaboration has happened because JPEG, too, is using ISOBMFF for its standards.

Collaboration need not be the exception. It is an undertaking to reach a common goal – when it exists. Otherwise it is just a way to further knowledge. This is certainly a useful thing, but not if it is done within a standards committee, because it becomes a distraction.

Collaboration is important and if it is to happen successfully, the following steps are needed:

  • Verify the joint industrial interest in the common project
  • Define requirements of the potential common standard
  • Draft the Terms of Reference for joint effort
  • Agree on
    • Work plan and timeline of the joint effort
    • Who lead(s) the work: just one or one from each committee
    • Independent or collocated meetings with either parent committee
    • Which committee provides the secretariat
    • How ballot comments are resolved

Conclusion: collaboration must be done for what it means ethymologically – doing work together.

Conclusions

The prosecutor and the defence attorney have made their pleas. Now it is for the jury to reach a verdict.

Posts in this thread