Can we improve MPEG standards’ success rate?


I am very proud of the high scientific level of the work that MPEG does with its standards. I think that many universities would enhance the value and effectiveness of the education they provide if they took an MPEG standard, not necessarily a successful one, and have students redo the same process that led MPEG to develop that standard.

Because of their high scientific value so many MPEG standards have been wildly successful, as The impact of MPEG standards tells. More standards – more successes – more failures tells about the other side of the coin and argues that as much as certain products from companies are successful and others less so, some MPEG standards have a level of success that does not match the expectations of the experts who developed them.

What went wrong in those cases? My answer to this question is that not necessarily something went wrong and I will try and explain why.

Standards and market needs

MPEG is an excellent environment to develop standards because it has hundreds of the best experts in its field of endeavour. Several hundreds more are in the laboratories who do not, or only seldom attend MPEG meetings. MPEG experts make their proposals of new standards because someone in their companies has envisaged a business enabled by the idea they propose. MPEG is very receptive of new proposals and has a very good process to  develop requirements for a new standard. However, that assumes that the market needs that standard because those who develop the requirements are often the same technical people who will later develop the standard.

Let’s see how another successful environment developing specification – the Digital Video Broadcasting (DVB) project – handles the matter. A proposal for new work goes first to the Commercial Module. If the proposal is not accepted by the Commercial Module, it will not be considered by the Technical Module.

This is possibly a good arrangement for DVB, a group that works apparently in a similar area as MPEG. Similarities, however, may be misleading without considering the entire picture.

  1. MPEG addresses matters regarding the compressed digital representation of media. Therefore MPEG is a “horizontal” committee, while the DVB addresses the entire broadcasting protocol stack, from the physical layer – e.g. DVB-T (for terrestrial), DVB-S (for satellite) and DVB-C (for cable) up to the application layer (e.g. the Multimedia Home Platform or MHP).
  2. MPEG hosts experts from all industries with a stake in digital media while DVB members are predominantly companies interested in broadcasting applications. In all fairness . the separation of broadcasting from other ICT applications in an age of convergence is no longer what it used to be at the technical level, but the business context is different and this still matters.
  3. Some MPEG standards have the luxury of being “long term” (say, 5 years). Examples are the investigations in 3DoF+ and Point Cloud Compression, the former of which is on the launching pad to become a standards and the latter pretty close to the end. Both activities have been incubated for some years. DVB is more short-term oriented and does not want to invest resources in projects that may pay off only years from now.

MPEG already tried something

In 2015 I proposed to MPEG to establish a new activity with the preliminary name of “Industry liaison” and I even developed a concept of the new organisation of MPEG groups (Figure 1). My idea was that this activity could become a new MPEG group, if proved successful.

Figure 1 – Concept of MPEG organisation with industry liaison

Rob Koenen, then of TNO and José Alvarex, then of Huawei America (both tell you something about the dynamics of MPEG experts) were kind enough to take the challenge. They and a group of interested members created a presentation of MPEG activities.that was used at several exhibitions and conferences, e.g. NAB and SMPTE, to stimulate reactions from industry participants. The last slide of the presentation asked three questions:

  • Which needs do you see for media standardisation, between now and years out?
  • What MPEG standardisation roadmap would best meet your needs?
  • To accommodate your use cases, what should MPEG’s priorities be for the delivery of specific standards? For example, do you urgently need something that may enable basic functionality now, or can you wait for a more optimal solution to be released later?

That was a big effort and I am thankful to Rob and José for spending so much effort for the good of MPEG. Unfortunately the goal to create a momentum to the process of getting more information from industry about their needs was not achieved as a continuous and self-sustaining process.

This does not mean that the effort was not useful. Today Rob is in charge of keeping up to date the MPEG Standardisation Roadmap, a public document that provides a graphical representation of the entire MPEG work plan and its timeline, a very important function as Figure 2 related to March 2019 shows.

Figure 2 – MPEG Standardisation Roadmap

Let’s give it another try

Why did the proposal not work out as expected? One reason is intrinsic to the MPEG constitution. MPEG is a large community of experts, but there are very few if any market people, even from the larger companies. We have tried and reached out to the industry but we have been unable to get market people to join MPEG.

My assessment is that, as long as MPEG remains what it is, a wonderful forge of new standard technologies, it will be hard to achieve that goal.

In Which future for MPEG? I have proposed that MPEG become a Subcommittee (SC). This change of status is not intended to be a “promotion” but a better set up that allows MPEG to play a wider role than purely technical.

My proposal is reproduced here in an improved form as

Figure 3 – Proposed new structure of MPEG as an SC

The “Market needs” box represents the new attempt at getting market requirements into the MPEG process. However, it is quite different than Figure 1 because in that figure the Industry liaison function precedes Technical requirements, while here they work in parallel.

How should this new arrangement work?

Proposals for new work should be assessed both from their technical feasibility and their market value. The Technical requirements and Market needs Advisory Groups (AG) should make their assessments driven by “how much technology supports this proposal” and “how much market needs this proposal”, respectively. So, in a sense, the two groups will co-compete. In general Technical requirements will be for innovation, while Market needs will be for conservation. But it will not necessarily be always like that.

These groups should have, as all MPEG groups do have, joint meetings to align their views and they should, if they can, develop a single report. If that cannot happen, there will be two separate, possibly conflicting, reports. In both cases the SC plenary will asses the report(s), take responsibility and make a decision on the basis of a country-based vote.

What will be the meaning of this vote? It should certainly not be a replacement of the vote taken via ballot on a New Work Item Proposal (NWIP), but a vote on whether MPEG should acquire – via Calls for Proposals – the technologies that may be used to develop a standard. The actual NWIP vote will only take place when the Technical requirements AG and the appropriate WG(s) will agree that there is enough material to develop a standard.

So far I have talked of Market needs and Technical requirements having a passive role, i.e. waiting for proposals coming either from experts (as it has regularly happened so far) or from National Bodies.

The operation of the AGs, however, needs not be constrained to be in response to a proposal. They could very well, by their initiative or because they have been requested by the SC plenary, to engage in the study of new areas. Such studies should always be conducted from the point of view of what technology enables and what market needs, or what could be the reaction of the market to a certain standard technology.

The mission of the Market needs and Technical requirements AGs should not be considered accomplished when the NWIP has been approved and the development of the standard starts. MPEG must have the means to review the steps that have brought to success or failure of a standard.

You should not expect that this post mortem analysis can be done 6 months after MPEG has released a standard for publication. It is not unusual that 6 months after release by MPEG a standard has not been published yet. It is true that the world a quarter of century ago moved more slowly that today, but MP3, released in 1992 became a product only 5 years later, MPEG-1 Video (with MP2) became a product at the end of last century. We are talking here of an infrastructure to be put in place now whose first results we can probably see in five years.


This proposal should not be seen as a roadblock to the capability to innovate that MPEG has shown to be capable of in the last 30 years.

It is usually said that if commercialisation costs 100, product development costs 10 and research costs 1. I add that, probably, an MPEG standard costs a company 0.1 (excluding research because that is already accounted for).

So this proposal is not about cost saving. It is about making an excellent and successful MPEG process better, without running the risk highlighted by Monsieur de Montesquieu: Le mieux est l’ennemi du bien (better is the enemy of good).

MPEG is known for its innovative standards, but it should be better known for its process innovation.

Posts in this thread

Can you “clone” MPEG?


The publication of A vision made real – Past, present and future of MPEG has triggered many reactions coming from people in other industries. They are facing the same problem that MPEG started facing 31 years ago when it wanted to create digital media standards that were industry-agnostic and with a global scope.

My answer to this people has been that, indeed, the MPEG model can be exported to other domains where industries with different backgrounds, technologies, stakes and business models want to join forces and develop standards that enable new businesses.

But it won’t be like a piece of cake. Try to inject a new philosophy of work in an environment with its own history, people, relationships and the task is close to impossible. This is not because of technical reasons but because it is a matter of philosophy. MPEG had an easy (so to speak) task because it was a new group without history, with new people with a limited degree of relationships.

In this article I will examine some of the main aspects that one needs to care about to “clone” MPEG to make standards that are industry-agnostic, with a global scope and are intended to enable new businesses.

Going from here to there

The first task is to define what the common project is about. Clarifying the purpose of an undertaking is a good practice that should apply to any human endeavour. This good practice is even more necessary when a group of like-minded people work on a common project – a standard. When the standard is not designed by and for a single industry but by and for many, keeping this rule is vital for the success of the effort. When the standard involves disparate technologies, whose practitioners are not even accustomed to talk to one another, complying with this rule is a prerequisite.

When MPEG was established (late 1980’s) many industries, regions and countries had realised that the state of digital technologies had made enough progress to enable a switch from analogue to digital. Several companies had developed prototypes, regional initiatives were attempting to develop formats for specific countries and industries, some companies were planning products and some standards organisations were actually developing standards for their industries.

MPEG jumped in the scene at a time the different trials had not had the time to solidify. Therefore, it had a unique opportunity to execute its plan of an epochal analogue-to-digital transition of media.

Similarly, industries who wish to create standards that are industry agnostic need to understand what is exactly the world that they want to establish and share a common vision of it.

By the way, you need a business model

Of course, I am not talking about how a hypothetical group working for industry-agnostic standards is going to make money. I am talking about how companies participating in this common effort can develop standards that bring actual benefits to each of them in spite of their differences, market positioning etc.

In the case of MPEG the definition of the business model had to take into account the fact that industry and acad­emia had worked on video compression technologies for some 3 decades filing many patents (at that time they could already be counted by the thousands) which covered a wide range of basic video coding aspects. Video coding standard that are loosely called “royalty free” (in ISO language, for which only Option 1 patent declar­ations are made) were certainly possible but would probably have been unattractive because of their low performance compared with the state-of-the-art codecs.

Therefore, MPEG decided that it would develop standards with the best performance, without consideration of the IPR involved. Patent holders would get royalties from the use of MPEG standards widely adopted by the market. If a patent holder did not want to allow that to happen, they could make an Option 3 declaration and MPEG would remove the infringing tech­nologies.

The MPEG business model is certainly not a prerequisite for developing industry agnostic standards, but it has worked well for the industry. More than that, most patent holders have been and keep on re-investing the royalties they get from existing standards in more technologies for future standards. The MPEG “business model” has created a standard-producing machine (MPEG) that feeds itself with new technologies.

Standards must be industry-friendly

A primary – and obvious – goal of the effort is that the standards produced by the collaborating industries should serve the needs of the participating industries.

The following points describe the three main MPEG targets:

  1. Display formats: Since the appearance of television cameras and displays in the 1920’s, industry and govern­ments have created tens of television formats, mostly around the basic NTSC, PAL and SECAM families. Already in the late 1960’s, when the Picturephone service was deployed, the tradition was hard to die: AT&T invented a new 267-line format, with no obvious connection with any of the existing video formats. As MPEG wanted to serve all markets, it decided that it would just support any display format, leaving display formats outside MPEG standards.
  2. Serving one without encumbering others. An industry may like the idea of sharing the cost of an enabling technology but not at the cost of compromising their individual needs. MPEG standards share some basic technologies but provide the necessary flexibility to its many different users with the notion of Prof­iles (subsets of general interoperability) and Levels (grades of performance within a Profile).
  3. Standards apply only to decoders; encoders are only implicitly defined, and their implementation leaves ample margins of freedom. By restricting standardisation to the decoding functionality, MPEG extends the life of its standards and, at the same time, allows industry players to compete on the basis of their constantly improved encoders.

The conclusion is that standards are great because they enable interoperability but should leave meaningful room to individual participants to exercise their business. Even better, as MPEG did with standards defining only the encoder, this opens the way to research-enabled competition.

Standards for the market, not the other way around

Before MPEG, a company with a success­ful product would try to get a “standard” stamp on it, share the technology with its competitors and enjoy the economic benefits of its “standard” technology.

This process may still be in place for some industries but is not an option when different industries team up to define common industry-agnostic standards with a global scope.

Under the MPEG regime, companies do not wait for the market to decide which technology wins, an outcome that very often has little to do with the value of the technol­ogy or the product but wait for the “best stan­dard” to be developed based on a set of technologies each of which is collectively selected based on a priori defined criteria. Then the technology package – the standard – developed by MPEG is taken over by the industry.

In a multi-industry environment, standards must anticipate the future. The alternative is to stop making standards because if the body waits until market needs are clear, the market is already full of incompatible solutions and there is no room left for standards, certainly not industry-agnostic and with a global scope.

Anticipating market needs is in the DNA of MPEG standards. With each of its standards MPEG is betting that a certain standard technology will be adopted. This explains why some MPEG standards are extremely successful and other less so.

Integrated standards as toolkits

Today’s systems comprise many functions. Some users of the standards are keen to have the complete package of functions, while others want to keep the freedom to cherry-pick other solutions that hopefully fit in the package as shown in the figure.

If interfaces are kept, say the one between System B and System C, the complete system continues to work. Depending on the specific case, however, the level of performance (not the functionality) of the entire system may change and a degree of interoperability may be lost.

Most MPEG standards are composed of the 3 key elements – audio, video and systems – that make an audio-visual system and some, such as MPEG-4 and MPEG-I, even include 3D Graphic information and the way to combine all the media. However, the standards allow maximum usage flexibility:

  1. A standard can be directly used as complete solutions, e.g. like in VCD where Systems, Video and Audio are used
  2. The components of the standard can be used individually, e.g. like in ATSC A/53 where Systems and Video are from MPEG, and Audio is from and external source
  3. The standard does not specify a technology but only an interface to different implementations of the technology, e.g. like in the case of MPEG-I, for which MPEG will likely not standardise a Scene Description technology but just indicate how externally defined technologies can be plugged into the system
  4. A standard does not specify the solution but only the components of a solution, e.g. like in the case of Reconfigurable Video Coding (RVC) where a non-standard video codec can be assembled using an MPEG standard.

A multi industry effort must satisfy the needs of all customers, even those who do not want to use its standards in their entirety but other specifications as well.

Compete and collaborate

Competition is the engine of progress, but standards are the result of a collaboration. How to combine competition and collaboration?

MPEG favours competition to the maximum extent possible. This is achieved by calling for solutions that respondents must comprehensively describe, i.e. without black boxes, in order to qualify for consideration. MPEG experts, including other proponents, assess the merit of prop­osed technologies.

Exte­nding competition beyond a certain point, however, is counterproductive and prevents the group from reaching the goal with the best results.

MPEG develops and uses software platforms that assemble the candidate components selected by its experts – called Test Models – as the platforms where participants can work on improving the different areas of the Test Models.

Core Experiments is the tool that allows experts to improve the Test Model by adding step by step the software that implements the accepted technologies. A Core Experiment is “a technical experiment where the alter­nat­ives considered are fully documented as part of the test model, ensuring that the results of independent experimenters are consistent”.

MPEG mission is to provide the best standards to industry via competition, but MPEG standards should not be shielded from competition Probably the earliest example of application of this principle is provided by MPEG-2 part 3 (Audio). When backward compatibility requirements did not allow the standard to yield a performance comparable to that of algorithms not constrained by compatibility, MPEG issued a Call for Proposals and developed MPEG-2 part 7 (Advanced Audio Codec). Later the algorithms evolved and became the now ubiquitous MPEG-4 AAC. Had MPEG not made this decision, probably we would still have MP3 everywhere, but no other MPEG Audio standards. The latest example is Essential Video Coding (EVC), a standard not designed to offer the best performance, but a good performance with good licensability prospects.

Working on generic standards means that reasonable requests – say, the best unconstrained multichannel audio quality – cannot be dismissed. MPEG tried to achieve that with the technology it was working on – backward-compatible multichannel audio coding – and failed. The only way to respond to the request was to work on a new – competing – technology.

One step at a time

An obvious principle, but it is better to keep it in mind, is that one must fine tune the engine first before engaging in a car race. If in 1988 the newly born MPEG had proposed itself as the developer of an ambitious generic digital media technology standard applicable to all indus­tries on a global scale, the proposal would have been seen as far-fetched and most likely the initiative would have gone no­where.

Instead, MPEG started with a moderately ambitious project: a video coding standard for interactive applications on digital storage media (CD-ROM) at a rather low bitrate (1.5 Mbit/s) targeting the market covered by the video cassette (VHS/Beta) with the addition of interactivity.

Moving one step at a time has been MPEG policy for MPEG-1 and all its subsequent standards and so should do any effort comparable to MPEG’s.

Separate wheat from chaff

In human societies parliaments make laws and tribunals decide if a specific human action conforms to the law. In certain regulated environments (e.g. terrestrial broadcasting in many countries) there are standards and entities (authorised test laboratories) who decide whether a specific implementation conforms to the standard. MPEG has neither but, in keeping with its “industry-neutral” mission, it provides the technical means – namely, tools for conformance assessment, e.g. bitstreams and reference software – for industries to use in case they want to establish authorised test laboratories for their own purposes.

Providing the tools for testing the standard is vital in a multi-industry environment. The ecosystem is owned by all and should not be polluted by non-conforming implementations.

Technology is always on the move

The Greek philosopher Heraclitus is reported to have said: τὰ πάντα ῥεῖ καὶ οὐδὲν μένει (every­thing flows and nothing stays). The fate of any technology field today is that technologies not only do not stay but move fast and actually accelerate.

MPEG is well aware that the technology landscape is constantly changing, and this awareness informs its standards. Until HEVC – one can even say, including the upcoming Versatile Video Coding (VVC) standard – video meant a rectangular area (in MPEG-4, a flat area of any shape, in HEVC it can be a video projected on a sphere). The birth of immersive visual experiences is not without pain, but they are happening, and MPEG must be ready with solutions that take this basic assumption into account. This means that, in the technology scenario that is taking shape, the MPEG role of “anticipatory standards” is ever more important and challenging to achieve.

This has happened for most of its video and audio compression standards. A paradigmatic case of a standard addressing a change of context is MPEG Media Transport (MMT) that MPEG designed having in mind a broadcasting system for which the layer below it is IP, unlike MPEG-2 Transport Stream, originally designed for a digitised analogue channel (but also used for trans­port over IP as in IPTV).

Research for standards

The wild pace of technology progress requires an engine capable to constantly feed new technologies.

MPEG is not in the research business. However, without a world of researchers working with MPEG in mind there, would be no MPEG. The MPEG work plan promotes corporate/academic research because it pushes com­panies to improve their technologies to enable them to make successful responses to Calls for Proposals.

One of the reasons of MPEG success, but also of some of its difficulties, is that MPEG standardisation is a process closer to research than to product design.

Roughly speaking, in the MPEG standardisation process, research happens in two phases:

  1. In companies, in preparation for Calls for Evidence (CfE) or Calls for Proposals (CfP), that MPEG calls competitive phase
  2. In MPEG in what is called collaborative phase, i.e. during the development of Core Exper­iments (of course this research phase is still done by the companies, but in the framework of an MPEG standard under development).

The MPEG collaborative phase offers another opportunity to do more research. This has apparently a more lim­ited scope, because it is in the context of optimising a subset of the entire scope of the standard, but the sum of many small optimisations can provide big gains in performance. The shortcoming of this process is the possible introduction of a large number of IP items for a gain that some may well consider not to justify the added IP onus to complexity. With its MPEG-5 EVC project, MPEG is trying to see if a suitably placed lower limit to performance improvements can help solve the problems identified in the HEVC standard.

Standards as enablers, not disablers

A standard intended for use by many industries cannot be “owned” by a specific industry. Therefore MPEG, keeping faith to its “generic standards” mission, tries to accommodate all legitimate functional requirements when it develops a new standard. MPEG assesses each requirement for its merit (value of functionality, cost of implementation, possibility to aggregate the functionality with others etc.). Profiles and Lev­els are then used to partition the application space in response to specific industry needs.

The same happens if an industry comes with a legitimate request to add a functionality to an existing standard. The decision to accept or reject a request is only driven by the value brought by the proposal, as substantiated by use cases, not because an industry gets an advantage, or another is penalised.


This article has given some hints drawn from MPEG’s 30 years long experience to those who intend to undertake an effort to develop standards for a multi-industry environment.

It is a significant but doable task if the effort is supported by new people without common history because a completely new philosophy of work must be adopted.

It is a close to impossible task if the effort is supported by people who already had a common history. This is true also in the case that the effort is about cloning MPEG to develop digital media standards.

Posts in this thread

Who “owns” MPEG?


The title of this article contains three elements: the verb own pre- and postfixed with a quotation mark which conveys the notion of having “a control of, a stake in or an influence on”; the acronym MPEG with its multiple meanings; and the pronoun who which represents several entities.

In this article I will try and clarify the nature of these three elements and draw some conclusions that may be useful for some impending decisions.

The different uses of the word “MPEG”

In MPEG: what it did, is doing, will do I have reported evidence that even the public at large widely knows the word “MPEG”. But what do they mean when they use it? Most often they intend some “media technologies” present in a device or used to create, store or transmit media content. Therefore, the word “MPEG” in this context is owned by all.

The domain is obviously connected to the word “MPEG”. As I have written in the same MPEG: what it did, is doing, will do, the domain is owned by an individual who is not using the domain, is not inclined to donate it and does not answer when asked how much will make him inclined. So is privately owned.

A small number of people use the word “MPEG” to indicate one of more of the ISO/IEC standards listed in ISO numbers of MPEG standards. In this context the ownership is clear because ISO and IEC own the copyright of the standard, as stated in the second page of all MPEG standards. But what about other rights?

A small but important number of people use the word MPEG to indicate the working group (WG) whose official name is ISO/ IEC JTC 1/SC 29/WG 11 (I still have to meet someone who is not in the ISO/IEC circle and knows the avronym). To understand what “ownership” means in this context we have to make a digression that I hope will be perceived as short.

The place of the MPEG WG in ISO

The MPEG WG is the green box in the ISO organisation chart of Figure 1. I know that I should also say something about IEC, but the explanation I am going to make will discourage a sufficient number of well-intentioned readers to read further and I do not want to risk losing them all.

Figure 1 – MPEG sits at the bottom of the ISO organisation

In the hierarchical organisation of Figure 1 there are plenty of opportunities to claim “ownership”.

  1. The General Assembly, made up of all standards organisations members of ISO called National Bodies (NB);
  2. The ISO Council, the core governance body made up of 20 NBs, the ISO Officers and the Chairs of 4 committees;
  3. The Technical Management Board (TMB), in charge of managing the structure and activities of the Technical Committee (TC), made up of a chair and 15 members from NBs;
  4. The Joint ISO/IEC Technical Committee 1 Information technology (JTC 1), the largest Techical Committee in ISO with 15 Advisory Groups, 2 Working Groups, and 22 Subcommittees (SC);
  5. The Subcommittee 29, in charge of Coding of audio, picture, multimedia and hypermedia information and has 2 Working Groups;
  6. WG 11 aka MPEG in charge of Moving Pictures and Audio.

“Ownership” of MPEG standards


The MPEG WG is populated by experts who are accredited by the ISO NBs they are members of. They are typically employes of a company or university but there are some students, consultants or simply individuals.

Experts attending MPEG produce standards using the process described in … ISO and IEC sell copies of the standards whose copyright they own. These can be purchased online from Some standards (e.g. those common with ITU) can be freely downloaded.

Patent holders

The buyer of an MPEG standard does not necessarily have the right to practice the standard because of rights to essential patents that may be owned by third parties, in most cases employers of MPEG experts. Therefore, these third parties also “own” the MPEG standard in which they have essential patents. Since the very beginning, MPEG standards a large number of patent holders and patent pools were established to license packages of patents considered essential to practice MPEG standards. Therefore, patent pools, too, have a sort of “ownership”.

A court of law may declare that a patent is essential to practice a standard in a jurisdiction. This kind of declaration is available for a limited number o patents (compared to the number of patents granted). Therefore, the identity of those who really “own” something in an MPEG standard is typically wrapped in mystery.

ISO and IEC (and ITU when relevant) keep patent declarations spontaneously made by those who believe they have rights in the standards. Patent declarations are also made by those who submit technologies for consideration by MPEG, either as part of responses to a Call for Proposal or as independent submissions (e.g. when submitting results of a Core Experiment). ISO and IEC simply record those declarations and take no stance regarding their validity. Therefore, the actual “ownership” claimed by those declarations or agreements to recognise them has to be determined outside of ISO.

Ownership of reference software

MPEG prides itself to have the policy to develop two “versions” of most of its standards. One version is expressed in natural language (English, later translated into French and Spanish). The second version is expressed in a computer language and is called the reference software of the standard. Both versions have normative status.

In the second half of the 1990’s, MPEG developed the MPEG-4 “copyright disclaimer” whereby users could get a free license from ISO/IEC to use and modify individual modules of the reference software for products claiming conformance to the specific MPEG standard. Of course, the copyright disclaimer included the usual disclaimer about third parties’ rights.

In the mid 2000s MPEG adopted a slightly modified version of the Berkeley Software Distribution (BSD) available as MXM Licence), a licence originally used to distribute a Unix-like operating system. The licence simply says that the reference software may be used for anything the uses wishes, with some obvious exceptions. MPEG has added the usual disclaimer (the “modification” above) about possible third-party rights.

Ownership of conformance testing suites

The notion of “ownership”, possibly a more complicated one, is also applicable to conformance testing suites. These consist of bitstreams designed and produced by MPEG experts to test particularly critical parts of a decoder. They have a normative value and are declared to conform to a specific MPEG standard. Bitstreams can be downloaded from the ISO web site and used to check whether a decoder implementation conforms to the standard.

Ownership of test sequences

There are other parties who have some “ownership” of MPEG standards, the companies (in some cases individuals) who have contributed content – images, audio, video and point clouds – used to carry out evaluations of submissions or verification tests. The very first test sequences used by MPEG were the so-called “CCIR sequences” donated by CCIR (now ITU-R) and used in MPEG-1 and other Calls for Proposals. For years the entire video coding community widely used “Table tennis”, “Mobile and calendar” etc. More recently, rights holders have begun to license content to individual MPEG experts for the purpose of developing MPEG standards.

ISO’s is a different organisation chart

Many companies have similar organisation charts with similar boxes as those of Figure 1. They are populated in the real world by employees and managers often with the support of secretariats. Unlike companies, however, ISO does not have its own employees populating the boxes located under the Technical Management Board. Who populates them?

Working Groups are populated by experts sent by companies and chaired by convenors, normally coming from industry. The role of convenors is to facilitate consensus and to detect achievement of consensus. As part of their duty to convene experts, convenors may need to organise the work. This gives convenors a limited level of “ownership”.

Subcommittees and Technical Committees are populated by delegations of National Bodies made up of NB officers often complemented by industry representatives. SC and TC secretariats have a level of “ownership” because the ISO/IEC Directives assign to them several important tasks such as the management of document balloting, e.g. proposals of new work items or approval of standards under development. Committee chairs, too, have a level of ownership.

Another level of “ownership” comes from the fact that a Convenor is nominated by the SC secretariat and elected by NB delegates in the SC. The SC Chair is elected by NB delegates in the SC and confirmed by NB delegates in the TC.

National Bodies “own” ISO

As we have seen, National Bodies permeate the ISO structure at all levels. Decisions at TC and SC level are made by NB votes. It is a one country, one vote systems but, sometimes, some votes weigh more than others. Votes are cast on the basis of the opinion formed within the NBs, ostensibly to further “national interests”.

National Bodies usually have a structure mirroring that of ISO. Industrial interests of national companies which are members of the National Body, often subsidiaries of multinational companies, intermingle to determine national positions on matters to be decided at the international level.


In this article I have tried to identify and describe the different forms of MPEG-generated “ownership”. Clearly the most important “ownership” is represented by MPEG standards because they generate three revenue streams: from products/services/applications enabled by MPEG standards (companies) to satisfy end user needs, from those who practice MPEG standards (patent or, more generally, intellectual property holders) and from the sale of standards (ISO/IEC and National Bodies).

Unfortunately, this clearcut logic is polluted by the different forms of “ownership” that I have described above, especially the desire of National Bodies to hold committee secretariats and nominate chairs. These may be nice-looking elements in National Body panoplies, but have nothing to do with the intensity of the three streams, the raison d’être of a standards organisation.

The intensity of revenue streams exclusively depends on the productivity of the “machine” (the committee) that creates the standards. “Owners” acting according to logic should leverage – not meddle with – a working group whose standards enable a global turnover of devices worth more than 1 trillion USD and services revenues of ~230 billion USD for pay-tv alone (data of 2018).

It will be interesting to see if the next decisions will follow a revenue stream maximisation logic, a curio collection logic or a survival logic.

Posts in this thread

Which future for MPEG


For three decades MPEG has designed multi-threaded work programs, cooperated with tens of organisations and developed close to 200 specifications and hundreds of amendments. MPEG standards have transformed the global media industry from analogue to digital, enabled participation of new industries in the media business, provided standards serving all industries without favours and ensured unrelenting expansion of the entire media business.

MPEG achievements look reassuring: in 2018 the global turnover of MPEG-enabled devices is worth more than 1 trillion USD per annum and the global Pay-Tv revenues are 228 billion USD per annum, without mentioning other industries.

Should MPEG rest and look forward to a bright future built on the successes achieved? It would be so easy to answer “tout va très bien madame la marquise” but my answer is no.

The MPEG landscape has changed

In the first 10 years of its existence MPEG ruled the field of audio and video coding. In the following 10 years some proprietary solutions popped up, but the field was still largely dominated by MPEG standards. In the last 10 years MPEG has seen proprietary solutions getting significant strength. Today large areas that used to be exclusively covered by MPEG standards are satisfied by other solutions and the path to satisfy the next needs is not at all clear.

We do not have the complete picture yet of the extent the market will turn its back to MPEG standards. Market will speak and market is right by definition. If we do not like it, it is just because we did not try hard enough.

The next few years will be very challenging. It will not be a time of “business as usual”. MPEG needs to rethink itself and take appropriate measures. This article lays down some ideas and presents a proposal.

MPEG is about compression

So far MPEG has produced 5 generations of video compression standards, each generation offering more compression and more features. More of the same is expected from the 6th generation (VVC). MPEG has produced an equivalent number of audio coding standards. Will industry keep on asking for more video compression? I would like to answer with a resolute yes as I believe that there will always be need for more compression of visual information, but not always and not necessarily of the “old way”. More importantly, the need will not always be satisfied by MPEG standards because “The Price Is Right” applies to compression standards, too.
The answer to the question “Do we need more audio compression?” is, at least in the current time frame, that the currently available compression engine (MPEG-H 3D Audio) is good enough but we need new standards for other non-compression features, e.g. 6 Degrees of Freedom (6DoF). In the future this trend will also apply to video, as 3DoF+ video moves in the same direction as 6DoF audio (see The MPEG drive to immersive visual experiences).
Point cloud compression definitely needs compression but the convergence between 3D visual information captured by means of traditional video capturing devices and Point Cloud is still a matter for investigation.

MPEG is also about systems aspects

Systems aspects have been the enabling factor of the success of many MPEG standards and the future will not make those aspect less, but more important. The trend toward immersive media will require an even deeper integration between compressed media and the systems aspects that permeate them.
This can be seen from the requirements that are being identified in an activity called “Immersive Media Access and Delivery” where four dimensions are identified:

  1. Time (the usual one)
  2. Space (how to retrieve just the media parts of interest)
  3. Quality (how to access portions of media with the quality desired)
  4. Object (how to access specific parts of specific objects of interest).

MPEG in not just about media

In the past 30 years MPEG has shown that it could address new domains of expertise and learn their language. The fact that today all digital media speak the same (technical) language, is also due to the efforts made by MPEG to understand the needs of different industries, convert them to requirements, develop the technologies and quantise the standards into profiles and levels. This workflow has been in operation for 27 years, starting from the moment MPEG invented profiles and levels, and consistently applied them to talk to different communities using the same language.
This does not mean that there are no challenges when we talk to a new industry. MPEG has spent more than 3 years talking to, and identifying and validating requirements with the genomic community before starting the development of the MPEG-G standard. This significant effort has paid off: three International Standards on Genomic Data Compression have been developed jointly with TC 276 and 3 more are in the pipeline.

Governance is important

MPEG achieved its results as a working group, i.e. as the lowest organisational unit in ISO/IEC that the ISO/IEC directives recommend to be “reasonably limited in size”. Rightly, the ISO/IEC Directives do not define the term “reasonable”, but in 1989 MPEG had already 100 members, in 1999 it had 300 members and in 2019 it has 1500 members 500 of which attend its quarterly meetings. For the work it has done, MPEG has been reasonably limited in size.

For 30 years MPEG has played a role much above its status and I do not think there should be complaints about the results. In normal conditions MPEG could continue to operate as a working group for another 30 years but, as I said above, these are not normal conditions.

MPEG should become a Subcommittee (SC). Why? Because an SC has a solid governance administered by delegates appointed by National Bodies under the leadership of a chair. On the other hand, the design of the organisation must be properly done, if we do not want to do more harm than good.

Design of the SC is important

To be successful, the organisation of the SC should be conservative because of the importance of the industries served by MPEG standards. Therefore, the organisation of the SC should leverage the existing successful MPEG organisation by retaining and strengthening:

  1. The existing well-honed and demonstrably effective MPEG organisation. MPEG has fed the global media industry with technology that has allowed its growth for the last 30 years. It would be irresponsible to do anything that jeopardises such a large industry, the millions of jobs that go with it and the billions of consumers.
  2. The MPEG collaborative stance with other bodies. The major reason of the success of MPEG standards is MPEG’s open collaboration stance with its many client industries as represented by their standard organisation or industry fora. Collaboration is a must for MPEG because compression is always part of a bigger system with many interactions with other components. However, what was good 30, 15 or even 5 years ago is not necessarily sufficient today.
  3. The match of new MPEG standards to market needs. MPEG has produced hugely successful standards. However, other standards are less so. This is inevitable for an organisation that develops anticipatory standards that sometimes target the next 5 years. MPEG’s ability to engage in standards that are better matches of market needs has to be enhanced because conditions have changed.
  4. The strong MPEG brand. The new organisation is an internal matter designed to give MPEG a better chance to face a complex situation and should not create confusion in the MPEG client industries.

Figure 1 represents the proposed organisation.

Figure 1 – Structure of Subcommittee “MPEG Coding of Moving Pictures and Audio”

Meeting design criteria

Criterion #1: the organisational chart of Figure 1 retains the current MPEG organisational structure where existing informal subgroups become (in italic the names of the existing MPEG entities):

Advisory Groups (AG) if they do not develop standards (orange blocks):

  1. Technical Requirements (Requirements)
  2. Liaison and Communication (Communication)
  3. Technical Coordination (Chairs meetings)

Working Groups (WG) if they develop standards (light green blocks):

  1. Systems Coding (Systems)
  2. Video Coding (Video)
  3. Audio Coding (Audio)
  4. 3D Graphics Coding (3D Graphics)
  5. Quality Assessment (Test)
  6. Genomic Data Coding (Genomic activity in Requirements)

Criterion #2: The chart preserves and extends MPEG’s collaboration stance: Joint Teams with ITU-T (JCT-VC and JVET). The new organisation will be able to establish JWGs to develop standards on well-identified common areas of standardisation, e.g. JPEG, SC 24, SC 41 (IoT), SC 42 (Artificial Intelligence), TC 276 (Bioinformatics).

Criterion #3: The SC will now be able to carry out activities with high strategic value by assigning to the Market Needs AG Technical Requirements AG the task to investigate 1) existing or new areas of work and 2) proposals for new areas of work. Both AGs will produce coordinated reports that will be used by the MPEG SC to make an informed decision on new work.

Criterion #4: The SC should be called “MPEG Coding of Moving Pictures and Audio” prefixing the word MPEG to the current title of ISO/IEC JTC 1/SC 29/WG 11 (MPEG).


In a few weeks industry will decide the future of MPEG.

Will industry decide to give itself a safe and prosperous path by adopting the organisation proposed in this article or will it opt for the Japanese saying “出る釘は打たれる” (The nail that sticks out gets hammered down)? Will industry allow the MPEG nail to keep on sticking out or will it hammer it down?

Stay tuned to this block for further news.

Posts in this thread

Why MPEG is part of ISO/IEC


In July 1987 the plan to create a group that would develop industry-neutral standards was formed. But problem to be tackled was that the MPEG “digital baseband” (see The discontinuity of digital technologies) had to based on international international standards because they had to have global validity.

The question then was: where should those standards be developed? The answer is provided by the following sections:

  1. Standards describes the 3 international standards organisations;
  2. ISO and IEC standards describes the ISO structure and the ISO/IEC standardisation process;
  3. A home for MPEG describes how an independent home for MPEG was found.


Standards have a special place in industry because they represent convergence points where the parties involved, who typically are in competition, find it convenient to agree on a single solution.

Standards bodies exists at the international level:

  1. International Telecommunication Union (ITU) for matters related to telecommunication and broadcasting
  2. International Electrotechnical Commission (IEC) for electrotechnical matters
  3. International Organisation for Standardisation (ISO) for everything else.


The International Telecommunication Union (ITU) is the result of the 1934 merge between the International Telegraph Convention of 1865 and the International Radiotelegraph Convention of 1906, and today is an agency of the United Nations. This is reflected in the two main branches of the ITU: ITU-T and ITU-R. The former deals with standards for global telecommunications excluding radio communication because this is the purview of ITU-R.


The International Electrotechnical Commission (IEC) is a not-for-profit organisation founded in 1906. It develops International Standards in the fields of electrotechnology, e.g. power gener­ation, transmission and distribution to home appliances and office equipment, semiconductors, fibre optics, batteries, solar energy, nanotechnology and marine energy.


The International Organization for Standardization (ISO) is an international non-governmental standard-setting organisation founded in 1947 and composed of representatives from various national standards organizations.

ISO is well known for its family of quality management systems standards (ISO 9000), environ­mental management standards (ISO 14000) and Information Security Management Systems standards (ISO 27000). There are more than 20,000 ISO published standards.

ISO is a huge organisation whose technical branch is structured, as is the IEC’s, in Technical Committees (TC). The first 3 active TCs are: TC 1 Screw threads, TC 2 Fasteners and TC 4 Rolling bearings. The last 3 TCs in order of establishment are TC 322 Sustainable finance, TC 323 Circular economy and TC 324 Sharing economy.

Between these two extremes there is a large number of TCs, e.g., TC 35 Paints and varnishes, TC 186 Cutlery and table and decorative metal hollow-ware, TC 249 Traditional Chinese med­icine, TC 282 Water reuse, TC 297 Waste collection and transportation management, etc.

Most TCs are organised in working groups (WG). They are tasked to develop standards while TCs retain key functions such as strategy and management. In quite a few cases the area of res­ponsibility is so broad that a horizontal organisation would not be functional. In this case a TC may decide to establish Subcommittees (SC) which include WGs tasked develop standards.

Figure 1 is an organigram of ISO.

Figure  1 – ISO governance structure

ISO and IEC standards

The development process

ISO and IEC share the standard development process which can be summarised as follows:

  1. Submission and balloting of a New Work Item Proposal (NWIP) of a new project meant to lead to an International Standard (IS) or Technical Report (TR). The former contains normative clauses, the latter is informative
  2. Development of a Working Draft (WD, possibly several versions of it
  3. Balloting of the Committee Draft (CD, when the WD has achieved sufficient maturity)
  4. Balloting of the Draft International Standard (DIS, after resolving comments made by National Bodies)
  5. Balloting of the Final Draft International Standard (FDIS, after resolving comments made by National Bodies)

The last ballot is yes/no. No comments allowed.

Amendments (AMD) extend a standard. The same steps as above are carried out with the names Proposed Draft Amendment (PDAM), Draft Amendment (DAM) and Final Draft Amendment (FDAM).

If an error is discovered, a Corrigendum (COR) is produced. This only goes through two stages: Draft Corrigendum (DCOR) and Corrigendum (COR).

A Technical Report, a document without normative clauses, goes through two stages of approval: Proposed Draft Technical Report (PDTR) and Technical Report (TR).


ISO/IEC mandates that in the development of stan­dards working groups operate based on consensus. This is defined as

General agreement characterised by the absence of sustained opposition to substantial issues by any important part of the concerned interests and by a process that involves seeking to take into account the views of all parties concerned and to reconcile any conflicting arguments.

NOTE — Consensus need not imply unanimity.

Patent policy

ISO, IEC and ITU share a common policy vis-à-vis patents in their standards. Using few im­precise but hopefully clear words (as opposed to many precise but unclear words), the policy is:

  1. It is good if a standard has no patents or if the patent holders allow use of their patents for free (with an “Option 1” declaration);
  2. It is accepted if a standard has patents, but the patents holders only allow use of their patents on fair and reasonable terms and non-discriminatory conditions (with an “Option 2” declaration);
  3. It is not permitted to have a standard with patents whose holders do not allow use of their patents (with an “Option 3” declaration).

A home for MPEG

When the MPEG idea took shape in July 1987, the selection of a home to implement the idea was the primary concern. The idea was spoilt for choices as shown by the list of international committees in  Table 1 that were created for various reasons – regulation or simply need for an independent technical reference – to cater to the needs of standards by the different industries.

Table  1 – Media-related standards committees (1980’s)

ITU-T Speech SG XV WP 1
Video SG XV WP 2
ITU-R Audio SG 10
Video SG 11
IEC Recording of audio SC 60 A
Recording of video SC 60 B
Audio-visual equipment TC 84
Receivers SC 12A and G
ISO Photography TC 42
Cinematography TC 36

Since MPEG was conceived to be industry-neutral, committees already developing standards in the “media” area were considered unsuitable because the represented “vested interests”. The choice fell on ISO TC 97 Data Processing who had SC 2 Character sets and Information Coding who included WG 8 Coding of Audio and Picture Information.

In 1987 ISO/TC 97 Data Processing merged with IEC/TC 83 Information technology equipment. The resulting (joint) technical committee was called ISO/IEC JTC 1 Information Technology. SC 2 with its WGs, including WG 8, became part of JTC 1. MPEG was established as an Experts Group on Moving Pictures of ISO/IEC JTC 1/SC 2/WG 8 in 1988.

Note that Experts Group is an organisational entity not officially recognised in the ISO organ­igram. In 1991 SC 2/WG 8 seceded from SC 2 and became SC 29. WG 8’s Moving Picture Experts Group (MPEG) became WG 11 Coding of audio, picture, multimedia and hypermedia information (but everybody in the industry, and even in the general public, calls it MPEG).

Posts in this thread