What to do with a jammed machine?


In this article I recall how the success story of the MPEG business model took shape, why that wonderful machine has jammed and how the competition is prospering. There could be a future that continues the successes of the past but more likely there will be another future that is costly for industry, research and consumers. This situation should prompt a call for action, but…

A success story

Starting with MPEG-1 (1992), MPEG standards have provided the industry with the means to carry out the transformation of media from analogue to digital. Subsequently, MPEG standards have enabled the industry to consolidate the transformation by giving the means to offer to consumers products and services of constantly improved performance. Globally, the annual value of products and services that rely on MPEG standards is ~1.5 trillion USD, or ~2% of the world gross product (GWP).

This success story is based on the “MPEG business model”, designed to reward innovation by developing standards that offer the best performance at a given time, without considering the impact of the IPRs introduced. As the model assumed, the market found the way to remunerate IPR holders and the royalties that the IPR holders obtained were typically reinvested in innovation that generated new IP for future standards.

The machine has jammed

Recently, this success story has been slowed down, if not interrupted. Some of the reasons are:

    1. Increased number of Non-Practicing Entities (NPE)
    2. Constant reduction in the percentage of Practicing Entities (PE)
    3. Valuable IP from NPEs added to MPEG standards
    4. Different PE and NPE motivations with respect to standards
    5. Inadequacy of ISO IPR rules for a fast-evolving market.

The issue with item 5 is that ISO requires patent holders to only declare how they will allow use of their IP: for free (option 1), FRAND assignment (option 2) or no assignment (option 3). This is inadequate for MPEG standards. I saw item 5 coming long time ago, I wrote about it and I even attempted to signal the IPR problem to higher ISO layers. I was stopped by a handful of National Bodies and then I turned to “silent mode”.

That the machine has jammed is shown by the situation of the latest approved video compression standard (HEVC): ~45 IPR holders, of which ~2/3 are part of 3 patent pools with divergent licences and ~1/3 is not part of any patent pool. Over the years the 3 patent pools made only atomic-level convergence and, 7.5 years after the approval of the standard, HEVC has a level of use only in broadcasting, but limited use in streaming.

This is not the only source of problems. The immersive audio compression standard MPEG-H 3D Audio has a significantly smaller number of IPR holders than HEVC. Still, there is no MPEG-H 3D Audio licence. When Sony wanted to use MPEG-H 3D Audio in a service, it was forced to ask MPEG to define a profile whose IP is expected to be owned by just one patent holder.

There is competition

Starting from MPEG-4 Visual, there has always been a level of competition. However, that competition had a limited impact compared to the competition of Alliance for Open Media (AOM), established by Amazon, Cisco, Google, Intel, Microsoft, Mozilla and Netflix in 2016. AOM has released its AV1 video compression specification in 2018. The AV1 license can be defined as “royalty free”. AOM has also established a fund that can be used by AOM members if they are attacked by a third party for IPR infringement.

AV1 is currently widely used for streaming. However, AOM is actively working to get its specification adopted in other application areas, such as broadcasting and mobile.

What the future could be

Economists like to say that having more solutions is good for competition Ask consumers and they will remind you of Betamax and VHS. In the past 30 years, there has been a lot of competition in the field of digital media but that was between proponents of different technologies at MPEG meetings. Thanks to that competition consumers were offered a constantly improved “user experience”. I like to think that MPEG made both economists and consumers happy, a record.

If the MPEG business model were not beset by the difficulties I mentioned, the market could continue to be powered by MPEG standards for a few more decades. This is proved by just two cases. Traditional audio-video compression (for TV and mobile) has reached levels compatible with the development of networks, but more can be done. Coding of immersive audio-visual scenes is a wide research and innovation field.

The costs of the other future

In this section I will analyse the cost that the stalled situation will have on the industry, the research community and the consumers.

Costs for industry

If the stalled situation described above will drag on, it can be expected that the markets will converge to royalty free solutions produced by a source dominated by a few big players. This source will not have an internal mechanism pushing for continuous innovation like MPEG used to have. Therefore, the source will hardly be able to offer the same rate of new technologies that we have witnessed in past decades.

Drawing a comparison with the situation of the 1970s and 1980s when the television industry was stagnating because it had all the technology it needed to make products and services, the media industry may face a future of stagnation.

Costs for research

I do not have direct experience but I have no difficulty believing what knowledgeable people told me, namely that AV1 has been developed by a handful of (very competent) engineers and a handful of (even more competent) patent attorneys.

The new future is one where thousands of well-paid jobs in media compression and the academic research that feeds it will go because there is no longer a pressing demand for new compression technologies.

 Costs for consumers

This situation will obviously be reflected on consumers. They will probably enjoy a bonanza, like television users of the 1970s and 1980s did, because manufacturers will compete on price. But they will not enjoy new shiny devices or services.

And then?

The above looks like a call for action…

Posts in this thread

Posts in the previous thread

List of all related articles


Stop here if you want to know about MPEG (†)

Over the last two years I have written ~100 articles in defence of MPEG, its legacy and its future. The articles were written without a particular organisation in mind.

During these two years the number of readers has grow and I thought it useful to group the articles according to the following categories.

Here are the articles

Posts in this thread

Posts in the previous thread

List of all related articles


This is ISO – A hypocritical organisation


ISO and IEC have developed the ISO/IEC Directives Part 1: Procedures for the technical work about procedures to be followed in developing and maintaining an International Standard. This is complemented by the Consolidated ISO Supplement,  about procedures specific to ISO and by the JTC 1 Supplement, about procedures specific to JTC 1. Finally, for the purpose of this article, I will mention the ISO/IEC Directives Part 2: Principles and rules for the structure and drafting of ISO and IEC documents.

We are talking of hundreds of pages in total. They have costed hundreds of person-years for their development and extension over the decades.

In one of my previous articles, This is ISO – A feudal organisation, I have complained about the fact that ISO does not have a document even remotely similar to those above to regulate the activities at Technical Committee and Subcommittee level where political decisions are made. At meetings, the secretariat has the power to interpret the few rules that can be found in the directives and to invent new ones if needed. For instance, I was not allowed to put on record a statement with the justification that “only the secretariat has the right to put anything in the meeting report”. I believe that the right to have recorded a position in the minutes is most basic right of a representative system.

In this article I would like to highlight an aspect of the directives that has been made to apply to both ISO and IEC. in my opinion, it is debatable whether this aspect should even be regulated, while it is clear that this rule has devastating impacts on the development of standards and forces ISO participants to formally comply with it while finding all sorts of subterfuges to avoid it.

My question is: should the minimum – in my opinion the trivial – be regulated, when essential aspects such as regulating the power of committee secretariats or of the ISO central secretariat, take second place, actually no place at all?

The issue

I do not have direct experience of the case that triggered the issue I am going to address. I learned from knowledgeable persons that, at a time when a New Work Item Proposal (or another document) was out for ballot in JTC 1, a National Body used an official mailing list to lobby for the defeat of the document.

Personally, I would not think this is a crime as it was considered. Often there are intense communications among National Bodies behind the scenes when a document is out for ballot, so why not have it in public? One could even argue that it is preferable that National Bodies know all arguments.

Instead, this action drew the ire of the JTC 1 secretariat who obtained that the following rule be introduced: “Documents out for ballot at NP, Committee Stage or any later stage shall not be subject to formal discussion at any working level of JTC 1 during the balloting period. Therefore, National Body positions on a document under ballot are not to be formally discussed at any working level” (wording may have changed over the years).

The impact

For MPEG, this decision was a disaster waiting to happen, another proof that MPEG is the “odd committee out” in ISO (see The MPEG exception). One effect of the decision wouls be that MPEG standards, typically very complex documents, could no longer be collectively reviewed at meetings, one of the main activities that allow MPEG standards to be released in a timely manner and with few or no errors. While errors can be corrected, they require the publication of corrigenda to take effect, not desirable for many reasons.

That rule notwithstanding, however, MPEG went on unconcerned because, this was the reasoning, input documents identifying errors in draft standards were submitted by experts, not National Bodies. So, MPEG kept on discussing and fixing bugs in draft standards out for ballots at its meetings, as any company does for its specifications. National Bodies were informed that bugs had been discovered and could be fixed as documented in MPEG-produced documents titled “Study on Draft CD, DIS etc…”.

But the JTC 1 crusade to straighten things out in the world of international standards did not stop at JTC 1. After a few years, this rule, so far valid only for JTC 1, was made to apply to all of ISO. This was not enough for the JTC 1 secretariat. After a few years of efforts, the maneuvres to extend the rule to IEC as well succeeded. Finally, ISO and IEC could tread in the ways of righteousness!

Unfortunately the attention, already turned to MPEG for other matters, was further focused on MPEG. The group realised that the interpretation of “documents identifying errors submitted by experts and not by National Bodies” would not be accepted. I began asking for advice to several National Bodies. Some were kind enough to offer specific advices like “You should meet in a session outside of the official meeting to discuss bug fixing of documents out for ballot” or “Create a copy under different document number – even if the content is more or less exactly the same, possibly titled “continuous improvement of ………….” and so on.


The story above is a fair description of the atmosphere in which people operate in ISO. Rules are established haphazardly with language, subject to interpretation by a secretariat, to achieve undefined goals of righteousness disregarding the impact on the real mission that should concern ISO, namely, to serve the industry with timely and bug-free standards. The “workers” (or should I call them “peasants”?) who ensure that the ISO mission is reached, not only should work, but are also forced to behave hypocritically by upholding the sacrosanct value of the Directives in words while finding subterfuges to circumvent them in practice.

All this while nothing is done to set order where order is much needed: regulating the power of secretariats.

In Matthew 7:5 we find “Thou hypocrite, first cast out the beam out of thine own eye; and then shalt thou see clearly to cast out the mote out of thy brother’s eye”.

Posts in this thread

Posts in the previous thread

List of all related articles


The MPEG Hall of fame

As MPEG is no more, it is time to open and populate the MPEG Hall of fame. It is not going to be an easy task because so many people have contributed to make MPEG what it was.

The MPEG subgroup chairs will not be mentioned here. They are in the Hall of fame by default because they have been among those most committed to the success of MPEG. If you want to know their names, responsibilities and years of services, please look here.

In this article I will mention some of those who made a particular contribution outside of standards development. If you think that someone whose name is not here should be added to MPEG Hall of fame, please send me an email.

Temporally, the first entry is Tsuneyoshi Hidaka. He took care of defining, planning, executing and processing the data of the MPEG-1 and MPEG-2 subjective tests carried out in November 1989 and November 1991, respectively. The execution part was particularly important because JVC had a recently built research lab at Kurihama with an excellent video quality testing facility. Without the benefit of Tsuneyoshi’s expertise and the access to testing facilities MPEG could very well stumbled at its first project because at that time it was an unknown group whose only asset was its plan.

The second entry of the MPEG Hall of fame is Ken Davies (CBC). Ken was a very respected participant in CCIR (now ITU-R) activities. He was the first to make available video test sequences to MPEG by giving access to the CCIR library. Two video sequences – “Table Tennis” and “Flower Garden” – were used and watched by thousands of people engaged in video coding research in the 1990s both inside and outside of MPEG. I am unable to tell the number of individuals and companies who permitted the use of their content – video, audio and, more recently, point cloud data – used to develop many MPEG standards since Ken’s first donation.

As the Hall of Fame attempts to list people in temporal order, it is appropriate to mention two figures whose support and assistance have been fundamental in the first 20 years of MPEG. They are Mike Smith, then the ITTF chief and Keith Brannon, his right hand. They both understood the importance of bringing the MPEG activity under the folds of ISO and both always available to help and solve problems. Today I would say: Quantum mutatus ab illo.

On a different plane is the role of another entry of the MPEG Hall of fame. Arian Koster was a young engineer at KPN Research who, one morning of July 1990, called me to make a suggestion: “What if MPEG developed a software implementation of the MPEG-1 standard?”. This was a radical new idea because at that time every participant in MPEG-1 Audio or Video had their own software implementations of the Video Simulation Model. His suggestion was the beginning of what became the consolidated practice of the MPEG Reference Software. Both MPEG-1 and MPEG-2 did not have significant examples of common reference software, but the practice took root with MPEG-4 and continued ever since.

On a similar plan I would like to propose another entry to the MPEG Hall of fame, myself Leonardo Chiariglione (CEDEO.net). In general, ISO standards do not make a clear distinction between specification (“the law”) and conformance testing (“the tribunal”). My job at Telecom Italia, however, made me sensitive to the latter because Testing Laboratories have always been part of the culture of that industry. MPEG-1 was a “telco – consumer electronics” project, hence it included an industry not very sensitive to formal conformance testing. Therefore, the adoption of my idea that there should be an ad hoc part of the standard that would serve the purpose of testing an implementation for conformance was not discounted. Conformance testing has been ever since a necessary component of all MPEG standards.

On yet another plane was the role of two other entries in the MPEG Hall of fame: Dick Green and Baryn Futa. Dick had worked at PBS had a long experience in attending ITU meeting. In 1988 he had established CableLabs of which he was President. In 1992 he visited my lab at CSELT (later Telecom Italia Lab) to know more about the MPEG-2 work and gave the task to Baryn Futa, CableLabs CEO, to take care of MPEG-2 IPR discussions. Eventually Baryn created MPEG LA (LA=Licensing Authority, no relation with MPEG) overcoming important legislative hurdles.Without a patent pool, MPEG-2 would most likely have never flown.

On yet another plane was the role of another entry of the MPEG Hall of fame, Pete Schirling. Pete worked at IBM and acted as US Head of Delegation (when we could still use that term). In July 1995 he offered MPEG to develop a system that would allow MPEG members to upload their documents and download other members’ documents. I believe that was the first system of this kind deployed for a standards committee, certainly for a committee of the MPEG size which was already hovering around 300. It took many years to get rid of paper, but after Pete’s system we no longer had examples of 1 million photocopies at the November 1993 meeting in Seoul. When Pete could no longer run the system, Wo Chang (NIST) took over. He redesigned the system and ran it for 5 years. When Wo left MPEG, Christian Tulvan (Institut Mines Télécom) took over, redesigned the system and kept on adding incredibly useful features that have made MPEG meetings constantly more efficient and enjoyable. Pete, Wo and Christian can deservedly enter the MPEG Hall of fame.

On the communication front, Pete Schirling (again), Arianne Hinds (IBM and then CableLabs) and Christian Timmerer (University of Klagenfurt) deserve to enter the MPEG Hall for fame because for many years in different periods, they collected and edited the main news and made them the press release of each meeting. Pete, Arianne and Christian have carried out the commendable and quite engaging task of chasing the chairs and extracting the news from them.

On the education front Alexis Tourapis (Apple) enters the MPEG Hall of fame because he shot and edited the video presentations made by MPEG members on different MPEG standards.

On the promotion front, Rob Koenen (TNO and then Tiled Media) and José Alvarez (Huawei) enter the MPEG Hall of fame because they accepted the challenge to be the heralds of MPEG industry liaison. They organised events at MPEG meetings and at several places in the world where they presented the MPEG work plan and asked industry representatives to provide feedback. Besides that, Rob took care of maintaining a visual representation of the MPEG work work plan at every meeting, like the one you find at MPEG status report (Jan 20202).

I am pretty sure I am forgetting worthwhile entries in the MPEG Hall of fame. If you notice one, please send me an email.

Posts in this thread

Posts in the previous thread

List of all related articles


Top-down or bottom-up?

On 2020/06/02 SC 29, the parent group of MPEG, decided to erase the group. This was the result of a combination of reasons:

  1. One National Body (NB) wanted to retain the MPEG activity within SC 29
  2. Another NB craved to get control of international standards for digital media
  3. Other NBs were “convinced” that it was better to go this way
  4. Some individuals coveted “positions”
  5. Certain segments of the IP (Intellectual Property) industry were determined to perpetuate the outdated business model of “put any IP in the standard and let the market sort out the mess”
  6. Some entities could have been motivated by elements drawn from the content of of my papers exemplified by Some have it straightThe MPEG exceptionWhy is there a battle around MPEG? etc.
  7. and more…

I let the readers make their own opinion about the wisdom of making a decision supporting the desire of the two NBs unidentified above and commiserate those seeking positions while unfit, but I cannot avoid noting the stubbornness of those who persevere on a path that is marginalising the use of HEVC, and possibly other MPEG standards as well, and negatively impacting the MPEG name, while the use of AVC is universal.

I have no doubt that SC 29 will continue, by sheer inertia, the work that MPEG had under way before being erased. However, the long term prospects are not positive. Today I will analyse the following reason: as a Subcommittee, SC 29 will make top-down decisions as opposed to MPEG’s bottom-up consensus building.

For 30 years MPEG has developed standards starting from inputs coming from its members (individuals). Nothing came ever down from SC 29, i.e. from NBs. For instance in July 2019 a proposal called Video Coding for Machines was made whose main elements I had the opportunity to introduce. Some 100 people are currently working on that proposal.

Those who think that MPEG’s successful standards have been proposed by NBs have got it completely wrong.

In MPEG, a formal process used to start form an idea. If successful, the idea could become a candidate for a standard. It was an engaging process where everybody was entitled to bring arguments pro and con. If the idea survived and prospered, it would become the target of a call for proposals of technologies. The submitted technologies, typically revised by MPEG, would be converted to a standard. All the process was managed by a group called Requirements.

Those bringing a new idea to MPEG were like entrepreneurs trying to get funding for their startup. They could get “funded” (i.e. they could get to the point of issuing a Call for Proposals, finding the human resources believing in their idea and then developing the standard) but eventually the company (the standard) could very well not fly. Actually, only ~1/10 of all MPEG standards did actually fly.

Sounds familiar?

Conversely, the process in some NBs is murky. Often, the NB membership does not represent the national reality. Often, large multinational players dictate the agenda. Often, employees of large companies are the only ones with the time and money required to attend high-level committees where important decisions are made. Often, “instructions” come from the above “communicating” positions taken by higher level bodies and to be executed by lower layer committees. Often, the chair of a committee may bring personal positions into other committees, call “friends”, without informing the committee, to get their motion approved. Often, a higher level committee may take a position diametrically opposite to the position unanimously agreed by a lower level committee on matters affecting the lower level committee.

My answer to the argument “this is how business works” is “from whom would you buy a second-hand car?”

The MPEG model has proved its worth. Logically, if something works – like MPEG – it should not be changed, aka if it ain’t broke don’t fix it. If the people in some committee fix what is not broken for one of more of the reasons above, they should bear responsibility for the misdeed. But the system has been designed to operate exactly in such a way that people who made the decisions will never be held accountable.

If there will be a new MPEG – clearly not in ISO – it should operate exactly as the old MPEG used to operate. Everybody should be entitled to make proposals, bring evidence, argument on the goodness of their proposal, find others who share the view that the proposal is good, and fight those opposing their idea by the force of their arguments.

Nothing murky should get in the way.

A preview of things to come: the old MPEG Requirements group is the only group in SC 29 that has not a chair.

Posts in this thread

Posts in the previous thread

List of all related articles


This is ISO – a chaotic organisation

Human organisations that are created and develop based on a master plan discover that the world is more complicated than assumed by the master plan they started from. Sooner or later they must flexibly adapt the plan to the conditions on the field. Other human organisations start without a master plan and develop on the basis of actual conditions. Sooner or later they discover that the organisation has a dysfunctional structure that must be rationalised.

I was not in Geneva in 1947 when ISO was established, but I am pretty sure that there was no master plan for creating standardisation areas. This can be easily seen from the way Technical Committees (TC) were established, i.e. on an as needed basis.

TC 1: Screw threads, TC 2: Fasteners, TC 3: Limits and fits, TC 4: Rolling bearings, TC 5: Ferrous metal pipes and metallic fittings, TC 6 Paper, board and pulps, TC: 7 Rivets, TC 8: Ships and marine technology

Indeed, after responding to the needs of “Ferrous metal pipes and metallic fittings”, ISO created a TC for “Paper, board and pulps”. After addressing rivets, ISO responded to the needs of “Ships and marine technology”.

I believe this was the right things to do. You may attempt to create a rational picture of the industry (good luck!). Even if you succeed to develop one, a few years later the evolution of technology will make the picture obsolete.

ISO disbanded TC 3: Limits and fits and TC: 7 Rivets because they were no longer needed. Good that rivets were still in need of standards in 1941 or Jane would have had a problem.

In ISO as well as in the market, new TCs/companies are born and die. But there are also mergers and acquisitions, and spin offs. TC 97 Data Processing is a good example of this:

  1. Formed in 1960 with ANSI as secretariat and originally responsible for computers, peripherals, and computer systems.
  2. In 1981 TC 97 absorbed TC 95 Office Equipment.
  3. In 1983 SC 8, Numerical Control of Machines, and SC 9, Programming Languages for Numerical Control were spun off to create TC 184, Industrial Automation Systems.
  4. In 1984, TC 97 was reorganised to group similar activities with 3 Vice-Chairpersons coordinating the activities of groups of SCs with similar interests.
  5. In 1987 TC 97 merged with IEC/TC 83 Information technology equipment (and later IEC/SC 47 B Microprocessor systems) and became the Joint ISO/IEC Technical Committee on Information Technology (JTC 1).

From that moment on JTC 1 witnessed an impressive organic growth. Today JTC 1 includes 22 Subcommittees (SCs) and many Advisory Groups and Working Groups reporting directly to it.

The JTC 1 SCs cover a broad range of areas. Here are some examples: Character Sets, Telecommunication, Software and System Engineering, Security Devices, Digitally Recorded Media, Office Equipment, Digital Media, Security, IT for learning, Biometrics, User Interfaces, Cloud Computing, Internet of Things, Artificial intelligence etc.

Is this situation OK? I do not think it is, because JTC 1 is no longer a TC in charge of an area, it is a collection of many disconnected areas which do not have an obvious reason to stay together. Today JTC 1 does not even attempt to group areas as it commendably did in 1984 but then discontinued.

Let me give examples: What is the relationship of digitally recorded media with artificial intelligence? What is the relationship of IT for learning with internet of things? What is the relationship of software and system engineering with digital media?

In JTC 1 there is a mix of basic information technologies (e.g. Software and systems engineering, Programming languages), devices (e.g. Cards and security devices, Digitally Recorded Media), communication (e.g. information exchange between systems, interconnection of IT equipment), applications (e.g. IT for learning, biometrics) and many other areas without any thread connecting them.

There is another issue, and an even more serious one, because it affects the governance. Out of the 22962 standards produced by ISO, JTC 1 has produced 3249 standards, i.e. ~14% of all ISO standards. JTC 1 is the dominating TC in terms of standards produced (by the way, more than 500, i.e. ~16%, were produced by MPEG), but the actual membership in JTC 1 entities (SCs, AGs and WGs) accounts for probably 30% of the entire membership in ISO entities (TCs, SCs, AGs and WGs). ~80% of patent declarations received by ISO relate to JTC 1 standards (and ~71% of those declarations relate to MPEG standards).

Still JTC 1 has the same governance as TC 291 Domestic gas cooking appliances that has produced no standards so far.

I have already expressed my opinion about ISO governance in TCs and SCs.

JTC 1 Subcommittees as of June 2020

  • SC 2    Coded character sets
  • SC 6    Telecommunications and information exchange between systems
  • SC 7    Software and systems engineering
  • SC 17  Cards and security devices for personal identification
  • SC 22  Programming languages, their environments and system software interfaces
  • SC 23  Digitally Recorded Media for Information Interchange and Storage
  • SC 24  Computer graphics, image processing and environmental data representation
  • SC 25  Interconnection of information technology equipment
  • SC 27  IT Security techniques
  • SC 28  Office equipment
  • SC 29  Coding of audio, picture, multimedia and hypermedia information
  • SC 31  Automatic identification and data capture techniques
  • SC 32  Data management and interchange
  • SC 34  Document description and processing languages
  • SC 35  User interfaces
  • SC 36  Information technology for learning, education and training
  • SC 37  Biometrics
  • SC 38  Cloud Computing and Distributed Platforms
  • SC 39  Sustainability for and by Information Technology
  • SC 40  IT Service Management and IT Governance
  • SC 41  Internet of Things and related technologies
  • SC 42  Artificial intelligence

Posts in this thread

Posts in the previous thread

List of all related articles


A future without MPEG

Table of contents

  1. Introduction
  2. The situation today
  3. The competitors
  4. What is/will be available
  5. What the market offers
  6. Licensing of other media coding standards
  7. My views on the next steps of media coding standards
  8. A disclosure
  9. My forecast
  10. So, what will happen next?
  11. And now?


My plan this week was to talk about the prospects of MPEG for the next few years. But there is a big news: MPEG passed away on 2020/06/02T16:30 CEST. The agony was long, but the result was predictable. The causes have partly to do with the nature of ISO whose main feature I describe in This is ISO – A feudal organisation.

What used to be the reference standards group driving close to 2% of the Gross World Product, i.e. 1.5 T$, and affecting the daily lives of billions of people is now reduced to splinters. The radical innovation of putting in one committee all media components, copied by most standards bodies and companies, is gone. Now, ISO media standards are treated separately. Godspeed, when someone will need to put the pieces together.

The passing away of MPEG is another hurdle added to a big hurdle on which I have published several articles: A crisis, the causes and a solution, Can MPEG overcome its Video “crisis”?, Business model based ISO/IEC standards, and IP counting or revenue counting?

After publishing those articles, I was forced to silence, but now it is time to speak again, this time loud and clear.

I need to make 3 strong statements:

  1. Those who have created a new intellectual object have to right to exploit it
  2. Technology innovation is well rewarded by a system that identifies, determines and protects intellectual property (IP)
  3. Those who make available their IP to a technical specification declared standard by an official standards body have additional benefits compared to those who do not. They should have more obligations as well.

The situation today

I will not spend much time recalling how companies who had their IP in MPEG standards have handsomely been rewarded. According to a knowledgeable person, whose statement I could not get confirmed, MPEG-2 patent holders shared revenues of 1 B$ p.a. during the validity of the MPEG-2 standard. Most IP holders have reinvested in new technologies that have feed the MPEG virtuous cycle of more than 180 standards including 6 video coding standards and 6 audio coding standards.

Unfortunately, only in fairy tales the story ends with “…and they all lived happily ever after”. MPEG did not live happily after MPEG-2 because it saw that most MPEG-2 IP holders held IP on the following (video) coding standards and had difficulty adjusting to the internet video distribution paradigm. Adding to that, the number of IP holders on HEVC has skyrocketed to ~45, 2/3 of which belong to one of the 3 existing patent pools and 1/3 belong to none. It should be no surprise that the HEVC standard has some use in broadcasting, but its use on the web is estimated to be at 12%. If one considers that broadcasting is a rich but declining market and video on the web is constantly rising, one understands that ISO standards will be gradually relegated to a more and more marginal market.

The competitors

When the MPEG-4 Visual licence sentenced the death penalty to the MPEG-4 Visual standard, competitors popped up. Four famous names are Real Networks, Microsoft, On 2 and Google. The fact that private companies could assemble a video codec of commercial value signaled that video coding was a maturing technology and that “anybody” could assemble a reasonably performing solution.

With many video coding patents reaching 20 years after filing, it was reasonable for MPEG to try and assemble a video coding specification that could be used without a licence or, if possible, with a free licence. The rationale for MPEG to do this was: do you want to let your customers be served by a competitor – and not see them back – or do you want to offer a solution that responds to the customer’s needs?

MPEG tried 3 different routes to achieve the goal

  1. Define a royalty free AVC baseline (called WebVC). Some AVC patent holders confirmed their royalty bearing patent declarations
  2. Develop a royalty free standard from scratch (called Internet Video Coding or IVC). Declarations from some companies that they held patents were received
  3. Develop a royalty free standard starting from an existing solution (called Video Coding for Browsers or VCB). A declaration from a company that it held many patents was received.

What did we learn from the 3 experiences? Of course that a company should not be forced to give away a patent for free (experience #1) but also that ISO rules allow a company to prevent a no-licence/free licence standard from happening by simply making a cautionary “I may have patents that I am willing to licence” declaration.

What is/will be available

A group of companies came up with an original proposal of making a standard that would avoid the uncontrolled flow of IP, sometimes of dubious value, with the attached burden of agreeing to a license. The proposal requested

  1. A two-layer coding scheme.
    1. The base layer must contain technology that is more than 20 years old or is accessible at no cost and provide a significant improvement on AVC.
    2. The second layer should provide significant improvement over HEVC
  2. Encouragement to declare that licencing terms will be published within two years after FDIS approval.

The Standard, called Essential Video Coding (EVC), has reached FDIS in April 2020. Final performance data are not available yet, however, it is expected that EVC base layer will be some 20% better than AVC and full EVC will be 30% better than HEVC. Licensing terms are not known yet.

In 4 weeks (July 2020) it is expected that Versatile Video Coding (VVC), the latest video coding standard, will reach FDIS.

To complete the line up of video coding standards, Low Complexity Enhancement Video Coding (LCEVC) is expected to reach FDIS. In appropriate conditions the combination of LCEVC and a generation N codec should provide a quality comparable to the quality from a generation N+1 codec.

What the market offers

The market has not been idle. In 2015 an initial group of companies (Amazon, Cisco, Google, Intel, Microsoft, Mozilla, Netflix) have created Alliance for Open Media (AOM), initially targeting video. The AOM about page mentions 4 main features of the first specification AV1:

  1. Royalty-free ecosystem
  2. Patent review process and legal defense fund
  3. Cutting edge technologies
  4. Collaborative open source development.

Licensing of other media coding standards

The information provided in this section is believed to be accurate. If you intend to make a business decision, however, please seek advice from counsel.


No licensing of MPEG-1 Video is known. Still the standard was widely used.

MPEG LA developed the MPEG-2 Video licence. MPEG-2 was widely used.

MPEG-4 Visual had a hard life because there was an unsuitable licence for streaming, the main target of the standard.

MPEG-4 AVC is a very successful standard that can proudly bear the “generic” attribute because it is used for broadcasting and online streaming as well.

After 7 years, MPEG-H HEVC patent holders could not get their acts together and propose a decently unified licence. HEVC is used in broadcasting, however, use for streaming is limited at best. For years an MPEG officer tried to convince MPEG that HEVC was doing well in the market.

In 2017 I took the initiative of proposing to bring the situation to the attention of JTC 1. A document was prepared and agreed. I went to JTC 1 and presented it. However, a couple of countries objected on procedural grounds. Today, ISO is still officially unaware of the patent problem because of that opposition.


MP3 had a very simple and effective licence that allowed the standard to be widely adopted. The number of hardware and software MP3 devices is counted by the billions.

AAC has a patent pool with an effective licence as shown by the number of hardware and software AAC devices estimated to be 10 billion units.

MPEG-H 3D Audio has been around for a few years and adopted for ATSC 3.0. However, there is no known licence available. Recently a profile that is expected to contain IP just from one company has been started. There is hope that a licence for 3D Audio will be possible.

3D graphics

Past standards in this area did not have much traction. The opposite is expected for point cloud compression (PCC). The standard will become FDIS in a month time (July 2020). The number of patent holders is expected to be significantly less than HEVC.

My views on the next steps of video coding standards

The void left by HEVC has been filled by AOM with their AV1 specification which is widely used for streaming.

EVC is promising because it provides a quality that is comparable with or better than AV1, although less than VVC. EVC may have a chance if a licence will be published. However, this has not happened yet.

Most likely the number of VVC patent holders is much larger than HEVC’s. A Media Coding Industry Forum (MC-IF) was established in 2018 with the goal

“to further the adoption of MPEG Standards, initially focusing on VVC (Versatile Video Coding), by establishing them as well accepted and widely used standards for the benefit of consumers and the industry”

So far, however, no concrete results have been made known outside, if we leave aside the events organised at conferences. MC-IF has 31 members, 7 of which are licensing entities (i.e. a little less than ¼ of all members). The “industry” members account for just ½ of the HEVC patent holders. In these conditions, it is hard to believe that VVC will fare better than HEVC. It could very well fare worse because VVC adoption in broadcasting will take years, if ever.

The success of EVC could help VVC succeed. Seeing the threat of EVC (and AV1), VVC patent holders could get their act together and provide a decent licence. While I would welcome such a development, I consider it as having low likelihood.

A decent licence for LCEVC could be a game changer and another threat to VVC that could mean additional pressure on VVC patent holders to provide a single and decent licence.

A disclosure

I can finally disclose my strategy to save what used to be MPEG Video coding from disappearance:

  1. HEVC and VVC will go nowhere if left by themselves
  2. EVC will be a mezzanine standard between HEVC and VVC
  3. LCEVC on top of EVC can compete with VVC in terms of quality, in appropriate conditions
  4. If EVC will succeed, and patent holders will feel the competition and agree to a decent licence, VVC can have a chance.

Few people, if any, have understood that my goal was never to promote the success of EVC per se. By pushing EVC, I intended to create internal competition and thus promote the success of VVC. If that will not happen, loyal MPEG customers still have a chance to buy the only competitive MPEG video codec available – EVC. By adding LCEVC, they can get state of the art performance.

My forecast

My forecast is grey for the following reasons

  1. There is no longer a united MPEG, actually, MPEG does not exist anymore. That was inevitable, it was just waiting for people to do it because MPEG, with all its influence on the industry, was like a free city in the Middle Ages, ready to fall under the attack of powerful armies. As the Romance of Three Kingdoms (三國演義) said 700 years ago: 話說天下大勢.分久必合,合久必分 (in the world things divided for a long time shall unite, things united for a long time shall divide), there  will be division for a long time, say, around 30 years, the time MPEG has operated united.
  2. The putative head of SC 29 has not shown a minimum of vision for the future. This is a death knell because digital media is developing at an aggressive rate in a mature market. If ISO waits for matters to consolidate in the market, and then develop a standard, it will lose its time with the standard. MPEG has gambled for 30 years, sometimes winning, sometimes losing. What matters, however, is that the industry has not been left without standard technologies for its necessities.
  3. Most importantly, the market has come up with a solution – AV1 by the Alliance for Open Media – that is capable to provide decently performing specifications that are royalty free and promises a defence to an attacked member (with strings attached, but everybody looks at the next quarter, and does not consider the implications of a choice 5 years from now).

So, what will happen next?

If EVC/LCEVC will fly

There is a hope that VVC will also fly. Patent holders will get the expected flow of royalties and may feel assured that supporting the model still pays off and may be willing to further invest in the environment. As this, however, will only be seen in 2-3 years, patent holders will have to give the environment credit for another 2-3 years.

If EVC/LCEVC will not fly

I will consider five cases

  1. A miracle happens. VVC patent holders get their acts together and provide a decent licence. However, miracles are not very common because they indicate sainthood.
  2. NPEs/Universities/Industrial research try to keep the environment alive by pretending that industry will use the “standards” produced. This will last for some time until higher-level  management realise they are wasting time and money.
  3. The environment becomes a special “conference” (not a standards meeting) assessing “papers” (not contributions) to produce “documents” (not standards). The value of a “document” is in the “technology screening process” that assesses the “value” of technologies, so that those who want to make a specification (e.g. AOM) know that they are good screened technologies. This may save the NPE business model.
  4. The environment is in charge of maintaining the large number of standards. This may become difficult because the environment will not be attractive enough to experts who are ready to perform their duty maintaining old standards in exchange for some fun in new standards (that will no longer be there).
  5. The environment becomes the place where Publicly Available Specifications (PAS) suppliers, e.g. AOM, send their specifications to be converted to ISO standards. This will be the final sanction of the end of The MPEG exception. I hope not to be in this world anymore to witness such a degradation if that happens .

And now

Something is going to happen…

See the Russian translation Будущее без MPEG

Posts in this thread

Posts in the previous thread

List of all related articles


This is ISO – A feudal organisation


The “International Organisation for Standardisation” – ISO – was established in 1947 on the ruins of World War II to accomplish one of the noblest activities that can be performed for humankind: making standards. I call this a noble activity because standards make the society of humans possible and better.

When I first encountered ISO in 1987, I found it unique because the national ISO counterparts, so-called National Standards Bodies, even in countries not particularly bent on it, gave senior positions to women. That characteristic, not so common in business at that time, largely continues today.

However, having operated in ISO for some 33 years, with many opportunities to see it at work, I have found positive characteristics along with aspects that are odd survivors of an age that most people I know would think is gone for good. Those surviving aspects, that in this paper I describe with the word feudal, still impregnate the organisation. It is a pity, and an inexcusable one, because the role of standards in a society that gets more numerous and integrated every day, is bound to increase.

Countries reject products that have been produced by, say, workers in unhealthy conditions, not because they are not good, but because they object to the way the products have been made. Many users are unaware of the way ISO standards are made. If they knew, they would probably question rules and customs that are a relic of the past today. I intend to provide elements of knowledge in the hope that things will change for the better.

I have two disclaimers. The first is that nothing of what I say in these articles is intended to apply to other standards developing organisations (SDO). While my experience in ISO is quite deep and pervasive, my experience of other SDOs is too shallow to attempt to characterise them in any way. What I know of some of them tells me they do not have the attributes that make, in my opinion, ISO unfit to the name it bears. The second is that what I say is intended to apply to ISO as the organisation registered in Geneva. They are not meant to apply, unless I will say explicitly so, to specific people working in the organisation or in its national counterparts.

The organisation

ISO is Non-Governmental Organisation (NGO). However, it is international in nature. It is hierarchical as depicted in the figure below.

It should not be a surprise that ISO is also bureaucratic. Many governmental organisations and private corporations are hierarchical and bureaucratic. The main difference here is in the way authority is exercised at each layer. This is what makes the functioning of the ISO structure look more like that of the Holy Roman Empire than of a company.

The ISO structure at work

A standardisation area is assigned to a Technical Committee (TC) managed by a secretariat. The secretariat is run by one or more employees from an ISO member body headed by a Committee Manager (used to be called Secretary) from the same member body. Secretariat is typically selected from a country playing an important role in the particular industry served by the TC.

This is the first feudal characteristic of an organisation that is supposed to produce standards that boast the “international” adjective. The fact that a country with a prominent role in an industry is also in charge of making standards for that industry does not just sound right. Not only the country has a dominant role, hence market power, in an industry, but the country, by exercising the extensive powers that the directives assign to the secretariat, can ultimately control the standards of the domain as well.

Of course it is my duty to say that the ISO/IEC directives, which all elements of the ISO structure must follow, mandates “A secretariat shall act in a purely international capacity, divesting itself of a national point of view”. This is laudable, but it is an intention. Why put intentions to the test of reality and why not have someone less engaged in that particular field to act as secretariat? I am saying this because, typically, the cost of the secretariat is borne by the domain industry of that country. More intentions at work, more intentions to test.

One example of the “extensive powers” of the secretariat is provided by the ISO/IEC directives. The secretariats of TCs and their SCs have the power to nominate the chair of the TC or of the SC, to be formally approved by the layer above. This chair-secretariat dualism is made more questionable by the rules of some TCs that make it a custom, if not the rule, that the chair be from the same country as the secretariat. More intentions at work, more intentions to test.

Some examples

Here are some examples of personal experiences about feudalism in ISO.

At one JTC 1 meeting where I represented Italy, I introduced myself to a Subcommittee chair as MPEG Convenor because I thought MPEG should cooperate with that SC in matters of common interest. The chair looked down on me with the attitude of “how do you dare, you lowly Working Group convenor, talk to me, Subcommittee chair?”. Then, he condescendingly talked to me for a half a minute.

The second case is the assignment of a prize to the chair of an SC whose most notable achievement was to chair the shortest yearly meetings of the SC plenary. This is again feudalism at work: the foot soldiers bear the brunt, the officers get the medals.

Let me now briefly explain how the rules governing the organisation (the directives) are managed. The principal difference of a feudal regime versus a modern regime is the separation of powers. The legislative, executive and judiciary powers are what makes us proud of our statehoods.

Does ISO apply “separation of powers”? Not really. There is a kind of legislative power, embodied in the people who develop and maintain the directives. Then there is a kind of executive power (chairs and secretariats). But where is the judiciary power? The secretariat of the committee in which one operates. Not only have secretariats extensive executive powers, but also the power to decide whether what they and others do conforms with the directives. The directives are very rich in details about what everybody else should do and should not do, but very poor in details about what the executive power, i.e. a secretariat, can do. Then, of course, a higher layer secretariat can overrule a lower layer secretariat in a perfect implementation of a hierarchical organisation.

One fresh example. Imagine you have a divisive issue at a meeting that must be sorted out. Even those with limited experience know that you can get different results depending on how you ask a question. Well, you can well not believe me, but the secretariat has the power to decide which type of question the committee will vote. If you ask to record a statement in the minutes, they tell you that only the Committee Manager can write the report. So your statement gets unrecorded.

How can this organisation do the work it expected to do, then? Simple. ISO is populated by excellent people operating in Working Groups while the rest plays the power game in the higher layer, possibly getting medals. For the people in the Working Group the Japanese maxim applies: 出る釘は打たれる (The nail that sticks out gets hammered down).

Not everything is feudal

It is important to remark that important aspects of the ISO society are not feudal.

For instance, in the Holy Roman Empire there were kingdoms, duchies, margraviates, counties and the like. It could happen that a king wanted to take direct control of a margraviate and get rid of a margrave, say, because he was too successful in his drive to conquer the eastern territories or because he wanted to give a fiefdom to one of his vassals. Most likely the margrave would not agree to be taken over, claiming his achievements and the fact that he had complied with his feudal duties of allegiance. Undeterred, the king would send emissaries in hiding (called “missi dominici”) to bribe some ambitious vassals of the duke and get them to support the king. Then the emissaries would also sweet talk other vassals of the duke and would threaten retaliations on the vassals who were still not bending. Finally, the king could easily take over the remaining supporters by sheer military force.

Feudal is an attribute that befits ISO in a number of cases (and I will report more in the future), but not in this one. An ISO TC would never try to get control of a duchy or of a margraviate, sorry, I meant a SC or WG. That is because, more than 1,000 years after the Early Middle Ages and the abolition of serfdom, one can rightly expect to see ISO populated by people with a straight back who would never be convinced to switch allegiance by promises, bribes, sweet talks, or threats of retaliation.

A case like the one described above for the Holy Roman Empire is simply inconceivable in ISO.

Posts in this thread

Posts in the previous thread

List of all related articles