Top-down or bottom-up?

On 2020/06/02 SC 29, the parent group of MPEG, decided to erase the group. This was the result of a combination of reasons:

  1. One National Body (NB) wanted to retain the MPEG activity within SC 29
  2. Another NB craved to get control of international standards for digital media
  3. Other NBs were “convinced” that it was better to go this way
  4. Some individuals coveted “positions”
  5. Certain segments of the IP (Intellectual Property) industry were determined to perpetuate the outdated business model of “put any IP in the standard and let the market sort out the mess”
  6. Some entities could have been motivated by elements drawn from the content of of my papers exemplified by Some have it straightThe MPEG exceptionWhy is there a battle around MPEG? etc.
  7. and more…

I let the readers make their own opinion about the wisdom of making a decision supporting the desire of the two NBs unidentified above and commiserate those seeking positions while unfit, but I cannot avoid noting the stubbornness of those who persevere on a path that is marginalising the use of HEVC, and possibly other MPEG standards as well, and negatively impacting the MPEG name, while the use of AVC is universal.

I have no doubt that SC 29 will continue, by sheer inertia, the work that MPEG had under way before being erased. However, the long term prospects are not positive. Today I will analyse the following reason: as a Subcommittee, SC 29 will make top-down decisions as opposed to MPEG’s bottom-up consensus building.

For 30 years MPEG has developed standards starting from inputs coming from its members (individuals). Nothing came ever down from SC 29, i.e. from NBs. For instance in July 2019 a proposal called Video Coding for Machines was made whose main elements I had the opportunity to introduce. Some 100 people are currently working on that proposal.

Those who think that MPEG’s successful standards have been proposed by NBs have got it completely wrong.

In MPEG, a formal process used to start form an idea. If successful, the idea could become a candidate for a standard. It was an engaging process where everybody was entitled to bring arguments pro and con. If the idea survived and prospered, it would become the target of a call for proposals of technologies. The submitted technologies, typically revised by MPEG, would be converted to a standard. All the process was managed by a group called Requirements.

Those bringing a new idea to MPEG were like entrepreneurs trying to get funding for their startup. They could get “funded” (i.e. they could get to the point of issuing a Call for Proposals, finding the human resources believing in their idea and then developing the standard) but eventually the company (the standard) could very well not fly. Actually, only ~1/10 of all MPEG standards did actually fly.

Sounds familiar?

Conversely, the process in some NBs is murky. Often, the NB membership does not represent the national reality. Often, large multinational players dictate the agenda. Often, employees of large companies are the only ones with the time and money required to attend high-level committees where important decisions are made. Often, “instructions” come from the above “communicating” positions taken by higher level bodies and to be executed by lower layer committees. Often, the chair of a committee may bring personal positions into other committees, call “friends”, without informing the committee, to get their motion approved. Often, a higher level committee may take a position diametrically opposite to the position unanimously agreed by a lower level committee on matters affecting the lower level committee.

My answer to the argument “this is how business works” is “from whom would you buy a second-hand car?”

The MPEG model has proved its worth. Logically, if something works – like MPEG – it should not be changed, aka if it ain’t broke don’t fix it. If the people in some committee fix what is not broken for one of more of the reasons above, they should bear responsibility for the misdeed. But the system has been designed to operate exactly in such a way that people who made the decisions will never be held accountable.

If there will be a new MPEG – clearly not in ISO – it should operate exactly as the old MPEG used to operate. Everybody should be entitled to make proposals, bring evidence, argument on the goodness of their proposal, find others who share the view that the proposal is good, and fight those opposing their idea by the force of their arguments.

Nothing murky should get in the way.

A preview of things to come: the old MPEG Requirements group is the only group in SC 29 that has not a chair.

Posts in this thread

Posts in the previous thread

List of all related articles

 

This is ISO – a chaotic organisation

Human organisations that are created and develop based on a master plan discover that the world is more complicated than assumed by the master plan they started from. Sooner or later they must flexibly adapt the plan to the conditions on the field. Other human organisations start without a master plan and develop on the basis of actual conditions. Sooner or later they discover that the organisation has a dysfunctional structure that must be rationalised.

I was not in Geneva in 1947 when ISO was established, but I am pretty sure that there was no master plan for creating standardisation areas. This can be easily seen from the way Technical Committees (TC) were established, i.e. on an as needed basis.

TC 1: Screw threads, TC 2: Fasteners, TC 3: Limits and fits, TC 4: Rolling bearings, TC 5: Ferrous metal pipes and metallic fittings, TC 6 Paper, board and pulps, TC: 7 Rivets, TC 8: Ships and marine technology

Indeed, after responding to the needs of “Ferrous metal pipes and metallic fittings”, ISO created a TC for “Paper, board and pulps”. After addressing rivets, ISO responded to the needs of “Ships and marine technology”.

I believe this was the right things to do. You may attempt to create a rational picture of the industry (good luck!). Even if you succeed to develop one, a few years later the evolution of technology will make the picture obsolete.

ISO disbanded TC 3: Limits and fits and TC: 7 Rivets because they were no longer needed. Good that rivets were still in need of standards in 1941 or Jane would have had a problem.

In ISO as well as in the market, new TCs/companies are born and die. But there are also mergers and acquisitions, and spin offs. TC 97 Data Processing is a good example of this:

  1. Formed in 1960 with ANSI as secretariat and originally responsible for computers, peripherals, and computer systems.
  2. In 1981 TC 97 absorbed TC 95 Office Equipment.
  3. In 1983 SC 8, Numerical Control of Machines, and SC 9, Programming Languages for Numerical Control were spun off to create TC 184, Industrial Automation Systems.
  4. In 1984, TC 97 was reorganised to group similar activities with 3 Vice-Chairpersons coordinating the activities of groups of SCs with similar interests.
  5. In 1987 TC 97 merged with IEC/TC 83 Information technology equipment (and later IEC/SC 47 B Microprocessor systems) and became the Joint ISO/IEC Technical Committee on Information Technology (JTC 1).

From that moment on JTC 1 witnessed an impressive organic growth. Today JTC 1 includes 22 Subcommittees (SCs) and many Advisory Groups and Working Groups reporting directly to it.

The JTC 1 SCs cover a broad range of areas. Here are some examples: Character Sets, Telecommunication, Software and System Engineering, Security Devices, Digitally Recorded Media, Office Equipment, Digital Media, Security, IT for learning, Biometrics, User Interfaces, Cloud Computing, Internet of Things, Artificial intelligence etc.

Is this situation OK? I do not think it is, because JTC 1 is no longer a TC in charge of an area, it is a collection of many disconnected areas which do not have an obvious reason to stay together. Today JTC 1 does not even attempt to group areas as it commendably did in 1984 but then discontinued.

Let me give examples: What is the relationship of digitally recorded media with artificial intelligence? What is the relationship of IT for learning with internet of things? What is the relationship of software and system engineering with digital media?

In JTC 1 there is a mix of basic information technologies (e.g. Software and systems engineering, Programming languages), devices (e.g. Cards and security devices, Digitally Recorded Media), communication (e.g. information exchange between systems, interconnection of IT equipment), applications (e.g. IT for learning, biometrics) and many other areas without any thread connecting them.

There is another issue, and an even more serious one, because it affects the governance. Out of the 22962 standards produced by ISO, JTC 1 has produced 3249 standards, i.e. ~14% of all ISO standards. JTC 1 is the dominating TC in terms of standards produced (by the way, more than 500, i.e. ~16%, were produced by MPEG), but the actual membership in JTC 1 entities (SCs, AGs and WGs) accounts for probably 30% of the entire membership in ISO entities (TCs, SCs, AGs and WGs). ~80% of patent declarations received by ISO relate to JTC 1 standards (and ~71% of those declarations relate to MPEG standards).

Still JTC 1 has the same governance as TC 291 Domestic gas cooking appliances that has produced no standards so far.

I have already expressed my opinion about ISO governance in TCs and SCs.

JTC 1 Subcommittees as of June 2020

  • SC 2    Coded character sets
  • SC 6    Telecommunications and information exchange between systems
  • SC 7    Software and systems engineering
  • SC 17  Cards and security devices for personal identification
  • SC 22  Programming languages, their environments and system software interfaces
  • SC 23  Digitally Recorded Media for Information Interchange and Storage
  • SC 24  Computer graphics, image processing and environmental data representation
  • SC 25  Interconnection of information technology equipment
  • SC 27  IT Security techniques
  • SC 28  Office equipment
  • SC 29  Coding of audio, picture, multimedia and hypermedia information
  • SC 31  Automatic identification and data capture techniques
  • SC 32  Data management and interchange
  • SC 34  Document description and processing languages
  • SC 35  User interfaces
  • SC 36  Information technology for learning, education and training
  • SC 37  Biometrics
  • SC 38  Cloud Computing and Distributed Platforms
  • SC 39  Sustainability for and by Information Technology
  • SC 40  IT Service Management and IT Governance
  • SC 41  Internet of Things and related technologies
  • SC 42  Artificial intelligence

Posts in this thread

Posts in the previous thread

List of all related articles

 

A future without MPEG

Table of contents

  1. Introduction
  2. The situation today
  3. The competitors
  4. What is/will be available
  5. What the market offers
  6. Licensing of other media coding standards
  7. My views on the next steps of media coding standards
  8. A disclosure
  9. My forecast
  10. So, what will happen next?
  11. And now?

Introduction

My plan this week was to talk about the prospects of MPEG for the next few years. But there is a big news: MPEG passed away on 2020/06/02T16:30 CEST. The agony was long, but the result was predictable. The causes have partly to do with the nature of ISO whose main feature I describe in This is ISO – A feudal organisation.

What used to be the reference standards group driving close to 2% of the Gross World Product, i.e. 1.5 T$, and affecting the daily lives of billions of people is now reduced to splinters. The radical innovation of putting in one committee all media components, copied by most standards bodies and companies, is gone. Now, ISO media standards are treated separately. Godspeed, when someone will need to put the pieces together.

The passing away of MPEG is another hurdle added to a big hurdle on which I have published several articles: A crisis, the causes and a solution, Can MPEG overcome its Video “crisis”?, Business model based ISO/IEC standards, and IP counting or revenue counting?

After publishing those articles, I was forced to silence, but now it is time to speak again, this time loud and clear.

I need to make 3 strong statements:

  1. Those who have created a new intellectual object have to right to exploit it
  2. Technology innovation is well rewarded by a system that identifies, determines and protects intellectual property (IP)
  3. Those who make available their IP to a technical specification declared standard by an official standards body have additional benefits compared to those who do not. They should have more obligations as well.

The situation today

I will not spend much time recalling how companies who had their IP in MPEG standards have handsomely been rewarded. According to a knowledgeable person, whose statement I could not get confirmed, MPEG-2 patent holders shared revenues of 1 B$ p.a. during the validity of the MPEG-2 standard. Most IP holders have reinvested in new technologies that have feed the MPEG virtuous cycle of more than 180 standards including 6 video coding standards and 6 audio coding standards.

Unfortunately, only in fairy tales the story ends with “…and they all lived happily ever after”. MPEG did not live happily after MPEG-2 because it saw that most MPEG-2 IP holders held IP on the following (video) coding standards and had difficulty adjusting to the internet video distribution paradigm. Adding to that, the number of IP holders on HEVC has skyrocketed to ~45, 2/3 of which belong to one of the 3 existing patent pools and 1/3 belong to none. It should be no surprise that the HEVC standard has some use in broadcasting, but its use on the web is estimated to be at 12%. If one considers that broadcasting is a rich but declining market and video on the web is constantly rising, one understands that ISO standards will be gradually relegated to a more and more marginal market.

The competitors

When the MPEG-4 Visual licence sentenced the death penalty to the MPEG-4 Visual standard, competitors popped up. Four famous names are Real Networks, Microsoft, On 2 and Google. The fact that private companies could assemble a video codec of commercial value signaled that video coding was a maturing technology and that “anybody” could assemble a reasonably performing solution.

With many video coding patents reaching 20 years after filing, it was reasonable for MPEG to try and assemble a video coding specification that could be used without a licence or, if possible, with a free licence. The rationale for MPEG to do this was: do you want to let your customers be served by a competitor – and not see them back – or do you want to offer a solution that responds to the customer’s needs?

MPEG tried 3 different routes to achieve the goal

  1. Define a royalty free AVC baseline (called WebVC). Some AVC patent holders confirmed their royalty bearing patent declarations
  2. Develop a royalty free standard from scratch (called Internet Video Coding or IVC). Declarations from some companies that they held patents were received
  3. Develop a royalty free standard starting from an existing solution (called Video Coding for Browsers or VCB). A declaration from a company that it held many patents was received.

What did we learn from the 3 experiences? Of course that a company should not be forced to give away a patent for free (experience #1) but also that ISO rules allow a company to prevent a no-licence/free licence standard from happening by simply making a cautionary “I may have patents that I am willing to licence” declaration.

What is/will be available

A group of companies came up with an original proposal of making a standard that would avoid the uncontrolled flow of IP, sometimes of dubious value, with the attached burden of agreeing to a license. The proposal requested

  1. A two-layer coding scheme.
    1. The base layer must contain technology that is more than 20 years old or is accessible at no cost and provide a significant improvement on AVC.
    2. The second layer should provide significant improvement over HEVC
  2. Encouragement to declare that licencing terms will be published within two years after FDIS approval.

The Standard, called Essential Video Coding (EVC), has reached FDIS in April 2020. Final performance data are not available yet, however, it is expected that EVC base layer will be some 20% better than AVC and full EVC will be 30% better than HEVC. Licensing terms are not known yet.

In 4 weeks (July 2020) it is expected that Versatile Video Coding (VVC), the latest video coding standard, will reach FDIS.

To complete the line up of video coding standards, Low Complexity Enhancement Video Coding (LCEVC) is expected to reach FDIS. In appropriate conditions the combination of LCEVC and a generation N codec should provide a quality comparable to the quality from a generation N+1 codec.

What the market offers

The market has not been idle. In 2015 an initial group of companies (Amazon, Cisco, Google, Intel, Microsoft, Mozilla, Netflix) have created Alliance for Open Media (AOM), initially targeting video. The AOM about page mentions 4 main features of the first specification AV1:

  1. Royalty-free ecosystem
  2. Patent review process and legal defense fund
  3. Cutting edge technologies
  4. Collaborative open source development.

Licensing of other media coding standards

The information provided in this section is believed to be accurate. If you intend to make a business decision, however, please seek advice from counsel.

Video

No licensing of MPEG-1 Video is known. Still the standard was widely used.

MPEG LA developed the MPEG-2 Video licence. MPEG-2 was widely used.

MPEG-4 Visual had a hard life because there was an unsuitable licence for streaming, the main target of the standard.

MPEG-4 AVC is a very successful standard that can proudly bear the “generic” attribute because it is used for broadcasting and online streaming as well.

After 7 years, MPEG-H HEVC patent holders could not get their acts together and propose a decently unified licence. HEVC is used in broadcasting, however, use for streaming is limited at best. For years an MPEG officer tried to convince MPEG that HEVC was doing well in the market.

In 2017 I took the initiative of proposing to bring the situation to the attention of JTC 1. A document was prepared and agreed. I went to JTC 1 and presented it. However, a couple of countries objected on procedural grounds. Today, ISO is still officially unaware of the patent problem because of that opposition.

Audio

MP3 had a very simple and effective licence that allowed the standard to be widely adopted. The number of hardware and software MP3 devices is counted by the billions.

AAC has a patent pool with an effective licence as shown by the number of hardware and software AAC devices estimated to be 10 billion units.

MPEG-H 3D Audio has been around for a few years and adopted for ATSC 3.0. However, there is no known licence available. Recently a profile that is expected to contain IP just from one company has been started. There is hope that a licence for 3D Audio will be possible.

3D graphics

Past standards in this area did not have much traction. The opposite is expected for point cloud compression (PCC). The standard will become FDIS in a month time (July 2020). The number of patent holders is expected to be significantly less than HEVC.

My views on the next steps of video coding standards

The void left by HEVC has been filled by AOM with their AV1 specification which is widely used for streaming.

EVC is promising because it provides a quality that is comparable with or better than AV1, although less than VVC. EVC may have a chance if a licence will be published. However, this has not happened yet.

Most likely the number of VVC patent holders is much larger than HEVC’s. A Media Coding Industry Forum (MC-IF) was established in 2018 with the goal

“to further the adoption of MPEG Standards, initially focusing on VVC (Versatile Video Coding), by establishing them as well accepted and widely used standards for the benefit of consumers and the industry”

So far, however, no concrete results have been made known outside, if we leave aside the events organised at conferences. MC-IF has 31 members, 7 of which are licensing entities (i.e. a little less than ¼ of all members). The “industry” members account for just ½ of the HEVC patent holders. In these conditions, it is hard to believe that VVC will fare better than HEVC. It could very well fare worse because VVC adoption in broadcasting will take years, if ever.

The success of EVC could help VVC succeed. Seeing the threat of EVC (and AV1), VVC patent holders could get their act together and provide a decent licence. While I would welcome such a development, I consider it as having low likelihood.

A decent licence for LCEVC could be a game changer and another threat to VVC that could mean additional pressure on VVC patent holders to provide a single and decent licence.

A disclosure

I can finally disclose my strategy to save what used to be MPEG Video coding from disappearance:

  1. HEVC and VVC will go nowhere if left by themselves
  2. EVC will be a mezzanine standard between HEVC and VVC
  3. LCEVC on top of EVC can compete with VVC in terms of quality, in appropriate conditions
  4. If EVC will succeed, and patent holders will feel the competition and agree to a decent licence, VVC can have a chance.

Few people, if any, have understood that my goal was never to promote the success of EVC per se. By pushing EVC, I intended to create internal competition and thus promote the success of VVC. If that will not happen, loyal MPEG customers still have a chance to buy the only competitive MPEG video codec available – EVC. By adding LCEVC, they can get state of the art performance.

My forecast

My forecast is grey for the following reasons

  1. There is no longer a united MPEG, actually, MPEG does not exist anymore. That was inevitable, it was just waiting for people to do it because MPEG, with all its influence on the industry, was like a free city in the Middle Ages, ready to fall under the attack of powerful armies. As the Romance of Three Kingdoms (三國演義) said 700 years ago: 話說天下大勢.分久必合,合久必分 (in the world things divided for a long time shall unite, things united for a long time shall divide), there  will be division for a long time, say, around 30 years, the time MPEG has operated united.
  2. The putative head of SC 29 has not shown a minimum of vision for the future. This is a death knell because digital media is developing at an aggressive rate in a mature market. If ISO waits for matters to consolidate in the market, and then develop a standard, it will lose its time with the standard. MPEG has gambled for 30 years, sometimes winning, sometimes losing. What matters, however, is that the industry has not been left without standard technologies for its necessities.
  3. Most importantly, the market has come up with a solution – AV1 by the Alliance for Open Media – that is capable to provide decently performing specifications that are royalty free and promises a defence to an attacked member (with strings attached, but everybody looks at the next quarter, and does not consider the implications of a choice 5 years from now).

So, what will happen next?

If EVC/LCEVC will fly

There is a hope that VVC will also fly. Patent holders will get the expected flow of royalties and may feel assured that supporting the model still pays off and may be willing to further invest in the environment. As this, however, will only be seen in 2-3 years, patent holders will have to give the environment credit for another 2-3 years.

If EVC/LCEVC will not fly

I will consider five cases

  1. A miracle happens. VVC patent holders get their acts together and provide a decent licence. However, miracles are not very common because they indicate sainthood.
  2. NPEs/Universities/Industrial research try to keep the environment alive by pretending that industry will use the “standards” produced. This will last for some time until higher-level  management realise they are wasting time and money.
  3. The environment becomes a special “conference” (not a standards meeting) assessing “papers” (not contributions) to produce “documents” (not standards). The value of a “document” is in the “technology screening process” that assesses the “value” of technologies, so that those who want to make a specification (e.g. AOM) know that they are good screened technologies. This may save the NPE business model.
  4. The environment is in charge of maintaining the large number of standards. This may become difficult because the environment will not be attractive enough to experts who are ready to perform their duty maintaining old standards in exchange for some fun in new standards (that will no longer be there).
  5. The environment becomes the place where Publicly Available Specifications (PAS) suppliers, e.g. AOM, send their specifications to be converted to ISO standards. This will be the final sanction of the end of The MPEG exception. I hope not to be in this world anymore to witness such a degradation if that happens .

And now

Something is going to happen…

See the Russian translation Будущее без MPEG

Posts in this thread

Posts in the previous thread

List of all related articles

 

This is ISO – A feudal organisation

Introduction

The “International Organisation for Standardisation” – ISO – was established in 1947 on the ruins of World War II to accomplish one of the noblest activities that can be performed for humankind: making standards. I call this a noble activity because standards make the society of humans possible and better.

When I first encountered ISO in 1987, I found it unique because the national ISO counterparts, so-called National Standards Bodies, even in countries not particularly bent on it, gave senior positions to women. That characteristic, not so common in business at that time, largely continues today.

However, having operated in ISO for some 33 years, with many opportunities to see it at work, I have found positive characteristics along with aspects that are odd survivors of an age that most people I know would think is gone for good. Those surviving aspects, that in this paper I describe with the word feudal, still impregnate the organisation. It is a pity, and an inexcusable one, because the role of standards in a society that gets more numerous and integrated every day, is bound to increase.

Countries reject products that have been produced by, say, workers in unhealthy conditions, not because they are not good, but because they object to the way the products have been made. Many users are unaware of the way ISO standards are made. If they knew, they would probably question rules and customs that are a relic of the past today. I intend to provide elements of knowledge in the hope that things will change for the better.

I have two disclaimers. The first is that nothing of what I say in these articles is intended to apply to other standards developing organisations (SDO). While my experience in ISO is quite deep and pervasive, my experience of other SDOs is too shallow to attempt to characterise them in any way. What I know of some of them tells me they do not have the attributes that make, in my opinion, ISO unfit to the name it bears. The second is that what I say is intended to apply to ISO as the organisation registered in Geneva. They are not meant to apply, unless I will say explicitly so, to specific people working in the organisation or in its national counterparts.

The organisation

ISO is Non-Governmental Organisation (NGO). However, it is international in nature. It is hierarchical as depicted in the figure below.

It should not be a surprise that ISO is also bureaucratic. Many governmental organisations and private corporations are hierarchical and bureaucratic. The main difference here is in the way authority is exercised at each layer. This is what makes the functioning of the ISO structure look more like that of the Holy Roman Empire than of a company.

The ISO structure at work

A standardisation area is assigned to a Technical Committee (TC) managed by a secretariat. The secretariat is run by one or more employees from an ISO member body headed by a Committee Manager (used to be called Secretary) from the same member body. Secretariat is typically selected from a country playing an important role in the particular industry served by the TC.

This is the first feudal characteristic of an organisation that is supposed to produce standards that boast the “international” adjective. The fact that a country with a prominent role in an industry is also in charge of making standards for that industry does not just sound right. Not only the country has a dominant role, hence market power, in an industry, but the country, by exercising the extensive powers that the directives assign to the secretariat, can ultimately control the standards of the domain as well.

Of course it is my duty to say that the ISO/IEC directives, which all elements of the ISO structure must follow, mandates “A secretariat shall act in a purely international capacity, divesting itself of a national point of view”. This is laudable, but it is an intention. Why put intentions to the test of reality and why not have someone less engaged in that particular field to act as secretariat? I am saying this because, typically, the cost of the secretariat is borne by the domain industry of that country. More intentions at work, more intentions to test.

One example of the “extensive powers” of the secretariat is provided by the ISO/IEC directives. The secretariats of TCs and their SCs have the power to nominate the chair of the TC or of the SC, to be formally approved by the layer above. This chair-secretariat dualism is made more questionable by the rules of some TCs that make it a custom, if not the rule, that the chair be from the same country as the secretariat. More intentions at work, more intentions to test.

Some examples

Here are some examples of personal experiences about feudalism in ISO.

At one JTC 1 meeting where I represented Italy, I introduced myself to a Subcommittee chair as MPEG Convenor because I thought MPEG should cooperate with that SC in matters of common interest. The chair looked down on me with the attitude of “how do you dare, you lowly Working Group convenor, talk to me, Subcommittee chair?”. Then, he condescendingly talked to me for a half a minute.

The second case is the assignment of a prize to the chair of an SC whose most notable achievement was to chair the shortest yearly meetings of the SC plenary. This is again feudalism at work: the foot soldiers bear the brunt, the officers get the medals.

Let me now briefly explain how the rules governing the organisation (the directives) are managed. The principal difference of a feudal regime versus a modern regime is the separation of powers. The legislative, executive and judiciary powers are what makes us proud of our statehoods.

Does ISO apply “separation of powers”? Not really. There is a kind of legislative power, embodied in the people who develop and maintain the directives. Then there is a kind of executive power (chairs and secretariats). But where is the judiciary power? The secretariat of the committee in which one operates. Not only have secretariats extensive executive powers, but also the power to decide whether what they and others do conforms with the directives. The directives are very rich in details about what everybody else should do and should not do, but very poor in details about what the executive power, i.e. a secretariat, can do. Then, of course, a higher layer secretariat can overrule a lower layer secretariat in a perfect implementation of a hierarchical organisation.

One fresh example. Imagine you have a divisive issue at a meeting that must be sorted out. Even those with limited experience know that you can get different results depending on how you ask a question. Well, you can well not believe me, but the secretariat has the power to decide which type of question the committee will vote. If you ask to record a statement in the minutes, they tell you that only the Committee Manager can write the report. So your statement gets unrecorded.

How can this organisation do the work it expected to do, then? Simple. ISO is populated by excellent people operating in Working Groups while the rest plays the power game in the higher layer, possibly getting medals. For the people in the Working Group the Japanese maxim applies: 出る釘は打たれる (The nail that sticks out gets hammered down).

Not everything is feudal

It is important to remark that important aspects of the ISO society are not feudal.

For instance, in the Holy Roman Empire there were kingdoms, duchies, margraviates, counties and the like. It could happen that a king wanted to take direct control of a margraviate and get rid of a margrave, say, because he was too successful in his drive to conquer the eastern territories or because he wanted to give a fiefdom to one of his vassals. Most likely the margrave would not agree to be taken over, claiming his achievements and the fact that he had complied with his feudal duties of allegiance. Undeterred, the king would send emissaries in hiding (called “missi dominici”) to bribe some ambitious vassals of the duke and get them to support the king. Then the emissaries would also sweet talk other vassals of the duke and would threaten retaliations on the vassals who were still not bending. Finally, the king could easily take over the remaining supporters by sheer military force.

Feudal is an attribute that befits ISO in a number of cases (and I will report more in the future), but not in this one. An ISO TC would never try to get control of a duchy or of a margraviate, sorry, I meant a SC or WG. That is because, more than 1,000 years after the Early Middle Ages and the abolition of serfdom, one can rightly expect to see ISO populated by people with a straight back who would never be convinced to switch allegiance by promises, bribes, sweet talks, or threats of retaliation.

A case like the one described above for the Holy Roman Empire is simply inconceivable in ISO.

Posts in this thread

Posts in the previous thread

List of all related articles

 

Some have it straight

Developing standards is one of the greatest endeavours humans can do for their fellow humans. A well conceived and rightly implemented standard is the lubricant that makes society more efficient. Communication standards are even more valuable because they enhance the most powerful feature of humans to do things with other humans in a coordinated way.

In the 20th century public authorities and entrepreneurs vied in inventing new ways to communicate within a community to the exclusion of others. My best example is ITU-R Recommendation 624 “Television systems” where minuscule and large countries alike compete in adding footnotes stating that they reserve the right to adopt different frequency tolerances and the like.

MPEG was born in 1988 in reaction to the balkanised state of analogue media standards and to the attempts by countries and companies to carry over the analogue media paradigm to digital media.

The rest is history. MPEG standards have universal coverage, a result achieved without the intervention of public authorities because companies were convinced that a global market was not only larger than the sum of individual markets but also more efficient. Consumers reaped the benefits.

The reality of today, however, is that the MPEG model is not adopted in all markets and there is an interest on the part of some powerful parties to make the MPEG model a temporary exception to the rule, starting with “Nothing will change”.

If this will happen, it will not be because people have lost faith in the MPEG model, but because of ferocious arm-twisting behind the scenes.

Some people have a straight back and some have not. Some others get it only too well. Others simply don’t get it.

Posts in next thread

Posts in this thread

The MPEG exception

In the past 18 months, I have provided ample information on how MPEG was established, how it works, what it has done and is doing, and why it is different than other committees. In the last article Why is there a battle around MPEG?, I have identified another point of divergence. Patent declarations that relate to MPEG standards are 57.5% of all patent declarations submitted to ISO. MPEG is special because the next ISO committee in terms of number of patent declarations submitted to ISO has 13% share. JTC 1 itself has 81% of all patent declarations submitted to ISO.

In this article I would like to explore another difference, the nature of MPEG standards vs other JTC 1 standards.

Let’s start from an extended version of the table where the % refers to patent declarations submitted to ISO for all JTC 1 standards.

Committee Title # Pat. Decl. %
MPEG Coding of Moving Pictures and Audio 1857 69.76
SC 31 Automatic identification and data capture techniques 399 14.99
SC 27 Information security, cybersecurity and privacy protection 125 4.70
SC 25 Interconnection of information technology equipment 92 3.46
JPEG Coding of digital representations of images 80 3.01
SC 17 Cards and security devices for personal identification 56 2.10
SC 23 Digitally Recorded Media for Information Interchange and Storage 22 0.83
SC 6 Telecommunications and information exchange between systems 22 0.83
SC 34 Document description and processing languages 3 0.11
SC 35 User interfaces 2 0.08
SC 37 Biometrics 2 0.08
SC 28 Office equipment 1 0.04
SC 36 Information technology for learning, education and training 1 0.04

We see that Data (MPEG, SC 31 and JPEG), Security (SC 27), Communication (SC 25 abd SC 6) and Physical Media (SC 17 and SC 23) have a number of patent declarations expressed with more than 1 decimal digit.

This should be no surprise. At the risk of being criticised, I would state that the first 8 committees are not typical Information Technology (IT) committees, but Information and Communication Technology (ICT) committees because they handle information that does not only reside in a device, and hence is not typically the object of a standards, but is expected to move between devices. For instance there is no single patent declaration for SC 24 Computer graphics, image processing and environmental data representation.

Indeed, save for the first 8 committees in the table, the main target for standardisation in the other committees is architectures, frameworks, principles, guidelines etc.

Some JTC 1 standards are indeed developed outside of JTC 1. There are many organisations who request, e.g., JTC 1 to become Suppliers of Publicly Available Specifications (PAS). Those organisations who are granted that status have a preferential channel to make their specifications International Standards.

The Virtual Reality Modeling Language (VRML) was a PAS. JTC 1/SC 24 took VRML97 and made it ISO/IEC 14772-1:1997 Information Technology — Computer Graphics and Image Processing — The Virtual Reality Modeling Language — Part 1: Functional Specification and UTF-8 Encoding.

The justification for a making an ISO standard from a PAS is that the PAS can be developed with greater freedom and at a higher speed. The ISO process puts an ISO stamp on the PAS without changing the specification. Again at the risk of being criticised, I think that standards are too important to make them international standards by letting them cut corners.

MPEG has received a limited number of PAS with a request to convert them to ISO standards, but done so for none. MPEG has issued calls to which  some organisations that had produced PAS’s have responded. MPEG has developed standards based on those responses. An example is MPEG-M to which the Digital Media Project has been a major contributor. This is fine. The way technology is developed should be unconstrained, but there should be a single route through which technologies become international standards.

The Alliance for Open Media (AOM) specification known as AV1 is a PAS. One day, but not today, AV1 may became an MPEG standard.

MPEG is an exception, the question is whether it will continue to be so.

Posts in next thread

Posts in this thread

Why is there a battle around MPEG?

View powerpoint presentation

MPEG is known for a variety of reasons. It is the group that carried over analogue television and made it digital, multiplying by orders of magnitude the number of channels, it opened the way to new business models for audio, it enabled carriage of digital media on the internet, added media to the mobile experience, is poised to make immersive media real and genomics affordable, and more.

MPEG achieved these goals – and keeps on setting, working on and achieving more goals – with technical specifications of minuscule material or immaterial things that have a big impact on devices and service delivering infrastructures that are worth billions of USD.

I never tire of saying that every year products that rely on MPEG standards to function or to be attractive to buying global customers are worth more than 1 trillion USD. Similarly for services, which are worth more than 500 billion USD p.a.

There is something emblematically immaterial that has driven the success of MPEG – Patents, the engine that has allowed the MPEG machine to pile up records.

When MPEG developed its first and second standards it could draw from decades of research in audio and video coding. MPEG-1 and MPEG-2 were extremely successful and patent holders were handsomely rewarded. This encouraged more researchers and companies to file more patents so that, when MPEG engaged in new generations of coding standards it could draw from a wider and fresher set of technologies.

The story is not without hiccups. Overall, however, it is a success story that puts MPEG on a different league than any other ISO committee. This is shown by the figure below

You see that patent declarations made to ISO by entities believing they hold patents relevant to MPEG standards are 57.5% of ALL patent declarations received by ISO. The next committee is JTC 1/SC 31 Automatic identification and data capture techniques which totals less than ¼ of MPEG patent declarations (and it is a Subcommittee). All other committees mentioned in the table are JTC 1 SCs. Other ISO TCs have typically a low or even zero number of patent declarations.

You can get the data yourself from the ISO website.

MPEG should stay as the defender of a “business model” that has offered a relentless expansion of the business to all facets of the media industry and to consumers the possibility to enjoy newer and fresher experiences.

Probably you can answer yourself why there is a battle around MPEG.

Posts in next thread

Posts in this thread

Strategic planning for MPEG

In Leonardo: who I am, what I did so far and what I plan to do next, I have given some information on myself and listed my challenges in standardisation. In this article I intend to provide some more details about the challenges of a strategic planning for an organisation as special as MPEG. This is the list

  1. Uphold the role of MPEG as the unmastered and non-discriminatory source of digital media standards
  2. Expand the use of MPEG standards to insufficiently reached market areas
  3. Keep MPEG constantly abreast of technology and market development
  4. Develop standards in emerging fields of MPEG’s traditional media domain
  5. Collaborate with other committees by developing joint standards when mutually convenient
  6. Cultivate new and enhanced relationships with industry, MPEG client industries and academia
  7. Enhance the MPEG image and standing across the industry
  8. Reach out to other domains for which data compression may be a business booster
  9. Establish an ongoing process to assess MPEG’s relevance in fulfilling its mission
  10. Defend the MPEG business model by adapting it to changing conditions
  11. Define and establish a management-level succession process based on community preferences.

Uphold the role of MPEG as the unmastered and non-discriminatory source of digital media standards

Most ISO committees are the direct expression of a particular industry. Most secretariats are the expressions of the policies of the country in which the secretariat is located. MPEG was established having the opposite in mind, namely a committee that would develop standards that were industry and country agnostic. The first standard MPEG-1 targeted products and services for the Telecom and Consumer Electronics (CE) industries, the second standard MPEG-2 added the broadcasting industry. The third standard MPEG-4 added the Information Technology (IT) and mobile industries.

For 30 years MPEG has been remarkably able to keep its standards country-, industry-, company-, individual-independent. What will happen if, say, Information Technology took control of MPEG? What will happen if a major country took control of MPEG? What will happen if an alien business model took control of MPEG? What will happen if a major company took control of MPEG?

These are very serious questions and I am not sure if the global industry and world population would like the answer to these questions.

There is a second aspect, somehow related with the above, that need to be made explicit. This is the need to keep MPEG as the non-discriminatory source of digital media standards. For someone who has operated in the unmastered world of MPEG for many years, an MPEG standard is the result of the efforts of those who wanted to participate in the endeavour that produced the standards. No one should be denied participation. This does not mean that anybody can actually add technology to a standard, because MPEG is an extremely competitive environment striving to produce the best. Therefore only the best technologies are selected.

Once the standard has been produced, it should be accessible to all those willing to pay for the benefit of using excellent technology.

The idea that only some qualify to contribute to the standard or to use the standard seems to take hold in some quarters. This is an idea that MPEG should fight. MPEG should be unmastered and not discriminate against anybody.

Expand the use of MPEG standards to insufficiently reached market areas

I am not negative to the fact that in some market areas MPEG standards are not used to the extent the quality of the technology would promise. The worst that can happen to an organisation is that it occupies all the space and its members are left without goals to achieve. For instance digital cinema does not use MPEG standards and production uses only partly MPEG standards. These, however, are not real concerns because they are markets where devices or applications are counted by the thousands.

The real concern is video distribution over the internet, because distribution is what MPEG is mostly about, but MPEG never had an easy life in this segment. In the early 2000 there were companies eager to try video and audio distribution on this new medium using MPEG-4 Visual. No way, the MPEG Visual licensing terms were disappointing. MPEG-4 AVC fared definitely better with business-friendly licensing terms and a single patent pool. AVC is universally used even though competitors like P8 and VP9 did emerge. The problem is that AVC is a 15+ year old technology and what was due to be its successor HEVC is definitely underperforming with reportedly only 12% market share. Some blame licensing terms and multiple patent pools and a large number of patent holder not belonging to any patent pools. MPEG has developed DASH and CMAF and can offer the full protocol stack with its media standards. The streaming market, however, is taking different directions..

Keep MPEG constantly abreast of technology and market development

There is not doubt that MPEG knows how to stay abreast of technology development. Almost all of its standards are anticipatory of products, services and applications to come. Currently MPEG is engaged in several technology-deep investigations. One is Video Coding for Machines (VCM). VCM is expected to be the standard that allows remotely placed cameras to extract machine-understandable information, compress it and send it to a machine to use. The problem is that machines can be good for routine work, but what if something happens remotely and a human must intervene? The same information designed for machines should be usable by a human, possibly with the best quality that regular video coding can yield. Another exploration is video coding based on deep neural networks (DNN). We already know that some tools used e.g. in VVC could be replaced with more efficient DNN-based tools. The idea is to investigate the improvement of DNN based video coding technologies – both traditional schemes with some tools replaced or end-to-end DNN coding systems – over existing MPEG standards.

This is all good and no one can blame MPEG for developing the standards that have brought industry convergence, catered to its continuous development and made media a daily continuous experience. However, MPEG, as a group, is not aware of what is happening in the market. This should change. The idea is to create a Market needs group, the counterpart of the technology driven Requirements group. By having these two groups interacting we can still rely on a technology-based forward looking group mitigated by a market-savvy group, as depicted in the figure below.

Develop standards in emerging fields of MPEG’s traditional media domain

This strategy element confirms what we have been saying all along. MPEG is well aware of what the world of digital media technology can offer because is members are for most part (~3/4) from industry and for a smaller part (~1/4) from academia. The core of MPEG customers is interested in media standards and MPEG has much to offer in terms of standards that are close to availability or still under investigation, e.g.

  • Video-based Point Cloud Compression (V-PCC)
  • Graphics-based Point Cloud Compression (G-PCC)
  • Neural Network Compression
  • MPEG Immersive Video (MIV)
  • MPEG Immersive Audio (MIA)
  • Session based DASH operation
  • MPEG-I Scene Descriptions
  • Low Complexity Video Coding Enhancements
  • Deep Neural Network (DNN) based Video Coding
  • Video Coding for Machines

And many other standards.

Collaborate with other committees by developing joint standards when mutually convenient

In companies and standards alike the NIH (Not-Invented Here) principle rules. This is understandable, because if you have been working in a field for long enough you think you know everything in your field. The problem is that in our age technology is fluid. Something that is developed for a purpose, might equally well be usable for another purpose. I mention a case that MPEG is aware of where technologies for video coding or media file format have been successfully adapted for use in compressing the reads from high speed DNA sequencing machines. MPEG should embrace collaboration with other groups even more than it has done so far. Three examples are Big Data, for which MPEG has already developed Network Based Media Processing, Artificial Intelligence for which MPEG is developing Neural Network Compression and investigating AI-based video coding and Internet of Things for which MPEG is working on extending its Internet of Media Things standard keeping an eye on Video Coding for Machines.

Cultivate new and enhanced relationships with industry, MPEG client industries and academia

MPEG has not been shy to make the first step approaching the industry. MPEG can wave the flag of being a committee that is agnostic to countries and industries, but this comes at a high cost because your work can fall in a vacuum.

For years MPEG has been the merchant who devised all sorts of tricks to get the attention of different industries through their fora. The efforts have been rewarded: MPEG regularly talks with the main bodies representing broadcasting, fixed and mobile telecommunications, and package media at the international and regional level.

MPEG has also aggressively sought to collaborate with other bodies in the development of common standards, the best examples being the common video coding standards developed with ITU-T. MPEG has developed standards on commission from other bodies such as JPEG and 3GPP.

MPEG can also boast to have an extended community of followers in industrial research and academia. Indeed ¼ of its members are from academia.

Is this enough? No, it used to be enough, but it is no longer sufficient. MPEG makes in advance investigations that should involve academia and MPEG needs in advance information on the technologies that generate the data that it will be asked to reduce to a manageable form. It is a new form of deeper liaison that should be developed and implemented. Then, of course, you never talk often enough with your customers…

Enhance the MPEG image and standing across the industry

MPEG has a unique brand and its logo is universally known, but the fact that there are market segments where MPEG don’t play the role they could is an indication that the MPEG imaged must be beefed up. Of course, I am not talking of a marketing exercise, but of something that goes at the roots of what MPEG is: the principles that guide its activity that have kept making bigger the lowest organisational unit in ISO; its business model that guarantees that there is always new technology coming in because there is old technology that is being remunerated; its structure that combines flexibility with identification of responsibility; uniquely competent and dedicated personnel; a modus operandi that is the result of decades of honing; the academic and industrial network that has MPEG as its hub; the experience accumulated over the years and the ability to adapt itself to new conditions; and its technical and market achievements.

MPEG must define this multi-threaded presence and task a revamped Liaisons and Communication group to establish and maintain it.

Reach out to other domains for which data compression may be a business booster

The proposal to identify data compression as one of the axes of the information society failed, but the idea that generated it is pretty much alive. Countless application domains generate data: media, industry 4.0, health, automotive to mention a few. Most of these data are simply not used or seldom used because there is no economic means transmit, store or process them. The MPEG-G standard proves that MPEG technologies can be used to make an application domain flooded with data more efficient.

Establish an ongoing process to assess MPEG’s relevance in fulfilling its mission

MPEG likes to boast the fact that it is a working groups but it has produced more standards than any other JTC 1 subcommitee. Great, one could say, but that statement could be compared to the statement of a book publisher who publishes more book than its competitors, but its books pile up in its warehouses.

Fortunately that is not the case. Products and services that use in a determinant way MPEG standard are worth 1.5 trillion dollars a year.

This does not mean that the efficiency of MPEG has reached its limits. Before I have mentioned the need to create a Market needs group to compensate the exclusive technology driver of technology in MPEG. That, however, is not sufficient. MPEG must be able to answer the question: why did a standard A on which so many efforts have spent did not have the adoption that it deserved? Was the idea wrong? Were the requirements wrong? Were the technologies used in the standard unsuitable? Was profiling wrong? Was it because there was another better solution on the market? If so why were we not able to add the right requirements? Was it because the licensing terms were rejected? And you can add more.

If we will be able to answer these and other questions we will probably not be able to save the standard, but we can probably inject into our standard development process feedbacks that we got from unsuccessful standards.

Defend the MPEG business model by adapting it to changing conditions

I continue to believe that the MPEG business model of making good standards that reward good technologies is the best in the context of a rapidly evolving technology scenario. It is not the only one, however. You can develop option 1 (a.k.a. royalty free) standards that only use 20+ year old technologies or you can develop royalty free standards by buying a set of patents that allow you to build a codec with a performance that is sufficient for your needs etc.

It is clear that the MPEG business model is under threat and that something must be done to defend the MPEG business model – the only one that can provide the best standard at a given time – against those threats.

There are probably several ways. One has been attempted with Essential Video Coding (EVC). Under changed conditions, it is the same approached used in MPEG-2 times. Backward Compatible (BC) MPEG-2 Audio was not faring well, so a Non-Backward Compatible (NBC) MPEG-2 standard was developed in the hope that those engaged in BC would try harder to counter NBC. Eventually BC people lost but MPEG gained because MPEG-2 AAC is the progenitor of the extremely successful MPEG-4 AAC standard.

EVC is clearly a competitor to HEVC. If HEVC will have a good licence EVC will fail, but if HEVC continues have a complex situation, EVC will win. In both cases MPEG will gain.

Define and establish a management-level succession process based on community preferences.

MPEG is not a hierarchical organisation. This does not mean that it does not need leaders. ISO gives the secretariat a lot of power in nominating leaders. This may be good in other areas but it is not the ideal solution for MPEG. Such an ideal solution may not be a plain election, but for sure the community should have a say in how leaders are appointed.

Posts in next thread

Posts in this thread

Leonardo: who I am, what I did so far and what I plan to do next

I have written this article because I thought that, at this particular juncture, it would be useful to tell more about myself. I assume I am not unknown to quite a few people, but I think that it could be useful to put in a succinct form some information that summarises the responses to the who and what in the title.

Not that such information is “hidden”. Over the years I did spend some time to document some of the endeavours I happened to be part of or initiated or managed. Although a little out of style, https://www.chiariglione.org/ can be taken as the starting point of a navigation, in particular my web page and Riding the Media Bits.

Leonardo, the man

I am an Italian citizen by birth and a convinced supporter of the vision of a united Europe since my high scool days. However, I like to think I am a citizen of the world. I live in Villar Dora, province of Turin. Google maps provides a view of my house with a glimpse of the vineyard surrounding it. Originally, the house was the summer house of my grandfather, a farmer. At that time all farmer houses in town had a vineyard grown to produce grapes to make wine. There are still some farmers in town, but no one is left with a vineyard. While I am no farmer, I still produce some wine for my consumption.

I am father of three kids who have left the nest some time ago. My kids have 3 children, two girls and one boy.

Leonardo’s education

I graduated in Electronic Engineering from the Polytechnic University of Turin. At that time Italy had just one academic degree, equivalent to today’s Master’s degree. Before my graduation, I applied for a Japanese government scholarship, then went to Japan and got a Doctor’s degree in Electrical Communication from the University of Tokyo. I am told I was the first Caucasian to get a Doctor’s degree from the University.

Leonardo’s work experiences

CSELT, at the time the research centre of what is called today TIM-Telecom Italia, hired me to do research in video coding for visual telephony. When I left the company in 2003, I was Director of the Multimedia Division with 30+ researchers. Throughout my career at CSELT, I was involved at different levels of responsibility – leader, participant or manager of resources – in some 60 collaborative research projects, mostly at European level. This table provides an overview of all the research projects I was involved in.

Since 2003 I am the CEO of CEDEO.net, a technology company in Turin developing advanced services and products. The main services are WimTV and Stream4U. The main product is TVBridge, based on MPEG-A’s Multimedia Linking Application Format. The main product under development is DNASearch, based on the MPEG-G standard. Since 2011, I am also the CEO of WimLabs, a sister company of CEDEO.net focused on the commercialisation of products developed and the provision of services enabled by CEDEO.net. The WimLabs website provides an overview of all products and services commercialised by WimLabs.

Leonardo’s standards experiences

Over the years I participated in various standards committees. At the European level I joined CEPT committees (before ETSI was established) and then the ETSI JTC where I helped to create the path for acceptance of the MPEG-2 standard in Europe, before DVB had even started.

At the international level, I joined ITU-T and ITU-R committees. In particular I joined the first few meetings of the Okubo group who developed H.261 recommendtaion. I attended ITU-R meetings especially to promote adoption of the MPEG-2 standard. I was/am the Head of the Italian Delegation to JTC 1, SC 2 and SC 29. I attended JTC 1 meetings on several occasions, starting from the time I lobbied to convince SC 2 and then JTC 1 to create SC 29. I am the Chairman of the Italian SC 29 committee.

I founded and held the rapporteur role of the Moving Picture Experts Group of SC 2/WG 8 (1988-90) and the convenor role of SC 29/WG 11 (1990-) under SC 2 and then SC 29.

I founded and chaired DAVIC – Digital Audio-Visual Council (1994-95), FIPA – Foundation for Intelligent Physical Agents (1996-1988), OPIMA – Open Platform Initiative for Multimedia Access (1997- 1999) and DMP – Digital Media Project (2003-2015). I was also the Executive Director of the Secure Digital Music Initiative (1999-2001), an industry forum with the mission to develop specifications for secure digital music delivery.

Leonardo’s academic experiences

These are my academic experiences

  1. In 1986 I founded the International Workshop on HDTV which I chaired until 1994. I was the editor of all Proceedings of the HDTV Workshop until 1994.
  2. In 1989 I founded and was the Editor-in-Chief of Signal Processing – Image Communication, a EURASIP journal, until 1999.
  3. In 2004-2005 I was Professor at the Information and Communication University in Daejeon, Korea.
  4. In 2011 I was the Editor of the book The MPEG Representation of Digital Media published by Springer Science.
  5. I have organised a total of 18 conferences related to digital media, including one Picture Coding Symposium and one Packet Video Workshop,
  6. I have written some 200 technical and strategy papers, mostly invited, some submitted to technical journals and some delivered at conferences.

In 2002 I received the Doctorate “honoris causa” from Instituto Superior Técnico (Lisbon) and the EURASIP Meritorious Service Award. I am Chevalier de l’Ordre des arts et des lettres (France) since 2003 and Membre de l’Académies des technologies (France) since 2008.

Leonardo’s awards

I have received several awards (the text of the awards can be found here):

  1. the Information and Communication Society of Japan Award (1995)
  2. the International Institute of Communications Award (1997)
  3. the Society of Photo-Optical and Instrumentation Engineers Award (1998)
  4. the Kilby Foundation Award (1998)
  5. the IEEE Masaru Ibuka Consumer Electronics Award (1999)
  6. the IBC John Tucker Award (1999)
  7. the Edward-Rhein Foundation Award (1999)
  8. the SMPTE James Leitch Gold Medal Award (2002)
  9. the NAB Award of Honor (2003)
  10. the Charles F. Jenkins Lifetime Achievement Award (2018).

I am Honorary member of SMPTE since 2014.

Leonardo’s vision

In 1986, when the CCIR failed to approve the HDTV recommendation, I called the technical and business world and established the International Workshop on HDTV. I chaired the Workshop until 1994. The Workshop attracted hundreds of participants and acted as a technical forum that enabled eventual convergence of diverse regional views on the next form of television experience.

In 1987 I proposed the creation of an experts group on moving picture coding. The original idea was stimulated by my years of research in video coding and telecom standardisation that did not have an impact on the market. I tried to find a way to make available video communication end device by blending the manufacturing capability of the consumer electronics industry with the infrastructure of the telecom industry and the content offer of the broadcasting industry. All this crossing what were at the time strong industry barriers. In 1988 this idea became the Moving Picture Experts Group (MPEG). Today MPEG has become the established source of digital media standards that have caused industry convergence and made digital media available to billions of people. MPEG has received several Emmy Awards.

In 1994 I launched the Digital Audio-Visual Council (DAVIC), an industry forum with the mission to promote the success of emerging digital audio-visual applications and services, by the timely availability of internationally agreed specifications of open interfaces and protocols that maximise interoperability across countries and applications/services. DAVIC reached a membership of more than 200 companies and developed two editions of its comprehensive specifications that were transposed to ISO/IEC standards (the ISO/IEC 16500 suite).

In 1996 I launched the Foundation for Intelligent Physical Agents (FIPA), a non-profit organisation with the mission to promote the development of specifications of generic agent technologies that maximise interoperability within and across agent-based applications. FIPA developed 2 editions of its specifications. An open source software implementation of the FIPA specification (JADE), developed by one of my collaborators at CSELT, is used in several application domains.

In 2003 I launched the Digital Media Project (DMP), a not-for-profit organisation with the mission to promote continuing successful development, deployment and use of Digital Media that respect the rights of creators and rights holders to exploit their works, the wish of end users to fully enjoy the benefits of digital media and the interests of various value-chain players to provide products and services. DMP developed 3 editions of its specifications which were implemented in open source software (Chillout). DMP members proposed and made major contributions to MPEG-A’s Media Streaming Application Format (ISO/IEC 23000-5) and to MPEG-M suite of standards (ISO/IEC 23006).

Leonardo’s challenges

I see my next challenges in two domains: in my companies and in standardisation.

The challenges for my companies are to make WimTV the natural choice for those who want to do business with video on the web, TVBridge the normal way for broadcasters to augment their viewers’ experience and DNASearch the indispensable companion of all humans following their lives from cradle to grave.

I am sure that I can meet the challenge because I can rely on the support of my family and CEDEO personnel.

My challenges in standardisation are to:

  1. Uphold the role of MPEG as the unmastered and non-discriminatory source of digital media standards
  2. Expand the use of MPEG standards to insufficiently reached market areas
  3. Keep MPEG constantly abreast of technology and market development
  4. Develop standards in emerging fields of MPEG’s traditional media domain
  5. Collaborate with other committees by developing joint standards when mutually convenient
  6. Cultivate new and enhanced relationships with industry, MPEG client industries and academia
  7. Enhance the MPEG image and standing across the industry
  8. Reach out to other domains for which data compression may be a business booster
  9. Establish an ongoing process to assess MPEG’s relevance in fulfilling its mission
  10. Defend the MPEG business model by adapting it to changing conditions
  11. Define and establish a management-level succession process based on community preferences.

I am sure these standardisation challenges can be met. My confidence comes from the fact that MPEG is a large organisation that relies on the best and most dedicated experts a standards committee can hope to attract, has a solid organisational structure, enjoys an excellent reputation, and boast a universally known brand and loyal following both in academy and industry.

MPEG Future has published a vision that includes standardisation goals and an assessment of MPEG’s organisational capabilities. Industry is actively reacting to MPEG Future’s vision.

Yes, there will be challenges. There have always been challenges in the last 32 years. The next may be the biggest ever, but I think I know how to face them, as a group.

Posts in next thread

Posts in this thread

The value of MPEG standards

Introduction

MPEG can certainly be proud of the valuable standards it has produced: MPEG-1, MPEG-2, MPEG-4 AVC, MPEG-4 File Format and, more recently HEVC, MMT, DASH and many more.

Why are MPEG standards valuable and what can we do to add more value to them?

Why are MPEG standards valuable?

There is no single answer to this question. So, let’s analyse the causes that make MPEG standards valuable. While we are at it. later we wik let’s also analyse why some standards. on which great hopes were laid, did not deliver.

MPEG-1 (Systems, Video and Audio layer II)

The standard was designed for what would be the first device handling digital media, was interactive and and would serve the telecom and Consumer Electronics (CE) markets. Eventually that device did not fly, but the technology that made it possible was later reconfigured to make a “lower class” DVD player. Success was incredible, with a total of 1 billion devices manufactured by many companies.

MP3 (MPEG-1 Audio layer III)

MP3 turned out to be an incredible success from a totally unexpected quarter. When MPEG-1 was approved in 1992, MP3 offered transparency at 192 kbit/s for stereo music, ¼ less than MPEG-1 Audio layer II at 256 kbit/s (MP2) and at ½ of the MPEG-1 Audio layer I bitrate of 384 kbit/s (MP1). Complexity, however, was daunting (at that time). While MP2 and MP1 were implemented in ICs for mass market products, in the first years MP3 was implemented with sparse logic for professional equipment. Until, I mean, the impossible happened: software implementing the MP3 decoder in real time on a PC became available, non-real time encoding software became available, ripping a track from a CD became an OS feature of the PC and RIO became the first portable MP3 player with the size of a deck of cards. Most importantly a US court ruled that the RIO player was an IT, not a CE device.

With MPEG-G MPEG is taking the shot by addressing a conceptually similar situation as music tracks on CD converted to compressed digital 20 years ago. MPEG-G can potentially make the output file of a sequenced DNA “liquid” because it reduces a file of a few un-processable terabytes to a few tens of processable gigabytes. The difference is that the file of a sequenced DNA is a professional object, even though it can be produced in billions of instances and thus it looks like a consumer object.

MPEG-2 (Systems, Video and Audio)

MPEG-2 was a planned success. The early 1990s was a case of Brownian motion where particles (companies) were scrambling to come out with a new generation of TV. Japan with its MUSE and Europe with its HD-MAC were analogue by definition, and the USA put some order in its industrial activities with the Grand Alliance. However, all manufacturing companies, already global at that time, pushed for a digital solution. MPEG, the new kid in the block, announced it would develop a digital television standard and did so. The world followed.

There is much to learn from MPEG-2 because it is a standard that addressed the common needs of broadcasting industries – terrestrial, satellite, cable and package media – that until that time did not have much in common, if not the television experience eventually delivered to the end user. MPEG-2 was the catalyst of media industry convergence.

The development of the MPEG-2 4:2:2 profile was a natural follow-up and one that was constantly implemented in the video coding standards following MPEG-2 (including MPEG-4, RGB etc.). The 4:2:2 profile was a step itoward another industry – production – that has never been entirely fascinated by the MPEG offers.

MPEG-4 Binary Format for Scenes

MPEG-4 was the project that was meant to provide digital audio and video for yet another customer, the IT industry, actually the ICT industry, because video – and audio – for mobile was an important target from early on. BIFS had as foundation the Virtual Reality Modelling Language (VRML), a specification of the computer graphics world. MPEG extended VRML to support efficient transmission and real time. BIFS did not fly, and VRML, too, did not fly. Fascination with XML prompted both MPEG and VRML to introduce XMT and X3D, respectively. But success did not come. A later specification developed by the Korean Digital Media Broadcasting (DMB) made use of a relatively small portion of BIFS for complementary low bitrate services to digital audio broadcasting.

BIFS is an example of how commonality of technology between two radically different industries can well lead to no synergy. The origins of computer graphics are rooted in professional users using packaged content. The intended users of BIFS were consumers using real-time content via the network. The battle for the introduction of the technology was fought independently (it could hardly have been otherwise). Unfortunately, it was lost – independently.

MPEG learned that you cannot develop a standard in a top down fashion. Vision is needed, but vision must constantly face reality.

MPEG-4 Visual and Advanced Video Coding

MPEG-4 Visual inherited the vision of 2D visual objects in a 3D space. MPEG-4 Visual was a standard that supported moving 2D object in addition to extend the performance of MPEG-2 Visual. Industry needed the standard to enable services on the web and mobile. The hopes of a licence of MPEG-4 Visual patents were dashed when the licence included a charge based on the time an MPEG-4 Visual decoder was used.

MPEG-4 Visual was the sacrificial lamb that allowed MPEG-4 AVC to get a better deal in the use of relevant patents. Higher compression and acceptable licencing terms made AVC the universal video codec that it still is today.

MPEG-4 Audio

The MPEG Audio fortunes have different stories to tell. MP3 was and is an outstanding success. MP2 has been a good success and MP1 had a short span of use in the Digita Compact Cassette. MPEG-2 Audio did not have many takers because the performance was not adequate in a multi-channel environment. MPEG-2 AAC is in use in some countries. More importantly, MPEG-2 AAC it paved the way to MPEG-4 AAC.

MPEG-4 AAC is an outstanding success, with probably 10 billion devices in current use for all sorts of applications. Why? One element is the performance of the technology, another is the choice made by Apple to base their web-based music distribution service on MPEG-4 AAC. Yet another is the continuous performance improvement that has delivered MPEG-4 HE-AAC v2.

Open Font Format

OFF is a great example of standard development not for an industry forum but for a group of companies who had developed a successful specification that they were no longer keen on maintaining. In a sense, still, that was a request from an industry, conveyed by a National Body, that MPEG gladly accepted. OFF is now an extremely successful standard. This success is not entirely of MPEG’s doing, but the amount of extension MPEG has added to the original OpenType specification is such that MPEG can say that OFF is an MPEG standard of great success.

MPEG Media Transport (MMT)

MMT is an example of a solution that answers the questions that many (terrestrial) broadcasters ask: how can I face the competition of services on the web? The answer could have been strengthening broadcast delivery with some magi. Instead MMT offered the means to move broadcasting services, if not to the web, at least to the transport technology that enables the web . In this way it becomes easy to integrate broadcasting services with interactive services. The standard is deployed in several important countries, but acceptance of MMT is not universal. Why? Broadcasting is a conservative world and highly influenced by political considerations. Interestingly, streaming services that use MMT can offer short delay. However, there is still a limited use of MMT for streaming services.

High Efficiency Video Coding

HEVC continues the tradition of MPEG Video compression standards in terms of technical achievements. The HEVC Verification Tests showed that on average HEVC delivers the same quality as AVC with 40% of the bitrate required by AVC. MPEG’s traditional customers have jumped on HEVC which is widely deployed in their devices and services. A concern is the low level of adoption in online services which is rated at ~12%.

3D Audio Coding

This standard, part of the MMT and HEVC package in MPEG-H has been adopted by several broadcasting agencies. Its use has been announced by other industry sources also for distribution of high quality immersive audio via the web.

DASH

DASH is a remarkable example of how MPEG is able to respond to the need of a client industry, in this particular case 3GPP, representing the mobile telecommunication industry. MPEG developed DASH with a close connection with 3GPP. Since the standard has been developed, DASH has seen a broad acceptance by the industry.

Adding value to MPEG standards

The incomplete summary above shows that MPEG can develop high quality standards, the best in terms of performance in its area of work. Its standards do respond to industry requirements thanks to MPEG’s network of liaisons with the relevant client industries.

Unfortunately, this is no longer sufficient today. The HEVC case shows that MPEG should not only be able to deliver high quality standards but also that its users can access the technology. MPEG has no place in discussions about licensing, but it can operate to create the condition for easier licensing to happen,

A possible way forward is shown by the recently released MPEG-5 Essential Video Coding (EVC) standard. EVC offers improved compression efficiency compared to existing video coding standards (HEVC and its competitors) and relies on the statements of all contributors to the standard who have committed to announce their licensing terms no later than two years after the FDIS publication date.

Another route that will add value to MPEG standards is the acknowledgment that media technologies get more and more intertwined with other technologies present in media devices or relied on by media services. MPEG should be open to embrace more collaborations with other bodies and to develop joint standards, Far from being a limitation of MPEG’s role, this is an enhancement because MPEG standards will be able to reach more communities and hence more customers.

Ditto for the trend that MPEG has initiated some years ago to develop compression standards for data other than media. The collaboration with ISP TC 276 Biotechnology in the area of compression of reads from high speed sequencing machines has produced 5 parts of the MPEG-G standard. MPEG-G shows that MPEG compression technologies, developed for media, can be adapted to compress other data as well.

Posts in next thread

Posts in this thread