Some have it straight

Developing standards is one of the greatest endeavours humans can do for their fellow humans. A well conceived and rightly implemented standard is the lubricant that makes society more efficient. Communication standards are even more valuable because they enhance the most powerful feature of humans to do things with other humans in a coordinated way.

In the 20th century public authorities and entrepreneurs vied in inventing new ways to communicate within a community to the exclusion of others. My best example is ITU-R Recommendation 624 “Television systems” where minuscule and large countries alike compete in adding footnotes stating that they reserve the right to adopt different frequency tolerances and the like.

MPEG was born in 1988 in reaction to the balkanised state of analogue media standards and to the attempts by countries and companies to carry over the analogue media paradigm to digital media.

The rest is history. MPEG standards have universal coverage, a result achieved without the intervention of public authorities because companies were convinced that a global market was not only larger than the sum of individual markets but also more efficient. Consumers reaped the benefits.

The reality of today, however, is that the MPEG model is not adopted in all markets and there is an interest on the part of some powerful parties to make the MPEG model a temporary exception to the rule, starting with “Nothing will change”.

If this will happen, it will not be because people have lost faith in the MPEG model, but because of ferocious arm-twisting behind the scenes.

Some people have a straight back and some have not. Some others get it only too well. Others simply don’t get it.

Posts in next thread

Posts in this thread

The MPEG exception

In the past 18 months, I have provided ample information on how MPEG was established, how it works, what it has done and is doing, and why it is different than other committees. In the last article Why is there a battle around MPEG?, I have identified another point of divergence. Patent declarations that relate to MPEG standards are 57.5% of all patent declarations submitted to ISO. MPEG is special because the next ISO committee in terms of number of patent declarations submitted to ISO has 13% share. JTC 1 itself has 81% of all patent declarations submitted to ISO.

In this article I would like to explore another difference, the nature of MPEG standards vs other JTC 1 standards.

Let’s start from an extended version of the table where the % refers to patent declarations submitted to ISO for all JTC 1 standards.

Committee Title # Pat. Decl. %
MPEG Coding of Moving Pictures and Audio 1857 69.76
SC 31 Automatic identification and data capture techniques 399 14.99
SC 27 Information security, cybersecurity and privacy protection 125 4.70
SC 25 Interconnection of information technology equipment 92 3.46
JPEG Coding of digital representations of images 80 3.01
SC 17 Cards and security devices for personal identification 56 2.10
SC 23 Digitally Recorded Media for Information Interchange and Storage 22 0.83
SC 6 Telecommunications and information exchange between systems 22 0.83
SC 34 Document description and processing languages 3 0.11
SC 35 User interfaces 2 0.08
SC 37 Biometrics 2 0.08
SC 28 Office equipment 1 0.04
SC 36 Information technology for learning, education and training 1 0.04

We see that Data (MPEG, SC 31 and JPEG), Security (SC 27), Communication (SC 25 abd SC 6) and Physical Media (SC 17 and SC 23) have a number of patent declarations expressed with more than 1 decimal digit.

This should be no surprise. At the risk of being criticised, I would state that the first 8 committees are not typical Information Technology (IT) committees, but Information and Communication Technology (ICT) committees because they handle information that does not only reside in a device, and hence is not typically the object of a standards, but is expected to move between devices. For instance there is no single patent declaration for SC 24 Computer graphics, image processing and environmental data representation.

Indeed, save for the first 8 committees in the table, the main target for standardisation in the other committees is architectures, frameworks, principles, guidelines etc.

Some JTC 1 standards are indeed developed outside of JTC 1. There are many organisations who request, e.g., JTC 1 to become Suppliers of Publicly Available Specifications (PAS). Those organisations who are granted that status have a preferential channel to make their specifications International Standards.

The Virtual Reality Modeling Language (VRML) was a PAS. JTC 1/SC 24 took VRML97 and made it ISO/IEC 14772-1:1997 Information Technology — Computer Graphics and Image Processing — The Virtual Reality Modeling Language — Part 1: Functional Specification and UTF-8 Encoding.

The justification for a making an ISO standard from a PAS is that the PAS can be developed with greater freedom and at a higher speed. The ISO process puts an ISO stamp on the PAS without changing the specification. Again at the risk of being criticised, I think that standards are too important to make them international standards by letting them cut corners.

MPEG has received a limited number of PAS with a request to convert them to ISO standards, but done so for none. MPEG has issued calls to which  some organisations that had produced PAS’s have responded. MPEG has developed standards based on those responses. An example is MPEG-M to which the Digital Media Project has been a major contributor. This is fine. The way technology is developed should be unconstrained, but there should be a single route through which technologies become international standards.

The Alliance for Open Media (AOM) specification known as AV1 is a PAS. One day, but not today, AV1 may became an MPEG standard.

MPEG is an exception, the question is whether it will continue to be so.

Posts in next thread

Posts in this thread

Why is there a battle around MPEG?

View powerpoint presentation

MPEG is known for a variety of reasons. It is the group that carried over analogue television and made it digital, multiplying by orders of magnitude the number of channels, it opened the way to new business models for audio, it enabled carriage of digital media on the internet, added media to the mobile experience, is poised to make immersive media real and genomics affordable, and more.

MPEG achieved these goals – and keeps on setting, working on and achieving more goals – with technical specifications of minuscule material or immaterial things that have a big impact on devices and service delivering infrastructures that are worth billions of USD.

I never tire of saying that every year products that rely on MPEG standards to function or to be attractive to buying global customers are worth more than 1 trillion USD. Similarly for services, which are worth more than 500 billion USD p.a.

There is something emblematically immaterial that has driven the success of MPEG – Patents, the engine that has allowed the MPEG machine to pile up records.

When MPEG developed its first and second standards it could draw from decades of research in audio and video coding. MPEG-1 and MPEG-2 were extremely successful and patent holders were handsomely rewarded. This encouraged more researchers and companies to file more patents so that, when MPEG engaged in new generations of coding standards it could draw from a wider and fresher set of technologies.

The story is not without hiccups. Overall, however, it is a success story that puts MPEG on a different league than any other ISO committee. This is shown by the figure below

You see that patent declarations made to ISO by entities believing they hold patents relevant to MPEG standards are 57.5% of ALL patent declarations received by ISO. The next committee is JTC 1/SC 31 Automatic identification and data capture techniques which totals less than ¼ of MPEG patent declarations (and it is a Subcommittee). All other committees mentioned in the table are JTC 1 SCs. Other ISO TCs have typically a low or even zero number of patent declarations.

You can get the data yourself from the ISO website.

MPEG should stay as the defender of a “business model” that has offered a relentless expansion of the business to all facets of the media industry and to consumers the possibility to enjoy newer and fresher experiences.

Probably you can answer yourself why there is a battle around MPEG.

Posts in next thread

Posts in this thread

Strategic planning for MPEG

In Leonardo: who I am, what I did so far and what I plan to do next, I have given some information on myself and listed my challenges in standardisation. In this article I intend to provide some more details about the challenges of a strategic planning for an organisation as special as MPEG. This is the list

  1. Uphold the role of MPEG as the unmastered and non-discriminatory source of digital media standards
  2. Expand the use of MPEG standards to insufficiently reached market areas
  3. Keep MPEG constantly abreast of technology and market development
  4. Develop standards in emerging fields of MPEG’s traditional media domain
  5. Collaborate with other committees by developing joint standards when mutually convenient
  6. Cultivate new and enhanced relationships with industry, MPEG client industries and academia
  7. Enhance the MPEG image and standing across the industry
  8. Reach out to other domains for which data compression may be a business booster
  9. Establish an ongoing process to assess MPEG’s relevance in fulfilling its mission
  10. Defend the MPEG business model by adapting it to changing conditions
  11. Define and establish a management-level succession process based on community preferences.

Uphold the role of MPEG as the unmastered and non-discriminatory source of digital media standards

Most ISO committees are the direct expression of a particular industry. Most secretariats are the expressions of the policies of the country in which the secretariat is located. MPEG was established having the opposite in mind, namely a committee that would develop standards that were industry and country agnostic. The first standard MPEG-1 targeted products and services for the Telecom and Consumer Electronics (CE) industries, the second standard MPEG-2 added the broadcasting industry. The third standard MPEG-4 added the Information Technology (IT) and mobile industries.

For 30 years MPEG has been remarkably able to keep its standards country-, industry-, company-, individual-independent. What will happen if, say, Information Technology took control of MPEG? What will happen if a major country took control of MPEG? What will happen if an alien business model took control of MPEG? What will happen if a major company took control of MPEG?

These are very serious questions and I am not sure if the global industry and world population would like the answer to these questions.

There is a second aspect, somehow related with the above, that need to be made explicit. This is the need to keep MPEG as the non-discriminatory source of digital media standards. For someone who has operated in the unmastered world of MPEG for many years, an MPEG standard is the result of the efforts of those who wanted to participate in the endeavour that produced the standards. No one should be denied participation. This does not mean that anybody can actually add technology to a standard, because MPEG is an extremely competitive environment striving to produce the best. Therefore only the best technologies are selected.

Once the standard has been produced, it should be accessible to all those willing to pay for the benefit of using excellent technology.

The idea that only some qualify to contribute to the standard or to use the standard seems to take hold in some quarters. This is an idea that MPEG should fight. MPEG should be unmastered and not discriminate against anybody.

Expand the use of MPEG standards to insufficiently reached market areas

I am not negative to the fact that in some market areas MPEG standards are not used to the extent the quality of the technology would promise. The worst that can happen to an organisation is that it occupies all the space and its members are left without goals to achieve. For instance digital cinema does not use MPEG standards and production uses only partly MPEG standards. These, however, are not real concerns because they are markets where devices or applications are counted by the thousands.

The real concern is video distribution over the internet, because distribution is what MPEG is mostly about, but MPEG never had an easy life in this segment. In the early 2000 there were companies eager to try video and audio distribution on this new medium using MPEG-4 Visual. No way, the MPEG Visual licensing terms were disappointing. MPEG-4 AVC fared definitely better with business-friendly licensing terms and a single patent pool. AVC is universally used even though competitors like P8 and VP9 did emerge. The problem is that AVC is a 15+ year old technology and what was due to be its successor HEVC is definitely underperforming with reportedly only 12% market share. Some blame licensing terms and multiple patent pools and a large number of patent holder not belonging to any patent pools. MPEG has developed DASH and CMAF and can offer the full protocol stack with its media standards. The streaming market, however, is taking different directions..

Keep MPEG constantly abreast of technology and market development

There is not doubt that MPEG knows how to stay abreast of technology development. Almost all of its standards are anticipatory of products, services and applications to come. Currently MPEG is engaged in several technology-deep investigations. One is Video Coding for Machines (VCM). VCM is expected to be the standard that allows remotely placed cameras to extract machine-understandable information, compress it and send it to a machine to use. The problem is that machines can be good for routine work, but what if something happens remotely and a human must intervene? The same information designed for machines should be usable by a human, possibly with the best quality that regular video coding can yield. Another exploration is video coding based on deep neural networks (DNN). We already know that some tools used e.g. in VVC could be replaced with more efficient DNN-based tools. The idea is to investigate the improvement of DNN based video coding technologies – both traditional schemes with some tools replaced or end-to-end DNN coding systems – over existing MPEG standards.

This is all good and no one can blame MPEG for developing the standards that have brought industry convergence, catered to its continuous development and made media a daily continuous experience. However, MPEG, as a group, is not aware of what is happening in the market. This should change. The idea is to create a Market needs group, the counterpart of the technology driven Requirements group. By having these two groups interacting we can still rely on a technology-based forward looking group mitigated by a market-savvy group, as depicted in the figure below.

Develop standards in emerging fields of MPEG’s traditional media domain

This strategy element confirms what we have been saying all along. MPEG is well aware of what the world of digital media technology can offer because is members are for most part (~3/4) from industry and for a smaller part (~1/4) from academia. The core of MPEG customers is interested in media standards and MPEG has much to offer in terms of standards that are close to availability or still under investigation, e.g.

  • Video-based Point Cloud Compression (V-PCC)
  • Graphics-based Point Cloud Compression (G-PCC)
  • Neural Network Compression
  • MPEG Immersive Video (MIV)
  • MPEG Immersive Audio (MIA)
  • Session based DASH operation
  • MPEG-I Scene Descriptions
  • Low Complexity Video Coding Enhancements
  • Deep Neural Network (DNN) based Video Coding
  • Video Coding for Machines

And many other standards.

Collaborate with other committees by developing joint standards when mutually convenient

In companies and standards alike the NIH (Not-Invented Here) principle rules. This is understandable, because if you have been working in a field for long enough you think you know everything in your field. The problem is that in our age technology is fluid. Something that is developed for a purpose, might equally well be usable for another purpose. I mention a case that MPEG is aware of where technologies for video coding or media file format have been successfully adapted for use in compressing the reads from high speed DNA sequencing machines. MPEG should embrace collaboration with other groups even more than it has done so far. Three examples are Big Data, for which MPEG has already developed Network Based Media Processing, Artificial Intelligence for which MPEG is developing Neural Network Compression and investigating AI-based video coding and Internet of Things for which MPEG is working on extending its Internet of Media Things standard keeping an eye on Video Coding for Machines.

Cultivate new and enhanced relationships with industry, MPEG client industries and academia

MPEG has not been shy to make the first step approaching the industry. MPEG can wave the flag of being a committee that is agnostic to countries and industries, but this comes at a high cost because your work can fall in a vacuum.

For years MPEG has been the merchant who devised all sorts of tricks to get the attention of different industries through their fora. The efforts have been rewarded: MPEG regularly talks with the main bodies representing broadcasting, fixed and mobile telecommunications, and package media at the international and regional level.

MPEG has also aggressively sought to collaborate with other bodies in the development of common standards, the best examples being the common video coding standards developed with ITU-T. MPEG has developed standards on commission from other bodies such as JPEG and 3GPP.

MPEG can also boast to have an extended community of followers in industrial research and academia. Indeed ¼ of its members are from academia.

Is this enough? No, it used to be enough, but it is no longer sufficient. MPEG makes in advance investigations that should involve academia and MPEG needs in advance information on the technologies that generate the data that it will be asked to reduce to a manageable form. It is a new form of deeper liaison that should be developed and implemented. Then, of course, you never talk often enough with your customers…

Enhance the MPEG image and standing across the industry

MPEG has a unique brand and its logo is universally known, but the fact that there are market segments where MPEG don’t play the role they could is an indication that the MPEG imaged must be beefed up. Of course, I am not talking of a marketing exercise, but of something that goes at the roots of what MPEG is: the principles that guide its activity that have kept making bigger the lowest organisational unit in ISO; its business model that guarantees that there is always new technology coming in because there is old technology that is being remunerated; its structure that combines flexibility with identification of responsibility; uniquely competent and dedicated personnel; a modus operandi that is the result of decades of honing; the academic and industrial network that has MPEG as its hub; the experience accumulated over the years and the ability to adapt itself to new conditions; and its technical and market achievements.

MPEG must define this multi-threaded presence and task a revamped Liaisons and Communication group to establish and maintain it.

Reach out to other domains for which data compression may be a business booster

The proposal to identify data compression as one of the axes of the information society failed, but the idea that generated it is pretty much alive. Countless application domains generate data: media, industry 4.0, health, automotive to mention a few. Most of these data are simply not used or seldom used because there is no economic means transmit, store or process them. The MPEG-G standard proves that MPEG technologies can be used to make an application domain flooded with data more efficient.

Establish an ongoing process to assess MPEG’s relevance in fulfilling its mission

MPEG likes to boast the fact that it is a working groups but it has produced more standards than any other JTC 1 subcommitee. Great, one could say, but that statement could be compared to the statement of a book publisher who publishes more book than its competitors, but its books pile up in its warehouses.

Fortunately that is not the case. Products and services that use in a determinant way MPEG standard are worth 1.5 trillion dollars a year.

This does not mean that the efficiency of MPEG has reached its limits. Before I have mentioned the need to create a Market needs group to compensate the exclusive technology driver of technology in MPEG. That, however, is not sufficient. MPEG must be able to answer the question: why did a standard A on which so many efforts have spent did not have the adoption that it deserved? Was the idea wrong? Were the requirements wrong? Were the technologies used in the standard unsuitable? Was profiling wrong? Was it because there was another better solution on the market? If so why were we not able to add the right requirements? Was it because the licensing terms were rejected? And you can add more.

If we will be able to answer these and other questions we will probably not be able to save the standard, but we can probably inject into our standard development process feedbacks that we got from unsuccessful standards.

Defend the MPEG business model by adapting it to changing conditions

I continue to believe that the MPEG business model of making good standards that reward good technologies is the best in the context of a rapidly evolving technology scenario. It is not the only one, however. You can develop option 1 (a.k.a. royalty free) standards that only use 20+ year old technologies or you can develop royalty free standards by buying a set of patents that allow you to build a codec with a performance that is sufficient for your needs etc.

It is clear that the MPEG business model is under threat and that something must be done to defend the MPEG business model – the only one that can provide the best standard at a given time – against those threats.

There are probably several ways. One has been attempted with Essential Video Coding (EVC). Under changed conditions, it is the same approached used in MPEG-2 times. Backward Compatible (BC) MPEG-2 Audio was not faring well, so a Non-Backward Compatible (NBC) MPEG-2 standard was developed in the hope that those engaged in BC would try harder to counter NBC. Eventually BC people lost but MPEG gained because MPEG-2 AAC is the progenitor of the extremely successful MPEG-4 AAC standard.

EVC is clearly a competitor to HEVC. If HEVC will have a good licence EVC will fail, but if HEVC continues have a complex situation, EVC will win. In both cases MPEG will gain.

Define and establish a management-level succession process based on community preferences.

MPEG is not a hierarchical organisation. This does not mean that it does not need leaders. ISO gives the secretariat a lot of power in nominating leaders. This may be good in other areas but it is not the ideal solution for MPEG. Such an ideal solution may not be a plain election, but for sure the community should have a say in how leaders are appointed.

Posts in next thread

Posts in this thread

Leonardo: who I am, what I did so far and what I plan to do next

I have written this article because I thought that, at this particular juncture, it would be useful to tell more about myself. I assume I am not unknown to quite a few people, but I think that it could be useful to put in a succinct form some information that summarises the responses to the who and what in the title.

Not that such information is “hidden”. Over the years I did spend some time to document some of the endeavours I happened to be part of or initiated or managed. Although a little out of style, https://www.chiariglione.org/ can be taken as the starting point of a navigation, in particular my web page and Riding the Media Bits.

Leonardo, the man

I am an Italian citizen by birth and a convinced supporter of the vision of a united Europe since my high scool days. However, I like to think I am a citizen of the world. I live in Villar Dora, province of Turin. Google maps provides a view of my house with a glimpse of the vineyard surrounding it. Originally, the house was the summer house of my grandfather, a farmer. At that time all farmer houses in town had a vineyard grown to produce grapes to make wine. There are still some farmers in town, but no one is left with a vineyard. While I am no farmer, I still produce some wine for my consumption.

I am father of three kids who have left the nest some time ago. My kids have 3 children, two girls and one boy.

Leonardo’s education

I graduated in Electronic Engineering from the Polytechnic University of Turin. At that time Italy had just one academic degree, equivalent to today’s Master’s degree. Before my graduation, I applied for a Japanese government scholarship, then went to Japan and got a Doctor’s degree in Electrical Communication from the University of Tokyo. I am told I was the first Caucasian to get a Doctor’s degree from the University.

Leonardo’s work experiences

CSELT, at the time the research centre of what is called today TIM-Telecom Italia, hired me to do research in video coding for visual telephony. When I left the company in 2003, I was Director of the Multimedia Division with 30+ researchers. Throughout my career at CSELT, I was involved at different levels of responsibility – leader, participant or manager of resources – in some 60 collaborative research projects, mostly at European level. This table provides an overview of all the research projects I was involved in.

Since 2003 I am the CEO of CEDEO.net, a technology company in Turin developing advanced services and products. The main services are WimTV and Stream4U. The main product is TVBridge, based on MPEG-A’s Multimedia Linking Application Format. The main product under development is DNASearch, based on the MPEG-G standard. Since 2011, I am also the CEO of WimLabs, a sister company of CEDEO.net focused on the commercialisation of products developed and the provision of services enabled by CEDEO.net. The WimLabs website provides an overview of all products and services commercialised by WimLabs.

Leonardo’s standards experiences

Over the years I participated in various standards committees. At the European level I joined CEPT committees (before ETSI was established) and then the ETSI JTC where I helped to create the path for acceptance of the MPEG-2 standard in Europe, before DVB had even started.

At the international level, I joined ITU-T and ITU-R committees. In particular I joined the first few meetings of the Okubo group who developed H.261 recommendtaion. I attended ITU-R meetings especially to promote adoption of the MPEG-2 standard. I was/am the Head of the Italian Delegation to JTC 1, SC 2 and SC 29. I attended JTC 1 meetings on several occasions, starting from the time I lobbied to convince SC 2 and then JTC 1 to create SC 29. I am the Chairman of the Italian SC 29 committee.

I founded and held the rapporteur role of the Moving Picture Experts Group of SC 2/WG 8 (1988-90) and the convenor role of SC 29/WG 11 (1990-) under SC 2 and then SC 29.

I founded and chaired DAVIC – Digital Audio-Visual Council (1994-95), FIPA – Foundation for Intelligent Physical Agents (1996-1988), OPIMA – Open Platform Initiative for Multimedia Access (1997- 1999) and DMP – Digital Media Project (2003-2015). I was also the Executive Director of the Secure Digital Music Initiative (1999-2001), an industry forum with the mission to develop specifications for secure digital music delivery.

Leonardo’s academic experiences

These are my academic experiences

  1. In 1986 I founded the International Workshop on HDTV which I chaired until 1994. I was the editor of all Proceedings of the HDTV Workshop until 1994.
  2. In 1989 I founded and was the Editor-in-Chief of Signal Processing – Image Communication, a EURASIP journal, until 1999.
  3. In 2004-2005 I was Professor at the Information and Communication University in Daejeon, Korea.
  4. In 2011 I was the Editor of the book The MPEG Representation of Digital Media published by Springer Science.
  5. I have organised a total of 18 conferences related to digital media, including one Picture Coding Symposium and one Packet Video Workshop,
  6. I have written some 200 technical and strategy papers, mostly invited, some submitted to technical journals and some delivered at conferences.

In 2002 I received the Doctorate “honoris causa” from Instituto Superior Técnico (Lisbon) and the EURASIP Meritorious Service Award. I am Chevalier de l’Ordre des arts et des lettres (France) since 2003 and Membre de l’Académies des technologies (France) since 2008.

Leonardo’s awards

I have received several awards (the text of the awards can be found here):

  1. the Information and Communication Society of Japan Award (1995)
  2. the International Institute of Communications Award (1997)
  3. the Society of Photo-Optical and Instrumentation Engineers Award (1998)
  4. the Kilby Foundation Award (1998)
  5. the IEEE Masaru Ibuka Consumer Electronics Award (1999)
  6. the IBC John Tucker Award (1999)
  7. the Edward-Rhein Foundation Award (1999)
  8. the SMPTE James Leitch Gold Medal Award (2002)
  9. the NAB Award of Honor (2003)
  10. the Charles F. Jenkins Lifetime Achievement Award (2018).

I am Honorary member of SMPTE since 2014.

Leonardo’s vision

In 1986, when the CCIR failed to approve the HDTV recommendation, I called the technical and business world and established the International Workshop on HDTV. I chaired the Workshop until 1994. The Workshop attracted hundreds of participants and acted as a technical forum that enabled eventual convergence of diverse regional views on the next form of television experience.

In 1987 I proposed the creation of an experts group on moving picture coding. The original idea was stimulated by my years of research in video coding and telecom standardisation that did not have an impact on the market. I tried to find a way to make available video communication end device by blending the manufacturing capability of the consumer electronics industry with the infrastructure of the telecom industry and the content offer of the broadcasting industry. All this crossing what were at the time strong industry barriers. In 1988 this idea became the Moving Picture Experts Group (MPEG). Today MPEG has become the established source of digital media standards that have caused industry convergence and made digital media available to billions of people. MPEG has received several Emmy Awards.

In 1994 I launched the Digital Audio-Visual Council (DAVIC), an industry forum with the mission to promote the success of emerging digital audio-visual applications and services, by the timely availability of internationally agreed specifications of open interfaces and protocols that maximise interoperability across countries and applications/services. DAVIC reached a membership of more than 200 companies and developed two editions of its comprehensive specifications that were transposed to ISO/IEC standards (the ISO/IEC 16500 suite).

In 1996 I launched the Foundation for Intelligent Physical Agents (FIPA), a non-profit organisation with the mission to promote the development of specifications of generic agent technologies that maximise interoperability within and across agent-based applications. FIPA developed 2 editions of its specifications. An open source software implementation of the FIPA specification (JADE), developed by one of my collaborators at CSELT, is used in several application domains.

In 2003 I launched the Digital Media Project (DMP), a not-for-profit organisation with the mission to promote continuing successful development, deployment and use of Digital Media that respect the rights of creators and rights holders to exploit their works, the wish of end users to fully enjoy the benefits of digital media and the interests of various value-chain players to provide products and services. DMP developed 3 editions of its specifications which were implemented in open source software (Chillout). DMP members proposed and made major contributions to MPEG-A’s Media Streaming Application Format (ISO/IEC 23000-5) and to MPEG-M suite of standards (ISO/IEC 23006).

Leonardo’s challenges

I see my next challenges in two domains: in my companies and in standardisation.

The challenges for my companies are to make WimTV the natural choice for those who want to do business with video on the web, TVBridge the normal way for broadcasters to augment their viewers’ experience and DNASearch the indispensable companion of all humans following their lives from cradle to grave.

I am sure that I can meet the challenge because I can rely on the support of my family and CEDEO personnel.

My challenges in standardisation are to:

  1. Uphold the role of MPEG as the unmastered and non-discriminatory source of digital media standards
  2. Expand the use of MPEG standards to insufficiently reached market areas
  3. Keep MPEG constantly abreast of technology and market development
  4. Develop standards in emerging fields of MPEG’s traditional media domain
  5. Collaborate with other committees by developing joint standards when mutually convenient
  6. Cultivate new and enhanced relationships with industry, MPEG client industries and academia
  7. Enhance the MPEG image and standing across the industry
  8. Reach out to other domains for which data compression may be a business booster
  9. Establish an ongoing process to assess MPEG’s relevance in fulfilling its mission
  10. Defend the MPEG business model by adapting it to changing conditions
  11. Define and establish a management-level succession process based on community preferences.

I am sure these standardisation challenges can be met. My confidence comes from the fact that MPEG is a large organisation that relies on the best and most dedicated experts a standards committee can hope to attract, has a solid organisational structure, enjoys an excellent reputation, and boast a universally known brand and loyal following both in academy and industry.

MPEG Future has published a vision that includes standardisation goals and an assessment of MPEG’s organisational capabilities. Industry is actively reacting to MPEG Future’s vision.

Yes, there will be challenges. There have always been challenges in the last 32 years. The next may be the biggest ever, but I think I know how to face them, as a group.

Posts in next thread

Posts in this thread

The value of MPEG standards

Introduction

MPEG can certainly be proud of the valuable standards it has produced: MPEG-1, MPEG-2, MPEG-4 AVC, MPEG-4 File Format and, more recently HEVC, MMT, DASH and many more.

Why are MPEG standards valuable and what can we do to add more value to them?

Why are MPEG standards valuable?

There is no single answer to this question. So, let’s analyse the causes that make MPEG standards valuable. While we are at it. later we wik let’s also analyse why some standards. on which great hopes were laid, did not deliver.

MPEG-1 (Systems, Video and Audio layer II)

The standard was designed for what would be the first device handling digital media, was interactive and and would serve the telecom and Consumer Electronics (CE) markets. Eventually that device did not fly, but the technology that made it possible was later reconfigured to make a “lower class” DVD player. Success was incredible, with a total of 1 billion devices manufactured by many companies.

MP3 (MPEG-1 Audio layer III)

MP3 turned out to be an incredible success from a totally unexpected quarter. When MPEG-1 was approved in 1992, MP3 offered transparency at 192 kbit/s for stereo music, ¼ less than MPEG-1 Audio layer II at 256 kbit/s (MP2) and at ½ of the MPEG-1 Audio layer I bitrate of 384 kbit/s (MP1). Complexity, however, was daunting (at that time). While MP2 and MP1 were implemented in ICs for mass market products, in the first years MP3 was implemented with sparse logic for professional equipment. Until, I mean, the impossible happened: software implementing the MP3 decoder in real time on a PC became available, non-real time encoding software became available, ripping a track from a CD became an OS feature of the PC and RIO became the first portable MP3 player with the size of a deck of cards. Most importantly a US court ruled that the RIO player was an IT, not a CE device.

With MPEG-G MPEG is taking the shot by addressing a conceptually similar situation as music tracks on CD converted to compressed digital 20 years ago. MPEG-G can potentially make the output file of a sequenced DNA “liquid” because it reduces a file of a few un-processable terabytes to a few tens of processable gigabytes. The difference is that the file of a sequenced DNA is a professional object, even though it can be produced in billions of instances and thus it looks like a consumer object.

MPEG-2 (Systems, Video and Audio)

MPEG-2 was a planned success. The early 1990s was a case of Brownian motion where particles (companies) were scrambling to come out with a new generation of TV. Japan with its MUSE and Europe with its HD-MAC were analogue by definition, and the USA put some order in its industrial activities with the Grand Alliance. However, all manufacturing companies, already global at that time, pushed for a digital solution. MPEG, the new kid in the block, announced it would develop a digital television standard and did so. The world followed.

There is much to learn from MPEG-2 because it is a standard that addressed the common needs of broadcasting industries – terrestrial, satellite, cable and package media – that until that time did not have much in common, if not the television experience eventually delivered to the end user. MPEG-2 was the catalyst of media industry convergence.

The development of the MPEG-2 4:2:2 profile was a natural follow-up and one that was constantly implemented in the video coding standards following MPEG-2 (including MPEG-4, RGB etc.). The 4:2:2 profile was a step itoward another industry – production – that has never been entirely fascinated by the MPEG offers.

MPEG-4 Binary Format for Scenes

MPEG-4 was the project that was meant to provide digital audio and video for yet another customer, the IT industry, actually the ICT industry, because video – and audio – for mobile was an important target from early on. BIFS had as foundation the Virtual Reality Modelling Language (VRML), a specification of the computer graphics world. MPEG extended VRML to support efficient transmission and real time. BIFS did not fly, and VRML, too, did not fly. Fascination with XML prompted both MPEG and VRML to introduce XMT and X3D, respectively. But success did not come. A later specification developed by the Korean Digital Media Broadcasting (DMB) made use of a relatively small portion of BIFS for complementary low bitrate services to digital audio broadcasting.

BIFS is an example of how commonality of technology between two radically different industries can well lead to no synergy. The origins of computer graphics are rooted in professional users using packaged content. The intended users of BIFS were consumers using real-time content via the network. The battle for the introduction of the technology was fought independently (it could hardly have been otherwise). Unfortunately, it was lost – independently.

MPEG learned that you cannot develop a standard in a top down fashion. Vision is needed, but vision must constantly face reality.

MPEG-4 Visual and Advanced Video Coding

MPEG-4 Visual inherited the vision of 2D visual objects in a 3D space. MPEG-4 Visual was a standard that supported moving 2D object in addition to extend the performance of MPEG-2 Visual. Industry needed the standard to enable services on the web and mobile. The hopes of a licence of MPEG-4 Visual patents were dashed when the licence included a charge based on the time an MPEG-4 Visual decoder was used.

MPEG-4 Visual was the sacrificial lamb that allowed MPEG-4 AVC to get a better deal in the use of relevant patents. Higher compression and acceptable licencing terms made AVC the universal video codec that it still is today.

MPEG-4 Audio

The MPEG Audio fortunes have different stories to tell. MP3 was and is an outstanding success. MP2 has been a good success and MP1 had a short span of use in the Digita Compact Cassette. MPEG-2 Audio did not have many takers because the performance was not adequate in a multi-channel environment. MPEG-2 AAC is in use in some countries. More importantly, MPEG-2 AAC it paved the way to MPEG-4 AAC.

MPEG-4 AAC is an outstanding success, with probably 10 billion devices in current use for all sorts of applications. Why? One element is the performance of the technology, another is the choice made by Apple to base their web-based music distribution service on MPEG-4 AAC. Yet another is the continuous performance improvement that has delivered MPEG-4 HE-AAC v2.

Open Font Format

OFF is a great example of standard development not for an industry forum but for a group of companies who had developed a successful specification that they were no longer keen on maintaining. In a sense, still, that was a request from an industry, conveyed by a National Body, that MPEG gladly accepted. OFF is now an extremely successful standard. This success is not entirely of MPEG’s doing, but the amount of extension MPEG has added to the original OpenType specification is such that MPEG can say that OFF is an MPEG standard of great success.

MPEG Media Transport (MMT)

MMT is an example of a solution that answers the questions that many (terrestrial) broadcasters ask: how can I face the competition of services on the web? The answer could have been strengthening broadcast delivery with some magi. Instead MMT offered the means to move broadcasting services, if not to the web, at least to the transport technology that enables the web . In this way it becomes easy to integrate broadcasting services with interactive services. The standard is deployed in several important countries, but acceptance of MMT is not universal. Why? Broadcasting is a conservative world and highly influenced by political considerations. Interestingly, streaming services that use MMT can offer short delay. However, there is still a limited use of MMT for streaming services.

High Efficiency Video Coding

HEVC continues the tradition of MPEG Video compression standards in terms of technical achievements. The HEVC Verification Tests showed that on average HEVC delivers the same quality as AVC with 40% of the bitrate required by AVC. MPEG’s traditional customers have jumped on HEVC which is widely deployed in their devices and services. A concern is the low level of adoption in online services which is rated at ~12%.

3D Audio Coding

This standard, part of the MMT and HEVC package in MPEG-H has been adopted by several broadcasting agencies. Its use has been announced by other industry sources also for distribution of high quality immersive audio via the web.

DASH

DASH is a remarkable example of how MPEG is able to respond to the need of a client industry, in this particular case 3GPP, representing the mobile telecommunication industry. MPEG developed DASH with a close connection with 3GPP. Since the standard has been developed, DASH has seen a broad acceptance by the industry.

Adding value to MPEG standards

The incomplete summary above shows that MPEG can develop high quality standards, the best in terms of performance in its area of work. Its standards do respond to industry requirements thanks to MPEG’s network of liaisons with the relevant client industries.

Unfortunately, this is no longer sufficient today. The HEVC case shows that MPEG should not only be able to deliver high quality standards but also that its users can access the technology. MPEG has no place in discussions about licensing, but it can operate to create the condition for easier licensing to happen,

A possible way forward is shown by the recently released MPEG-5 Essential Video Coding (EVC) standard. EVC offers improved compression efficiency compared to existing video coding standards (HEVC and its competitors) and relies on the statements of all contributors to the standard who have committed to announce their licensing terms no later than two years after the FDIS publication date.

Another route that will add value to MPEG standards is the acknowledgment that media technologies get more and more intertwined with other technologies present in media devices or relied on by media services. MPEG should be open to embrace more collaborations with other bodies and to develop joint standards, Far from being a limitation of MPEG’s role, this is an enhancement because MPEG standards will be able to reach more communities and hence more customers.

Ditto for the trend that MPEG has initiated some years ago to develop compression standards for data other than media. The collaboration with ISP TC 276 Biotechnology in the area of compression of reads from high speed sequencing machines has produced 5 parts of the MPEG-G standard. MPEG-G shows that MPEG compression technologies, developed for media, can be adapted to compress other data as well.

Posts in next thread

Posts in this thread