Does success breed success?

Introduction

Most readers will answer yes to the question asked in the title. Indeed, very often we see that success of a human organisation breeds success. Until, I mean, the machine that looked like it could produce results forever “seizes”. But don’t look elsewhere for the causes of failure: it’s not the machine, the causes are humans inside and/or outside.

In an age when things move fast and change, MPEG has been in operation for three decades. Its standards have achieved and continue achieving enormous success serving billions of human beings: consumers, service providers and manufacturers.

This article makes some considerations on the best way for MPEG success to breed success – unless success to breed failure is the goal. Apparently unrelated considerations are made in The Imperial Diet is facing a problem.

Recalling the MPEG story

MPEG started in 1988 as an “experts group” with the task to develop video coding standards for storage media at a rate of about 1.5 Mbit/s, like the compact disc (CD). This was because, in the second half of the 1980’s, the Consumer Electronics and telco industries imagined that interactive video – local or via the network – was a killing application.

Within 6 months MPEG had already started working on audio coding because – it looks obvious now, but it was not so obvious at that time – if you have video you also need audio and, if you do not compress stereo audio at 1.41 Mbit/s, the output bitrate of a CD, there will be no space left for video. In another 6 months MPEG had started working on “systems” aspects, those allowing a receiver to reproduce synchronised audio and video information.

These were the first steps in the MPEG drive to make standards that had no “holes” for implementors. Thanks to these efforts, the scope of use of MPEG standards, still within the scope of “coding of moving pictures and audio”, have expanded like wildfire: starting from coding of moving pictures at 1.5 Mbit/s and expanded to more video, audio, transport, protocols, API and more. With its standards, MPEG is handling all technologies that facilitate enhanced use of digital media.

The MPEG expansion is a joyous phenomenon that has created an expanding global brotherhood of digital media researchers – in industry and academia – for which MPEG and its standards are the motivation for more research. If research results are good, they can make their way into some MPEG standard.

MPEG needs a structure

Clearly you cannot have hundreds of people discussing such a broad scope of technologies at the same time and place. You can split the work because technologies can be considered independent up to a point. Eventually, however, like in a puzzle, all pieces have to find a place in the global picture. The MPEG structure has been implemented to allow the creation of ever more complex puzzles.

In its 31 years of activity MPEG has developed a unique organisation capable of channeling the efforts of thousands of researchers working at any one time on MPEG standards – only a fraction of which actually show up at MPEG meetings – into the suites of integrated standards that industry uses to churn out products and services worth trillions of USD a year.

The figure below depicts the MPEG structure from the viewpoint of the standard development workflow.

The MPEG workflow

Typically, new ideas come from members’ contributions, but can also be generated from inside MPEG. The Requirements group assesses and develops ideas and may go as far as to request “evidence” of existence and performance of technologies (Calls for Evidence – CfE) or actual “proposals” for fully documented technologies (Call for Proposals – CfP).

MPEG has never had a “constituency” because it develops horizontal standards cutting across industries. It has established liaisons with tens of industries and communities through their standards committees or trade associations. We call many of them as “client industries” in the sense that they provide their requirements to MPEG against which MPEG produces standards. At every meeting, several tens of input liaisons are received and about the same amount of output liaisons are issued.

Many CfPs cover a broad range of technologies that are within the competence of the different MPEG groups. The adequacy of submitted technologies is tested by the Test Group. The submitted proposals and the test results are provided to the appropriate technical groups – Systems, Video, Audio and 3D Graphics.

The Chairs group includes the chairs of all groups. It has the task to assess the progress of work, uncover bottlenecks, identify needs to discuss shared interests between groups and organise joint meetings to resolve issues.

An MPEG week is made of intense days (sometimes continuing until midnight). Coordinated work, however, does not stop when the meeting ends. At that time MPEG establishes tens of ad hoc groups with precise goals for collaborative development to be reported at the next meeting.

The Communication group has the task to keep the world informed of the progress of the work and to produce white papers, investigations and technical notes.

MPEG is not an empire

From the above, one may think that MPEG is an empire, but it is not. MPEG is a working group, the lowest layer of the ISO hierarchy, in charge of developing digital media standards. It formally reports to a Subcommittee called SC 29 but, as I have explained in Dot the i’s and cross the t’s, SC 29 has ended up with a laissez-faire attitude that has allowed MPEG to autonomously develop strategy, organisational structure and network of client industries. MPEG standards have given client industries the tools to make their analogue infrastructures digital and, subsequently, to leverage successive generations of standard digital media technologies to expand their business. With some success, one could say.

The MPEG organisation is robust. Virtually the same organisation has been in place since – 25 years ago – MPEG had an attendance of 300. Groups have come and gone and the structure currently in operation has been refined multiple times in response to actual needs. Changes have been effected, and there will be more changes in the future. However, they all have been and, as far as I can see, will be incremental adaptations, to perfect one aspect or another of the structure. With this structure, more than 150 standards have been produced, some of which have been wildly successful.

MPEG can count on three assets: the logic of the structure, the experience gained in all those years, its membership and its client industries. With these, MPEG success can breed more success in the years to come.

The Imperial Diet is facing a problem

I said before that MPEG is not an empire. In the imperial context of the Holy Roman Empire, MPEG could be defined as a Margraviate in charge of defending and extending a portion of the frontiers of the Empire. A Margraviate reported to a Kingdom who reported to the Imperial Diet.

Now, let’s suppose that the Imperial Diet has requested the S Kingdom to review the status of its two J and M Margraviates and propose a new arrangement. The main element in the decision is the size of the two Margraviates: 10% of the territory of the S Kingdom for the J Margraviate and 90% for the M Margraviate. Ruling out other fancy ideas, the S Kingdom has two options: request that the M Margraviate be elevated to Kingdom status or create a few smaller Margraviates inside the S Kingdom out of the M Margraviate.

There is a problem, though, if the M Margraviate is cut in smaller Margraviates: the Margraviates of the Holy Roman Empire are not domino game pawns. For decades the M Margraviate has fought hard extending its territory – hence the Holy Roman Empire’s territory – to lands that until then were occupied by unruly tribes. It has been successful in its endeavours because it had large armies with different skills: archers, knights, foot soldiers and more. By skillfully coordinating these specialised troops, the M Margraviate was able to conquer new lands and make them faithful fiefdoms.

But there is another important consideration: there are wild hordes coming from the steppes of Central Asia with a completely new warfare technique. Some armies of M Margraviate are having a hard time dealing with them, even though they are learning a trick or two to fight back.

How could the new armies of the different Margraviates created out of the M Margraviate defend – never mind extend – the frontier, when the S Kingdom does not know the territory, having lived all time in its castle, and has never led an army?

The Holy Roman Empire lasted 1,000 years. There is no doubt that the Imperial Diet would make the M Margraviate a Kingdom keeping its armies and structure unchanged. Warfare is a serious business and the effective defence of the frontiers is the priority.

Conclusions

Fortunately, today there is no Margraviates and Kingdoms anymore, much less the Holy Roman Empire. There are also no new territories to conquer by force of arms and there are no frontiers to defend against rebellious hordes.

I realise now that at the beginning of this article I have promised that I would make some considerations on the best way for MPEG success to breed success and not failure. Maybe I will do that next time.

Posts in this thread

 

 

Dot the i’s and cross the t’s

Introduction

In Book 2 of the Georgics, the Latin poet Virgil writes “Felix, qui potuit rerum cognoscere causas”. This maxim was true in 29 BC when the Georgics was written and remains true 2030 years later. For those who did not have the chance to study Latin, the verse means “Fortunate who could know the causes of things”.

Virgil’s maxim will be used in this article to draw some conclusions on current matters. Before getting to the things, however, I need to talk about the the causes. Those who cannot wait can jump to the conclusions, at their own risk.

The need for standards and standards bodies

At the climax of Belle Époque, Europe realised that a properly functioning industry needed standards. Britain – at that time for sure and, from now on, most likely not really part of Europe – was the first to establish an Engineering Standards Committee (1901).

In 1906 the most culturally advanced industrial field of the time – electrical technologies – was the first to recognise the need, not just for standards, but international ones, and established the International Electrotechnical Commission (IEC). Electrical technologies were second only to telecommunications (actually third, if we consider the Universal Postal Union) in recognising that need. Indeed, 41 years before governments had established the International Telegraph Union. But governments is one thing and private industry quite another.

The rest of the world took a quarter of a century and a war to realise the need for international standards. Finally in 1926 the International Federation of the National Standardising Associations (ISA) started, only to stop 16 years later when governments has other priorities (killing millions of people in WW II). In 1946 the idea was revived and the International Organisation for Standardisation (ISO) was created as a not-for-profit association. National Standards Associations (or National Bodies) – not governments – are represented in ISO.

How can you govern an international organisation that issues standards that are, yes, voluntary but, if you do not conform, you’re in a whole world of hurt? The answer is: hierarchy and scope. In ISO there are 4 layers (actually more, if you want to know how many read Who owns MPEG?): Technical Management Board (TMB), Technical Committees (TC), Subcommittee (SC) and Working Group (WG). Each entity is administered by a secretariat run by a National Body and has a scope that defines what the entity is expected and entitled to do.

Some scopes

Delimitation of territory is one of the most engaging human activities. According to Standards for computers and information processing, by T. B. Steel, Jr, page 103 et seqq. (in Advances in computers, Franz L. Alt and Morris Rubinoff (Editors) Volume 8) in 1967 the scope of TC 97 Data processing was: the standardisation of the terminology, problem definition, programming languages, communication characteristics, and physical (i.e. non electrical) characteristics of computers, and information processing devices, equipment and systems.

According to the same source, TC 97/SC 2 Characters set and coding at that time was about Standardisation of character sets, character meanings, the grouping of character sets into information, coded representation, and the identification of it for the interchange of information between data processing systems and associated equipments…

Typically working groups develop standards. They do so with a major constraint: decisions may only be made by “consensus”, defined as

general agreement where there is no sustained opposition to substantial issues by any important part of the concerned interests, in a process that seeks to take into account the views of all parties concerned.

The definition is supplemented by the note: consensus does not imply unanimity.

Obviously this text can only hint at the complexity of other environments where decisions are made not by consensus but by voting. One can image that these other environments are such that, in comparison, a horse-trading market is a place that boarding school pupils can safely visit.

JPEG and MPEG

In the 1980’s Videotex was a service telcos wanted to offer as a competing service to broadcasters’ Teletext service. One limitation of videotex and teletext, however, was that information could only be displayed with characters and rudimentary graphics (made as combinations of ad hoc characters). Telcos thought that videotex services could be enhanced by pictures transmitted at 64 kbit/s made available by Integrated Services Digital Network (ISDN).

In 1986 a joint group between TC 97 of ISO and SG XVIII of CCITT (ITU-T’s name of the time) was created to develop a compressed image format. As videotext was based on characters, TC 97/SC 2 was the natural place to develop that standard. SC 2 created WG 8 Coded representation of Picture and audio information. WG 8 hosted the Joint Photographic Coding Experts Group (JPEG).

Two years later, WG 8 created the Moving Picture Experts Group (MPEG), not joint with CCITT. In any case, if it had been joint, it would have been joint with SG XV, not with SG XVIII. There was nothing equivalent to the Treaty of Tordesillas, but in CCITT the digital world was divided between SG XVIII for audio and telematic services, and SG XV for video. MPEG – Coding for Moving Pictures and Audio – was a Copernican revolution.

Immediately, MPEG had a skyrocketing attendance: 100 members 18 months after its establishment and 200 members after two more years. That was because MPEG was working on such a high profile standard as digital television (actually only the baseband part of it, but that did not really matter).

Unlike other committees dealing with the “digital television”, who were populated by “advocates” accustomed to use “analogue” arguments to support their proposals, MPEG was populated by technical experts who made their cases with “digital” arguments in the framework of inflexibly digital Core Experiments rules. Some “advocates” did show up in the early MPEG-2 days, but they soon left never to come again.

The parent committee

In the years 1989-90-91, I had supported WG 8 Convenor’s bid to promote WG 8 to SC status (see here for more details) and in April 1990 the SC 2 plenary approved the following resolution:

JTC1/SC2 considering that

  1. The standardisation of the coded representation of picture, audio and multimedia information is considered to be one of the most important areas for standardization in the 1990’s;
  2. The work and scope of SC2/WG8 has expanded substantially beyond the scope of SC2;
  3. The work of SC2/WG8 has developed into a critical mass largely significant to warrant SC status;

Recommeds to JTC1

  1. To establish a new JTC1 Subcommittee for the purpose of developing standards in the area of Coded Representation of Picture, Audio and Multimedia Information;

It took another 18 months for SC 29, the entity WG 8 had morphed into, to hold its inaugural meeting.

A role for SC 29

In ISO a Subcommittee is part of the formal hierarchy. What was SC 29’s role ?

  1. Playground for “advocates”. Having found a hard time in MPEG, one could think that “advocates” should move to SC 29 to find a more consonant “breeding ground”. Indeed, SC decisions are made by voting, but only after a lot of “analogue haggling” in the hallways. This did not happen because, once MPEG had settled the algorithm, the standard was done, save the need to to cross some t’s and to dot some i’s. There could have been room for some “analogue discussions” on some business-related issues as profiles and levels. SC 29, however, was not the right place to hold such discussions because only MPEG experts could handle the technical aspects.
  2. Playground of large company representatives. At that time some ISO committees were populated by some national body representatives who worked for some large companies. They were interested in committees NOT to develop some standards and sent their representatives to act accordingly. But as fate would have it, in the years immediately following the establishment of SC 29 there was a serious economic crisis that forced some large companies to lay off those professional participants to cut “unnecessary” expenses.
  3. Strategic planner. In 1993, the time of John Malone’s “500 channels”, the Italian National Body proposed to investigate standardisation opportunities for content metadata (see here for more details). SC 29 established an ad hoc group to study the needs for users who wanted to find content in those 500 channels. One year later, however, the convenor of the ad hoc group reported that there had been no activity. MPEG then developed the suite of content metadata standards called MPEG-7.

No one should be surprised that, for the next 25 years, SC 29 held yearly meetings to discuss such strategic issues as progression of work items, consolidations and minor revisions, and liaisons. Of course with no “advocates” in sight.

MPEG as a virtual subcommittee

The space left empty by SC 29 was occupied by MPEG. Continuing its initial drive, MPEG developed a modus operandi that has allowed it to produce the integrated digital media standards that have changed and keep on changing the media industry.

The four figures below depict the main elements of MPEG’s modus operandi.

  1. Top-left depicts the adaptation of ISO’s standard development process to acquire technology elements suitable to the development of a standard and to verify that the standard developed matches the original requirements. More on this at How does MPEG actually work?
  2. Top-right depicts the industries contributing technologies (right-hand side), the means to acquire them, the assets accumulated in MPEG standards and the client/implementation industries (bottom of figure). More on this at The MPEG ecosystem.
  3. Bottom-left depicts the unfolding of the MPEG workflow: ad hoc groups; “MPEG week” with its components: plenaries, subgroups, break-out groups, joint meeting and chairs meetings; and creation of new ad hoc groups. More on this at Looking inside an MPEG meeting.
  4. Bottom-right depicts the integrated nature of MPEG standards. The parts are developed by different groups who come to agree on the glue that is needed to keep the parts independent and interworking. Moreon this  in Hamlet in Gothenburg: one or two ad hoc groups?

Conclusions

MPEG has been fortunate to have been able to operate in a paradise island for 31 years.

It has devised strategies and defined work plans. It has sought and established liaison with outside industries. It has added industries as members of the MPEG digital media community. It has called for technologies and integrated them into standards. It has been the ear industries could talk to to have their needs satisfied

All this while MPEG meetings have grown to 600 participants and “advocates” have been kept at bay.

The MPEG digital media community has thrived on a continuous flow of evolving standards with an impact measured in trillions of USD and billions of people.

In Paradise Lost John Milton writes: Better to reign in Hell, than to serve in Heaven.

In Paradise MPEG a reborn John Milton could write: Better to serve in Heaven than to reign in Hell.

SC 29 was kind enough to handle administration. Mindless industry elements should memorise Virgil’s maxim before they engage in their adventures.

Posts in this thread

 

The MPEG frontier

Introduction

MPEG has developed standards in many areas. One the latest is compression of DNA reads from high-speed sequencing machines and is now working on Compression of Neural Networks for Multimedia Content Description and Analysis.

How could a group who was originally tasked to develop standards for video coding for interactive applications on digital storage media (CD-ROM) get to this point?

This article posits that the answer is in the same driving force that pushed the original settlers on the East Coast of the North American continent to reach the West Coast. This article also posits that, unlike the ocean after the West Coast that put an end to the frontier and forced John F. Kennedy to propose the New Frontier, MPEG has an endless series of frontiers in sight. Unless, I mean, some mindless industry elements will declare that there is no longer a frontier to overcome.

The MPEG “frontiers”

The ideal that made MPEG experts work over the last 31 years finds its match in the ideal that defined the American frontier. As much as “the frontier”, according to Frederick Jackson Turner, was the defining process of American civilisation, so the development of a series of 180 standards for Coding of Moving Pictures and Audio that extended the capability of compression to deliver better and new services has been the defining process of MPEG, the Moving Picture Experts Group.

The only difference is that the MPEG frontier is a collection of frontiers held together by the title “Coding of Moving Pictures and Audio”. It is difficult to give a rational order in a field undergoing a tumultous development, but I count 10 frontiers:

  1. Making rewarding visual experiences possible by reducing the number of bits required to digitally represent video while keeping the same visual quality and adding more features (see Forty years of video coding and counting and More video with more features)
  2. Making rewarding audio experiences possible by reducing the number of bits required to digitally represent audio with enhanced user experiences (see Thirty years of audio coding and counting)
  3. Making rewarding user experiences by reducing the number of bits required by other non-audio-visual data such as computer-generated or sensor data
  4. Adding infrastructure components to 1, 2 and 3 so as to provide a viable user experience
  5. Making rewarding spatially remote or time-shifted user experiences possible by developing technologies that enable the transport of compressed data of 1, 2 and 3 (What would MPEG be without Systems?)
  6. Making possible user experiences involving combinations of different media
  7. Giving users the means to search for the experiences of 1, 2, 3 and 4 of interest to them
  8. Enabling users to interact with the experiences made possible by 1, 2, 3 and 4
  9. Making possible electronic commerce of user experiences made possible by 1, 2, 3, 4 5 and 6
  10. Defining interfaces to facilitate the development of services.

The table below provides a mapping between MPEG frontiers and MPEG standards, both completed and under development.

Legend: DA=MPEG DASH, CI=CICP, Io=IoMT

No end to the MPEG frontier

Thirty-one years and 180 standards later, MPEG has not accomplished its mandate, yet. That is not because it has not tried hard enough, but because there is an unceasing stream of new technologies providing new opportunities to better accomplish its mandate with improved user satisfaction.

While developing its standards, MPEG has substantially clarified the content of its mandate, still within the title of “Coding of Moving Pictures and Audio”. The following topics highlight what will be the main directions of work in the next few (I must say, quite a few) years to come.

More of the same

The quest for new solutions that do better or just simply different than what has been done in the past 31 years will continue unabated. The technologies that will achieve the stated goals will change and new ones will be added. However, old needs are not going to disappear because solutions exist today.

Immersive media

This is the area that many expect will host the bulk of MPEG standards due to appear in the next few years. Point Cloud Compression is one of the first standards that will provide immediately usable 3D objects – both static and dynamic – for a variety of applications. But other, more traditional, video based approached are also being investigated. Immersive Audio will also be needed to provide complete user experiences. The Omnidirectional MediA Format (OMAF) will probably be, at least for some time, the platform where the different technologies will be integrated. Other capturing and presentation technologies will possibly require new approaches at later stages.

Media for all types of users

MPEG has almost completed the development of the Internet of Media Things (IoMT) standard (in October 2019 part 1 will become FDIS, while parts 2 and 3 have already achieved FDIS in March 2019). IoMT is still an Internet of Things, but Media Things are much more demanding because the amount of information transmitted and processed is typically huge (Mbit/s if not Gbit/s) and at odds with the paradigm of distributed Things that are expected to stay unattended possibly for years. In the IoMT paradigm information is typically processed by machines. Sometimes, however, human users are also involved. Can we develop standards that provide satisfactory (compressed) information to human and machine users alike?

Digital data are natively unsuitable to processing

The case of Audio and Video compression has always been clear. In 1992/1994 industry could only thank MPEG  for providing standards that made it economically feasible to deliver audio-visual services to millions (at that time), and billions (today), of users. Awareness for other data types took time to percolate, but industry now realises that point clouds is an excellent vehicle for delivery of content for entertainment and of 3D environments for automotive; DNA reads from high-speed sequencing machines can be compressed and made easier to process; large neural networks can be compressed for delivery to millions of devices. There is an endless list of use cases all subject to the same paradigm: huge amount of data that can hardly be distributed but can be compressed with or without loss of information, depending on the application.

MPEG is now exploring the use case of machine tools where signals are sampled at a sampling rate > 40 kHz with 10 bit accuracy. In these conditions the machine tool generates 1 TByte/year. The data stored are valuable resources for machine manufactures and operators because they can be used for optimisation of machine operation, determination of on-demand maintenance and factory optimisation.

The password here is: industry 4.0. In order to limit the amount of data stored on the factory floor, a standard for data compression of machine tool data would be invaluable.

In you are interested in this new promising area please subscribe at https://lists.aau.at/mailman/listinfo/mpeg-mc.at and join the email reflector mpeg-mc@lists.aau.at.

Is MPEG going to change?

MPEG is going to change but, actually, nothing needs to change because the notions outlined above are already part of MPEG’s cultural heritage. In the future we will probably make use of neural networks for different purposes. This, however, is already a reality today because the Compact Descriptors for Video Analysis (CDVA) standard uses neural networks and many proposals to use neural networks in the Versatile Video Coding (VVC) standard have already been made. We will certainly need neural network compression but we are already working on it in MPEG-7 part 17 Neural Networks for Multimedia Content Description and Analysis.

MPEG has been working on MPEG-I Coded representation of immersive media for some time. A standard has already been produced (OMAF), two others (V-PCC and NBMP) have reached DIS level and two others have reached CD level (G-PCC and VVC). Many parallel activities are under way at different stages of maturity.

I have already mentioned that MPEG has produced standards in the IoMT space, but the 25 year old MPEG-7 notion of describing, i.e. coding, content in compressed form is just a precursor of the current exploratory work on Video Coding for Machines (reflector: mpeg-vcm@lists.aau.at, subscription: https://lists.aau.at/mailman/listinfo/mpeg-vcm).

In the Italian novel The Leopard, better known in the 1963 film version (director Luchino Visconti, starring Burt Lancaster, Claudia Cardinale and Alain Delon), the grandson of the protagonist says: “Se vogliamo che tutto rimanga come è, bisogna che tutto cambi” (if we want that everything stays the same, everything needs to change).

The MPEG version of this cryptic sentence is “if we want that everything changes, everything needs to stay the same”.

Guidance for the future

MPEG is driven by a 31-year long ideal that it has pursued using guidelines that it is good to revisit here while we are about to enter a new phase:

  1. MPEG standard are designed to serve multiple industries. MPEG does not – and does not want to – have a “reference industry”. MPEG works with the same dedication for all industries trying to extract the requirements of each without favouring any.
  2. MPEG standards are provided to the market, not the other way around. At times when de facto standards are popping up in the market, it is proper to reassert the policy that international standards should be developed by experts in a committee.
  3. MPEG standards anticipate the future. MPEG standard cannot trail technology development. If it did otherwise is would be forced to adopt solution that a particular company in a particular industry has already developed.
  4. MPEG standards are the result of a competition followed by collaboration. Competition is as the root of progress. MPEG should continue publishing its work plan so that companies can develop their solutions. MPEG will assess the proposals, select and integrate the best technologies and develop its standards in a collaborative fashion.
  5. MPEG standards thrive on industry research. MPEG is not in the research business, but MPEG would go nowhere is not constantly fed with research, in responses to Calls for Proposals and in the execution of Core Experiments.
  6. MPEG Standards are enablers, not disablers. As MPEG standards are not “owned” by a specific industry, MPEG will continue assessing and accommodating all legitimate functional requirements from whichever source they come.
  7. MPEG standards need a business model. MPEG standards has been successful also because those successful in contributing good technologies MPEG standard have been handsomely remunerated and could invest in new technologies. This business model will not be sufficient to sustain MPEG in its new endeavours.

Conclusions

Leonardo da Vinci, an accomplished performer in all arts and probably the greatest inventor of all times, lived in the unique age of history called Renaissance, when European literati became aware that knowledge was boundless and that they had the capability to know everything. Leonardo’s dictum “Homo sum, humani nihil a me alienum puto” (I am human, and nothing human I consider alien to me) well represents the new consciousness of the intellectual power of humans in the Renaissance age.

MPEG does not have the power to know everything – but it knows quite a few useful things for its mission. MPEG does not have the power to do everything – but it knows how to make the best standards in the area of Coding of Moving Pictures and Audio (and in a few nearby areas as well).

It would indeed be a great disservice if MPEG could not continue serving industry and humankind in the challenges to come as it has successfully done in the challenges of the last 31 years.

Posts in this thread

Tranquil 7+ days of hard work in Gothenburg

Introduction

Purpose of this article is to offer some glimpses of 7 (actually 12, counting JVET activity) days of hard work at the 127th MPEG meeting (8 to 12 July 2019) in Sweden.

MPEG 127 was an interesting conjunction of the stars because the first MPEG meeting in Sweden (Stockholm, July 1989) was #7 (111 binary) and the last meeting in Sweden (Gothenburg, July 2019) was #127 (111111 binary). Will there be a 255th (1111111 binary) meeting in Sweden in July 2049? Maybe not, looking at some odd – one would call suicidal – proposals for the future of MPEG.

Let’s first have the big – logistic, but emblematic – news. For a few years the number of MPEG participants has been lurking at the level of 500, but in Gothenburg the number of participants has crossed the 600 people mark for the first time. Clearly MPEG remains a highly attractive business proposition if it has mobilised such a huge mass of experts.

It is not my intention to talk about everything that happened at MPEG 127. I will concentrate on some major results starting from, guess what, video.

MPEG-I Versatile Video Coding (VVC)

Versatile Video Coding (VVC) reached the first formal stage in the ISO/IEC standards approval process: Committee Draft (CD). This is the stage of a technical document that has been developed by experts but has not undergone any official scrutiny outside the committee.

The VVC standard has been designed to be applicable to a very broad range of applications, with substantial improvements compared to older standards but also with new functionalities. It is too early to announce a definite level of improvement in coding efficiency, but the current estimate is in the range of 35–60% bitrate reduction compared to HEVC in a range of video types going from 1080p HD to 4K and 8K, for both standard and high dynamic range video, and also for wide colour gamut.

Beyond these “flash news” like announcement, it is important to highlight the fact that, to produce the VVC CD, at MPEG 127 some 800 documents were reviewed. Many worked until close to midnight to process all input documents.

MPEG-5 Essential Video Coding (EVC)

Another video coding standard reached CD level at MPEG 127. Why is it that two video coding standards reached CD at the same meeting? The answer is simple: as a provider of digital media standards, MPEG has VVC as its top of the line video compression “product” but it is has other “products” under development that are meant to satisfy different needs.

One of them is “complexity”, a multi-dimensional entity. VVC is “complex” on several aspects. Therefore EVC does not have the goal to provide the best video quality money can buy, which is what VVC does, but a standard video coding solution for business needs that cover cases, such as video streaming, where MPEG video coding standards have hitherto not had the wide adoption that their technical characteristics suggested they should have.

Currently EVC includes two profiles:

  1. A baseline profile that contains only technologies that are over 20 years old or are otherwise expected to be obtainable royalty-free by a user.
  2. A main profile with a small number of additional tools, each providing significant performance gain. All main profile tools are capable of being individually switched off or individually switched over to a corresponding baseline tool.

Worth noting is the fact that organisations making proposals for the main profile have agreed to publish applicable licensing terms within two years of FDIS stage, either individually or as part of a patent pool.

MPEG-5 Low Complexity Enhancement Video Coding (LCEVC)

LCEVC is another video compression technology MPEG is working on. This is still at Working Draft (WD) level, but the plans call for achieving CD level at the next meeting.

LCEVC specifies a data stream structure made up of two component streams, a base stream decodable by a hardware decoder, and an enhancement stream suitable for software processing implem­entation with sustainable power consumption. The enhancement stream will provide new feat­ures, such as compression capability extension to existing codecs, lower encoding and decoding complexity. The standard is intended for on demand and live streaming applications.

It should be noted that LCEVC is not, stricto sensu, a video coding standard like VVC or EVC, but does cover the business need of enhancing a large number of deployed set top boxes with new capabilities without replacing them.

3 Degrees of Freedom+ (3DoF+) Video

This activity, still at an early (WD) stage, will reach CD stage in January 2020. The standard will allow an encoder to send a limited number of views of a scene so that a decoder can display specific views at the request of the user. If the request is for a view that is actually available in the bitstream, the decoder will simply display it. If the request is for a view that is not in the bitstream, the decoder will synthesise the view using all available information.

Figure 1 shows the effect of decreasing the number of views available at the decoder. With 32 views the image looks perfect, with 8 views there are barely visible artifacts on the tube on the floor, but with only two views artifacts become noticeable.

Of course this is an early stage result that will further be improved until the standard reaches Final Draft International Standard (FDIS) stage in October 2020.

Figure 1 – Quality of synthesised video as a function of the number of views

Video coding for machines

This is an example of work that looks like it is brand new to MPEG but has memories of the past.

In 1996 MPEG started working on MPEG-7, a standard to describe images, video, audio and multimedia data. The idea was that a user would tell a machine what was being looked for. The machine would then convert the request into some standard descriptors and use them to search in the data base where the descriptors of all content of interest had been stored.

I should probably not have to say that the descriptors had a compressed representation because moving data is always “costly”.

Some years ago, MPEG revisited the issue and developed Compact Descriptors for Visual Search (CVDS). The standard was meant to provide a unified and interoperable framework for devices and services in the area of visual search and object recognition.

Soon after CDVS, MPEG revisited the issue for video and developed Compact Descriptord for Video Analysis (CDVA). The standard is intended to achieve the goals of designing interoperable applications for object search, minimising the size of video descriptors and ensuring high matching performance of objects (both in accuracy and complexity).

As the “compact” adjective in both standards signals, CDVS and CDVA descriptors are compressed, with a user-selectable compression ratio.

Recently MPEG has defined requirements, issued a call for evidence and a call for proposals, and developed a WD of a standard whose long name is “Compression of neural networks for multimedia content description and analysis”. Let’s call it for simplicity Neural Network Representation (NNR).

Artificial neural networks are already used for extraction of descriptors, classification and encoding of multimedia content. A case in point is provided by CDVA that is already using several neural networks in its algorithm.

The efficient transmission and deployment of neural networks for multimedia applications require methods to compress these large data structures. NNR defines tools for compressing neural networks for multimedia applications and representing the resulting bitstreams for efficient transport. Distributing a neural network to billions of people may be hard to achieve if the neural network is not compressed.

I am now ready to introduce the new, but also old, idea behind video coding for machines. The MPEG “bread and butter” video coding technology is a sort of descriptor extraction: DCT (or other) coefficients provide the average value of a regions and frequency analysis, motion vectors describe how certain areas in the image move from frame to frame etc.

So far, video coding “descriptors” were designed to achieve the best visual quality – as assessed by humans – at a given bitrate. The question asked by video coding for machines is: “what descriptors provide the best performance for use by a machine at a given bitrate?”

Tough question for which currently there is no answer.

If you want to contribute to the answer, you can join the email reflector after subscribing here. MPEG 128 will be eager to know how the question has been addressed.

A consideration

MPEG is a unique organisation with activities covering the entire scope of a standard: idea, requirements, technologies, integration and specification.

How can mindless industry elements think they can do better than a Darwinian process that has shaped the MPEG machine for 30 years?

Maybe they think they are God, because only He can perform better than Darwin.

Posts in this thread

Hamlet in Gothenburg: one or two ad hoc groups?

In The Mule, Foundation and MPEG, the latest article published on this blog, I wrote: In 30 years of MPEG, and counting? I somehow referred to the MPEG Mule when I wrote “Another thirty years await MPEG, if some mindless industry elements will not get in the way”. We may be close to know the fate of the MPEG Mule.”

We are nowhere close to knowing the fate of MPEG and in this article I will tell another episode of the saga.

In Which future for MPEG? I exposed my ideas about the future of MPEG based on a very simple reasoning. MPEG has developed global industry-agnostic digital-media standards that have led the media industry from analogue to digital, given opportunities to develop new business models and enabled the continuous expansion of the media industry. This is not a tale of the past but a reality that continues today with sustained production of digital media standards. The proof is in the record attendance last week of more than 600 MPEG members in Gothenburg.

Finally, as I wrote in MPEG and ISO, even taxi drivers know MPEG, demonstrating that the name MPEG does not just refer to a technology hidden in devices no one knows about but is imprinted in people’s minds.

Next to my proposal to leverage such a unique organisation making official the strategic role that MPEG has played for the last 30 years, there are many other proposals that can be summarised as follows

The first of these other proposals says: JPEG and MPEG are two working groups in the parent Subcommittee (SC). The former is in charge of coding of images and the latter is in charge of coding of moving pictures. By making MPEG an SC, JPEG remains alone in the parent SC and there will be no more collaboration.

The problem of this argument is that, especially in the last few years, for whatever reasons, JPEG and MPEG have not collaborated. JPEG used to meet collocated with MPEG, but then decided to meet separately. This does not mean that MPEG has not worked for JPEG because it developed two standards for the transport of JPEG2000 and JPEG XS images on MPEG-2 Transport Stream (TS), the standard that transports digital television.

Starting from 1992 MPEG has developed 5 standards jointly with ITU-T Study Group 16 (SG16) and is now developing a 6th standard. Still ITU-T SG16 is not even part of ISO! Another example is that MPEG has developed 3 standards and is developing 3 more standards jointly with TC 276 Biotechnology. Here we are talking of an ISO Technical Committee whose mission is to develop standards for biotechnology that do not have anything to do with digital media (but the jointly developed standard – MPEG-G – is very much needed by TC 276 for their workflows)!

This proves that collaboration happens when there is a common interest, not because the parties in the collaboration belong to the same organisational structure. This a bureaucratic view of collaboration that is unfortunately prevalent in ineffective organisations. Indeed, for bureaucrats it is so difficult to understand the essence of a common problem across organisational borders, while it is so easy to understand what happens inside an organisation (if it is understood, I mean).

The second of these proposals is a further attempt at creating organisational bindings where none existed before because they were never needed. In a few words the proposal is: instead of becoming an independent SC of 600 members (larger than many Technical Committees) the MPEG subgroups should melt in the parent SC.

This proposal demonstrates that the proponents miss the basic understanding of what MPEG is. MPEG is an ecosystem of groups developing integrated standards whose parts can also be used independently. To achieve this result, MPEG has developed the organisation described in More standards – more successes – more failures.

Figure 1 – Independent parts making an integrated standard

The parts of an MPEG standard (Blue circles) are typically “owned” (i.e. developed) by different groups, but there is a need to provide a “glue” (red lines in Figure 1) between the different parts of a standard if the standard is to be used as an integrated whole. The glue is provided by MPEG subgroups assisted by ad hoc groups, breakout groups and joint meetings and orchestrated by studies made at chairs meetings.

Dumping the MPEG organisation to melt in the parent SC will lead to the loss of the glue that make MPEG standards uniquely effective and successful in the industry. The components of a disbanded MPEG will not deliver as before in the new environment. Sure, given time, a new structure can emerge, but it is vital that a new structure operate now at same level of performance of MPEG, not in some years. Competition to MPEG is at the gates.

The third of these proposals is to give the parent SC the role of strategic planning, technical coordination and external relations that MPEG has – successfully – carried out for the last 30 years. This proposal is so naïve that not many words are needed to kill it (in Japanese you would use the word 黙殺, literally meaning “killing with silence”). For 30 years the parent organisation has performed administrative functions and, as much as you cannot make a racehorse from a horse who has pulled a cart for years, because its master so decides, in the same way the parent SC cannot become a strategic planner, a technical coordinator or an external relation manager. After years a new structure and modus operandi can very well settle (MPEG did not become what it is in a day), but in the meantime the cohesion that has kept MPEG components together will wither never to come back again and industry will just spurn its standards.

The fourth and last proposal (in this article, because there are many more) comes from a Non-Performing Entity (NPE). Appoint a new parent committee chair, disbands what exists today and create a new organisation from scratch. Sure, if the intention is to keep with a leash a tame committee whose sole purpose is to add IP in standards without any consideration for their industrial value, this is an excellent proposal.

In Gothenburg these and other proposals were discussed. How to make progress? One proposal was to make two ad hoc groups: one studying the first, well documented, proposal and the other trying to put order in the patchwork of ideas parts of which I have described above. Another proposal was to create only one ad hoc group combining the mandates of the two.

The matter was discussed for hours. Hamlet had to be called from neighbouring Denmark to decide. Whose skull did he use?

Posts in this thread

The Mule, Foundation and MPEG

What do the three entities of the title have to do together?

The second entity is Isaac Asimov’s Foundation Trilogy,  the tale of an organisation, actually more than one, established by Hari Seldon, who had invented psychohistory. According to that fictional theory the responses of large human populations to certain stimuli will remain the same over time if conditions remain as planned. Then, according to Asimov, psychohistory can predict the main elements of the evolution of society over the centuries and the Foundation is the organisation created to make sure that the future remains as Hari Seldon had planned it.

The first element is the Mule, a character of the trilogy, a mutant that quickly conquers the Galactic Empire with the power of his mental capabilities. It is an element of that fictional society whose appearance Hari Seldon’s psychohistory could not predict. The Mule was not expected to appear, but did.

The third is the MPEG data compression – especially media – group I have been writing about for some time on this blog. a group whose appearance in the media industry could not be predicted because it was completely outside of the rules of that industry, maybe the best real-world equivalent of Hari Seldon’s psychohistory.

Which were those rules? At certain points in history, several discoveries were made that rendered a range of inventions possible. Very often the only merit of guy who made the invention was that he put together a process whose elements were either known or already “floating in the air”. Regularly the invention was patented and gave the inventor the right to exploit his invention for the number of years granted by the law of his country.

In spite of this often chaotic process, several media types converged to the same technology. The photographic industry settled on a limited number of film sizes and the cinematographic industry settled on a limited number of formats: number of frames per second and film sizes. The sound recorded on vinyl records that were played at a limited number of speeds. All this according  to a series of steps that could not individually be predicted, but whose general outcome could.

Use of magnetics and electronics allowed more effective recording and, more importantly, enabled the instantaneous transmission of sound and images to remote places. Here the chaos reigned supreme with a large and growing number of formats for sound and television, real time and stored. If there had been a Hari Seldon of the media industry he could have applied his psychohistory.

In the Media Empire yhe Foundation materialised as a growing number of standards organisations who Tried to keep some order in the field. Table 1 shows just those at the international level, but others popped up at regional, national and industry level.

Table  1 – Media-related standards committees (1980’s)

ITU-T Speech SG XV WP 1
Video SG XV WP 2
ITU-R Audio SG 10
Video SG 11
IEC Recording of audio SC 60 A
Recording of video SC 60 B
Audio-visual equipment TC 84
Receivers SC 12A and G
ISO Photography TC 42
Cinematography TC 36

In “Foundation”, Hari Seldon had anticipated a number of “crises”. In the Media Empire, too, one crisis was due, the advent of digital technologies. Normally, this crisis should have been absorbed by making some cosmetic changes while keeping the system unchanged.

This is not what happened in the Media Empire because The Mule appeared in the form of a wild group of experts banding together under the MPEG flag. In the early days their very existence was not even detected by the most sophisticated devices, but soon the Mule’s onslaught was unstoppable. In a sequence of strikes  the MPEG Mule conquered  the media Empire: interactive video on compact disc, portable music, digital audio broadcasting, digital televisions, audio and video on the internet, file format, common encryption, IP-based television, 3D Audio, streaming on the unreliable internet and more. Billions of people were lured, without complaint but with joy, into the new world.

The forces of the MPEG Mule have clearly triumphed over the forces of darkness and anarchy. The Mule – the ultimate outsider – has exploited the confusion and brought order to everybody’s satisfaction if not to the forces of the Foundation who have been .designing their comeback

What will then be the eventual fate of the MPEG Mule?

In the Foundation, the Mule is eventually wiped out, not because his powers disappear but because others learned some of the methods of the Mule and applied them for their own sake, i.e. to re-instate confusion.

In 30 years of MPEG, and counting? I somehow referred to the MPEG Mule when I wrote “Another thirty years await MPEG, if some mindless industry elements will not get in the way”.

We may be close to know the fate of the MPEG Mule.

Posts in this thread

Can we improve MPEG standards’ success rate?

Introduction

I am very proud of the high scientific level of the work that MPEG does with its standards. I think that many universities would enhance the value and effectiveness of the education they provide if they took an MPEG standard, not necessarily a successful one, and have students redo the same process that led MPEG to develop that standard.

Because of their high scientific value so many MPEG standards have been wildly successful, as The impact of MPEG standards tells. More standards – more successes – more failures tells about the other side of the coin and argues that as much as certain products from companies are successful and others less so, some MPEG standards have a level of success that does not match the expectations of the experts who developed them.

What went wrong in those cases? My answer to this question is that not necessarily something went wrong and I will try and explain why.

Standards and market needs

MPEG is an excellent environment to develop standards because it has hundreds of the best experts in its field of endeavour. Several hundreds more are in the laboratories who do not, or only seldom attend MPEG meetings. MPEG experts make their proposals of new standards because someone in their companies has envisaged a business enabled by the idea they propose. MPEG is very receptive of new proposals and has a very good process to  develop requirements for a new standard. However, that assumes that the market needs that standard because those who develop the requirements are often the same technical people who will later develop the standard.

Let’s see how another successful environment developing specification – the Digital Video Broadcasting (DVB) project – handles the matter. A proposal for new work goes first to the Commercial Module. If the proposal is not accepted by the Commercial Module, it will not be considered by the Technical Module.

This is possibly a good arrangement for DVB, a group that works apparently in a similar area as MPEG. Similarities, however, may be misleading without considering the entire picture.

  1. MPEG addresses matters regarding the compressed digital representation of media. Therefore MPEG is a “horizontal” committee, while the DVB addresses the entire broadcasting protocol stack, from the physical layer – e.g. DVB-T (for terrestrial), DVB-S (for satellite) and DVB-C (for cable) up to the application layer (e.g. the Multimedia Home Platform or MHP).
  2. MPEG hosts experts from all industries with a stake in digital media while DVB members are predominantly companies interested in broadcasting applications. In all fairness . the separation of broadcasting from other ICT applications in an age of convergence is no longer what it used to be at the technical level, but the business context is different and this still matters.
  3. Some MPEG standards have the luxury of being “long term” (say, 5 years). Examples are the investigations in 3DoF+ and Point Cloud Compression, the former of which is on the launching pad to become a standards and the latter pretty close to the end. Both activities have been incubated for some years. DVB is more short-term oriented and does not want to invest resources in projects that may pay off only years from now.

MPEG already tried something

In 2015 I proposed to MPEG to establish a new activity with the preliminary name of “Industry liaison” and I even developed a concept of the new organisation of MPEG groups (Figure 1). My idea was that this activity could become a new MPEG group, if proved successful.

Figure 1 – Concept of MPEG organisation with industry liaison

Rob Koenen, then of TNO and José Alvarex, then of Huawei America (both tell you something about the dynamics of MPEG experts) were kind enough to take the challenge. They and a group of interested members created a presentation of MPEG activities.that was used at several exhibitions and conferences, e.g. NAB and SMPTE, to stimulate reactions from industry participants. The last slide of the presentation asked three questions:

  • Which needs do you see for media standardisation, between now and years out?
  • What MPEG standardisation roadmap would best meet your needs?
  • To accommodate your use cases, what should MPEG’s priorities be for the delivery of specific standards? For example, do you urgently need something that may enable basic functionality now, or can you wait for a more optimal solution to be released later?

That was a big effort and I am thankful to Rob and José for spending so much effort for the good of MPEG. Unfortunately the goal to create a momentum to the process of getting more information from industry about their needs was not achieved as a continuous and self-sustaining process.

This does not mean that the effort was not useful. Today Rob is in charge of keeping up to date the MPEG Standardisation Roadmap, a public document that provides a graphical representation of the entire MPEG work plan and its timeline, a very important function as Figure 2 related to March 2019 shows.

Figure 2 – MPEG Standardisation Roadmap

Let’s give it another try

Why did the proposal not work out as expected? One reason is intrinsic to the MPEG constitution. MPEG is a large community of experts, but there are very few if any market people, even from the larger companies. We have tried and reached out to the industry but we have been unable to get market people to join MPEG.

My assessment is that, as long as MPEG remains what it is, a wonderful forge of new standard technologies, it will be hard to achieve that goal.

In Which future for MPEG? I have proposed that MPEG become a Subcommittee (SC). This change of status is not intended to be a “promotion” but a better set up that allows MPEG to play a wider role than purely technical.

My proposal is reproduced here in an improved form as

Figure 3 – Proposed new structure of MPEG as an SC

The “Market needs” box represents the new attempt at getting market requirements into the MPEG process. However, it is quite different than Figure 1 because in that figure the Industry liaison function precedes Technical requirements, while here they work in parallel.

How should this new arrangement work?

Proposals for new work should be assessed both from their technical feasibility and their market value. The Technical requirements and Market needs Advisory Groups (AG) should make their assessments driven by “how much technology supports this proposal” and “how much market needs this proposal”, respectively. So, in a sense, the two groups will co-compete. In general Technical requirements will be for innovation, while Market needs will be for conservation. But it will not necessarily be always like that.

These groups should have, as all MPEG groups do have, joint meetings to align their views and they should, if they can, develop a single report. If that cannot happen, there will be two separate, possibly conflicting, reports. In both cases the SC plenary will asses the report(s), take responsibility and make a decision on the basis of a country-based vote.

What will be the meaning of this vote? It should certainly not be a replacement of the vote taken via ballot on a New Work Item Proposal (NWIP), but a vote on whether MPEG should acquire – via Calls for Proposals – the technologies that may be used to develop a standard. The actual NWIP vote will only take place when the Technical requirements AG and the appropriate WG(s) will agree that there is enough material to develop a standard.

So far I have talked of Market needs and Technical requirements having a passive role, i.e. waiting for proposals coming either from experts (as it has regularly happened so far) or from National Bodies.

The operation of the AGs, however, needs not be constrained to be in response to a proposal. They could very well, by their initiative or because they have been requested by the SC plenary, to engage in the study of new areas. Such studies should always be conducted from the point of view of what technology enables and what market needs, or what could be the reaction of the market to a certain standard technology.

The mission of the Market needs and Technical requirements AGs should not be considered accomplished when the NWIP has been approved and the development of the standard starts. MPEG must have the means to review the steps that have brought to success or failure of a standard.

You should not expect that this post mortem analysis can be done 6 months after MPEG has released a standard for publication. It is not unusual that 6 months after release by MPEG a standard has not been published yet. It is true that the world a quarter of century ago moved more slowly that today, but MP3, released in 1992 became a product only 5 years later, MPEG-1 Video (with MP2) became a product at the end of last century. We are talking here of an infrastructure to be put in place now whose first results we can probably see in five years.

Conclusions

This proposal should not be seen as a roadblock to the capability to innovate that MPEG has shown to be capable of in the last 30 years.

It is usually said that if commercialisation costs 100, product development costs 10 and research costs 1. I add that, probably, an MPEG standard costs a company 0.1 (excluding research because that is already accounted for).

So this proposal is not about cost saving. It is about making an excellent and successful MPEG process better, without running the risk highlighted by Monsieur de Montesquieu: Le mieux est l’ennemi du bien (better is the enemy of good).

MPEG is known for its innovative standards, but it should be better known for its process innovation.

Posts in this thread

Can you “clone” MPEG?

Introduction

The publication of A vision made real – Past, present and future of MPEG has triggered many reactions coming from people in other industries. They are facing the same problem that MPEG started facing 31 years ago when it wanted to create digital media standards that were industry-agnostic and with a global scope.

My answer to this people has been that, indeed, the MPEG model can be exported to other domains where industries with different backgrounds, technologies, stakes and business models want to join forces and develop standards that enable new businesses.

But it won’t be like a piece of cake. Try to inject a new philosophy of work in an environment with its own history, people, relationships and the task is close to impossible. This is not because of technical reasons but because it is a matter of philosophy. MPEG had an easy (so to speak) task because it was a new group without history, with new people with a limited degree of relationships.

In this article I will examine some of the main aspects that one needs to care about to “clone” MPEG to make standards that are industry-agnostic, with a global scope and are intended to enable new businesses.

Going from here to there

The first task is to define what the common project is about. Clarifying the purpose of an undertaking is a good practice that should apply to any human endeavour. This good practice is even more necessary when a group of like-minded people work on a common project – a standard. When the standard is not designed by and for a single industry but by and for many, keeping this rule is vital for the success of the effort. When the standard involves disparate technologies, whose practitioners are not even accustomed to talk to one another, complying with this rule is a prerequisite.

When MPEG was established (late 1980’s) many industries, regions and countries had realised that the state of digital technologies had made enough progress to enable a switch from analogue to digital. Several companies had developed prototypes, regional initiatives were attempting to develop formats for specific countries and industries, some companies were planning products and some standards organisations were actually developing standards for their industries.

MPEG jumped in the scene at a time the different trials had not had the time to solidify. Therefore, it had a unique opportunity to execute its plan of an epochal analogue-to-digital transition of media.

Similarly, industries who wish to create standards that are industry agnostic need to understand what is exactly the world that they want to establish and share a common vision of it.

By the way, you need a business model

Of course, I am not talking about how a hypothetical group working for industry-agnostic standards is going to make money. I am talking about how companies participating in this common effort can develop standards that bring actual benefits to each of them in spite of their differences, market positioning etc.

In the case of MPEG the definition of the business model had to take into account the fact that industry and acad­emia had worked on video compression technologies for some 3 decades filing many patents (at that time they could already be counted by the thousands) which covered a wide range of basic video coding aspects. Video coding standard that are loosely called “royalty free” (in ISO language, for which only Option 1 patent declar­ations are made) were certainly possible but would probably have been unattractive because of their low performance compared with the state-of-the-art codecs.

Therefore, MPEG decided that it would develop standards with the best performance, without consideration of the IPR involved. Patent holders would get royalties from the use of MPEG standards widely adopted by the market. If a patent holder did not want to allow that to happen, they could make an Option 3 declaration and MPEG would remove the infringing tech­nologies.

The MPEG business model is certainly not a prerequisite for developing industry agnostic standards, but it has worked well for the industry. More than that, most patent holders have been and keep on re-investing the royalties they get from existing standards in more technologies for future standards. The MPEG “business model” has created a standard-producing machine (MPEG) that feeds itself with new technologies.

Standards must be industry-friendly

A primary – and obvious – goal of the effort is that the standards produced by the collaborating industries should serve the needs of the participating industries.

The following points describe the three main MPEG targets:

  1. Display formats: Since the appearance of television cameras and displays in the 1920’s, industry and govern­ments have created tens of television formats, mostly around the basic NTSC, PAL and SECAM families. Already in the late 1960’s, when the Picturephone service was deployed, the tradition was hard to die: AT&T invented a new 267-line format, with no obvious connection with any of the existing video formats. As MPEG wanted to serve all markets, it decided that it would just support any display format, leaving display formats outside MPEG standards.
  2. Serving one without encumbering others. An industry may like the idea of sharing the cost of an enabling technology but not at the cost of compromising their individual needs. MPEG standards share some basic technologies but provide the necessary flexibility to its many different users with the notion of Prof­iles (subsets of general interoperability) and Levels (grades of performance within a Profile).
  3. Standards apply only to decoders; encoders are only implicitly defined, and their implementation leaves ample margins of freedom. By restricting standardisation to the decoding functionality, MPEG extends the life of its standards and, at the same time, allows industry players to compete on the basis of their constantly improved encoders.

The conclusion is that standards are great because they enable interoperability but should leave meaningful room to individual participants to exercise their business. Even better, as MPEG did with standards defining only the encoder, this opens the way to research-enabled competition.

Standards for the market, not the other way around

Before MPEG, a company with a success­ful product would try to get a “standard” stamp on it, share the technology with its competitors and enjoy the economic benefits of its “standard” technology.

This process may still be in place for some industries but is not an option when different industries team up to define common industry-agnostic standards with a global scope.

Under the MPEG regime, companies do not wait for the market to decide which technology wins, an outcome that very often has little to do with the value of the technol­ogy or the product but wait for the “best stan­dard” to be developed based on a set of technologies each of which is collectively selected based on a priori defined criteria. Then the technology package – the standard – developed by MPEG is taken over by the industry.

In a multi-industry environment, standards must anticipate the future. The alternative is to stop making standards because if the body waits until market needs are clear, the market is already full of incompatible solutions and there is no room left for standards, certainly not industry-agnostic and with a global scope.

Anticipating market needs is in the DNA of MPEG standards. With each of its standards MPEG is betting that a certain standard technology will be adopted. This explains why some MPEG standards are extremely successful and other less so.

Integrated standards as toolkits

Today’s systems comprise many functions. Some users of the standards are keen to have the complete package of functions, while others want to keep the freedom to cherry-pick other solutions that hopefully fit in the package as shown in the figure.

If interfaces are kept, say the one between System B and System C, the complete system continues to work. Depending on the specific case, however, the level of performance (not the functionality) of the entire system may change and a degree of interoperability may be lost.

Most MPEG standards are composed of the 3 key elements – audio, video and systems – that make an audio-visual system and some, such as MPEG-4 and MPEG-I, even include 3D Graphic information and the way to combine all the media. However, the standards allow maximum usage flexibility:

  1. A standard can be directly used as complete solutions, e.g. like in VCD where Systems, Video and Audio are used
  2. The components of the standard can be used individually, e.g. like in ATSC A/53 where Systems and Video are from MPEG, and Audio is from and external source
  3. The standard does not specify a technology but only an interface to different implementations of the technology, e.g. like in the case of MPEG-I, for which MPEG will likely not standardise a Scene Description technology but just indicate how externally defined technologies can be plugged into the system
  4. A standard does not specify the solution but only the components of a solution, e.g. like in the case of Reconfigurable Video Coding (RVC) where a non-standard video codec can be assembled using an MPEG standard.

A multi industry effort must satisfy the needs of all customers, even those who do not want to use its standards in their entirety but other specifications as well.

Compete and collaborate

Competition is the engine of progress, but standards are the result of a collaboration. How to combine competition and collaboration?

MPEG favours competition to the maximum extent possible. This is achieved by calling for solutions that respondents must comprehensively describe, i.e. without black boxes, in order to qualify for consideration. MPEG experts, including other proponents, assess the merit of prop­osed technologies.

Exte­nding competition beyond a certain point, however, is counterproductive and prevents the group from reaching the goal with the best results.

MPEG develops and uses software platforms that assemble the candidate components selected by its experts – called Test Models – as the platforms where participants can work on improving the different areas of the Test Models.

Core Experiments is the tool that allows experts to improve the Test Model by adding step by step the software that implements the accepted technologies. A Core Experiment is “a technical experiment where the alter­nat­ives considered are fully documented as part of the test model, ensuring that the results of independent experimenters are consistent”.

MPEG mission is to provide the best standards to industry via competition, but MPEG standards should not be shielded from competition Probably the earliest example of application of this principle is provided by MPEG-2 part 3 (Audio). When backward compatibility requirements did not allow the standard to yield a performance comparable to that of algorithms not constrained by compatibility, MPEG issued a Call for Proposals and developed MPEG-2 part 7 (Advanced Audio Codec). Later the algorithms evolved and became the now ubiquitous MPEG-4 AAC. Had MPEG not made this decision, probably we would still have MP3 everywhere, but no other MPEG Audio standards. The latest example is Essential Video Coding (EVC), a standard not designed to offer the best performance, but a good performance with good licensability prospects.

Working on generic standards means that reasonable requests – say, the best unconstrained multichannel audio quality – cannot be dismissed. MPEG tried to achieve that with the technology it was working on – backward-compatible multichannel audio coding – and failed. The only way to respond to the request was to work on a new – competing – technology.

One step at a time

An obvious principle, but it is better to keep it in mind, is that one must fine tune the engine first before engaging in a car race. If in 1988 the newly born MPEG had proposed itself as the developer of an ambitious generic digital media technology standard applicable to all indus­tries on a global scale, the proposal would have been seen as far-fetched and most likely the initiative would have gone no­where.

Instead, MPEG started with a moderately ambitious project: a video coding standard for interactive applications on digital storage media (CD-ROM) at a rather low bitrate (1.5 Mbit/s) targeting the market covered by the video cassette (VHS/Beta) with the addition of interactivity.

Moving one step at a time has been MPEG policy for MPEG-1 and all its subsequent standards and so should do any effort comparable to MPEG’s.

Separate wheat from chaff

In human societies parliaments make laws and tribunals decide if a specific human action conforms to the law. In certain regulated environments (e.g. terrestrial broadcasting in many countries) there are standards and entities (authorised test laboratories) who decide whether a specific implementation conforms to the standard. MPEG has neither but, in keeping with its “industry-neutral” mission, it provides the technical means – namely, tools for conformance assessment, e.g. bitstreams and reference software – for industries to use in case they want to establish authorised test laboratories for their own purposes.

Providing the tools for testing the standard is vital in a multi-industry environment. The ecosystem is owned by all and should not be polluted by non-conforming implementations.

Technology is always on the move

The Greek philosopher Heraclitus is reported to have said: τὰ πάντα ῥεῖ καὶ οὐδὲν μένει (every­thing flows and nothing stays). The fate of any technology field today is that technologies not only do not stay but move fast and actually accelerate.

MPEG is well aware that the technology landscape is constantly changing, and this awareness informs its standards. Until HEVC – one can even say, including the upcoming Versatile Video Coding (VVC) standard – video meant a rectangular area (in MPEG-4, a flat area of any shape, in HEVC it can be a video projected on a sphere). The birth of immersive visual experiences is not without pain, but they are happening, and MPEG must be ready with solutions that take this basic assumption into account. This means that, in the technology scenario that is taking shape, the MPEG role of “anticipatory standards” is ever more important and challenging to achieve.

This has happened for most of its video and audio compression standards. A paradigmatic case of a standard addressing a change of context is MPEG Media Transport (MMT) that MPEG designed having in mind a broadcasting system for which the layer below it is IP, unlike MPEG-2 Transport Stream, originally designed for a digitised analogue channel (but also used for trans­port over IP as in IPTV).

Research for standards

The wild pace of technology progress requires an engine capable to constantly feed new technologies.

MPEG is not in the research business. However, without a world of researchers working with MPEG in mind there, would be no MPEG. The MPEG work plan promotes corporate/academic research because it pushes com­panies to improve their technologies to enable them to make successful responses to Calls for Proposals.

One of the reasons of MPEG success, but also of some of its difficulties, is that MPEG standardisation is a process closer to research than to product design.

Roughly speaking, in the MPEG standardisation process, research happens in two phases:

  1. In companies, in preparation for Calls for Evidence (CfE) or Calls for Proposals (CfP), that MPEG calls competitive phase
  2. In MPEG in what is called collaborative phase, i.e. during the development of Core Exper­iments (of course this research phase is still done by the companies, but in the framework of an MPEG standard under development).

The MPEG collaborative phase offers another opportunity to do more research. This has apparently a more lim­ited scope, because it is in the context of optimising a subset of the entire scope of the standard, but the sum of many small optimisations can provide big gains in performance. The shortcoming of this process is the possible introduction of a large number of IP items for a gain that some may well consider not to justify the added IP onus to complexity. With its MPEG-5 EVC project, MPEG is trying to see if a suitably placed lower limit to performance improvements can help solve the problems identified in the HEVC standard.

Standards as enablers, not disablers

A standard intended for use by many industries cannot be “owned” by a specific industry. Therefore MPEG, keeping faith to its “generic standards” mission, tries to accommodate all legitimate functional requirements when it develops a new standard. MPEG assesses each requirement for its merit (value of functionality, cost of implementation, possibility to aggregate the functionality with others etc.). Profiles and Lev­els are then used to partition the application space in response to specific industry needs.

The same happens if an industry comes with a legitimate request to add a functionality to an existing standard. The decision to accept or reject a request is only driven by the value brought by the proposal, as substantiated by use cases, not because an industry gets an advantage, or another is penalised.

Conclusions

This article has given some hints drawn from MPEG’s 30 years long experience to those who intend to undertake an effort to develop standards for a multi-industry environment.

It is a significant but doable task if the effort is supported by new people without common history because a completely new philosophy of work must be adopted.

It is a close to impossible task if the effort is supported by people who already had a common history. This is true also in the case that the effort is about cloning MPEG to develop digital media standards.

Posts in this thread

Who “owns” MPEG?

Introduction

The title of this article contains three elements: the verb own pre- and postfixed with a quotation mark which conveys the notion of having “a control of, a stake in or an influence on”; the acronym MPEG with its multiple meanings; and the pronoun who which represents several entities.

In this article I will try and clarify the nature of these three elements and draw some conclusions that may be useful for some impending decisions.

The different uses of the word “MPEG”

In MPEG: what it did, is doing, will do I have reported evidence that even the public at large widely knows the word “MPEG”. But what do they mean when they use it? Most often they intend some “media technologies” present in a device or used to create, store or transmit media content. Therefore, the word “MPEG” in this context is owned by all.

The domain mpeg.org is obviously connected to the word “MPEG”. As I have written in the same MPEG: what it did, is doing, will do, the domain is owned by an individual who is not using the domain, is not inclined to donate it and does not answer when asked how much will make him inclined. So mpeg.org is privately owned.

A small number of people use the word “MPEG” to indicate one of more of the ISO/IEC standards listed in ISO numbers of MPEG standards. In this context the ownership is clear because ISO and IEC own the copyright of the standard, as stated in the second page of all MPEG standards. But what about other rights?

A small but important number of people use the word MPEG to indicate the working group (WG) whose official name is ISO/ IEC JTC 1/SC 29/WG 11 (I still have to meet someone who is not in the ISO/IEC circle and knows the avronym). To understand what “ownership” means in this context we have to make a digression that I hope will be perceived as short.

The place of the MPEG WG in ISO

The MPEG WG is the green box in the ISO organisation chart of Figure 1. I know that I should also say something about IEC, but the explanation I am going to make will discourage a sufficient number of well-intentioned readers to read further and I do not want to risk losing them all.

Figure 1 – MPEG sits at the bottom of the ISO organisation

In the hierarchical organisation of Figure 1 there are plenty of opportunities to claim “ownership”.

  1. The General Assembly, made up of all standards organisations members of ISO called National Bodies (NB);
  2. The ISO Council, the core governance body made up of 20 NBs, the ISO Officers and the Chairs of 4 committees;
  3. The Technical Management Board (TMB), in charge of managing the structure and activities of the Technical Committee (TC), made up of a chair and 15 members from NBs;
  4. The Joint ISO/IEC Technical Committee 1 Information technology (JTC 1), the largest Techical Committee in ISO with 15 Advisory Groups, 2 Working Groups, and 22 Subcommittees (SC);
  5. The Subcommittee 29, in charge of Coding of audio, picture, multimedia and hypermedia information and has 2 Working Groups;
  6. WG 11 aka MPEG in charge of Moving Pictures and Audio.

“Ownership” of MPEG standards

ISO and IEC

The MPEG WG is populated by experts who are accredited by the ISO NBs they are members of. They are typically employes of a company or university but there are some students, consultants or simply individuals.

Experts attending MPEG produce standards using the process described in … ISO and IEC sell copies of the standards whose copyright they own. These can be purchased online from www.iso.org. Some standards (e.g. those common with ITU) can be freely downloaded.

Patent holders

The buyer of an MPEG standard does not necessarily have the right to practice the standard because of rights to essential patents that may be owned by third parties, in most cases employers of MPEG experts. Therefore, these third parties also “own” the MPEG standard in which they have essential patents. Since the very beginning, MPEG standards a large number of patent holders and patent pools were established to license packages of patents considered essential to practice MPEG standards. Therefore, patent pools, too, have a sort of “ownership”.

A court of law may declare that a patent is essential to practice a standard in a jurisdiction. This kind of declaration is available for a limited number o patents (compared to the number of patents granted). Therefore, the identity of those who really “own” something in an MPEG standard is typically wrapped in mystery.

ISO and IEC (and ITU when relevant) keep patent declarations spontaneously made by those who believe they have rights in the standards. Patent declarations are also made by those who submit technologies for consideration by MPEG, either as part of responses to a Call for Proposal or as independent submissions (e.g. when submitting results of a Core Experiment). ISO and IEC simply record those declarations and take no stance regarding their validity. Therefore, the actual “ownership” claimed by those declarations or agreements to recognise them has to be determined outside of ISO.

Ownership of reference software

MPEG prides itself to have the policy to develop two “versions” of most of its standards. One version is expressed in natural language (English, later translated into French and Spanish). The second version is expressed in a computer language and is called the reference software of the standard. Both versions have normative status.

In the second half of the 1990’s, MPEG developed the MPEG-4 “copyright disclaimer” whereby users could get a free license from ISO/IEC to use and modify individual modules of the reference software for products claiming conformance to the specific MPEG standard. Of course, the copyright disclaimer included the usual disclaimer about third parties’ rights.

In the mid 2000s MPEG adopted a slightly modified version of the Berkeley Software Distribution (BSD) available as MXM Licence), a licence originally used to distribute a Unix-like operating system. The licence simply says that the reference software may be used for anything the uses wishes, with some obvious exceptions. MPEG has added the usual disclaimer (the “modification” above) about possible third-party rights.

Ownership of conformance testing suites

The notion of “ownership”, possibly a more complicated one, is also applicable to conformance testing suites. These consist of bitstreams designed and produced by MPEG experts to test particularly critical parts of a decoder. They have a normative value and are declared to conform to a specific MPEG standard. Bitstreams can be downloaded from the ISO web site and used to check whether a decoder implementation conforms to the standard.

Ownership of test sequences

There are other parties who have some “ownership” of MPEG standards, the companies (in some cases individuals) who have contributed content – images, audio, video and point clouds – used to carry out evaluations of submissions or verification tests. The very first test sequences used by MPEG were the so-called “CCIR sequences” donated by CCIR (now ITU-R) and used in MPEG-1 and other Calls for Proposals. For years the entire video coding community widely used “Table tennis”, “Mobile and calendar” etc. More recently, rights holders have begun to license content to individual MPEG experts for the purpose of developing MPEG standards.

ISO’s is a different organisation chart

Many companies have similar organisation charts with similar boxes as those of Figure 1. They are populated in the real world by employees and managers often with the support of secretariats. Unlike companies, however, ISO does not have its own employees populating the boxes located under the Technical Management Board. Who populates them?

Working Groups are populated by experts sent by companies and chaired by convenors, normally coming from industry. The role of convenors is to facilitate consensus and to detect achievement of consensus. As part of their duty to convene experts, convenors may need to organise the work. This gives convenors a limited level of “ownership”.

Subcommittees and Technical Committees are populated by delegations of National Bodies made up of NB officers often complemented by industry representatives. SC and TC secretariats have a level of “ownership” because the ISO/IEC Directives assign to them several important tasks such as the management of document balloting, e.g. proposals of new work items or approval of standards under development. Committee chairs, too, have a level of ownership.

Another level of “ownership” comes from the fact that a Convenor is nominated by the SC secretariat and elected by NB delegates in the SC. The SC Chair is elected by NB delegates in the SC and confirmed by NB delegates in the TC.

National Bodies “own” ISO

As we have seen, National Bodies permeate the ISO structure at all levels. Decisions at TC and SC level are made by NB votes. It is a one country, one vote systems but, sometimes, some votes weigh more than others. Votes are cast on the basis of the opinion formed within the NBs, ostensibly to further “national interests”.

National Bodies usually have a structure mirroring that of ISO. Industrial interests of national companies which are members of the National Body, often subsidiaries of multinational companies, intermingle to determine national positions on matters to be decided at the international level.

Conclusions

In this article I have tried to identify and describe the different forms of MPEG-generated “ownership”. Clearly the most important “ownership” is represented by MPEG standards because they generate three revenue streams: from products/services/applications enabled by MPEG standards (companies) to satisfy end user needs, from those who practice MPEG standards (patent or, more generally, intellectual property holders) and from the sale of standards (ISO/IEC and National Bodies).

Unfortunately, this clearcut logic is polluted by the different forms of “ownership” that I have described above, especially the desire of National Bodies to hold committee secretariats and nominate chairs. These may be nice-looking elements in National Body panoplies, but have nothing to do with the intensity of the three streams, the raison d’être of a standards organisation.

The intensity of revenue streams exclusively depends on the productivity of the “machine” (the committee) that creates the standards. “Owners” acting according to logic should leverage – not meddle with – a working group whose standards enable a global turnover of devices worth more than 1 trillion USD and services revenues of ~230 billion USD for pay-tv alone (data of 2018).

It will be interesting to see if the next decisions will follow a revenue stream maximisation logic, a curio collection logic or a survival logic.

Posts in this thread

Which future for MPEG

Introduction

For three decades MPEG has designed multi-threaded work programs, cooperated with tens of organisations and developed close to 200 specifications and hundreds of amendments. MPEG standards have transformed the global media industry from analogue to digital, enabled participation of new industries in the media business, provided standards serving all industries without favours and ensured unrelenting expansion of the entire media business.

MPEG achievements look reassuring: in 2018 the global turnover of MPEG-enabled devices is worth more than 1 trillion USD per annum and the global Pay-Tv revenues are 228 billion USD per annum, without mentioning other industries.

Should MPEG rest and look forward to a bright future built on the successes achieved? It would be so easy to answer “tout va très bien madame la marquise” but my answer is no.

The MPEG landscape has changed

In the first 10 years of its existence MPEG ruled the field of audio and video coding. In the following 10 years some proprietary solutions popped up, but the field was still largely dominated by MPEG standards. In the last 10 years MPEG has seen proprietary solutions getting significant strength. Today large areas that used to be exclusively covered by MPEG standards are satisfied by other solutions and the path to satisfy the next needs is not at all clear.

We do not have the complete picture yet of the extent the market will turn its back to MPEG standards. Market will speak and market is right by definition. If we do not like it, it is just because we did not try hard enough.

The next few years will be very challenging. It will not be a time of “business as usual”. MPEG needs to rethink itself and take appropriate measures. This article lays down some ideas and presents a proposal.

MPEG is about compression

So far MPEG has produced 5 generations of video compression standards, each generation offering more compression and more features. More of the same is expected from the 6th generation (VVC). MPEG has produced an equivalent number of audio coding standards. Will industry keep on asking for more video compression? I would like to answer with a resolute yes as I believe that there will always be need for more compression of visual information, but not always and not necessarily of the “old way”. More importantly, the need will not always be satisfied by MPEG standards because “The Price Is Right” applies to compression standards, too.
The answer to the question “Do we need more audio compression?” is, at least in the current time frame, that the currently available compression engine (MPEG-H 3D Audio) is good enough but we need new standards for other non-compression features, e.g. 6 Degrees of Freedom (6DoF). In the future this trend will also apply to video, as 3DoF+ video moves in the same direction as 6DoF audio (see The MPEG drive to immersive visual experiences).
Point cloud compression definitely needs compression but the convergence between 3D visual information captured by means of traditional video capturing devices and Point Cloud is still a matter for investigation.

MPEG is also about systems aspects

Systems aspects have been the enabling factor of the success of many MPEG standards and the future will not make those aspect less, but more important. The trend toward immersive media will require an even deeper integration between compressed media and the systems aspects that permeate them.
This can be seen from the requirements that are being identified in an activity called “Immersive Media Access and Delivery” where four dimensions are identified:

  1. Time (the usual one)
  2. Space (how to retrieve just the media parts of interest)
  3. Quality (how to access portions of media with the quality desired)
  4. Object (how to access specific parts of specific objects of interest).

MPEG in not just about media

In the past 30 years MPEG has shown that it could address new domains of expertise and learn their language. The fact that today all digital media speak the same (technical) language, is also due to the efforts made by MPEG to understand the needs of different industries, convert them to requirements, develop the technologies and quantise the standards into profiles and levels. This workflow has been in operation for 27 years, starting from the moment MPEG invented profiles and levels, and consistently applied them to talk to different communities using the same language.
This does not mean that there are no challenges when we talk to a new industry. MPEG has spent more than 3 years talking to, and identifying and validating requirements with the genomic community before starting the development of the MPEG-G standard. This significant effort has paid off: three International Standards on Genomic Data Compression have been developed jointly with TC 276 and 3 more are in the pipeline.

Governance is important

MPEG achieved its results as a working group, i.e. as the lowest organisational unit in ISO/IEC that the ISO/IEC directives recommend to be “reasonably limited in size”. Rightly, the ISO/IEC Directives do not define the term “reasonable”, but in 1989 MPEG had already 100 members, in 1999 it had 300 members and in 2019 it has 1500 members 500 of which attend its quarterly meetings. For the work it has done, MPEG has been reasonably limited in size.

For 30 years MPEG has played a role much above its status and I do not think there should be complaints about the results. In normal conditions MPEG could continue to operate as a working group for another 30 years but, as I said above, these are not normal conditions.

MPEG should become a Subcommittee (SC). Why? Because an SC has a solid governance administered by delegates appointed by National Bodies under the leadership of a chair. On the other hand, the design of the organisation must be properly done, if we do not want to do more harm than good.

Design of the SC is important

To be successful, the organisation of the SC should be conservative because of the importance of the industries served by MPEG standards. Therefore, the organisation of the SC should leverage the existing successful MPEG organisation by retaining and strengthening:

  1. The existing well-honed and demonstrably effective MPEG organisation. MPEG has fed the global media industry with technology that has allowed its growth for the last 30 years. It would be irresponsible to do anything that jeopardises such a large industry, the millions of jobs that go with it and the billions of consumers.
  2. The MPEG collaborative stance with other bodies. The major reason of the success of MPEG standards is MPEG’s open collaboration stance with its many client industries as represented by their standard organisation or industry fora. Collaboration is a must for MPEG because compression is always part of a bigger system with many interactions with other components. However, what was good 30, 15 or even 5 years ago is not necessarily sufficient today.
  3. The match of new MPEG standards to market needs. MPEG has produced hugely successful standards. However, other standards are less so. This is inevitable for an organisation that develops anticipatory standards that sometimes target the next 5 years. MPEG’s ability to engage in standards that are better matches of market needs has to be enhanced because conditions have changed.
  4. The strong MPEG brand. The new organisation is an internal matter designed to give MPEG a better chance to face a complex situation and should not create confusion in the MPEG client industries.

Figure 1 represents the proposed organisation.

Figure 1 – Structure of Subcommittee “MPEG Coding of Moving Pictures and Audio”

Meeting design criteria

Criterion #1: the organisational chart of Figure 1 retains the current MPEG organisational structure where existing informal subgroups become (in italic the names of the existing MPEG entities):

Advisory Groups (AG) if they do not develop standards (orange blocks):

  1. Technical Requirements (Requirements)
  2. Liaison and Communication (Communication)
  3. Technical Coordination (Chairs meetings)

Working Groups (WG) if they develop standards (light green blocks):

  1. Systems Coding (Systems)
  2. Video Coding (Video)
  3. Audio Coding (Audio)
  4. 3D Graphics Coding (3D Graphics)
  5. Quality Assessment (Test)
  6. Genomic Data Coding (Genomic activity in Requirements)

Criterion #2: The chart preserves and extends MPEG’s collaboration stance: Joint Teams with ITU-T (JCT-VC and JVET). The new organisation will be able to establish JWGs to develop standards on well-identified common areas of standardisation, e.g. JPEG, SC 24, SC 41 (IoT), SC 42 (Artificial Intelligence), TC 276 (Bioinformatics).

Criterion #3: The SC will now be able to carry out activities with high strategic value by assigning to the Market Needs AG Technical Requirements AG the task to investigate 1) existing or new areas of work and 2) proposals for new areas of work. Both AGs will produce coordinated reports that will be used by the MPEG SC to make an informed decision on new work.

Criterion #4: The SC should be called “MPEG Coding of Moving Pictures and Audio” prefixing the word MPEG to the current title of ISO/IEC JTC 1/SC 29/WG 11 (MPEG).

Conclusions

In a few weeks industry will decide the future of MPEG.

Will industry decide to give itself a safe and prosperous path by adopting the organisation proposed in this article or will it opt for the Japanese saying “出る釘は打たれる” (The nail that sticks out gets hammered down)? Will industry allow the MPEG nail to keep on sticking out or will it hammer it down?

Stay tuned to this block for further news.

Posts in this thread