Month: December 2019

The true history of MPEG’s first steps

Introduction

Purely based on logic, MPEG should not exist.

In the 1980s the Galactic Empire of media standardisation was firmly in the hands of ITU (videocommunication and speech), IEC (audio and television) and ISO (photography and cinematography), not to mention its kingdoms and duchies – the tens of regional and national standards committees.

How could this little Experts Group, not even recognised in the ISO hierarchy, become the reference standards group of the media industry (and more to come)?

Like the Mule in Asimov’s Foundation, MPEG came to the fore with unstoppable force. In this article I will tell the (short) story of how the impossible happened.

Video coding in the 1970’s and 1980’s

I was hired to do research on videophone at CSELT, the research center of Telecom Italia (at that time STET/SIP) and in 1978 I joined the European Action COST 211. The Action’s goal was to specify, implement and trial a 2 Mbit/s videoconference system based on DPCM and Conditional Replenishment. In those years that was as much as sparse electronics and MSI chips could implement.

In 1984 the results of the project were standardised as ITU Recommendation H.120 – Codecs for videoconferencing using primary digital group transmission. A few industrial implementations were made (one from my lab prototype, at that time reduced to 2 16U racks), but the service had a limited use.

COST 211 was followed by COST 211 bis which became a European coordination forum for next ITU-T project. I attended the first few meetings of what was then called the Okubo group, from the name of its chairman. The group developed the specification that in 1988 became H.261 – Video codec for audiovisual services at p x 64 kbit/s.

I discontinued my participation because I did not agree with some of the decisions that were made by the group. My bête noire was the Common Intermediate Format (CIF). As the name implies, CIF was a video format defined as a sequence of frames with a resolution of 352×288 pixels, (1/4 of a PAL frame) at a rate of ~ 29.97 frames/s (the NTSC rate). I failed to convince my colleagues that defining video formats was history – an encoder issue. A decoder designed for flexibility could handle different spatial and temporal resolutions.

In 1984 the European Commission (EC) launched the trial version of its research program Research & development on Advanced Communication for Europe (RACE). I set up a consortium called Integrated Video Codec (IVICO) which was awarded a 1 year grant. The project, a scaled-up version of the COMIS projects, came up with a work plan based on a general coding architecture – DCT + motion compensation, at that time not a discounted solution.

At the end of the year funding of the project was discontinued, I believe that was done because a project whose aim was to design VLSI chips for a consumer market of video decoders was at odds with the EC policy of that time: analogue HD-MAC as the next television format, and digital for the next century (the one we live).

In December 1986 I went to Globecom, held in Houston, TX, to present a paper on the IVICO project. At the conference I met Hiroshi Yasuda and other University of Tokyo alumni I had known during my PhD there. Hiroshi invited me to attend the next meeting of the Joint ISO-CCITT Photographic Coding Experts Group (JPEG) in March.

Learning from JPEG

Going to JPEG meetings made sense to me because my group at CSELT was involved in another European project called Photographic Image Compression Algorithm (PICA). This had been launched by British Telecom to coordinate European participation in JPEG. So, in March 1987 I went to Darmstadt where the FTZ research centre of Deutsche Bundespost hosted the JPEG meeting that finalised the procedure to test proposals to be submitted in July 1987.

I was favourably impressed by the heterogeneous nature of the group. Unlike the various European standards groups I had attended since the late 1970s – all composed of telecommunication people – JPEG was populated by representatives of a wide range of international companies such as telecommunication operators (British Telecom, Deutsche Telekom, KDD, NTT), broadcasting companies (CCETT, IBA), computer manufacturers (IBM, Digital Equipment), terminal equipment manufacturers (NEC, Mitsubishi), integrated circuits (Zoran) and others.

The reality of a multi-industry standards group resonated well to me because the idea of a group on video coding standards not uniquely connected to telecommunications had been stuck in my head for a long while. I worked for a telecommunication company and I realised that the time scale of the telecom infrastructure did not match the scale of the media market and that the industry that provided equipment to the telecommunication industry followed the same reasoning as its customers.

In those years I was particularly attracted by the project announced by Philips called CD-i (compact disc interactive) that targeted interactive video on compact disc, hence for a bitrate of ~1.5 Mbit/s. I thought that consumer electronics (CE) should be the industry developing the video equipment that the telecom industry could use, if only the CE industry could be convinced that it was in their interest to develop products based on international standards, instead of proprietary solutions.

At the July meeting, hosted by Kjøbenhavns Telefon Aktieselskab (KTAS) in Copenhagen, one of the PICA proposals coordinated by Alain Léger of CCETT, with the participation of my group, performed the best and became the basis on which the well know JPEG standard was developed. At that meeting I proposed to Hiroshi, who was the convenor of SC 2/WG 8, of which JPEG was an Expert Group that he chaired, to create another Experts Group on Moving Pictures in WG 8.

The birth of MPEG

In January 1988, at another meeting hosted by KTAS in Copenhagen, WG 8 established two more Experts Groups: the Joint ISO-CCITT Binary Image Coding Experts Group (JBIG) and the ISO-only Moving Picture Coding Experts Group (MPEG). The latter had the mandate to develop standards for coded representation of moving pictures (audio and systems came later).

The first project concerned video coding at a bitrate of about 1.5 Mbit/s for storage and retrieval applications “on digital storage media”. It was the realisation of my idea of an international standard developed by the researchers of all industries, implemented by the consumer electronics industry and exploited by the telecommunication and broadcasting industries for audio-visual services.

Having achieved this result, I set up a European project called COding of Moving Images for Storage (COMIS) where I called European telecom companies, CE companies, chip manufacturers, Universities, etc. The intention was not only to create a forum for a more effective participation in MPEG, but also to give more visibility to the MPEG group. Hiroshi did the same in Japan with its Digital Audio Pictures Architecture (DAPA).

The first MPEG meeting took place in Ottawa in May 1988 with 29 people in attendance. At the 4th MPEG meeting in December 1988 in Hannover, the Audio group was established. In a few meetings, the Video, Test, Digital Storage Media, Systems, VLSI groups were also established. The responses to the MPEG-1 Call for Proposals were considered in November 1989 in Kurihama, hosted by JVC, where the attendance reached 99.

I am pleased to say that, at the following Eindhoven meeting (February 1989), MPEG introduced Source Input Format (SIF) obtained by sampling the 625 lines and 525 lines frames yielding 288×352 pixels and 240×352 pixels, respectively, at different frame rates. Simulation results could be shown with the 625 or 525 version of the format. Making decoders disconnected from the video format of the encoder has been an MPEG constant ever since.

The creation of SC 29

In Kurihama the Multimedia and Hypermedia Experts Group (MHEG), an ISO-only group,  was added to WG 8. This meant that in less than two years after adding JBIG and MPEG to JPEG, the membership of WG 8 was already exceeding 150.  I assessed that WG 8 was too small a container for all the activities that were booming and pushed Hiroshi to create a Subcommittee out of WG 8.

At the Washington, DC meeting of SC 2 in April 1991, a coalition of National Bodies approved the secession of WG 8 from SC 2 and SC 29 held its first meeting in November 1991, just after the MPEG-2 tests, where the MPEG attendance reached 200.

Conclusions

MPEG “happened” against all odds, like the Mule in Foundation. My idea of combining the manufacturing capability of the consumer electronics industry to the needs of telecom and broadcasting services succeeded. Today the CE industry of the MPEG-1 times no longer exists as it has largely merged with the IT industry/ However, thanks to its MPEG-4 project, MPEG has been able to continue to play the role of reference standards body also for the “extended” media industry.

The successful MPEG story is a reminder that standardisation has all to gain from fostering the birth of “Mules”. Within MPEG everybody has the chance to become the next Mule. They just have to bring a plan (and convince the rest of MPEG ;-).

Posts in this thread

 

Let’s put MPEG on trial

Introduction

An Italian proverb says “those who do things make mistakes” to mean that only if you are inactive you don’t make mistakes. According to that proverb, then, MPEG, who has done a lot of things, must have done a lot of mistakes.

So, in this article I imagine there is a prosecutor making charges to MPEG and a defence attorney speaking on behalf of the defendant. Whoever wants to be part of the jury is welcome to join.

Charge #1: MPEG standards are losing market share

Prosecutor: In several markets the importance and adoption of MPEG standards is decreasing.

Attorney: MPEG standards has never been “catch all” solutions. Originally MPEG-1 Video went nowhere because interactive video went nowhere. MPEG-1 Audio layer 2 had a very slow adoption, but MP3 was a huge success. When some Far East markets needed disc-based video distribution without using DVD, MPEG-2 Video and Audio layer 2 became a huge success (1 billion Video CD players). MPEG-2 Video was universally adopted but Audio faced strong competition from proprietary solutions. The 4:2:2 profile of MPEG-2 Video faced strong competition from a market solution. Companies vying for video distribution on the nascent internet turned their back to MPEG-4 Visual and looked at proprietary solutions when they learned that they would have to pay for the amount of video streamed. A more market friendly licensing of MPEG-4 AVC gave a big boost to the adoption of the standards.

Conclusion? As the then candidate Bill Clinton said: “It’s the economy, stupid”.

That was a conclusion, but maybe too hasty. It is true that, with MPEG-2 Video and MPEG-4 AVC, MPEG standards have dominated the video distribution market. Things, however, are changing. The appearance of bigplayers on the internet distribution market has favoured the appearance of proprietary solutions. VP8, VP9 and AV1 had/have millions of users, while the penetration of MPEG-H HEVC is still low. Some, including myself, think that a confused HEVC licensing situation, not the usability and quality of HEVC, is the cause of this situation. MPEG has nothing to say or do about licensing or about the rules that originate the current state of affairs, but MPEG-5 EVC, a standard due to be approved in 4.5 months, can unlock the situation. EVC is expected to have significant more compression performance than HEVC (still less than VVC), but a more streamlined licencing landscape.

Obviously MPEG cannot be involved in the MPEG Future initiative, but one of its action points reads:

Enhance the value of intellectual property that make MPEG standards unique, while facilitating their use

Conclusion: MPEG has its hands tied but the market has not. The people of good will who are part of MPEG Future need not have their hands tied, either, when they operate within the initiative.

Charge #2: too many standards are less than successful

Prosecutor: in 30 years the defendant has produced some 180 standards, but the really successful ones out of these are maybe 1/6 of the total. This has been a waste of resources.

Attorney: it is a matter of where you stop. MPEG could have decided to place the bar of acceptance of new standard work higher and would thus have produced a smaller number of less successful standards. If it had done so, however, most likely MPEG could also not have produced some of its successful standards. Which is better: more attempts or lost opportunities?

One should not forget that, if making a market-ready product costs 1000, designing a product costs 100, making research for a product costs 10 and making a standard for that product costs 1. Companies should concentrate on the 1000s and 100s, not on the 1s.

This does not mean that MPEG has not been aware of the need to increase the number of successful standards. In 2014 MPEG did try to address the problem. It dreamed of a new structure where communication with the the world was not only at the end of the process (“look, this is the standard we’ve developed for you”) but also at the beginning (“look, we plan to develop this standard”).

Figure 1 – The MPEG organisation designed in 2014

The plan went nowhere. As it is perceived as a technical body, MPEG was unable to find the leader of the industry liaison subgroup, never mind the members.

The problem of raising the rate of successful standards, however, does not go away. The MPEG Future initiative has this as a key element of its proposal to make MPEG a subcommittee. Unlike the structure of Figure 1, however, the Industry Liaison and Requirements functions are no longer sequential. They operate in co-competition to provide proposals for action that take into account the unique positioning of MPEG as the body producing standards that are anticipatory but aim to address markets of million or billion products, services and applications. It does that by blending technology evolution trends (as assessed by Technical Requirements) against the expected market need (as assessed by Market Needs).

Figure 2 – The MPEG organisation proposed for MPEG as an SC

Conclusion: applying the English proverb “you cannot have your cake and eat it”, one can say that if you want to have many successful standards you must make many attempts. Nevertheless, injecting market awareness in the standards process will help.

Charge #3: the MPEG structure is too static

Prosecutor: In the last ten years, the organisation of MPEG has not changed appreciably.

Attorney: I am not sure that an organisation that changes its structure frequently is necessarily a healthy organisation. There are many companies out there who are in constant restructuring and their performance can hardly be described as excellent. In the last few years, however, MPEG has concentrated in making its organisation more flexible and suitable for the development of its standards.

Since its early days MPEG had Subgroups and, within it, Units addressing specific areas who are typically temporary, but can also be permanent. Today MPEG can claim to have a flat organisation whose elementary Units belong to specific Subgroups and carry out work for a particular standard interacting with other units under the supervision the Technical Coordination made up of Convenor and Subgroup Chairs. This is represented in Figure 3 where the letter before the Unit indicates the Subgroup the Unit belongs to.

Figure 3 – The flat MPEG organisation

This organisation is functional to the development of the highly integrated MPEG standards.

Conclusion: an organisational chart is typically viewed the sexy part of a company. In MPEG it is not necessarily the most important one.

Charge #4: MPEG does not collaborate with JPEG

Prosecutor: a sizeable part of MPEG deals with (moving) pictures and JPEG deals with (static) pictures. There are a lot of technical commonalities, still the two groups do not collaborate.

Attorney: Collaboration and brotherhood are nice words. Both, however, have to be put in a concrete setting to give them a practical meaning. Here I will only talk about the former.

The word collaboration can be easily applied to a group of researchers belonging to different organisations when they write a paper where each author brings different contributions.

The raison d’être of a standards committee is, well, … making standards. Therefore, “collaboration between two standards committees” can only be invoked when there is a need for a standard serving the constituencies of both committees. MPEG serves the industries engaged in the “distribution of moving pictures and audio on broadcast, broadband, mobile and physical media” while JPEG serves the industries engaged in “digital photography and distribution of still image sequences”. There is little intersection between the two as I am going to explain.

MPEG and JPEG may very well share some technologies, but the needs of their constituencies are different. For instance, JPEG is close to approving the FDIS of a standard for compression of light field images because its industries feel that they are ready to implement such a standard. On the other hand MPEG is just carrying out an exploration on dynamic light fields, because its industries are nowhere ready to do the same. If and when MPEG will develop a standard for dynamic light fields, it will most likely use a different capture format (JPEG has used plenoptic 1 in its standard and MPEG is using plenoptic 2 in its exploration). Therefore, little if anything of what has been developed by JPEG for its light field image compression standards, will be used by MPEG.

This is not a new story. In all 6 generations of MPEG moving picture coding standards no need or significant benefit has ever been found that justifies the adoption of a JPEG standard as the still picture coding mode of an MPEG standard. This assessment holds also for point clouds and, as said above, is expected to hold for other digital moving picture representation technologies such as light fields.

On the other hand MPEG has developed the ISO Base Media File Format (ISOBMFF), a standard used also by several non-ISO bodies such as 3GPP and AOM. Collaboration has happened because JPEG, too, is using ISOBMFF for its standards.

Collaboration need not be the exception. It is an undertaking to reach a common goal – when it exists. Otherwise it is just a way to further knowledge. This is certainly a useful thing, but not if it is done within a standards committee, because it becomes a distraction.

Collaboration is important and if it is to happen successfully, the following steps are needed:

  • Verify the joint industrial interest in the common project
  • Define requirements of the potential common standard
  • Draft the Terms of Reference for joint effort
  • Agree on
    • Work plan and timeline of the joint effort
    • Who lead(s) the work: just one or one from each committee
    • Independent or collocated meetings with either parent committee
    • Which committee provides the secretariat
    • How ballot comments are resolved

Conclusion: collaboration must be done for what it means ethymologically – doing work together.

Conclusions

The prosecutor and the defence attorney have made their pleas. Now it is for the jury to reach a verdict.

Posts in this thread

 

An action plan for the MPEG Future community

Introduction

A group of people caring about the future of MPEG has established MPEG Future. According to the MPEG Future Manifesto, the group plans to

  1. Support and expand the academic and research community which provides the life blood of MPEG standards;
  2. Enhance the value of intellectual property that make MPEG standards unique while facilitating their use;
  3. Identify and promote the development of new compression-related standards benefitting from the MPEG approach to standardisation;
  4. Further improve the connection between industry and users, and the MPEG standards group
  5. Preserve and enhance the organisation of MPEG, the standards group who can achieve the next goals because it brought the industry to this point.

In this article I would like to contribute to item 4 “Further improve the connection between industry and users, and the MPEG standards group”. I will examine what MPEG currently does or plans to do in the area of Video and Point Cloud compression and will solicit feedbacks and comments from the MPEG Future community.

Video and Point Cloud compression in MPEG

Video and Point Cloud compression are two of the most important MPEG work areas. In MPEG Video and Point Cloud compression is quite articulated because 6 ongoing projects. Tn temporal order of standard approval they are:

  1. Video-based Point Cloud Compression (V-PCC), part 5 of MPEG-I. This standard, to be approved by MPEG in April 2020, will provide a standard means to compress 3D objects whose points on the surface are “densely” captured with (x,y,z) coordinates, colour and depth (and potentially other attributes as well). V-PCC is the natural response to market needs for a standard that compress point clouds without requiring the deployment of new hardware. Here you can see an example of a dynamic point cloud suitable for V-PCC compression.
  2. Essential Video Coding (EVC), part 1 of MPEG-5. This standard, to be approved by MPEG in April 2020, will provide an improved compression efficiency over HEVC, that is less than VVC’s, but with promises to offer a simplified IP landscape. EVC is thus designed to provide a solution to those who need compression but are not ready or capable to access the powerful but sophisticated VVC technology.
  3. Versatile Video Coding (VVC), part 3 of MPEG-I. This standard, to be approved by MPEG in July 2020, is expected to provide the usual improvement of an MPEG video coding standard over the next generation’s. VVC natively supports 3 degrees of freedom (3DoF) viewing (HEVC did already), especially in conjunction with Omnidirectional Media Format (OMAF), part 2 of MPEG-I. VVC is the latest entry of the line MPEG video compression standards and crowns 30 years of efforts by reaching a compression of 1,000 over uncompressed video.
  4. Low Complexity Enhancement Video Coding (LCEVC), part 2 of MPEG-5. This standard, to be approved by MPEG in July 2020, will specify an enhancement stream, suitable for software processing implementation, that enables an existing device to offer higher resolution at competitive quality and with sustainable power consumption.
  5. Geometry-based Point Cloud Compression (G-PCC), part 9 of MPEG-I. This standard, to be approved by MPEG-I in October 2020, will also provide a standard means to compress 3D objects whose points on the surface are “sparsely” captured with (x,y,z) coordinates, colour and depth (and potentially other attributes as well). G-PCC, is designed to compressed point clouds that cannot be compressed satisfactorily with V-PCC. Here you can see an example of a dynamic point cloud suitable for G-PCC compression.
  6. MPEG Immersive Video Coding (MIV), part 12 MPEG-I. This standard, to be approved in January 2021, will provide an immersive video experience to users when they navigate the scene with limited displacements (so-called 3DoF+) because the reduced number of views compressed with standard video coding technologies sent by the encoder are supplemented by the specific view requested by the view synthesised by the decoder.

Why is MPEG engaged in 6 visual coding projects?

In 30 years, the audio-visual market has evolved a lot and the mantra of “one single video compression (albeit with profiles and levels) fits all needs” no longer holds true. Users have different and often incompatible requirements:

  1. High quality is desirable, and you get more quality by increasing bitrate or the amount of technology. Bitrate has a cost but technology, too, has a cost. Therefore, users should be able to achieve the quality they need with different combinations of bitrate and technology,
  2. 2D video has always been the bread and butter of visual applications, but 3D is an attractive mermaid that continuously morphs depending on the capturing technology that becomes available.
  3. Silicon has always been considered a must for constrained devices. This statement continues to be true but using software that implements clever technology allowing the extension of the basic silicon capabilities without changing the device (e.g. V-PCC and LCEVC).

More to come

In 12 months the 6 MPEG projects described will all gradually achieve FDIS level and industry will take over. But MPEG does not only have these 6 projects in store. In the exploration pipeline MPEG has 4 more activities that may become projects.

More Point Cloud Compression

The release of V-PCC and G-PCC does not mean that point cloud compression activitieswill stop. MPEG must listen to what industry and users will say about the first release of these standards and will add what is required to achieve more satisfactory user experiences.

As MPEG in Point Cloud Compression is still in the learning curve, expert more breakthroughs.

Six Degrees of Freedom (6DoF)

With this standard MPEG expects to be able to provide “unrestricted” immersive navigation where users can not only yaw, pitch and roll their heads (as in 3DoF) but move in the 3D space as well. The project is connected with the current 3DoF+ project in ways that are still to be fully determined.

Light fields

MPEG has been exploring the use of light field cameras to generate light field data. Technology, however, is evolving fast form plenoptic-1 cameras (microlens array placed at the image plane of the main lens) to plenoptic-2 cameras (microlens array placed behind the image plane of the main lens to re-image the captured scene). Read here about the challenges light field compression has to overcome for the technology to be deployable to millions of users.

Video coding for machines (VCM)

This exploration is driven by the increasing number of applications that capture video for (remote) consumption by a machine, such as connected vehicles, video surveillance systems, and video capure for smart cities. The fact that the user is a remote machine does not change the fact that transmission bandwidth is often at a premium and compression is required. However, unlike traditional video coding where users expect high visual fidelity, in video coding for machines, “users” expect high fidelity of the parameters that will be processed by the machine users.  The bold idea of VCM is to have a unified video coding system for machines, where one type of machine is human. Technology is expected to be uniformly based on neural networks.

As noted here, however, VCM is not an entirely new field for MPEG. Twenty years ago, MPEG was working on MPEG-7 (multimedia description) and a figure of that time (below) clearly depicts what MPEG had in mind: MPEG-7 was eminently targeted to searching and browsing applications but the users were both humans and machines.

The same figure with minor modifications can be used for VCM. Feature extraction is probably going to be automatic (but manual feature extraction is not excluded because this is an encoder matter). Storage is probably not needed but cannot be excluded either. Search/query and browse are not specifically the target of VCM but cannot be excluded either.

The MPEG Future community should react

MPEG Future participants have a role to play in providing feedbacks on the future of video and point cloud coding. They can be in 3 different directions

  1. Feedbacks on the suitability of the 6 standards that MPEG will release in the next 12 months to market needs;
  2. Comments on the current explorations that will hopefully be converted to MPEG standards, Comments can be on 1) the identified applications, 2) their requirements and 3) the technologies most suitable to support them;
  3. Proposals of new applications.

This will not be the end of the involvement of the MPEG Future community. However, it will be a big step forward.

Posts in this thread

Which company would dare to do it?

To get an explanation of the question in the title and an answer, the reader of this article should read the first 5 chapters. If in a hurry, the reader can jump here.

  1. MPEG is synonym of growth
  2. Manage growth with organisation
  3. MPEG does not only make, but “sells” standards
  4. A different approach to standards
  5. Interoperability is not just for commercials
  6. Time to answer the question

MPEG is synonym of growth

MPEG is a unique group in the way it has grown to what it is today. Thirty-one years ago it started as an experts group for video coding for interactive applications on compact disc (it became a working group only 3.4 years after its establishment). A few months after its establishment it added audio coding and systems aspects related to the handling of compressed audio and video. Two years later it moved to digital television (MPEG-2) shedding the “interactive applications on compact disc” attribute, and then it moved to audio-visual applications for the nascent fixed and mobile internet (MPEG-4), then to media delivery in heterogeneous environments (MPEG-H), then to media delivery on the unreliable internet (MPEG-DASH), then to immersive media (MPEG-I), then to genome compression (MPEG-G).

The list of MPEG projects given above is a heavily subsampled version of all MPEG projects (21 in total). However, it gives a clear proof of the effectiveness of the MPEG policy to extend to and cover fields that are related to compression of media. Now MPEG is even working on DNA read compression and has already approved 3 parts of the MPEG-G standard.

The MPEG story, however, is not really in line with the ISO policy on Working Groups (WG). WGs are supposed to be established and run like projects, i.e. disbanded after the task has been achieved. MPEG, however, has been in operation for 31 years because the field of standards for moving pictures and audio is far from being exhausted.

Today MPEG standards cover the following domains: Video Coding, Audio Coding, 3D Graphics Coding, Font Coding, Digital Item Coding, Sensors and Actuators Data Coding, Genome Coding, Neural Network Coding, Media Description, Media Composition, Systems support, Intellectual Property Management and Protection (IPMP), Transport, Application Formats, Application Programming Interfaces (API), Media Systems, Reference implementation and Conformance.

Manage growth with organisation

The MPEG story is also not really in line with another ISO policy on WGs. WGs are supposed to be “limited in size”. However, MPEG counts 1500 experts registered and an average 500 experts attending its quarterly meetings, These last one weeks and are preceded by almost another week of joint video projects and ad hoc group meetings.

The size of MPEG, however, is not the result of a deliberate will to be outside of the rules. It is the natural result of the expansion of its programme of work.

Since the early days, MPEG had a structure: first a Video subgroup, then an Audio subgroup, then a Systems subgroups. The Test subgroup was formed because there was a need to test the submissions in response to the MPEG-1 and MPEG-2 Video Calls for Proposals, the Requirements subgroup came with the need to manage the requirements of all industries interested in MPEG-2 and the 3D Graphics subgroup came with MPEG-4. The Communication subgroup came later prompted by the need to have regular communication with the outside world.

The MPEG organisation is not the traditional hierarchical organisation. Inside most subgroups there are units – permanent or established on an as needed basis – that address specific areas. Units report to subgroups but interact with other units, inside or outside their subgroups, prompted by a group that includes all subgroup chairs.

This unique flat organisation allows MPEG’s 500 experts to work on multiple projects, while allowing those interested in specific one to stay focussed. At the last October 2019 meeting experts worked on 60 parallel activity tracks and produced ~200 output documents. This is also possible because, since 1995, MPEG has been using a sophisticated online document management system now extended to make available information on other parallel meetings, to support the development of plenary results, to manage the MPEG work plan etc.

MPEG not only makes, but “sells” standards

Formally MPEG was not the result of a conscious decision by National Bodies (eventually it became so) but it was prompted by a vision. Therefore, MPEG never had a “guaranteed market”: its standards had to “sell” for MPEG to continue to exist. For that to happen, MPEG standards had to be perfectly tuned to market needs. In other words there had to be a closed relationship between supplier (MPEG) and customers (industries).

Therefore, searching for and listening to customers is in the MPEG DNA:

  1. For its first MPEG-1 standard MPEG found as customers consumer electronics (interactive video on compact disc and eventually Video CD), telcos (video distribution on ADSL-1), radio broadcasters (digital audio broadcasting). MP3 is a different story…
  2. For its MPEG-2 standard MPEG had to convince all types of broadcasters – terrestrial, cable and satellite – all consumer electronics companies and the first IT companies
  3. For its MPEG-4 standard MPEG has to convince the first wave of IT companies and to start talking to mobile communication companies
  4. And so on… Every new subsequent MPEG project brought some new companies of existing industries or new industries.

Above I have used the verb “brought”, but this is not the right verb, certainly not in the early years. MPEG made frenetic efforts to talk to companies, industry fora and standards organisations, presented its plans and sought requirements for its standards. The result of those efforts is that today MPEG can boast a list of several tens of major industry fora and standards organisations such as (in alphabetic order): 3GPP, ATSC, DVB, EBU, SCTE, SMPTE and more.

A different approach to standards

This intense customer-oriented approach is not what other JTC 1 committees do. If you want to build Internet of Things (IoT) solutions and you turn to standards produced by JTC 1/SC 41 – Internet of Things and related technologies, you are likely to only find models and frameworks. If you want to implement a Big Data solution and you turn to JTC 1/SC 42 – Artificial intelligence, you will find again models and frameworks but no trace of API, data formats or protocols.

Both MPEG and other JTC 1 subcommittees develop standards before industry has a declared need for them. However, they differ in what they deliver: implementable standards (MPEG) vs models and frameworks (other committees).

MPEG commits resources to develop specifications when implementation technologies may not be fully developed. Therefore, MPEG can assemble and possibly improve its specifications by adding more technologies to the extent it can be shown that they provide measurable technical merits.

The danger is that MPEG may very well spot the right standard, but its development may happen too early with the risk that the standard may be superseded by a better technology at the time industry really needs the standard. Conversely, the standard may arrive too late, at a time when companies have made real investments for their own solutions and are not ready to discount their investments.

This is one reason why, within JTC 1, MPEG (a WG) has produced more standards than any other subcommittee (note that a subcommittee includes several WGs). MPEG’s policy is to try and develop a new “business” area with a standard that may turn out not to be adopted by industry, not to risk losing an opportunity.

By providing models and frameworks, other committees take the approach of creating an environment where industry players share some basic elements on top of which an ecosystem of solutions with certain degrees of interoperability may eventually and gradually appear.

It is very difficult to make general comparisons, but it is clear that MPEG standards create markets because industry knows how to make products and users know they buy products that provide full interoperability. A proof of this? In 2018 MPEG-enabled devices had a global market value in excess of 1 T$ and MPEG-enabled services generated global revenues in excess of 500 B$. In the same year MPEG standards had a far reaching impact on people at the global level: at the global level there are 2.8 billion smartphones of which 1,56 billions were sold in 2018 alone (see here) and there are ~1.6 billion TV sets in global use by ~1.42 billion households serving a TV viewing audience of ~4.2 billion in 2011 (see here).

Interoperability is not just for commercials

MPEG is also unique because it has succeeded in converting what would otherwise be a naïve approach to interoperability to a practical and effective implementation of this notion. Back in 1992 MPEG was struggling with a problem created by the success of its own marketing efforts: many industries from many countries and regions had believed in the MPEG promise of a single international standard for audio and video compression, but actually industries and countries or regions had different agendas. Telcos wanted a scalable video coding solution, American broadcasters wanted digital HDTV, European broadcasters wanted digital TV scalable to higher resolutions, some industries sought a cheap solution (RAM was an important cost element at that time) while others could afford more expensive solutions.

The solution was obvious but making a standard that included all requirements for all users was out of question. MPEG learned from the notion of profile developed by JTC 1/SC 21 – Open systems interconnection (OSI):

“set of one or more base standards, and, where applicable, the identification of chosen classes, subsets, options and parameters of those base standards, necessary for accomplishing a particular function”

MPEG madeit practically implementable for its own purposes by interpreting “base standards” and “chosen classes, subsets, options and parameters of those base standards” as “coding tools”, e.g. a type of prediction or quantisation. Starting from MPEG-2 (but actually the profile and level notion – at that time not crystal clear yet – was already present in MPEG-1) MPEG standards conceptually specify collections of tools and descriptions of different combinations of tools, i.e. the “profiles”.

Next to quality, profile is the feature that has contributed the most to the success of MPEG standards. Today, OSI profiles are nowhere to be seen, but the world is full of products and services that implement MPEG profiles.

Time to answer the question

I will try now to give a meaning to “it” in the question of the title of this article “Which company would dare to do it?”

MPEG is a standards group, not a company. Still MPEG operates like a company: it produces standards to maximise the number of its customers by satisfying their needs. The measure of MPEG performance used here would probably not satisfy the criteria of an accountant, but I consider enabling a market of 1.5 T$ p.a. to be something akin to “profits” while expenses can be measured to be 500 (experts)*4 (meetings/year)*7.5 (meeting days)*1000 ($/day) = 15 M$.

What would you think of board of directors who wanted to reorganise a “company” whose Operating Expense Ratio (OER) is 0.001% (or its Revenues/Expenses ratio is 100,000)?

Posts in this thread