Design and build the future of MPEG

You are currently viewing Design and build the future of MPEG

Design and build the future of MPEG

Established in 1988, the Moving Picture Experts Group (MPEG) has produced some 180 standards in the area of efficient distribution on broadcast, broadband, mobile and physical media, and consum­ption of digital moving pictures and audio, both conventional and immersive, using analysis, compression and trans­port technologies. It has also produced 5 standards for efficient storage, processing and delivery of genomic data.

With its membership of ~1500 registered and ~600 attending experts, MPEG has produced more standards than any other JTC 1 subcommittee. The standards enable a manufacturing and service business worth more than 1.5 T$ p.a.

We expect that in the future industry will require more MPEG standards for the following areas

  1. Rectangular video coding. As ~80% of world communication traffic is video-based, new more efficient video coding standards will continue to be of high interest. These will likely incor­porate new video compression technologies, driven by extensive research both in algo­rithm and hardware domain, e.g. in the area of Machine Learning. The market will also demand standards with higher quality user experience such as high-fidelity near-live video coding.
  2. Audio coding. As for video, future audio coding standards could be in the wake of the MPEG Audio coding family, be Machine Learning based or a be mix of the two. High fidelity near live audio coding standards will also be required to provide new forms of user experience.
  3. Immersive media will require coding standards that allow a user to move unres­tricted in a 3D space, for Point Clouds for human and machine use, for realistic meshes and textures and for light fields. Immersive audio coding standards providing an even more intensely satisfying user experience than the MPEG-I Audio standard currently being developed will also be required.
  4. 3D scene representation standards will be needed to recreate realistic 3D scenes populated by media objects en­coded with other MPEG immersive media coding standards. It is expected that standards for effective human interaction with media will also be required.
  5. Quality of experience will likely differentiate the market of products and services through the use of sophis­ticated technologies that reduce the amount of information needed to transmit future media types. MPEG must support the manufacturing and service industries by offering standards with features beyond those of the currently available standards, such as machine analytics for quality measure.
  6. Video Coding for Machines (VCM) is a new area of endeavour primarily designed to serve remotely located machines and, potentially, human users as well. Audio Coding for Machines will likely be a necessary complement to VCM. Machine vision standards, e.g. for object classification, will continue the tradition of CDVS (image search) and CDVA (video analysis) standards and will mostly be based on similar technologies as VCM. Internet of Media Things standards will also serve as a framework for the above standards.
  7. Application formats will be required by industry to facilitate the use of the wide range of available and future MPEG basic media coding standards by configuring MPEG technologies to suit application needs.
  8. Transport of compressed media standards have always been enablers of the use of MPEG media coding standards and we expect the need for them will continue. Ways to cope with emerging constraints on video and audio delivery over internet, such as those witnessed during the Covid-19 emergency, should be considered. New networks, e.g. 5G and other future networks, will likely drive the requests and MPEG should actively work together with the relevant bodies.
  9. Compression of genomic data, initiated 5 years ago in collaboration with TC 276, has shown that MPEG technologies could be successfully applied to new non-media areas. Standards will continue to be produced to respond to this new client industry.
  10. MPEG has realised that Compression of other data is a broad new area that can benefit from its approach to standardisation in support to new industries such as Medical, Industry 4.0, Automotive etc. As MPEG does not obviously have domain specific competence, it plans to identify the needs and develop standards for these new areas jointly with the relevant committees.

The above is an ambitious program of work in an expanding area of standardisation that heavily depends on sophisticated and evolving technologies. MPEG is confident it will be able to execute this program because it has learnt how to foster and actually fostered innovation with its standard and because, since its early days, it has innovated the way a standards committee operates, and developed a set of procedures and operations that MPEG calls its “modus operandi”.

Below we will examine how MPEG’s “modus operandi” will be exercised and adapted to the new challenges of the work program.

No one controls MPEGConnection with industries
Connection with academiaTechnology-driven standards meeting market needs
Standards with and for other committeesPlans for future common standards
Standard (co-)developmentEffectiveness of MPEG role
Role of ICT tools in standardisation 

No one controls MPEG. The key differentiating element is that no entity or industry “controls”  MPEG. At all levels, proposals follow a bottom-up path and are accepted, not because of their origin but because of their value, by a mostly flat internally dynamic organisation that fosters the constructive flow and interaction of ideas. The manag­ement monitors the activities of the different organisational units, identifies points of conver­gence, interaction or contention, and creates opportunities to resolve issues. This element of the modus operandi must be preserved.

Connection with industries. The success of MPEG standards is largely based on the establishment of efficient connections with the client industries. MPEG will continue to interact with them providing solutions to their cus­tomers’ needs. However, we expect that formal liaisons may soon become insufficient because MPEG standards heavily depend on rapidly evolving techno­logies, such as capture and presentation, which have a strong impact on its standards but have a scope outside of the MPEG purview. MPEG will need to identify and activ­ate relationships with the industries who are developing the new media technologies that will drive the development of its new standards. At the same time MPEG should enhance its relationships with the content industry that provides the ultimate user experience using MPEG standards, again using a network with varying capabilities to address devices that could have graded tiers of computational complexity.

Connection with academia. Another reason for the success of MPEG standards is MPEG’s strong links with academia and research organisations, an area that accounts for ~¼ of its total mem­bership and contributes to sustaining the vitality of its standards activities. MPEG has always conducted exploration on promising new technologies, before standardisation can begin. However, the number of directions that require explorations are growing fast and MPEG resources may not be sufficient in the future to assess the potential of all candidate technologies. MPEG will need to establish more organic links with academia and create more opportunities to stimulate, get input from and possibly collaborate with academia.

Technology-driven standards meeting market needs. MPEG’s modus operandi has been inspired by applications as supported by technology and we think that in the future this must continue. However, it is imperative that this be complemented by a form of internal competition with new MPEG members who are market-driven. The goal is to strike the right balance between tech­nol­ogy-push and market-pull, i.e. to be able to identify targets that are both technology-driven and attractive to the market. MPEG should create a market needs group who dialectically interacts with the requirem­ents group to fortify proposals for new work, keeping timely delivery of its standards as a high priority issue. MPEG should better assert its active presence on all aspects of the standardisation workflow: industry, market, research and academia.

Standards with and for other committees. MPEG’s modus operandi has allowed MPEG to manage the development of technology-heavy stan­dards, some of which have involved hundreds of person-years. Most standards have been developed with internal resources. However, other standards have been developed jointly with entities outside MPEG at different level of integration: e.g. with ITU-T SG 16, with SC 24 for augmented reality, with ISO TC 276 for genomics etc. Several MPEG standard extensions have been developed in response to the need of other committees (e.g. JPEG, 3GPP, SCTE etc.). This fluid interaction must be enhanced by broadening MPEG’s operation, as media technologies interact more and more with other technologies in products and services and use of compression spreads to more domains.

Plans for future common standards. We envisage the need for MPEG to continue to develop common standards with other committees, in partic­ular with ITU-T SG 16 on video coding and with TC 276 Biotechnologies on gen­omic data compression. Augmented Reality will provide opportunities to develop common standards with JTC 1/SC 24. We also see more work with other commi­ttees as an essential component of MPEG future. Examples are JTC 1/SC 41 on a new phase of Internet of Media Things (IoMT), JTC 1/SC 42 on Machine Learning for compression and Big Data, starting from MPEG’s Network-Based Media Processing (NBMP) standard, and TC 215 Health Informatics on data compression. In a not-so-distant future MPEG should seek new standardisation opportunities in the area of Industry 4.0 and automotive. It is an ambitious program that can be implemented using MPEG experience and representation.

Standard development models. MPEG has learned that effective standardisation can be achieved if experts work shoulder-to-shoulder. How can this be achieved with an expanded scope of work and increased number of collaboration parties? One answer lies in the experience MPEG has gathered from its long history of online collaborations and the fully online 130th meeting attended by ~600 experts. In normal conditions, a large part of MPEG meetings will continue to be held face-to-face. However, many standards can and should be developed jointly with other organisations mostly via online meetings. These can be easily integrated with the rest of the MPEG workflow because they deal with specific topics.

Effectiveness of MPEG role. MPEG will need to develop new metrics, beyond the number of standards produced (MPEG is already the largest producer of standards in JTC 1) and apply them in a regular fashion to revise the effectiveness of its role. While MPEG standards continue to have loyal customers, their importance in other domains is less significant. The past is often so sticky that MPEG must make sure that its past successes do not make it an “also-ran” in the future. This does not mean that MPEG must renege its practices, but that it must adapt its business model to the new challenges. MPEG needs to reassert its mission to develop the best standards satisfying agreed requir­ements, thus continuing to provide opportunities to remunerate good IP for good standards. At the same time, MPEG should develop a strategy to handle the growing competition to its standards. One way is by enhancing the role of reference software, as an equivalent normative version of MPEG standards. Another is to envisage conditions that may facilitate expeditious development of licensing terms by the market.

Role of ICT tools in standardisation. Many aspects of the MPEG modus operandi heavily rely on ICT support. In 1995 MPEG was probably the first committee of its size and structure to adopt electronic document upload and distribution. In the last 15 years MPEG had the privilege to benefit from the support of Institut Mines Telecom and especially Christian Tulvan who has continuously designed new solutions that allow MPEG to be a fluid and flat organisation where every member can contribute by knowing what happens where. MPEG expects to be able to rely on their unique support to implement more features that will enhance the quality and quantity of its standards through enhanced and qualified participation.

Posts in next thread

Posts in this thread