MPEG communicates

Introduction

MPEG standards are great communication tools and MPEG itself is – or tries to be – a good communicator, of course using all available media.

This article is for those who want to be informed about MPEG, without having to digest one of its handy 500-page standards 😊

MPEG web site

Almost everything that will be mentioned in this post can be found, not necessarily in an easy way, in the MPEG web page: press releases, MPEG column, video tutorials, White papers, investigations and technical notes, ad hoc groups, events, social networks, liaisons and meetings. The next meeting will be held at Marrakesh, MA from 14 to 18 January 2019

Press releases

Since its early days, MPEG took care of informing the world of its work. At the beginning this was done only on major occasions. Now MPEG does that systematically at every meeting. The news considered the most important news gets the headlines and all other achievements get a note. The rule is to mention in a press release all Calls for Evidence (CfE) and Call for Proposals (CfP), and the standards under development that reach the stage of Committee Draft (CD) or Final Draft International Standard (FDIS). News are prepared by the relevant group chairs, edited and distributed to the press by Prof. Christian Timmerer of University of Klagenfurt.

Let’s take as look at the press release from MPEG 124 (Macau SAR, CN) to have an example.

  1. The headline news is “Point Cloud Compression – MPEG promotes a video-based point cloud compression technology to the Committee Draft stage” because we believe this will be a very important standard.
  2. The second news is “MPEG issues Call for Proposals on Compressed Representation of Neural Networks”.
  3. Two video compression-related CfPs “MPEG issues Call for Proposals on Low Complexity Video Coding Enhancements” and “MPEG issues Call for Proposals for a New Video Coding Standard expected to have licensing terms timely available”
  4. News on “Multi-Image Application Format (MIAF) promoted to Final Draft International Standard”
  5. Anticipation of a new CfP “3DoF+ Draft Call for Proposal goes Public”.

The press release page containing all press release since January 2006 here.

If you want to be added to the distribution list, please send an email to Christian.

MPEG Column

When possible the news in the press releases have links to documents relevant to the news. We try to avoid links to the mythical 500-pages documents but clearly knowing more about a news item typically has a higher entry level.

With the MPEG column MPEG tries to be facilitate understanding of what its standards do. At every meeting, brief notes are published that explain the purpose and, to some extent, the working of MPEG standards.

Want to know about High Dynamic Range (HDR)? the Common Media Application Format (CMAF)? The new standard to view omnidirectional video (OMAF)? By going here you will see all articles published in the column and will have the opportunity to get answers to many questions on these and other MPEG standards.

The MPEG column is managed by the professional journalist Philip Merrill who is able to understand MPEG standards and explain them to the public.

Do you think an article on a particular standard would be of interest? Please send email to the chairman of the Communication Group Prof. Kyuheon Kim of Kyunhee University. We will do our best to satisfy your request.

Video tutorials

How could MPEG miss the opportunity to have its own series of “MPEG standards tutorials”, of course using its audio and video compression standards? By going to Video tutorials on MPEG standards you will be able to understand what MPEG has done to make its standards “green”, what is the Multimedia Preservation Application Format that manages multimedia content over the ages, what is the High Efficiency Video Coding (HEVC) standard, what is MPEG-H 3D Audio, what is MPEG Media Transport (MMT) used in ATSC 3.0, what is Dynamic Adaptive Streaming over HTTP (DASH) for audio-visual strteaming over the internet and much more.

The content is delivered by the best experts in the field. The videos that you see are the result of the shooting and post-processing performed by Alexis Tourapis.

White papers, investigations and technical notes

MPEG makes its best efforts to provide the smoothest entry path to its standards and several types of publicly accessible papers are functional to this strategy. White papers are published with the goal to

  1. Communicate that we are investigating some promising ideas as in Investigations about parts of standards
  2. Describe the purpose of an entire suite of standards, as in the case of White papers about standards or for a single part of a standard like in White papers about parts of standards
  3. Provide specific guidance about the use of standards as in Technical notes about parts of standards.

As a rule, the purpose of white papers is not to describe how the technology work but about what the standard is for, the problems it solves and the benefits that MPEG expects users will get by using it.

Investigations, White papers and Technical notes can be found here.

Standards

ISO is a private association registered in Switzerland. Standards are developed pro bono by participants in the working groups, but the cost of the organisation is covered by the sale of standards and other sources. Therefore you should not expect to find ISO standards on the public MPEG web page. If you need a standard, you should go to the ISO web site where your can easily buy online all the standards on sale. In some cases, MPEG requests ISO to make standard public because the standards is particularly relevant or because the standard is already publicly available (as is the case of all standards developed jointly with ITU-T).

MPEG posts all public documents at Standard documents from the last meeting, e.g. Use cases, Requirements, Calls for Evidence, Calls for Proposals, Working Drafts up to Committee Drafts, Verification Test results and more. MPEG does this because it wants to make sure that the industry is aware of, can comment on, and contribute to what to develop standards.

Ad hoc groups

Since 1990 MPEG has created ad hoc groups (AHG). According to the rules “AHGs are established for the sole purpose of continuing work between consecutive MPEG meetings”, but they are a unique way to have work done outside MPEG meetings in a coordinated way. The scope of an AHG, however, is limited by the rule: “The task of an AHG may only cover preparation of recommendations to be submitted to MPEG”.

AHGs are not permanent organisations but established at the end of a meeting to last until the following meeting, To have a feeling of what AHGs are about you can see the AHGs established at 124th MPEG meeting.

AHGs are mentioned as part of MPEG communication because anybody can join the email reflector of an AHG and even attend AHG meetings.

MPEG events

In certain cases, MPEG organises events open to the public. Some events are held during and co-located with an MPEG meeting, but events outside MPEG meetings are also held. These are some of the goals an MPEG event can have:

  1. To present a particular standard under development or just released as in Workshop on MPEG-G (Shenzhen, October 2018)
  2. To introduce the MPEG workplan such as MPEG Workshop on Immersive Services Roadmap (Gwangju, January 2018)
  3. To demonstrate what the industry is doing with an MPEG standards such as OMAF Developers’ Day (Gwangju, January 2018)
  4. To frame a particular segment of the MPEG activity in the general context of the industry such as Workshop on 5G/ Beyond UHD Media (La Jolla, February 2016).

Reference software and conformance

MPEG standards can be made public by a decision of the ISO Central Secretariat. MPEG requests ISO to make all reference software and conformance standards publicly available. The rationale of this request is that if developers look at the reference software, they need to buy the standard to make a conforming implementation. To create a healthy ecosystem of interoperable products, services and applications – the conformance testing suites, too, must make freely available. This entices more users to buy the standard.

Social networks

MPEG has a Twitter account @MPEGgroup (https://twitter.com/mpeggroup). This is used by a group of MPEG social champions to spread information on the currently hottest MPEG topics: MPEG-I Requirements, Neural Network Compression, MPEG-G, OMAF, File format, Network Based Media Processing, MPEG-I Visual (3DoF+ and 6DoF), Audio, Point Cloud Compression, Internet of Media Things.

Please subscribe to receive brief notes on MPEG-related news and events.

Liaisons

MPEG develops standards technologies that are used by many industries all over the world. MPEG requests to liaise with many standards committees and industry fora for several purposes: to get use cases and requirements, to jointly develop standards, to promote adoption of the standard once it has been developed and to receive further requests for new functionalities.

These are some of the organisations MPEG has liaisons with: the Third Generation Partnership Project (3GPP), Advanced Television Systems Committee, Inc. (ATSC), Internet Engineering Task Force (IETF), Society of Motion Picture and Television Engineers (SMPTE), Audio Engineering Society (AES), European Broadcast Union (EBU), Hybrid Broadcast Broadband TV (HbbTV), Society of Cable Telecommunications Engineers (SCTE), World Wide Web Consortium (W3C) and many more.

Administrative documents

At every meeting MPEG publishes several documents – that I call “administrative” for lack of a better name – but are very important because they include organisational information. The following documents relate to MPEG 124 (Macau SAR, October 2018):

  1. list of ad hoc groups established at the meeting
  2. call for patents related to standards under development
  3. list of MPEG standards produced since day 1 and those planned
  4. MPEG work plan with a summary description of all activities under way including explorations
  5. MPEG timeline with planned dates of development of all standard
  6. MPEG terms of reference
  7. MPEG schemas
  8. MPEG URIs.

Conclusions

MPEG is fully aware of the social and industrial implications of its standards. This post conveys the MPEG policy of being open to the world, to the extent that ISO rules allow.

 

Posts in this thread (in bold this post)

How does MPEG actually work?

Introduction

In Life inside MPEG I have described the breadth, complexity and interdependence of the  work program managed by the MPEG ecosystem that has fed the digital media industry for the last 30 years. I did not mention, however, the actual amount of work developed by MPEG experts during an MPEG meeting. Indeed, at every meeting, the majority of work items listed in the post undergo a review prompted by member submissions. At the Macau meeting in October 2018, there were 1238 such documents covering the entire work program described in the post, actually more because that covers only a portion of what we are actually doing .

What I presume readers cannot imagine – and I do not claim that I do much more than they do – is the amount of work that individual companies perform to produce their submissions. Some documents may be the result of months of work often based on days/weeks/months of processing of audio, video, point clouds or other types of visual test sequences carried out by expensive high performance computers.

It is  clear why this is done, at least in the majority of cases: the discovery of algorithms that enable better and/or new audio-visual user experiences may trigger the development and deployment of highly rewarding products, services and applications. By joining the MPEG process, such algorithms may become part of one or more standard helping the industry develop top performance _interoperable_ products, services and applications.

But what is the _MPEG process_? In this post I would like to answer this question. As I may not be able to describe everything, I will likely have to revisit this issue in the future. In any case the  figure below should be used as a guide in the following.

How it all starts

The key players of the MPEG process are the MPEG members: currently ~1,500, ~1/4 are from academia and ~3/4 from industry, of which ~500 attend quarterly MPEG meetings.

Members bring ideas to MPEG for presentation and discussion at MPEG meetings. If the idea is found interesting and promising, an ad hoc group is typically established to explore further the opportunity until the next meeting. The proposer of the idea is typically appointed as chair of the ad hoc group. In this way MPEG offers the proposer the opportunity to become the entretrepreneur who can convert an idea into a product, I mean, an MPEG standard.

To get to a standard it may take some time: the newer the idea, the more time it may take to make it actionable by the MPEG process. After the idea has been clarified, the first step is to understand: 1) the context for which the idea is relevant, 2) the “use cases” the idea offers advantages for and 3) the requirements that a solution should satisfy to support the use cases.

Even if idea, use context, use cases and requirements have been clarified, it does not mean that technologies necessarily exist out there that can be assembled to provide the needed solution. For this reason, MPEG typically produces and publishes a Call for Evidence (CfE) – sometimes more than one – with attached context, use cases and requirements. The CfE requests companies who think they have technologies satisfying the requirements to demonstrate what they can achieve, without requesting them to describe how the results have been achieved. In many cases respondents are requested to use specific test data to facilitate comparison. If MPEG does not have them, it will ask industry to provide them.

Testing technologies for the idea 

If the result of the CfE is positive, MPEG will move to the next step and publish a Call for Proposals (CfP), with attached context, use cases, requirements, test data and evaluation method. The CfP requests companies who have technologies satisfying the requirements to submit responses to the CfP where they demonstrate the performance _anddisclose the exact nature of the technologies that achieve the demonstrated results.

Let’s see how this process has worked in the specific case of neural network compression. Note that the last row in italic refers to a future meeting. Links point to public documents on the MPEG web site.

Mtg

YY MM

Actions

120 17 Oct 1.      Presentation of idea of compressing neural networks

2.      Approval of CNN in CDVA (document)

121 18 Jan 1.      Release of Use cases and requirements 
122 18 Apr 1.      New release of Use cases and requirements 

2.      Release of Call for Test Data 

3.      Approval of Draft Evaluation Framework

123 18 Jul 1.      Release of Call for Evidence, Use cases and requirements

2.      New release of Call for Test Data

3.      New version of Draft Evaluation Framework

124 18 Oct 1.      Approval of Report on the CfE

2.      Release of Call for Proposals, Use cases and requirements, Evaluation Framework

126 19 Mar 1.      Responses to Call for Proposals

2.      Evaluation of responses based on Evaluation Framework

We can see that Use Cases and Requirements are updated at each meeting and made public, Test data are requested to the industry and the Evaluation Framework is developed well in advance of the CfP. In this particular case it will take 18 months just to move from the idea to CfP responses.

MPEG gets its hands on technology

A meeting where CfP submissions are due is typically a big event for the community involved. Knowledgeable people say that such a meeting is more intellectually rewarding than attending a conference. How could it be otherwise if participants not only can understand and discuss the technologies but also see and judge their actual performance? Everybody feels like being part of an engaging process of building a “new thing”.

If MPEG comes to the conclusion that the technologies submitted and retained are sufficient to start the development of a standard, the work is moved from the Requirements subgroup, which typically handles the process of moving from idea to proposal submission and assessment, to the appropriate technical group. If not, the idea of creating a standard is – maybe temporarily – dropped or further studies are carried out or a new CfP is issued calling for the missing technologies.

What happens next? The members involved in the discussions need to decide which technologies are useful to build the standard. Results are there but questions pop up from all sides. Those meetings are good examples of the “survival of the fittest” principle, applied to technologies as well as to people.

Eventually the group identifies useful technologies from the proposals and builds an initial Test Model (TM) of the solution. This is the starting point of a cyclic process where MPEG experts

  1. Identify critical points of the RM
  2. Define which experiments – called Core Experiments (CE) – should be carried out to improve TM performance
  3. Review members’ submissions
  4. Review CE technologies and adopt those bringing sufficient improvements.

At the right time (which may very well be the meeting where the proposals are reviewed or several meetings after), the group produces a Working Draft (WD). The WD is continually improved following the 4 steps above.

The birth of a “new baby” is typically not without impact on the whole MPEG ecosystem. Members may wake up to the need to support new requirements or they realise that specific applications may require one or more “vehicle” to embed the technology in those application or they come to the conclusion that the originally conceived solution needs to be split in more than one standard.

These and other events are handled by convening joint meetings between the group developing the technology and all technical “stakeholders” in other groups.

The approval process

Eventually MPEG is ready to “go public” with a document called Committee Draft (CD). However, this only means that the solution is submitted to the national standard organisations – National Bodies (NB) – for consideration. NB experts vote on the CD with comments. If a sufficient number of positive votes are received (this is what has happened for all MPEG CDs so far), MPEG assesses the comments received and decides on accepting or rejecting them one by one. The result is a new version of the specification – called Draft International Standard (DIS) – that is also sent to the NBs where it is assessed again by national experts, who vote and comment on it. MPEG reviews NB comments for the second time and produces the Final Draft International Standard. This, after some internal processing by ISO, is eventually published as an International Standard.

MPEG typically deals with complex technologies that companies consider “hot” because they are urgently needed for their products. As much as in companies the development of a product goes through different phases (alpha/beta releases etc., in addition to internal releases), achieving a stable specification requires many reviews. Because CD/DIS ballots may take time, experts may come to a meeting reporting bugs found or proposing improvements to the document under ballot. To take advantage of this additional information that the group scrutinises for its merit, MPEG has introduced an unofficial “mezzanine” status called “Study on CD/DIS” where proposals for bug fixes and improvements are added to the document under ballot. These “Studies on CD/DIS” are communicated to the NBs to facilitate their votes on the official documents under ballot.

Let’s see in the table below how this part of the process has worked for the Point Cloud Compression (PCC) case. Only the most relevant documents have been retained. Rows in italic refer to future meetings.

Mtg

YY MM

Actions

120 17 Oct 1.      Approval of Report on PCC CfP responses

2.      Approval of 7 CEs related documents

121 18 Jan 1.      Approval of 3 WDs

2.      Approval of PCC requirements

3.      Approval of 12 CEs related documents

122 18 Apr 1.      Approval of 2 WDs

2.      Approval of 22 CEs related documents

123 18 Jul 1.      Approval of 2 WDs

2.      Approval of 27 CEs related documents

124 18 Oct 1.      Approval of V-PCC CD

2.      Approval of G-PCC WD

3.      Approval of 19 CEs related documents

4.      Approval of Storage of V-PCC in ISOBMFF files

128 19 Oct 1.      Approval of V-PCC FDIS
129 20 Jan 1.      Approval of G-PCC FDIS

I would like to draw your attention to “Approval of 27 CEs related documents” in the July 2018 row. The definition of each of these CE documents requires lengthy discussions by involved experts because they describe the experiments that will be carried out by different parties at different locations and how the results will be compared for a decision. It should not be a surprise if some experts work from Thursday until Friday morning to get documents approved by the Friday plenary.

I would also like to draw your attention to “Approval of Storage of V-PCC in ISOBMFF files” in the October 2018 row. This is the start of a Systems standards that will enable the use of PCC in specific applications, as opposed to being just a data compression specification.

The PCC work item is currently seeing the involvement of some 80 experts. If you think that there were more than 500 experts at the October 2018 meeting, you should have a pretty good idea of the complexity of the MPEG machine and the amount of energies poured in by MPEG members.

The MPEG process – more than just a standard

In most cases MPEG performs “Verification Tests” on a standard  produced to provide the industry with precise indications of the performance that can be expected from an MPEG compression standard. To do this the following is needed: specification of tests, collection of appropriate test material, execution of reference or proprietary software, execution of subjective tests, test analysis and reporting.

Very often, as a standard takes shape, new requirements for new functionalities are added. They become part of a standard either through the vehicle of “Amendments”, i.e. separate documents that specify how the new technologies can be added to the base standard, or “New Editions” where the technical content of the Amendment is directly integrated into the standard in a new document.

As a rule MPEG develops reference software for its standards. The reference software has the same normative value as the standard expressed in human language.

MPEG also develops conformance tests, supported by test suites, to enable a manufacturer to judge whether its implementation of 1) an encoder produces correct data by feeding them into the reference decoder or 2) a decoder is capable of correctly decoding the test suites.

Finally it may happen that bugs are discovered in a published standard. This is an event to be managed with great attention because industry may already have released some implementations.

Conclusions

MPEG is definitely a complex machine because it needs to assess if an idea is useful in the real world, understand if there are technologies that can be used to make a standard supporting that idea, get the technologies, integrate them and develop the standard. Often it also has to provide integrated audio-visual solutions where a line up of standards nicely fit to provide a system specification. At every meeting MPEG works simultaneously on tens of intertwined activities taking place at the same time.

MPEG needs to work like a company developing products, but it is not a company. Fortunately one can say, but also unfortunately because it has to operate under strict rules. ISO is a private organisation, but it is international in scope.

Posts in this thread (in bold this post)

Life inside MPEG

Introduction

In my earlier post 30 years of MPEG, and counting? I brought evidence of what MPEG has done in the last 30 years to create the broadcast, recording, web and mobile digital media world that we know. This document tries to make the picture more complete by looking at the current main activities and deliveries in the next few months/years.

MPEG at a glance

The figure below shows the main standards developed or under development by MPEG in the 2017-2023 period organised in 3 main sections:

  • Media Coding (e.g. MP3 and AVC)
  • Systems and Tools (e.g. MPEG-2 TS and File Format)
  • Beyond Media (currently Genome Compression and Neural Network Compression).

Video coding

In the Video coding area MPEG is currently handling 4 standards (MPEG-4, -H, -I and -CICP) and several Explorations.

MPEG-I ISO/IEC 23090 Coded representation of immersive media is the container of standards needed for the development of immersive media devices, applications and services.

MPEG is currently working on Part 3 Versatile Video Coding, the new video compression standard after HEVC. VVC is developed jointly with VCEG and is expected to reach FDIS stage in October 2020.

Explorations

  1. An exploration on a new video coding standard that combines coding efficiency (similar to that of HEVC), complexity (suitable for real time encoding and decoding) and usability (timely availability of licensing terms). In October 2018 a Call for Proposals was issued. Submissions are due in January 2019 and FDIS stage for the new standard is expected to be reached in January 2020.
  2. An exploration on a future standard that defines a data stream structure composed by two streams: a base stream decodable by a hardware decoder, and an enhancement stream suitable for software processing implementation with sustainable power consumption. This activity is supported by more than 30 major international players in the video distribution business. A Call for Proposal has been issued in October 2018. Submissions are due in March 2019 and FDIS stage is expected to be reached in April 2020.
  3. Several explorations on Immersive Video Coding
    • 3DoF+ Visual: a Call for Proposal will be issued in January 2019. Submissions are due in March 2019 and the FDIS is planned for July 2020. The result of this activity is not meant to be a video coding standard but a set of metadata that can be used to provide a more realistic user experience in OMAF v2. Indeed, 3DoF+ Visual will be a part of MPEG-I part 7 Immersive Media Metadata. Note that 3 Degrees of Freedom (3DoF) means that a user can only make yaw, pitch, roll movements, but 3DoF+ means that the user can also displace the head to a limited extent.
    • Several longer-term explorations on compression of 6DoF visual (Windowed-6DoF and Omnidirectional 6DoF) and Compression of Dense Representation of Light Fields. No firm timeline for standards in these areas has been set.

Audio coding

In the Audio coding area MPEG is handling 3 standards (MPEG-4, -D, and -I). Of particular relevance is the MPEG-I part 3 Immersive Audio activity. This is built upon MPEG-H 3D Audio – which already supports a 3DoF user experience – and will provide a 6DoF immersive audio VR experience. A Call for Proposal will be issued in March 2019. Submissions are expected in January 2020 and FDIS stage is expected to be reached in April 2021. As for 3DoF+ Visual, this standard will not be about compression, but about metadata.

3D Graphics coding

In the 3D Graphics coding area MPEG is handling two parts of MPEG-I.

  • Video-based Point Cloud Compression (V-PCC) for which FDIS stage is planned to be reached in October 2019. It must be noted that in July 2018 an activity was initiated to develop standard technology for integration of a 360 video and V-PCC objects.
  • Geometry-based Point Cloud Compression (G-PCC) for which FDIS stage is planned to be reached in January 2020.

The two PCC standards employ different technologies and target different application areas: entertainment and automotive/unmanned aerial vehicles, respectively.

Font coding

In the Font coding area MPEG is working on MPEG-4 part 22.

MPEG-4 ISO/IEC 14496 Coding of audio-visual objects is a 34-part standard that made possible large scale use of media on the fixed and mobile web.

Amendment 1 to Open Font Format will support complex layouts and new layout features. FDAM stage will be reached in April 2020.

Genome coding

MPEG-G ISO/IEC 23092 Genomic Information Representation is the standard developed in collaboration with TC 276 Biotechnology to compress files containing DNA reads from high speed sequencing machines.

In the Genome coding area MPEG plans to achieve FDIS stage for Part 2 Genomic Information Representation in January 2019. MPEG has started investigating additional genome coding areas that would benefit from standardisation.

Neural network coding

Neural network compression is an exploration motivated by the increasing use of neural networks in many applications that require the deployment of a particular trained network instance potentially to a large number of devices, which may have limited processing power and memory.

In the Neural network coding area MPEG has issued a Call for Evidence in July 2018, assessed the responses received in October 2018 and collected evidence that justifies the Call for Proposals issued at the same October 2018 meeting. The goal of the Call is to make MPEG aware of technologies to reduce the size of trained neural networks. Responses are due in March 2019. As it is likely that in the future more Calls will be issued for other functionality (e.g., incremental representation), an expected time for FDIS has not been identified yet.

Media description

Media description is the goal of the MPEG-7 standard which contains technologies for describing media, e.g. for the purpose of searching media.

In the Media description area MPEG has completed Part 15 Compact descriptors for video analysis (CDVA) in October 2018. By exploiting the temporal redundancy of video, CDVA extracts a single compact descriptor that represents a video clip rather than individual frames, which was the goal of Compact Descriptors for Visual search (CDVS).

Work in this area continues to complete reference software and conformance.

System support

In the System support area MPEG is working on MPEG-4, -B and -I. In MPEG-I MPEG is developing

  • Part 6 – Immersive Media Metrics which specifies the metrics and measurement framework to enhance the immersive media quality and experiences. 3DoF+ Visual metadata will be one component of this standard
  • Part 7 – Immersive Media Metadata which specifies common immersive media metadata focusing on immersive video (including 360° video), images, audio, and timed text.

Both parts are planned to reach FDIS stage in July 2020.

Intellectual Property Management and Protection

In the IPMP area MPEG is developing an amendment to support multiple keys per sample. FDAM stage is planned to be reached in March 2019. Note that IPMP is not about _defining_ but _employing_ security technologies to digital media.

Transport

In the Transport area MPEG is working on MPEG-2, -4, -B, -H, -DASH, -I, -G and Explorations.

MPEG-2 ISO/IEC 13818 Generic coding of moving pictures and associated audio information is the standard that enabled digital television.

Part 2 Systems continues to be an extremely lively area of work. After producing Edition 7, MPEG is working on two amendments to carry two different types of content

  • JPEG XS (a JPEG standard for low latency applications)
  • CMAF (an MPEG Application Format).

MPEG-4

Part 12 ISO Based Media File Format is another extremely lively area of work. Worth mentioning are two amendments

  • Compact Sample-to-Group, new capabilities for tracks, and other improvements – has reached FDAM stage in October 2018
  • Box relative data addressing – is expected to reach FDAM in March 2019.

The 7th Edition of the MP4 file format is awaiting publication.

MPEG-B ISO/IEC 23001 MPEG systems technologies is a collection of systems standards that are not specific to a given standard.

In MPEG-B MPEG is working on two new standards

  • Part 14 Partial File Format will provide a standard mechanism to store HTTP entities and the partial file in broadcast applications for later cache population. The standard is planned to reach FDIS stage in April 2020.
  • Part 15 Carriage of Web Resources in ISOBMFF will make it possible to enrich audio/video content, as well as audio-only content, with synchronised, animated, interactive web data, including overlays. The standard is planned to reach FDIS stage in January 2019.

MPEG-H ISO/IEC 23008 High efficiency coding and media delivery in heterogeneous environments is a 15-part standard for audio-visual compression and heterogeneous delivery.

Part 10 MPEG Media Transport FEC Codes is being enhanced by the Window-based FEC code an amendment. FDAM is expected to be reached in January 2020.

MPEG-DASH ISO/IEC 23009 Dynamic adaptive streaming over HTTP (DASH) is the standard for media delivery on unpredictable-bitrate delivery channels.

In MPEG-DASH MPEG is working on

  • Part 1 Media presentation description and segment formats is being enhanced by the Device information and other extensions amendment. FDAM is planned to be reached in July 2019.
  • Part 7 Delivery of CMAF content with DASH contains guidelines recommending some of the most popular delivery schemes for CMAF content using DASH. TR is planned to be reached in March 2019

MPEG-I

Part 2 Omnidirectional Media Format (OMAF) released in October 2017 is the first standard format for delivery of omnidirectional content. With OMAF 2nd Edition Interactivity support for OMAF, planned to reach FDIS in January 2020, MPEG is substantially extending the functionalities of OMAF. Future releases may add 3DoF+ functionalities.

MPEG-G

MPEG plans to achieve FDIS stage for Part 1 Transport and Storage of Genomic Information in January 2019.

Application Formats

MPEG-A ISO/IEC 23000 Multimedia Application Formats is a suite of standards for combinations of MPEG and other standards (when there are no suitable MPEG standard for the purpose).

MPEG is working on two new standards

  • Part 21 Visual Identity Management Application Format will provide a framework for managing privacy of users appearing on pictures or videos shared among users. FDIS is expected to be reached in January 2019.
  • Part 22 Multi-Image Application Format (MIAF) will enable precise interoperability points when creating, reading, parsing and decoding images embedded in HEIF (MPEG-H part 12).

Application Programming Interfaces

The Application Programming Interfaces area comprises standards developed in order to make possible effective use of some MPEG standards.

In the API area MPEG is working on MPEG-I, MPEG-G and MPEG-IoMT.

MPEG-I

MPEG is working on Part 8 Network-based Media Processing (NBMP), a framework that will allow users to describe media processing operations to be performed by the network. The standard is expected to reach FDIS stage in January 2020.

MPEG-G

MPEG is working on Part 3 Genomic Information Metadata and Application Programming Interfaces (APIs). The standard is expected to reach FDIS stage in March 2019.

MPEG-IoMT ISO/IEC 23093 Internet of Media Things is a suite of standards supporting the notion of Media Thing (MThing), i.e. a thing able to sense and/or act on physical or virtual objects, possibly connected to form complex distributed systems called Internet of Media Things (IoMT) where MThings interact between them and humans.

In MPEG-IoMT MPEG is working on

  • Part 2 IoMT Discovery and Communication API
  • Part 3 IoMT Media Data Formats and API.

Both expected to reach FDIS stage in March 2019.

Media Systems

Media Systems includes standards or Technical Reports targeting architectures and frameworks.

In Media Systems MPEG is working on Part 1 IoMT Architecture, expected to reach FDIS stage in March 2019. The architecture used in this standard is compatible with the IoT architecture developed by JTC 1/SC 41.

Reference implementation

MPEG is working on the development of 10 standards for reference software of MPEG-4, -7, -B, -V, -H, -DASH, -G, -IoMT

Conformance

MPEG is working on the development of 8 standards for conformance of MPEG-4, -7, -B, -V, -H, -DASH, -G, -IoMT.

Conclusions

Even this superficial overview should make evident to all the complexity of interactions in the MPEG ecosystem that has been ongoing for 30 years (note that the above only represents a part of what happened at the last meeting in Macau).

In 30 years of MPEG, and counting? I wrote “Another thirty years await MPEG, if some mindless industry elements will not get in the way”.

It might well have been a prophecy.

Posts in this thread (in bold this post)