Why is MPEG successful?

There are people who do not like MPEG (I wonder why), but so far I have not found anybody disputing the success of MPEG. Some people claim that only a few MPEG standards are successful, but maybe that is because some MPEG standards are_so_ successful.

In this article the reasons of MPEG success are identified and analysed by using the 18 elements of the figure below.

A standard for all. In the late 1980’s many industries, regions and countries had understood that the state of digital technologies justified the switch from analogue to digital (some acted against that understanding and paid dearly for it). At that time several companies had developed prototypes, regional initiatives were attempting to develop formats for specific countries and industries, some companies were planning products and some standards organisations were actually developing standards for their industries. The MPEG proposal of a generic standard, i.e. a common technology for all industries, caught the attention because it offered global interoperability, created a market that was global – geographically and across industries – and placed the burden of developing the very costly VLSI technology on a specific industry accustomed to do that. The landscape today has changed beyond recognition, but today the revolutionary idea of that time is taken as a matter of fact.

One step at a time. Even before MPEG came to the fore many players were trying to be “first” and “impose” their early solution on other countries or industries or companies. If the newly-born MPEG had proposed itself as the developer of an ambitious generic standard digital media technology for all industries, the proposal would have been seen as far fetched. So, MPEG started with a moderately ambitious project: a video coding standard for interactive applications on digital storage media (CD-ROM) at a rather low bitrate (1.5 Mbit/s) targeting the market covered by the video cassette (VHS/Beta). Moving one step at a time has been MPEG policy for all its subsequent standards.

Complete standards. In 6 months after its inception MPEG had already realised the obvious, namely that digital media is not just video (although this it the first component that catches the attention), but it is also audio (no less challenging and with special quality requirements). In 12 months it had realised that bits do not flow in the air but that a stream of bits needs some means to adapt the stream to the mechanism that carries it (originally the CD-ROM). If the transport mechanism is analogue (as was 25 years ago and, to large extent, still today), the adaptation is even more challenging. Later MPEG also realised that a user interacts with the bits (even though it is so difficult to understand what exactly is the interaction that the user wants). With its MPEG-2 standard MPEG was able to provide the industry with a complete Audio-Video-Systems (and DSM-CC) solution whose pieces could also be used independently. That was possible because MPEG could attract, organise and retain the necessary expertise to address such a broad problem area and provide not just a solution that worked, but the best that technology could offer at the time.

Requirements first. Clarifying to yourself the purpose of something you want to make is a rule that should apply to any human endeavour. This rule is a must when you are dealing with a standard developed by a committee of like-minded people. When the standard is not designed by and for a single industry but by many, keeping this rule is vital for the success of the effort. When the standard involves disparate technologies whose practitioners are not even accustomed to talk to one another, complying with this rule is a prerequisite. Starting from its early days MPEG has developed a process designed to achieve the common understanding that lies at the basis of the technical work to follow: describe the environment (context and objectives), single out a set of exemplary uses of the target standard (use cases), and identify requirements.

Leveraging research. In the late 1980s compression of video and audio (and other data, e.g. facsimile) had been the subject of research for a quarter of a century, but how could MPEG access that wealth of technologies and know-how? The choice was the mechanism of Call for Proposals (CfP) because an MPEG CfP is open to anybody (not just the members of the committee) – see How does MPEG actually work? All respondents are given the opportunity to present their proposals that can, at their choice, address individual technologies, subsystems or full systems, and defend them (by becoming MPEG member). MPEG does not do research, MPEG uses the best research results to assemble the system specified in the requirements that always accompany a CfP. Therefore MPEG has a symbiotic relationship with research. MPEG could not operate without a tight relationship with research and research would certainly lose a big customer if that relationship did not exist.

Minimum standards. Industries can be happy to share the cost of an enabling technology but not at the cost of compromising their individual needs. MPEG develops standards so that the basic technology can be shared, but it must allow room for customisation. The notion of Profiles and Level provides the necessary flexibility to the many different users of MPEG standards. With profiles MPEG defines subsets of the general interoperability, with levels it defines different levels of performance within a profile. Further, by restricting standardisation to the decoding functionality MPEG extends the life of its standards because it allows industry players to compete on the basis of their constantly improved encoders.

Best out of good. When the responses to a CfP are on the table, how can MPEG select the best from the good? MPEG uses five tools:

  1. Comprehensive description of the technology proposed in each response (no black box allowed)
  2. Assessment of the performance of the technology proposed (e.g. subjective or objective tests)
  3. Line-up of aggressive “judges” (meeting participants, especially other proponents)
  4. Test Model assembling the candidate components selected by the “judges”
  5. Core Experiments to improve the Test Model.

By using these tools MPEG is able to provide the best standard in a given time frame.

Competition & collaboration. MPEG favours competition to the maximum extent possible. Many participants in the meeting are actual proponents of technologies in response to a CfP and obviously keen to have their proposals accepted. Extending competition beyond a certain point, however, is counterproductive and prevents the group from reaching the goal with the best results. MPEG uses the Test Model as the platform that help participants to collaborate by improving different areas of the Test Model. Improvement are obtained through

  1. Core Experiments, first defined in March 1992 as “a technical experiment where the alternatives considered are fully documented as part of the test model, ensuring that the results of independent experimenters are consistent”, a definition that applies unchanged to the work being done today;
  2. Reference Software, which today is a shared code base that is progressively improved with the addition of all the software validated through Core Experiments.

IPR (un)aware. MPEG uses the process described above and seeks to produce the best performing standards that satisfy the requirements independently of the existence of IPR. Should an IPR in the standard turn out not to be available, the technology protected by that IPR should be removed. Since only the best technologies are adopted and MPEG members are uniquely adept at integrating them, industry knows that the latest MPEG standards are top of the range. Of course one should not think that the best is free. In general it has a cost because IP holders need to be remunerated. Market (outside of MPEG) decides how that can be achieved.

Internal competition. If competition is the engine of innovation, why should those developing MPEG standards be shielded from it? The MPEG mission is not to please its members but to provide the best standards to industry. Probably the earliest example of this tenet is provided by MPEG-2 part 3 (Audio). When backward compatibility requirements did not allow the standard to yield the performance of algorithms not constrained by compatibility, MPEG issued a CfP and developed MPEG-2 part 7 (Advanced Audio Codec) that eventually evolved and became the now ubiquitous MPEG-4 AAC. Had MPEG not made this decision, probably we would still have MP3 everywhere, but no other MPEG Audio standards. MPEG values all its standards but cannot afford not to provide the best technology to those who demand it.

Separation of concerns. Even for the purpose of developing its earliest standards such as MPEG-1 and MPEG-2, MPEG needed to assemble disparate technological competences that had probably never worked together in a project (with its example MPEG has favoured the organisational aggregation of audio and video research in many institutions where the two were separate). To develop MPEG-4 (a standard with 34 parts whose development continues unabated), MPEG has assembled the largest ever number of competences ranging from audio and video to scene description, to XML compression, to font, timed text and many more. MPEG keeps competences organisationally separate in different in MPEG subgroups, but retains all flexibility to combine and deploy the needed resources to respond to specific needs.

New ways for new standards. MPEG works at the forefront of digital media technologies and it would be odd if it had not innovated the way it makes its own standard. Since its early days, MPEG has made massive use of ad hoc groups to progress collaborative work, innovated the way input and output documents are shared in the community and changed the way documents are discussed at meetings and edited in groups.

Talk to the world. Using extreme words, MPEG does have an industry of its own. It only has the industry that develops the technologies used to make standards for the industries it serves. Therefore MPEG needs to communicate its plans, the progress of its work and the results achieved more actively than other groups. See MPEG communicates the many ways MPEG uses to achieve this goal.

Standards are for a time. Digital media is one of the most fast evolving digital technology areas because most of the developers of good technologies incorporated in MPEG standards invest the royalties earned from previous standards to develop new technologies for new standards. As soon as a new technology shows interesting performance (which MPEG assesses by issuing Calls for Evidence – CfE) or the context changes offering new opportunities, MPEG swiftly examines the case, develops requirements and issues CfPs. For instance this has happened for its many video and audio compression standards. A paradigmatic case of a standard addressing a change of context is MPEG Media Transport (MMT) that MPEG designed having in mind a broadcasting system for which the layer below it is IP, unlike MPEG-2 Transport Stream, originally designed for a digitised analogue channel (but also used today for transport over IP as in IPTV).

Standards lead. When technology moves fast, as in the case of digital media, waiting is a luxury MPEG cannot afford. MPEG-1 and MPEG-2 were standards whose enabling technologies were already considered by some industries and MPEG-4 (started in 1993) was a bold and successful attempt to bring media into the IT world (or the other way around). That it is no longer possible to wait is shown by MPEG-I, a challenging undertaking where MPEG is addressing standards for interfaces that are still shaky or just hypothetical. Having standard that lead as opposed to trail, is a tough trial-and-error game, but the only possible game today. The alternative is to stop making standards for digital media because if MPEG waits until market needs are clear, the market is already full of incompatible solutions and there is no room left for standards.

Standards as enablers. An MPEG standard cannot be “owned” by an industry. Therefore MPEG, keeping faith to its “generic standards” mission, tries to accommodate all legitimate functional requirements when it develops a new standard. MPEG assesses each requirement for its merit (value of functionality, cost of implementation, possibility to aggregate the functionality with others etc.). Ditto if an industry comes with a legitimate request to add a functionality to an existing standard. The decision to accept or reject a request is only driven by a value substantiated by use cases, not because an industry gets an advantage or another is penalised.

What is “standard”? In human societies there are laws and entities (tribunals) with the authority to decide if a specific human action conforms to the law. In certain regulated environments (e.g. terrestrial broadcasting in many countries) there are standards and entities (authorised test laboratories) with the authority to decide if a specific implementation conforms to the standard. MPEG has neither but, in keeping with its “industry-neutral” mission, it provides the technical means – tools for conformance assessment, e.g. bitstreams and reference software – for industries to use in case they want to establish authorised test laboratories for their own purposes.

Rethinking what we are. MPEG started as a “club” of Telecommunication and Consumer Electronics companies. With MPEG-2 the “club” was enlarged to Terrestrial and Satellite Broadcasters, and Cable concerns. With MPEG-4, IT companies joined forces. Later, a large number of research institutions and academia joined (today they count for ~25% of the total membership). With MPEG-I, MPEG faces new challenges because the demand for standards for immersive services and applications is there, but technology immaturity deprives MPEG of its usual “anchors”. Thirty years ago MPEG was able to invent itself and, subsequently, to morph itself to adapt to the changed conditions while keeping its spirit intact. If MPEG will be able to continue to do as it did in the last 30 years, it can continue to support the industry it serves in the future, no matter what will be the changes of context.
I mean, if some mindless industry elements will not get in the way.

Suggestions? If you have comments or suggestions about MPEG, please write to leonardo@chiariglione.org.

Posts in this thread (in bold this post)

 

 

MPEG can also be green

Introduction

MPEG has given humans the means to add significant more effectiveness and enjoyment to their lives. This comes at a cost, though. Giving billions of people the means to stream video streamed to anywhere at any time of the day, adds to global energy consumption. Enhanced experiences provided by newer featurers such as High Dynamic Range further adds energy consumption in the display. More sophisticated compression algorithms consume more energy, even though this can be mitigated by more advanced circuit geometry.

In 2013 MPEG issued a Call for Proposal on “Green MPEG” requesting technologies that enable reduction of energy consumption in video codecs. In 2016 MPEG released ISO/IEC 23001-11 Green Metadata, followed by a number of ancillary activities.

It should be clear that Green Metadata should not be seen as an attempt at solving the global problem of energy consumption. More modestly Green Metadata seeks to reduce power consumption in the encoding, decoding, and display process while preserving the user’s quality of experience (QoE). At worst Green Metadata can be used to reduce the QoE in a controlled way.

The standard does not require changing the operation of a given encoder or decoder (i.e. changing the video coding standard). It just requires to be able to “access” and “influence” appropriate operating points of any or the encoder, decoder or display.

A system view

Green Metadata has been developed having as target metadata suitable for influencing the video encoding, decoding and display process. The framework, however, could be easily generalised by replacing “video” and “display” with “media” and “presentation”. However, the numerical results obtained in the video case cannot be directly extrapolated to other media.

Let’s start from the figure representing a conceptual diagram of a green encoder-decoder pair.

Figure 1 – Conceptual diagram of a green encoder-decoder pair

The Green Video Encoder (GVE), is a regular video encoder that generates a compressed video bitstream and also a stream of metadata (G-Metadata) for use by a Green Video Decoder (GVD) to reduce power consumption. When a return channel is available (e.g. on the internet), the GVD may generate feedback information (G-Feedback) that the GVE may use to generate a compressed video bitstream that demands less power for the GVD to decode.

To understand what is actually standardised by Green Metadata, it is worth digging a little bit in the following high-level diagram and see what is the new “green component” that is added. The figure below helps to understand such green components.

Figure 2 – Inside a green encoder-decoder pair

The GVE generates G-Metadata packaged by the G-Metadata Generator for transmission to a GVD. The GDV G-Metadata Extractor extracts the G-Metadata payload and passes the GVE G-Metadata to the GVD Power Manager along with G-Metadata coming from the GVD. The GVD Power Manager, based on the two G-Metadata streams and possibly other input such as user’s input (not shown in figure), may send

  1. Power Control data to the Video Decoder to change its operation
  2. G-Feedback data to the G-Feedback Generator to package it for transmission to the GVE.

At the GVE side the G-Feedback Extractor extracts the G-Feedback data and passes them to the GVE Power Manager. This may send Power Control data to the Video Encoder to change its operation.

To examine a bit more in detail how G-Metadata can be used, it is helpful to dissect the Video Encoder and Decoder pair.

Figure 3 – Inside the encoder and decoder

The Video Encoder is composed of a Media Preprocessor (e.g. a video format converter) and a Media Encoder. The Video Decoder is made of a Media Decoder and a Presentation Subsystem (e.g. to drive the display). All subsystems send G-Metadata and receive Power Contro. The Presentation Subsystem only receives Power Control.

What is standardised in Green Metadata? As always, the minimum that is required for interoperability. This means the Encoder Green Metadata and the Decoder Green Feedback (in red in the figure) that are exchanged by systems which are potentially manufactured by different entities. Other data formats inside the GVE and the GVD are a matter for GVE and GVD manufacturers to decide because they do not affect interoperability but may affect performance. In particular, the logic of the Power Manager that generates Power Control is the differentiating factor beyween implementations.

Achieving reduced power consumption

In the following the 3 areas positively affected by the use of the Green Metadata standard – encoder, decoder and display – will be illustrated.

Encoder. By using a segmented delivery mechanism (e.g. DASH), encoder power consumption can be reduced by encoding video segments with alternate high/low quality. Low-quality segments are generated by using lower-complexity encoding (e.g. fewer encoding modes and reference pictures, smaller search ranges etc.). Green Metadata include the quality of the last picture of each segment. The video decoder enhances the low-quality segment by using the metadata and the last high-quality video segment.

Decoder. Lowering the frequency of a CMOS circuit implementing a video decoder reduces power consumption because this roughly increases linearly with the clock frequency and quadratically with the voltage applied. In a software decoder picture complexity can be used to control the CPU frequency.

One type of Green Metadata signals the duration and degree of complexity of upcoming pictures. This can be used to select the most appropriate setting and offer the best QoE for a desired power-consumption level.

Display. The display adaptation technique known as backlight dimming reduces power consumption by dimming the LCD backlight while RGB values are scaled in proportion to the dimming level (RGB values do not have a strong influence on power consumption).

Green Metadata need to be carried

ISO/IEC 23001-11 only specifies the Green Metadata. The way this information is transported depends on the specific use scenarios (some of them are described in Context, Objectives, Use Cases and Requirements for Green MPEG).

Two transports have been standardised by MPEG. In the first Green Metadata is transported by a Supplementary Enhancement Information (SEI) message embedded in the video stream. This is a natural solution since Green Metadata are due to be processed in a Green Video Decoder that includes a regular video decoder. In this case, however, transport is limited to decoder metadata, not display metadata. In the second, suitable for a broadcast scenario, all Green Metadata is transported in the MPEG-2 Transport Stream.

Conclusion

Power consumption is a dimension that had not been tackled by MPEG, but the efforts that have led to the Green Metadata standard have been rewarded: with the currently standardised metadata 38% of video decoder power and 12% of video encoder power can be saved without affecting QoE and up to 80% of power can be saved with some degradation of the QoE. Power saving data were obtained using the Google Nexus 7 platform and the Monsoon power monitor, and a selection of video test material.

Interested readers can know more by visiting the MPEG web site and, more so, by purchasing the Green Metadata standard from the ISO website or from a National Body.

Posts in this thread (in bold this post)

 

 

 

 

The life of an MPEG standard

Introduction

In How does MPEG actually work? I described the way MPEG develops its standards, an implementation of the ISO/IEC Directives for technical work. This article describes the life of one of MPEG most prestigious standards: MPEG-2 Systems, which has turned 26 in November 2018 and has played a major role in creating the digital world that we know.

What is MPEG-2 Systems?

When MPEG started, standards for compressed video and later audio was the immediate goal. But it was clear that the industry needed more than that. So, after starting MPEG-1 video compression and audio compression, MPEG soon started to investigate “systems” aspects. Seen with today’s eyes, the interactive CD-ROM target of MPEG-1 was an easy problem because all videos on a a CD-ROM are assumed to have the same time base, and bit delivery is error free and on-time because the time interval between a byte leaving the transmitter is the same as the time interval at its arrival at the receiver.

In July 1990, even before delivering the MPEG-1 standard (November 1992), MPEG started working on the much more challenging “digital television” problem. This can be described as: the deliver of a package of digital TV programs with different time bases and associated metadata over a variety of analogue channels – terrestrial, satellite and cable. Of course operators expected to be able to do the same operations in the network that the television industry had been accustomed to do in the several decades since TV distribution had become common place.

A unique group of experts from different – and competing – industries with their different cultural backgrounds and many countries, and the common experience of designing from scratch the MPEG-1 Systems standard, designed the MPEG-2 Systems standards, again from a blank sheet of paper.

The figure illustrates the high-level structure of an MPEG-2 decoder: waveforms are received from a physical channel (e.g. a Hertzian channel) and decoded to provide a bistream containing multiplexed TV programs. A transport stream demultipler and decoder extracts audio and video streams (and typically other streams not shown in the figure) and a clock that is used to drive the video and audio decoders.

The structure of the transport bitstream is depicted in the figure. The stream is organised in fixed-length packets of 188 bytes of which 184 bytes are used for the payload.

The impact of MPEG-2 Systems

MPEG-2 Systems is the container and adapter of the digital audio and video information to the physical world. It is used every day by billions of people who receive TV programs from a variety of sources, analogue and, often, digital as well (e.g. IPTV).

MPEG-2 Systems was approved in November 1994 while some companies who could not wait had already made implementations before the formal release of the standard. That date, however, far from marking the “end” of the standard, as it often happens, it signaled the beginning of a story that continues unabated today. Indeed, in the 26 years after its release, MPEG-2 Systems has been constantly evolving, while keeping complete backward compatibility with the original 1994 specification.

MPEG-2 Systems in action

So far MPEG has developed 34 amendments (ISO language to indicate the addition of functionality to a standard), 3 additional amendments are close to completion and one is planned. After a few amendments are developed, ISO requests that they be integrated in a new edition of the standard. So far 7 MPEG-2 Systems editions have been produced covering the transport of non-MPEG-2 native media and non-media data. This is an incomplete lists of the trasnport functionality added:

  1. Audio: MPEG-2 AAC, MPEG-4 AAC and MPEG-H 3D
  2. Video: MPEG-4 Visual, MPEG-4 AVC and its extensions (SVC and MVC), HEVC, HDR/WCG, JPEG2000, JPEG XS etc.
  3. Other data: streaming text, quality metadata, green metadata etc.
  4. Signalling: format descriptor, extensions of the transport stream format (e.g. Tables for splice parameters, DASH event signalling, virtual segment etc.), etc.

Producing an MPEG-2 Systems amendment is a serious job. You need experts with the full visibility of a 26 years old standard (i.e. don’t break what works) and the collaboration of experts of the carrier (MPEG-2 Systems) and of the data carried (audio, video etc.). MPEG can respond to the needs of the industry because it has available all component expertise.

Conclusions

MPEG-2 Systems is probably one of MPEG standards that is less “visible” to its users. Still it is one of the most important enablers of television distribution applications impacting the life of billions of people and tens of thousands of professionals. Its continuous support is vital for the well-being of the industry.

The importance of MPEG-2 Systems has been recognised by the Academy of Television Arts and Sciences who has awarded MPEG an Emmy for it.

MPEG-2 Systems Amendments

The table below reports the full list of MPEG-2 Systems amendments, The 1st column gives the edition, the 2nd column the sequential number of the amendment of that edition, the 3rd column the title of the amendment and the 4th the dates of the approval stages.

E

A Title

Date

1 1 Format descriptor registration 95/11
2 Copyright descriptor registration 95/11
3 Transport Stream Description 97/04
4 Tables for splice parameters 97/07
5 Table entries for AAC 98/02
6 4:2:2 @HL splice parameters
7 Transport of MPEG-4 content 99/12
2 1 Transport of Metadata 02/10
2 IPMP support 03/03
3 Transport of AVC 03/07
4 Metadata Application Format CP 04/10
5 New Audio P&L Signaling 04/07
3 1 Transport of Streaming Text 06/10
2 Transport of Auxiliary Video Data
3 Transport of SVC 08/07
4 Transport of MVC 09/06
5 Transport of JPEG2000 11/01
6 MVC operation point descriptor 11/01
7 Signalling of stereoscopic video 12/02
8 Simplified carriage of MPEG-4 12/10
4 1 Simplified carriage of MPEG-4 12/07
2 MVC view, MIME type etc. 12/10
3 Transport of HEVC 13/07
4 DASH event signalling 13/07
5 Transport of MVC depth etc. 14/03
5 1 Timeline for External Data 14/10
2 Transport of layered HEVC 15/06
3 Transport of Green Metadata 15/06
4 Transport of MPEG-4 Audio P&L 15/10
5 Transport of Quality Metadata 16/02
6 Transport of MPEG-H 3D Audio 16/02
7 Virtual segment 16/10
8 Signaling of HDR/WCG 17/01
9 Ultra-Low-Latency & JPEG 2000 17/07
10 Media Orchestration & sample variants
11 Transport of HEVC tiles
6 1 Transport of JPEG XS
  2 Carriage of associated CMAF boxes  

 

Posts in this thread (in bold this post)

Genome is digital, and can be compressed

Introduction

The well-known double helix carries the DNA of living beings. The human DNA contains about 3.2 billion nucleotide base pairs represented by the quaternary symbols (A, G, C, T). With high-speed sequencing machines today it is possible to “read” the DNA. The resulting file contains millions of “reads”, short segments of symbols, typically all of the same length, and weighs an unwieldy few Terabytes.

The upcoming MPEG-G standards, developed jointly by MPEG and ISO TC 276 Biotechnology, will reduce the size of the file, without loss of information, by exploiting the inherent redundancy of the reads and make at the same time the information in the file more easily accessible.

This article provides some context, and explains the basic ideas of the standard and the benefits it can yield to those who need to access genomic information.

Reading the DNA

There are two main obstacles preventing a direct use of files from sequencing machines: the position of a read on the DNA sample is unknown and the value of each symbol of the read is not entirely reliable.

The picture below represents a 17 reads with a read length of 15 nucleotides. These have been aligned to a reference genome (first line). Reads with a higher number start further down in the reference genome.

Reading column-wise, we see that in most cases the values have exactly the value of the reference genome. A single difference (represented by isolated red symbols) may be caused by read errors while a quasi completely different column (most symbols in red) may be caused by the fact that 1) a given DNA is unlikely to be exactly equal to a reference genome or 2) the person with this particular DNA may have health problems.

Use of genomics today

Genomics is already used in the clinical practice. An example of genomic workflow is depicted in the figure below which could very well represent a blood test workflow if “DNA” were replaced by “blood”. Patients go to a hospital where a sample of their DNA is taken and read by a sequencing machine. The files are analysed by experts who produce reports which are read and analysed by doctors who decide actions.

Use of genomics tomorrow

Today genomic workflows take time – even months – and are costly – thousands of USD per DNA sample. While there is not much room to cut the time it takes to obtain a DNA sample, sequencing cost has been decreasing and are expected to continue doing so.

Big savings could be achieved by acting on data transport and processing. If the size of a 3 Terabytes file is reduced by, say, a factor of 100, the transport of the resulting 30 Gigabytes would be compatible with today’s internet access speeds of 1 Gbit/s (~4 min). Faster data access, a by-product of compression, would allow doctors to get the information they are searching, locally or from remote, in a fraction of a second.

The new possible scenario is depicted in the figure below.

MPEG makes genome compression real

Not much had been done to make the scenario above real (zip is the oft-used compression technology today) until the time (April 2013) MPEG received a proposal to develop a standard to losslessly compress files from DNA sequencing machines.

The MPEG-G standard – titled Genomic Information Representation – has 5-parts: Parts 1 and 2 are expected to be approved at MPEG 125 (January 2018) and the other parts are expected to follow suit shortly after.

MPEG-G is an excellent example of how MPEG could apply its expertise to a different field than media. Part 1, an adaptation of the MP4 File Format present in all smartphones/tablets/PCs, specifies how to make and transport compressed files. Part 2 specifies how to compress reads and Part 3 how to invoke the APIs to access specific compressed portions of a file. Part 4 and 5 are Conformance and Reference Software, respectively.

The figure below depicts the very sophisticated operation specified in Part 2 in a simplified way.

An MPEG-G file can be created with the following sequence of operations:

  1. Put the reads in the input file (aligned or unaligned) in bins corresponding to segments of the reference genome
  2. Classify the reads in each bin in 6 classes: P (perfect match with the reference genome), M (reads with variants), etc.
  3. Convert the reads of each bin to a subset of 18 descriptors specific of the class: e.g., a class P descriptor is the start position of the read etc.
  4. Put the descriptors in the columns of a matrix
  5. Compress each descriptor column (MPEG-G uses the very efficient CABAC compressor already present in several video coding standards)
  6. Put compressed descriptors of a class of a bin in an Access Unit (AU) for a maximum of 6 AUs per bin

Therefore MPEG-G file contains all AUs of all bins corresponding to all segments of the reference genome. A file may contain the compressed reads of more than one DNA sample.

The benefits of MPEG-G

Compression is beneficial but is not necessarily the only or primary benefit. More important is the fact that while designing compression, MPEG has given a structure to the information. In MPEG-G the structure is provided by Part 1 (File and transport) and by Part 2 (Compression).

In MPEG-G most information relevant to applications is immediately accessible, locally and, more importantly, also from remote without the need to download the entire file to be able to access the information of interest. Part 3 (Application Programming Interfaces) makes this fast access even more convenient because it facilitates the work of developers of genomics applications who may not have in-depth information of the – certainly complex – MPEG-G standard.

Conclusions

In the best MPEG tradition, MPEG-G is a generic standard, i.e. a standard that can be employed in a wide variety of applications that require small footprint of and fast access to genomic information.

A certainly incomplete list includes: Assistance to medical doctors’ decisions; Lifetime Genetic Testing; Personal DNA mapping on demand; Personal design of pharmaceuticals; Analysis of immune repertoire; Characterisation of micro-organisms living in the human host; Mapping of micro-organisms in the environment (e.g. biodiversity).

Standards are living beings, but MPEG standards have a DNA that allows them to grow and evolve to cope with the manifold needs of its ever-growing number of users.

I look forward to welcoming new communities in the big family of MPEG users.

Posts in this thread (in bold this post)

 

Compression standards and quality go hand in hand

Introduction

When I described the MPEG workflow in How does MPEG actually work? I highlighted the role of quality assessment across the entire MPEG standard life cycle: at the time of issuing a Call for Evidence (CfE) or a Call for Proposals (CfP), carrying out Core Experiments (CE) or executing Verification Tests.

We should consider, however, that in 30 years the coverage of the word “media” has changed substantially.

Originally (1989-90) the media types tested were Standard Definition (SD) 2D rectangular video and stereo audio. Today the video data types include also High Definition (HD), Ultra High Definition (UHD), Stereoscopic Video, High Dynamic Range (HDR) and Wide Colour Gamut (WCG), and Omnidirectional (Video 360).

The video information can be 2D, but also multiview and 3D: stereoscopic, 3 degrees of freedom + (3DoF+), 6 degrees of freedom (6DoF) and various forms of light field. Audio has evolved to different forms of Multichannel Audio and 6DoF. Recently Point Clouds were added to the media types for which MPEG has applied subjective quality measurements to develop compression standards.

In this article I would like to go inside the work that comes together with subjectively assessing the quality of media compression.

Preparing for the tests

Even before MPEG decides to issue a CfE or CfP for compression of some type of media content, viewing or listening to content may take place to appreciate the value of a proposal. When a Call is issued MPEG has already reached a pretty clear understanding of the use cases and requirements (at the time of a CfE) or the final version of them (at the time of a CfP).

The first step is the availability of appropriate test sequences. Sequences may already be in the MPEG Content Repository or are spontaneously offered by members or are obtained from industry representatives by issuing a Call for Content.

Selection of test sequences is a critical step because we need sequences that are suitable for the media type and representative of the use cases, and in a number that allows us to carry out meaningful and realistic tests.

By the CfE or CfP time, MPEG has also decided what is the standard against which responses to the CfE or CfP should be tested. For example, in the case of HEVC, the comparison was with AVC and, in the case of VVC, the comparison was with HEVC. In the case of Internet Video Coding (IVC) the comparison was with AVC. When such a reference standard does not exist (this was the case for, e.g., all layers of MPEG-1 Audio and Point Cloud Compression) the codec built during the exploratory phase that groups together state of the art tools is used.

Once the test sequences have been selected, the experts in charge of the reference software are asked to run the reference software encoder and produce the “anchors”, i.e. the video sequences encoded according to the “old” standard. The anchors are made available in an FTP site so that anybody intending to respond to the CfE or CfP can download them.

The set up used to generate the anchors are documented in “configuration files” for each class of submission. In the case of video these are (SD/HD/UHD, HDR, 360º) and design conditions (low delay or random access). Obviously, in order to have comparable data, all proponents must use the same configuration files when they encode the test sequences using their technology.

As logistic considerations play a key role in the preparation of quality tests, would-be proponents must submit formal statements of intention to respond to the CfE or CfP to the Requirements group chair (currently Jörn Ostermann) and Test group chair (currently Vittorio Baroncini), and the chair of the relevant technology group, 2 months before the submission deadline.

At the meeting before the responses to the CfP/CfE are due, an ad hoc group (AHG) is established with the task of promoting awareness of the Call among the industry, to carry out the tests, draft a report and submit conclusions on the quality tests to the following MPEG meeting.

Carrying out the tests

The actual tests are entirely carried out by the AHG under the leadership of AHG chairs (typically the Test chair and a representative of the relevant technology group).

Proponents send on hard disk drives or on an FTP site their files containing encoded data to the Test Chair by the deadline specified in the CfE/CfP.

When all drives are received the Test chair performs the following tasks

  1. Acquire special hardware and displays for the tests (if needed)
  2. Verify that the submitted files are all on disk and readable
  3. Assign submitted files to independent test labs (sometimes even 10 test labs are concurrently involved is a test run)
  4. Make copies and distribute the relevant files to the test labs
  5. Specify the tests or provide the scripts for the test labs to carry out.

The test labs carry out a first run of the tests and provide their results for the Test chair to verify. If necessary, the Test chair requests another test run or even visits the test labs to make sure that the tests will run properly.

When this “tuning” phase has been successfully executed, all test labs run the entire set of tests assigned to them using test subjects. Tens of “non-expert” subjects may be involved for several days.

Here is a sample of what it means to attend subjective quality tests

Test report

Tests results undergo a critical revision according to the following steps

  1. The Test chair collects all results from all test labs performs a statistical analysis of the data, prepares and submits a final report to the AHG
  2. The report id discussed in the AHG and may be revised depending on the discussions
  3. The AHG draws and submits its conclusions to MPEG along with the report
  4. Report and conclusions are added to all the material submitted by proponents
  5. The Requirements and the technology group in charge of the media type evaluate the material and rank the proposals. Because of the sensitivity of some of the data all the material is not made public.

This signals the end of the competition phase and the beginning of the collaboration phase.

Other media types

In general the process above has been described having specifically rectangular 2D or 360º video in mind. Most of the process applies to other media types with some specific actions to be made for each of them, e.g.

  1. 3DTV: for the purpose of 3D HEVC tests, polarised glasses as in 3d movies and autostereoscopic displays were used;
  2. 3DoF+: a common synthesiser will be used in the upcoming 3D0F+ tests to synthesise views that are not available at the decoder;
  3. Audio: in general subjects need to be carefully trained for the specific tests;
  4. Point Clouds: videos generated by a common presentation engine consisting of point clouds animated by a script (by rotating the object and seeing it from different viewpoints) were tested for quality as if they were natural videos (NB: there were no established method to assess the quality of point clouds before. It was demonstrated that the subjective tests converged to the same results as the objective measurements).

Verification tests

Verification tests are executed with a similar process. Test sequences are selected and compressed by experts running reference software for the “old” and the “new” standard. Subjective tests are carried out as done in CfE/CfP subjective tests. Test results are made public to provide the industry with guidance on the performance of the new standard. See as examples the Verification Test Report for HDR/WCG Video Coding Using HEVC Main 10 Profile and the MPEG-H 3D Audio Verification Test Report.

Conclusions

Quality tests play an enabling role in all phases of development of a media compression standard. For 30 years MPEG has succeeded in mobilising – on a voluntary basis – the necessary organisational and human resources to perform this critical task.

I hope that, with this post, I have opened a window to an aspect of MPEG life that is instrumental to offer industry the best technology so that users can have the best media experience.

Posts in this thread (in bold this post)

Digging deeper in the MPEG work

Introduction

In How does MPEG actually work? I described the “MPEG standards work flow” from a proposed an idea to the release of the corresponding standard and its verification. I also highlighted the key roleplayed by MPEG experts as the real makers of MPEG standards.

In this article I would like to introduce the role played by the peculiar MPEG organisation and some of its members in facilitating the work of MPEG experts.

The MPEG membership

MPEG operates in the framework of the International Organization for Standardization (ISO). To be entitled to attend and actively participate in the MPEG work, experts or their companies must be members of one of the 162 national standards organisations who are members of ISO. Respondents to an MPEG Call for Evidence (CfE) or Call for proposals (CfP) are allowed to attend a meeting where their proposal is presented for the first time. To continue attending, however, proponents are required to become formal members (and they have better to do so if they want to defend their proposals).

The MPEG organisation

The figure below depicts the conceptual MPEG organisation as of today. In its 30 years MPEG has created and disbanded several groups created for specific needs. More about the history of the MPEG groups and their chairs can be found here.

 

New ideas are typically submitted to the Requirements group. If they are considered worth of further explorations they are typically discussed in ad hoc groups (AHG) after the meeting. As explained in How does MPEG actually work? the AHG will report its findings at the following MPEG meeting. After a few iterations, MPEG will produce a Call for Evidence (CfE) or Call for Proposals (CfP) with due publicity in the Press Release. At that time. the AHG is charged with the task to disseminate the CfE or CfP, prepare for the logistics of the tests and perform a first assessment of the responses.

This does not mean that the Requirements group is no longer involved in this standard because the group typically continues the development of Use Cases and Requirements while the technical work progresses. When necessary, requirements are reviewed with the appropriate technology groups and may give rise to new CfPs whose outcome is fed into the work of the relevant technology groups after an assessment.

Today the task of the Test group is no longer confined to the assessment of the quality of proposals at CfE and CfP time. Designing and executing appropriate quality tests in support to Core Experiments has become the norm especially in the Video group.

If a Call requests Proposals, the Requirements group, in conjunction with the Test group and possibly one or more technology group, reviews the result of the AHG and makes a final assessment. If the technologies submitted are judged to be sufficient to initiate the development of a standard, the activity is tranferred to that/those group(s) for standard development.

Current MPEG Chairs are

Requirements Jörn Ostermann Leibniz Univ. Hannover
Systems Youngkwon Lim Samsung
Video Lu Yu, Jens-Rainer Ohm, Gary Sullivan Zhejiang Univ., RWTH Aachen, Microsoft
Audio Schuyler Quackenbush Audio Research Labs
3DGC Marius Preda Institut Mines Télécom
Test Vittorio Baroncini GBTech
Communication Kyuheon Kim Kyunghee Univ.

The chairs come from a variety of countries (CN, DE. FR, IT, KR, US) and organisations (small/medium size and large companies, and Universities).

Joint meetings

A key MPEG feature is the immediate availability of the necessary technical expertise to discuss matters that cross organisational boundaries. The speed of development and quality of MPEG standards would hardly be possible if MPEG did not have the ability to timely deploy the necessary expertise to address multi-faceted issues.

Let’s take as an example this video where the basketball player is represented as a compressed dynamic point cloud in a 360ᵒ video. The matter is raised at a meeting, discussed at a Chairs meeting where the need for a joint meeting is identified. This is proposed to the MPEG plenary and held with the participation of Requirements, Systems and 3D Graphics Coding experts. The outcome of such a meeting may be anything from the acknowledgement that “we don’t know well enough yet”, to the identification of technologies that can simply be developed as a collaborative effort of MPEG experts or require a CfP.

As a typical example let’s consider some of the joint meetings held at MPEG 124 (20 in total):

  1. Anchors for 3DoF+; Participants from Video and Test meet to select the anchors (reference sequences) to be used in the 3DoF+ CfP (Monday, 15:00 to 16:00)
  2. 3DoF+ CfP; Participants from Requirements, Video and Test meet to discuss the text of the CfP that was published at MPEG 124 (Tuesday, 09:00 to 10:00)
  3. 3DoF+ CfP & viewing; Participants from Video and Test meet to review the text of and view the content for the CfP (Wednesday, 11:30 to 12:30)
  4. 3DoF+ viewing; Participants from Video and Test meet to view 3DoF+ content for the CfP (Thursday, 12:00 to 13:00)
  5. Systems aspects for Point Cloud Compression; Participants from 3DG and Systems meet to discuss the topic intruduced in this session (Tuesday, 09:00 to 10:00 and Thursday, 12:00 to 13:00)
  6. MPEG-I Scene graph; Participants from Requirements, Systems and Audio meet to discuss Audio needs for a scene description technology (Thursday, 09:00 to 10:00)
  7. Neural Network Compression CfE results; Participants from Requirements and Video meet to discuss the CfE results and decide whether there is room for a CfP (Tuesday, 16:00 to 17:00)

Meeting documents

How are documents fed into the MPEG process? This is not a marginal aspect if we consider that the number of submissions uploaded can easily cross 1000 documents. In an age of (almost) pervasive broadband internet, it is hard to imagine how MPEG could operate in a totally “paper-based” fashion when its membership was already around 300. In October 1995 Pete Schirling (then with IBM) made available to MPEG a document management system that allowed MPEG members to upload their documents and download those uploaded by the other members. Wo Chang (NIST) in 2000 and Christian Tulvan (Institut Mines Télécom) in 2005 have taken over the system that ensures the high efficiency of MPEG operation.

Recently ISO has developed its own document management system for use by all entities in ISO. However, MPEG has been exempted from using it because the traffic generated by uploading and downloading MPEG document in the weeks around an MPEG meeting would bring the system down. Incidentally, a requirement for hosting an MPEG meeting is the availability of internet access at 1 Gbit/s.

Meeting information

For MPEG experts (and those who do not attend yet) a place with so many “hotbeds” discussing requirements, and assessing, integrating and testing media technologies is exciting, but how can MPEG experts know what happens where and when at a meeting? Christian, the designer and maintainer of the MPEG document management system, has come to help again and designed a system that answers those questions. MPEG members can have a full view of which meetings are held where and when to discuss which topics or documents. The design is responsive so MPEG experts can get the information from their smartphones.

The figure shows a how the service looked like at MPEG 124 (October 2018). The page all meetings of all groups, but filtered views are possible. By clicking on each meeting details (room, documents etc.) can be obtained. 

Mission control

Another major development made by Christian provides the chairs with a series of functionalities to manage the MPEG workplan and timeline. Some of these, like the one below, are open to MPEG members.

The figure shows the MPEG timeline for the portion related to the MPEG-I, MPEG-CICP and MPEG-G standards. On the top there are icons to create new activities, show different table previews, provide project-wise views and more.

Another remarkable functionality is the creation of the document that collects all the results from a meeting. Group chairs enter their own results independently and the system organises them by standards for approval at the Friday plenary. This has made the review of decisions shorter and allowed MPEG to slash the time taken by plenaries.

MPEG assets

MPEG defines as assets the data associated with the development of standards: URIs, Schemas, Content, Software and Conformance testing data.

The first two are publicly available here and here.

By Content assets we mean the collection of all test data (content) used for CfEs and CfPs, and in subsequent Core Experiments. MPEG relies on the good will of companies interested in a standard to provide relevant test data. These are typically licensed for exclusive use in MPEG standardisation because they typically have high value, e.g. from the content and technology viewpoints. Content owners license their content directly to individual MPEG experts.

Software assets is the collection of all software developed or under development as reference software of MPEG standards. With the approach taken by MPEG to develop most of its standards from reference software and give it a normative status, its importance can hardly be overstated.

Conformance testing data is the collection of all data that can be used to test an implementation for conformance to an MPEG standard, published or under development. The process of developing conformance testing suites is painstaking and depends on the good will of MPEG members and their companies. On the other hand conformance testing suites are vital for the creation of ecosystems of interoperable implementations.

Content, Software and Conformance testing data for most data used in the development of MPEG standards in the last 30 years are stored in the Media Repository, Software Repository and Conformance Testing Repository hosted by Institut Mines Télécom. We are talking of a total of several TeraBytes of data.

Conclusions

This article can only give hints at the level of commitment and dedication of so many MPEG members – with the support of their companies. Of course, MPEG standards are produced by MPEG experts, but the MPEG process works because so many other people and organisations contribute, some in a visible and some others in a less visible form to make MPEG what it is.

Posts in this thread

MPEG communicates

Introduction

MPEG standards are great communication tools and MPEG itself is – or tries to be – a good communicator, of course using all available media.

This article is for those who want to be informed about MPEG, without having to digest one of its handy 500-page standards 😊

MPEG web site

Almost everything that will be mentioned in this post can be found, not necessarily in an easy way, in the MPEG web page: press releases, MPEG column, video tutorials, White papers, investigations and technical notes, ad hoc groups, events, social networks, liaisons and meetings. The next meeting will be held at Marrakesh, MA from 14 to 18 January 2019

Press releases

Since its early days, MPEG took care of informing the world of its work. At the beginning this was done only on major occasions. Now MPEG does that systematically at every meeting. The news considered the most important news gets the headlines and all other achievements get a note. The rule is to mention in a press release all Calls for Evidence (CfE) and Call for Proposals (CfP), and the standards under development that reach the stage of Committee Draft (CD) or Final Draft International Standard (FDIS). News are prepared by the relevant group chairs, edited and distributed to the press by Prof. Christian Timmerer of University of Klagenfurt.

Let’s take as look at the press release from MPEG 124 (Macau SAR, CN) to have an example.

  1. The headline news is “Point Cloud Compression – MPEG promotes a video-based point cloud compression technology to the Committee Draft stage” because we believe this will be a very important standard.
  2. The second news is “MPEG issues Call for Proposals on Compressed Representation of Neural Networks”.
  3. Two video compression-related CfPs “MPEG issues Call for Proposals on Low Complexity Video Coding Enhancements” and “MPEG issues Call for Proposals for a New Video Coding Standard expected to have licensing terms timely available”
  4. News on “Multi-Image Application Format (MIAF) promoted to Final Draft International Standard”
  5. Anticipation of a new CfP “3DoF+ Draft Call for Proposal goes Public”.

The press release page containing all press release since January 2006 here.

If you want to be added to the distribution list, please send an email to Christian.

MPEG Column

When possible the news in the press releases have links to documents relevant to the news. We try to avoid links to the mythical 500-pages documents but clearly knowing more about a news item typically has a higher entry level.

With the MPEG column MPEG tries to be facilitate understanding of what its standards do. At every meeting, brief notes are published that explain the purpose and, to some extent, the working of MPEG standards.

Want to know about High Dynamic Range (HDR)? the Common Media Application Format (CMAF)? The new standard to view omnidirectional video (OMAF)? By going here you will see all articles published in the column and will have the opportunity to get answers to many questions on these and other MPEG standards.

The MPEG column is managed by the professional journalist Philip Merrill who is able to understand MPEG standards and explain them to the public.

Do you think an article on a particular standard would be of interest? Please send email to the chairman of the Communication Group Prof. Kyuheon Kim of Kyunhee University. We will do our best to satisfy your request.

Video tutorials

How could MPEG miss the opportunity to have its own series of “MPEG standards tutorials”, of course using its audio and video compression standards? By going to Video tutorials on MPEG standards you will be able to understand what MPEG has done to make its standards “green”, what is the Multimedia Preservation Application Format that manages multimedia content over the ages, what is the High Efficiency Video Coding (HEVC) standard, what is MPEG-H 3D Audio, what is MPEG Media Transport (MMT) used in ATSC 3.0, what is Dynamic Adaptive Streaming over HTTP (DASH) for audio-visual strteaming over the internet and much more.

The content is delivered by the best experts in the field. The videos that you see are the result of the shooting and post-processing performed by Alexis Tourapis.

White papers, investigations and technical notes

MPEG makes its best efforts to provide the smoothest entry path to its standards and several types of publicly accessible papers are functional to this strategy. White papers are published with the goal to

  1. Communicate that we are investigating some promising ideas as in Investigations about parts of standards
  2. Describe the purpose of an entire suite of standards, as in the case of White papers about standards or for a single part of a standard like in White papers about parts of standards
  3. Provide specific guidance about the use of standards as in Technical notes about parts of standards.

As a rule, the purpose of white papers is not to describe how the technology work but about what the standard is for, the problems it solves and the benefits that MPEG expects users will get by using it.

Investigations, White papers and Technical notes can be found here.

Standards

ISO is a private association registered in Switzerland. Standards are developed pro bono by participants in the working groups, but the cost of the organisation is covered by the sale of standards and other sources. Therefore you should not expect to find ISO standards on the public MPEG web page. If you need a standard, you should go to the ISO web site where your can easily buy online all the standards on sale. In some cases, MPEG requests ISO to make standard public because the standards is particularly relevant or because the standard is already publicly available (as is the case of all standards developed jointly with ITU-T).

MPEG posts all public documents at Standard documents from the last meeting, e.g. Use cases, Requirements, Calls for Evidence, Calls for Proposals, Working Drafts up to Committee Drafts, Verification Test results and more. MPEG does this because it wants to make sure that the industry is aware of, can comment on, and contribute to what to develop standards.

Ad hoc groups

Since 1990 MPEG has created ad hoc groups (AHG). According to the rules “AHGs are established for the sole purpose of continuing work between consecutive MPEG meetings”, but they are a unique way to have work done outside MPEG meetings in a coordinated way. The scope of an AHG, however, is limited by the rule: “The task of an AHG may only cover preparation of recommendations to be submitted to MPEG”.

AHGs are not permanent organisations but established at the end of a meeting to last until the following meeting, To have a feeling of what AHGs are about you can see the AHGs established at 124th MPEG meeting.

AHGs are mentioned as part of MPEG communication because anybody can join the email reflector of an AHG and even attend AHG meetings.

MPEG events

In certain cases, MPEG organises events open to the public. Some events are held during and co-located with an MPEG meeting, but events outside MPEG meetings are also held. These are some of the goals an MPEG event can have:

  1. To present a particular standard under development or just released as in Workshop on MPEG-G (Shenzhen, October 2018)
  2. To introduce the MPEG workplan such as MPEG Workshop on Immersive Services Roadmap (Gwangju, January 2018)
  3. To demonstrate what the industry is doing with an MPEG standards such as OMAF Developers’ Day (Gwangju, January 2018)
  4. To frame a particular segment of the MPEG activity in the general context of the industry such as Workshop on 5G/ Beyond UHD Media (La Jolla, February 2016).

Reference software and conformance

MPEG standards can be made public by a decision of the ISO Central Secretariat. MPEG requests ISO to make all reference software and conformance standards publicly available. The rationale of this request is that if developers look at the reference software, they need to buy the standard to make a conforming implementation. To create a healthy ecosystem of interoperable products, services and applications – the conformance testing suites, too, must make freely available. This entices more users to buy the standard.

Social networks

MPEG has a Twitter account @MPEGgroup (https://twitter.com/mpeggroup). This is used by a group of MPEG social champions to spread information on the currently hottest MPEG topics: MPEG-I Requirements, Neural Network Compression, MPEG-G, OMAF, File format, Network Based Media Processing, MPEG-I Visual (3DoF+ and 6DoF), Audio, Point Cloud Compression, Internet of Media Things.

Please subscribe to receive brief notes on MPEG-related news and events.

Liaisons

MPEG develops standards technologies that are used by many industries all over the world. MPEG requests to liaise with many standards committees and industry fora for several purposes: to get use cases and requirements, to jointly develop standards, to promote adoption of the standard once it has been developed and to receive further requests for new functionalities.

These are some of the organisations MPEG has liaisons with: the Third Generation Partnership Project (3GPP), Advanced Television Systems Committee, Inc. (ATSC), Internet Engineering Task Force (IETF), Society of Motion Picture and Television Engineers (SMPTE), Audio Engineering Society (AES), European Broadcast Union (EBU), Hybrid Broadcast Broadband TV (HbbTV), Society of Cable Telecommunications Engineers (SCTE), World Wide Web Consortium (W3C) and many more.

Administrative documents

At every meeting MPEG publishes several documents – that I call “administrative” for lack of a better name – but are very important because they include organisational information. The following documents relate to MPEG 124 (Macau SAR, October 2018):

  1. list of ad hoc groups established at the meeting
  2. call for patents related to standards under development
  3. list of MPEG standards produced since day 1 and those planned
  4. MPEG work plan with a summary description of all activities under way including explorations
  5. MPEG timeline with planned dates of development of all standard
  6. MPEG terms of reference
  7. MPEG schemas
  8. MPEG URIs.

Conclusions

MPEG is fully aware of the social and industrial implications of its standards. This post conveys the MPEG policy of being open to the world, to the extent that ISO rules allow.

Posts in this thread (in bold this post)

How does MPEG actually work?

Introduction

In Life inside MPEG I have described the breadth, complexity and interdependence of the  work program managed by the MPEG ecosystem that has fed the digital media industry for the last 30 years. I did not mention, however, the actual amount of work developed by MPEG experts during an MPEG meeting. Indeed, at every meeting, the majority of work items listed in the post undergo a review prompted by member submissions. At the Macau meeting in October 2018, there were 1238 such documents covering the entire work program described in the post, actually more because that covers only a portion of what we are actually doing .

What I presume readers cannot imagine – and I do not claim that I do much more than they do – is the amount of work that individual companies perform to produce their submissions. Some documents may be the result of months of work often based on days/weeks/months of processing of audio, video, point clouds or other types of visual test sequences carried out by expensive high performance computers.

It is  clear why this is done, at least in the majority of cases: the discovery of algorithms that enable better and/or new audio-visual user experiences may trigger the development and deployment of highly rewarding products, services and applications. By joining the MPEG process, such algorithms may become part of one or more standard helping the industry develop top performance _interoperable_ products, services and applications.

But what is the _MPEG process_? In this post I would like to answer this question. As I may not be able to describe everything, I will likely have to revisit this issue in the future. In any case the  figure below should be used as a guide in the following.

How it all starts

The key players of the MPEG process are the MPEG members: currently ~1,500, ~1/4 are from academia and ~3/4 from industry, of which ~500 attend quarterly MPEG meetings.

Members bring ideas to MPEG for presentation and discussion at MPEG meetings. If the idea is found interesting and promising, an ad hoc group is typically established to explore further the opportunity until the next meeting. The proposer of the idea is typically appointed as chair of the ad hoc group. In this way MPEG offers the proposer the opportunity to become the entretrepreneur who can convert an idea into a product, I mean, an MPEG standard.

To get to a standard it may take some time: the newer the idea, the more time it may take to make it actionable by the MPEG process. After the idea has been clarified, the first step is to understand: 1) the context for which the idea is relevant, 2) the “use cases” the idea offers advantages for and 3) the requirements that a solution should satisfy to support the use cases.

Even if idea, use context, use cases and requirements have been clarified, it does not mean that technologies necessarily exist out there that can be assembled to provide the needed solution. For this reason, MPEG typically produces and publishes a Call for Evidence (CfE) – sometimes more than one – with attached context, use cases and requirements. The CfE requests companies who think they have technologies satisfying the requirements to demonstrate what they can achieve, without requesting them to describe how the results have been achieved. In many cases respondents are requested to use specific test data to facilitate comparison. If MPEG does not have them, it will ask industry to provide them.

Testing technologies for the idea 

If the result of the CfE is positive, MPEG will move to the next step and publish a Call for Proposals (CfP), with attached context, use cases, requirements, test data and evaluation method. The CfP requests companies who have technologies satisfying the requirements to submit responses to the CfP where they demonstrate the performance _anddisclose the exact nature of the technologies that achieve the demonstrated results.

Let’s see how this process has worked in the specific case of neural network compression. Note that the last row in italic refers to a future meeting. Links point to public documents on the MPEG web site.

Mtg

YY MM

Actions

120 17 Oct 1.      Presentation of idea of compressing neural networks

2.      Approval of CNN in CDVA (document)

121 18 Jan 1.      Release of Use cases and requirements 
122 18 Apr 1.      New release of Use cases and requirements 

2.      Release of Call for Test Data 

3.      Approval of Draft Evaluation Framework

123 18 Jul 1.      Release of Call for Evidence, Use cases and requirements

2.      New release of Call for Test Data

3.      New version of Draft Evaluation Framework

124 18 Oct 1.      Approval of Report on the CfE

2.      Release of Call for Proposals, Use cases and requirements, Evaluation Framework

126 19 Mar 1.      Responses to Call for Proposals

2.      Evaluation of responses based on Evaluation Framework

We can see that Use Cases and Requirements are updated at each meeting and made public, Test data are requested to the industry and the Evaluation Framework is developed well in advance of the CfP. In this particular case it will take 18 months just to move from the idea to CfP responses.

MPEG gets its hands on technology

A meeting where CfP submissions are due is typically a big event for the community involved. Knowledgeable people say that such a meeting is more intellectually rewarding than attending a conference. How could it be otherwise if participants not only can understand and discuss the technologies but also see and judge their actual performance? Everybody feels like being part of an engaging process of building a “new thing”.

If MPEG comes to the conclusion that the technologies submitted and retained are sufficient to start the development of a standard, the work is moved from the Requirements subgroup, which typically handles the process of moving from idea to proposal submission and assessment, to the appropriate technical group. If not, the idea of creating a standard is – maybe temporarily – dropped or further studies are carried out or a new CfP is issued calling for the missing technologies.

What happens next? The members involved in the discussions need to decide which technologies are useful to build the standard. Results are there but questions pop up from all sides. Those meetings are good examples of the “survival of the fittest” principle, applied to technologies as well as to people.

Eventually the group identifies useful technologies from the proposals and builds an initial Test Model (TM) of the solution. This is the starting point of a cyclic process where MPEG experts

  1. Identify critical points of the RM
  2. Define which experiments – called Core Experiments (CE) – should be carried out to improve TM performance
  3. Review members’ submissions
  4. Review CE technologies and adopt those bringing sufficient improvements.

At the right time (which may very well be the meeting where the proposals are reviewed or several meetings after), the group produces a Working Draft (WD). The WD is continually improved following the 4 steps above.

The birth of a “new baby” is typically not without impact on the whole MPEG ecosystem. Members may wake up to the need to support new requirements or they realise that specific applications may require one or more “vehicle” to embed the technology in those application or they come to the conclusion that the originally conceived solution needs to be split in more than one standard.

These and other events are handled by convening joint meetings between the group developing the technology and all technical “stakeholders” in other groups.

The approval process

Eventually MPEG is ready to “go public” with a document called Committee Draft (CD). However, this only means that the solution is submitted to the national standard organisations – National Bodies (NB) – for consideration. NB experts vote on the CD with comments. If a sufficient number of positive votes are received (this is what has happened for all MPEG CDs so far), MPEG assesses the comments received and decides on accepting or rejecting them one by one. The result is a new version of the specification – called Draft International Standard (DIS) – that is also sent to the NBs where it is assessed again by national experts, who vote and comment on it. MPEG reviews NB comments for the second time and produces the Final Draft International Standard. This, after some internal processing by ISO, is eventually published as an International Standard.

MPEG typically deals with complex technologies that companies consider “hot” because they are urgently needed for their products. As much as in companies the development of a product goes through different phases (alpha/beta releases etc., in addition to internal releases), achieving a stable specification requires many reviews. Because CD/DIS ballots may take time, experts may come to a meeting reporting bugs found or proposing improvements to the document under ballot. To take advantage of this additional information that the group scrutinises for its merit, MPEG has introduced an unofficial “mezzanine” status called “Study on CD/DIS” where proposals for bug fixes and improvements are added to the document under ballot. These “Studies on CD/DIS” are communicated to the NBs to facilitate their votes on the official documents under ballot.

Let’s see in the table below how this part of the process has worked for the Point Cloud Compression (PCC) case. Only the most relevant documents have been retained. Rows in italic refer to future meetings.

Mtg

YY MM

Actions

120 17 Oct 1.      Approval of Report on PCC CfP responses

2.      Approval of 7 CEs related documents

121 18 Jan 1.      Approval of 3 WDs

2.      Approval of PCC requirements

3.      Approval of 12 CEs related documents

122 18 Apr 1.      Approval of 2 WDs

2.      Approval of 22 CEs related documents

123 18 Jul 1.      Approval of 2 WDs

2.      Approval of 27 CEs related documents

124 18 Oct 1.      Approval of V-PCC CD

2.      Approval of G-PCC WD

3.      Approval of 19 CEs related documents

4.      Approval of Storage of V-PCC in ISOBMFF files

128 19 Oct 1.      Approval of V-PCC FDIS
129 20 Jan 1.      Approval of G-PCC FDIS

I would like to draw your attention to “Approval of 27 CEs related documents” in the July 2018 row. The definition of each of these CE documents requires lengthy discussions by involved experts because they describe the experiments that will be carried out by different parties at different locations and how the results will be compared for a decision. It should not be a surprise if some experts work from Thursday until Friday morning to get documents approved by the Friday plenary.

I would also like to draw your attention to “Approval of Storage of V-PCC in ISOBMFF files” in the October 2018 row. This is the start of a Systems standards that will enable the use of PCC in specific applications, as opposed to being just a data compression specification.

The PCC work item is currently seeing the involvement of some 80 experts. If you think that there were more than 500 experts at the October 2018 meeting, you should have a pretty good idea of the complexity of the MPEG machine and the amount of energies poured in by MPEG members.

The MPEG process – more than just a standard

In most cases MPEG performs “Verification Tests” on a standard  produced to provide the industry with precise indications of the performance that can be expected from an MPEG compression standard. To do this the following is needed: specification of tests, collection of appropriate test material, execution of reference or proprietary software, execution of subjective tests, test analysis and reporting.

Very often, as a standard takes shape, new requirements for new functionalities are added. They become part of a standard either through the vehicle of “Amendments”, i.e. separate documents that specify how the new technologies can be added to the base standard, or “New Editions” where the technical content of the Amendment is directly integrated into the standard in a new document.

As a rule MPEG develops reference software for its standards. The reference software has the same normative value as the standard expressed in human language.

MPEG also develops conformance tests, supported by test suites, to enable a manufacturer to judge whether its implementation of 1) an encoder produces correct data by feeding them into the reference decoder or 2) a decoder is capable of correctly decoding the test suites.

Finally it may happen that bugs are discovered in a published standard. This is an event to be managed with great attention because industry may already have released some implementations.

Conclusions

MPEG is definitely a complex machine because it needs to assess if an idea is useful in the real world, understand if there are technologies that can be used to make a standard supporting that idea, get the technologies, integrate them and develop the standard. Often it also has to provide integrated audio-visual solutions where a line up of standards nicely fit to provide a system specification. At every meeting MPEG works simultaneously on tens of intertwined activities taking place at the same time.

MPEG needs to work like a company developing products, but it is not a company. Fortunately one can say, but also unfortunately because it has to operate under strict rules. ISO is a private organisation, but it is international in scope.

Posts in this thread (in bold this post)

Life inside MPEG

Introduction

In my earlier post 30 years of MPEG, and counting? I brought evidence of what MPEG has done in the last 30 years to create the broadcast, recording, web and mobile digital media world that we know. This document tries to make the picture more complete by looking at the current main activities and deliveries in the next few months/years.

The MPEG work plan at a glance

The figure below shows the main standards developed or under development by MPEG in the 2017-2023 period organised in 3 main sections:

  • Media Coding (e.g. MP3 and AVC)
  • Systems and Tools (e.g. MPEG-2 TS and File Format)
  • Beyond Media (currently Genome Compression and Neural Network Compression).

Navigating the MPEG work plan

Video Coding Audio Coding 3D Graphics Coding
Font Coding Genome Coding Neural Network Coding
Media Description Systems support IPMP
Transport Application Formats API
Media Systems Conformance Reference Implementation

Video coding

In the Video coding area MPEG is currently handling 4 standards (MPEG-4, -H, -I and -CICP) and several Explorations.

MPEG-I ISO/IEC 23090 Coded representation of immersive media is the container of standards needed for the development of immersive media devices, applications and services.

MPEG is currently working on Part 3 Versatile Video Coding, the new video compression standard after HEVC. VVC is developed jointly with VCEG and is expected to reach FDIS stage in October 2020.

Explorations

  1. An exploration on a new video coding standard that combines coding efficiency (similar to that of HEVC), complexity (suitable for real time encoding and decoding) and usability (timely availability of licensing terms). In October 2018 a Call for Proposals was issued. Submissions are due in January 2019 and FDIS stage for the new standard is expected to be reached in January 2020.
  2. An exploration on a future standard that defines a data stream structure composed by two streams: a base stream decodable by a hardware decoder, and an enhancement stream suitable for software processing implementation with sustainable power consumption. This activity is supported by more than 30 major international players in the video distribution business. A Call for Proposal has been issued in October 2018. Submissions are due in March 2019 and FDIS stage is expected to be reached in April 2020.
  3. Several explorations on Immersive Video Coding
    • 3DoF+ Visual: a Call for Proposal will be issued in January 2019. Submissions are due in March 2019 and the FDIS is planned for July 2020. The result of this activity is not meant to be a video coding standard but a set of metadata that can be used to provide a more realistic user experience in OMAF v2. Indeed, 3DoF+ Visual will be a part of MPEG-I part 7 Immersive Media Metadata. Note that 3 Degrees of Freedom (3DoF) means that a user can only make yaw, pitch, roll movements, but 3DoF+ means that the user can also displace the head to a limited extent.
    • Several longer-term explorations on compression of 6DoF visual (Windowed-6DoF and Omnidirectional 6DoF) and Compression of Dense Representation of Light Fields. No firm timeline for standards in these areas has been set.

Audio coding

In the Audio coding area MPEG is handling 3 standards (MPEG-4, -D, and -I). Of particular relevance is the MPEG-I part 3 Immersive Audio activity. This is built upon MPEG-H 3D Audio – which already supports a 3DoF user experience – and will provide a 6DoF immersive audio VR experience. A Call for Proposal will be issued in March 2019. Submissions are expected in January 2020 and FDIS stage is expected to be reached in April 2021. As for 3DoF+ Visual, this standard will not be about compression, but about metadata.

3D Graphics coding

In the 3D Graphics coding area MPEG is handling two parts of MPEG-I.

  • Video-based Point Cloud Compression (V-PCC) for which FDIS stage is planned to be reached in October 2019. It must be noted that in July 2018 an activity was initiated to develop standard technology for integration of a 360 video and V-PCC objects.
  • Geometry-based Point Cloud Compression (G-PCC) for which FDIS stage is planned to be reached in January 2020.

The two PCC standards employ different technologies and target different application areas: entertainment and automotive/unmanned aerial vehicles,

Font coding

In the Font coding area MPEG is working on MPEG-4 part 22.

MPEG-4 ISO/IEC 14496 Coding of audio-visual objects is a 34-part standard that made possible large scale use of media on the fixed and mobile web.

Amendment 1 to Open Font Format will support complex layouts and new layout features. FDAM stage will be reached in April 2020.

Genome coding

MPEG-G ISO/IEC 23092 Genomic Information Representation is the standard developed in collaboration with TC 276 Biotechnology to compress files containing DNA reads from high speed sequencing machines.

In the Genome coding area MPEG plans to achieve FDIS stage for Part 2 Genomic Information Representation in January 2019. MPEG has started investigating additional genome coding areas that would benefit from standardisation.

Neural network coding

Neural network compression is an exploration motivated by the increasing use of neural networks in many applications that require the deployment of a particular trained network instance potentially to a large number of devices, which may have limited processing power and memory.

In the Neural network coding area MPEG has issued a Call for Evidence in July 2018, assessed the responses received in October 2018 and collected evidence that justifies the Call for Proposals issued at the same October 2018 meeting. The goal of the Call is to make MPEG aware of technologies to reduce the size of trained neural networks. Responses are due in March 2019. As it is likely that in the future more Calls will be issued for other functionality (e.g., incremental representation), an expected time for FDIS has not been identified yet.

Media description

Media description is the goal of the MPEG-7 standard which contains technologies for describing media, e.g. for the purpose of searching media.

In the Media description area MPEG has completed Part 15 Compact descriptors for video analysis (CDVA) in October 2018. By exploiting the temporal redundancy of video, CDVA extracts a single compact descriptor that represents a video clip rather than individual frames, which was the goal of Compact Descriptors for Visual search (CDVS).

Work in this area continues to complete reference software and conformance.

System support

In the System support area MPEG is working on MPEG-4, -B and -I. In MPEG-I MPEG is developing

  • Part 6 – Immersive Media Metrics which specifies the metrics and measurement framework to enhance the immersive media quality and experiences. 3DoF+ Visual metadata will be one component of this standard
  • Part 7 – Immersive Media Metadata which specifies common immersive media metadata focusing on immersive video (including 360° video), images, audio, and timed text.

Both parts are planned to reach FDIS stage in July 2020.

Intellectual Property Management and Protection

In the IPMP area MPEG is developing an amendment to support multiple keys per sample. FDAM stage is planned to be reached in March 2019. Note that IPMP is not about _defining_ but _employing_ security technologies to digital media.

Transport

In the Transport area MPEG is working on MPEG-2, -4, -B, -H, -DASH, -I, -G and Explorations.

MPEG-2 ISO/IEC 13818 Generic coding of moving pictures and associated audio information is the standard that enabled digital television.

Part 2 Systems continues to be an extremely lively area of work. After producing Edition 7, MPEG is working on two amendments to carry two different types of content

  • JPEG XS (a JPEG standard for low latency applications)
  • CMAF (an MPEG Application Format).

MPEG-4

Part 12 ISO Based Media File Format is another extremely lively area of work. Worth mentioning are two amendments

  • Compact Sample-to-Group, new capabilities for tracks, and other improvements – has reached FDAM stage in October 2018
  • Box relative data addressing – is expected to reach FDAM in March 2019.

The 7th Edition of the MP4 file format is awaiting publication.

MPEG-B ISO/IEC 23001 MPEG systems technologies is a collection of systems standards that are not specific to a given standard.

In MPEG-B MPEG is working on two new standards

  • Part 14 Partial File Format will provide a standard mechanism to store HTTP entities and the partial file in broadcast applications for later cache population. The standard is planned to reach FDIS stage in April 2020.
  • Part 15 Carriage of Web Resources in ISOBMFF will make it possible to enrich audio/video content, as well as audio-only content, with synchronised, animated, interactive web data, including overlays. The standard is planned to reach FDIS stage in January 2019.

MPEG-H ISO/IEC 23008 High efficiency coding and media delivery in heterogeneous environments is a 15-part standard for audio-visual compression and heterogeneous delivery.

Part 10 MPEG Media Transport FEC Codes is being enhanced by the Window-based FEC code an amendment. FDAM is expected to be reached in January 2020.

MPEG-DASH ISO/IEC 23009 Dynamic adaptive streaming over HTTP (DASH) is the standard for media delivery on unpredictable-bitrate delivery channels.

In MPEG-DASH MPEG is working on

  • Part 1 Media presentation description and segment formats is being enhanced by the Device information and other extensions amendment. FDAM is planned to be reached in July 2019.
  • Part 7 Delivery of CMAF content with DASH contains guidelines recommending some of the most popular delivery schemes for CMAF content using DASH. TR is planned to be reached in March 2019

MPEG-I

Part 2 Omnidirectional Media Format (OMAF) released in October 2017 is the first standard format for delivery of omnidirectional content. With OMAF 2nd Edition Interactivity support for OMAF, planned to reach FDIS in January 2020, MPEG is substantially extending the functionalities of OMAF. Future releases may add 3DoF+ functionalities.

MPEG-G

MPEG plans to achieve FDIS stage for Part 1 Transport and Storage of Genomic Information in January 2019.

Application Formats

MPEG-A ISO/IEC 23000 Multimedia Application Formats is a suite of standards for combinations of MPEG and other standards (when there are no suitable MPEG standard for the purpose).

MPEG is working on two new standards

  • Part 21 Visual Identity Management Application Format will provide a framework for managing privacy of users appearing on pictures or videos shared among users. FDIS is expected to be reached in January 2019.
  • Part 22 Multi-Image Application Format (MIAF) will enable precise interoperability points when creating, reading, parsing and decoding images embedded in HEIF (MPEG-H part 12).

Application Programming Interfaces

The Application Programming Interfaces area comprises standards developed in order to make possible effective use of some MPEG standards.

In the API area MPEG is working on MPEG-I, MPEG-G and MPEG-IoMT.

MPEG-I

MPEG is working on Part 8 Network-based Media Processing (NBMP), a framework that will allow users to describe media processing operations to be performed by the network. The standard is expected to reach FDIS stage in January 2020.

MPEG-G

MPEG is working on Part 3 Genomic Information Metadata and Application Programming Interfaces (APIs). The standard is expected to reach FDIS stage in March 2019.

MPEG-IoMT ISO/IEC 23093 Internet of Media Things is a suite of standards supporting the notion of Media Thing (MThing), i.e. a thing able to sense and/or act on physical or virtual objects, possibly connected to form complex distributed systems called Internet of Media Things (IoMT) where MThings interact between them and humans.

In MPEG-IoMT MPEG is working on

  • Part 2 IoMT Discovery and Communication API
  • Part 3 IoMT Media Data Formats and API.

Both expected to reach FDIS stage in March 2019.

Media Systems

Media Systems includes standards or Technical Reports targeting architectures and frameworks.

In Media Systems MPEG is working on Part 1 IoMT Architecture, expected to reach FDIS stage in March 2019. The architecture used in this standard is compatible with the IoT architecture developed by JTC 1/SC 41.

Reference implementation

MPEG is working on the development of 10 standards for reference software of MPEG-4, -7, -B, -V, -H, -DASH, -G, -IoMT

Conformance

MPEG is working on the development of 8 standards for conformance of MPEG-4, -7, -B, -V, -H, -DASH, -G, -IoMT.

Conclusions

Even this superficial overview should make evident to all the complexity of interactions in the MPEG ecosystem that has been ongoing for 30 years (note that the above only represents a part of what happened at the last meeting in Macau).

In 30 years of MPEG, and counting? I wrote “Another thirty years await MPEG, if some mindless industry elements will not get in the way”.

It might well have been a prophecy.

Posts in this thread (in bold this post)

Data Compression Technologies – A FAQ

This post attempts to answer some of the most frequent questions received on the proposed ISO Data Compression Technologies Technical Committee (DCT TC). See here and here. If you have a question drop an email to leonardo@chiariglione.org.

Q: What is the difference between MPEG and DCT?

A: Organisation-wise MPEG is a Working Group (WG), reporting to a Subcommittee (SC), reporting to a Technical Committee (TC), reporting to the Technical Management Board (TMB). Another important difference is that a WG makes decision by consensus, i.e. with a high standard of agreement – not necessarily unanimity – while a TC makes decisions by voting.

Content-wise DCT is MPEG with a much wider mandate than media compression involving many more industries than just media.

Q: What is special of DCT to need a Technical Committee?

A: Data compression is already a large industry even if one only looks at the single media industry. By serving more client industries, it will become even larger and more differentiated. Therefore, the data compression industry will require a body with the right attributes of governance and representation. As an example, data compression standards are “abstract” as they are collections of algorithms, and intellectual property intensive (today most patent declarations submitted to ISO are from MPEG). The authority to regulate these matters resides in ISO but the data compression industry will need an authoritative body advocating its needs because the process will only intensify in the future.

Q: Why do you think that a Technical Committee will work better than a Working Group?

A: With its 175 standards produced and its 1400 members, over 500 of which attend quarterly meetings, MPEG – a working group – has demonstrated that it can deliver revolutionising standards affecting a global industry. DCT intends to apply the MPEG successful business model to many other non-homogeneous industries. Working by consensus is technically healthy and intellectually rewarding. Consensus will continue to be the practice of DCT, but policy decisions affecting the global data compression industry often require a vote. Managing a growing membership (on average one member a day joins the group) requires more than an ad hoc organisation like today. The size of MPEG is 20-30 times the normal size of an ISO WG.

Q: Which are the industries that need data compression standards?

According to Forbes, 163 Zettabytes of data, i.e. 163 billion Terabytes, will be produced worldwide in 2025. Compression is required by the growing number of industries that are digitising their processes and need to store, transmit and process huge amounts of data.

The media industry will continue its demands for more compression and for more media. The industries that capture data from the real world with RADAR-like technologies (earth-bound and unmanned aerial vehicles, civil engineering, media etc.), genomics, healthcare (all sorts of data generated by machines sensing humans), automotive (in-vehicle and vehicle-to-vehicle communication, environment sensing etc.), industry 4.0 (data generated, exchanged and processed by machines), geographic information and more.

Q: What are the concrete benefits that DCT is expected to bring?

A: MPEG standards have provided the technical means for the unrelenting growth of the media industry over the last 30 years. This has happened because MPEG standards were 1) timely (i.e. the design is started in anticipation of a need), 2) technically excellent (i.e. MPEG standards are technically the best at a given time), lively (i.e. new features improving the standard are continuously added) and 4) innovative (i.e. a new MPEG standard is available when technology progress enables a significant delta in performance). DCT is expected to do the same for the other industries targeted by DCT.

Q: Why should DCT solutions be better than solutions that are individually developed by industries?

A: “Better” is a multi-faceted word and should be assessed from different angles: 1) technical: compression is an extremely sophisticated domain and MPEG has proved that relying on the best experts produces the best results; 2) ecosystem: implementors are motivated to make products if the market is large and sharing technologies means that implementors face a lower threshold to enter. ON the contrary a small isolated market may not provide enough motivation to implementors; 3) maintenance: an industry may well succeeds in assembling a group of experts to develop a standard. However, a living body like a standard needs constant attention that can no longer be provided if  the experts have left the committee; 4) interoperability: industry-specific data compression methods are dangerous when digital convergence drives data exchange across apparently unrelated industry segments because they create non-communicating islands.

Posts in this thread (in bold this post)