Standards and collaboration

You are currently viewing Standards and collaboration

Standards and collaboration

The hurdles of standardisation today

Making standards is not like any other tasks. In most cases it is technical in nature because it is about agreeing on and documenting how certain things should be done to claim to be conforming to the standard. Standards can be developed unilaterally by someone powerful enough to tell other people how they should do things. More often, however, standards are developed collaboratively by people who share an interest in a standard, i.e. in enabling those who are willing to do certain things in the same way to have an agreed reference.

Many years ago, making a standard required that those who developed it just talk to people in their environment. Before MPEG all television distribution industries were silos sharing at most some technologies here and there. This is shown in Figure 1.

Figure 1 – The video industry – Before MPEG

By specifying a common “digital baseband” layer, MPEG standards prompted industry convergence, as shown in Figure 2.

Figure 2 – The video industry – After MPEG

Today, and especially in the domain of digital media, it is common not to have the luxury of defining a standard in isolation. Systems get more and more complex and their individual elements – which may be implementations of  standards – have to interact with other elements – which are again possibly implementations of other standards.

Some of these standards are produced by the same standards organisation while other standards are produced by different organisations. How is it possible to make sure that the “standard” elements used to make the system fit nicely, if there is no one overseeing the overall process?

The answer is that, indeed, it is not possible. If it happens it is because of luck or because there were enough people of good will who cared to attend the different groups to ensure coordination.

In some cases, all standards used to make the systems are produced by groups belonging to the same standards organisation. Some of these organisations, however, think that they can solve the problem of interoperability of standards by defining precise borders (“scopes”) within which a group of experts is allowed to develop standards.

This approach probably worked decently well in the past that is represented by Figure 1. However, this approach is destined to become less and less practical to implement and the result to become less and less satisfactory and reliable.

Many standards for use today must be conceived more on their ability to integrate or interface to technologies from different sources than on the traditional “territory” delimited by the “scope” or “terms of reference” etc. of the group that created it. This trend will only continue in the future. A new approach to standardisation must be developed and put to work.

A “systems-based” approach to standardisation

That the scope-based approach to standardisation is no longer serving its original purpose does not mean that it should be abandoned. It should just be given a different purpose. So far, the “scope” was more like the ring of walls that protected medieval towns against invasions. However, the scope should become an area of competence where “gates” can be “opened” so that “alliances” with other groups can be stipulated.

MPEG has put this attitude into practice for many years. The success of MPEG standards is largely based on this attitude.

Here follows a list of cases.

Collaboration with ISO/TC 276 for the creation of a standard for DNA read compression

In the first half of 2010’s MPEG identified “compression of DNA reads” generated by high-speed sequencing machines as an area where its coding expertise can be put to good use. MPEG investigated the field and identified a first set of requirements. As DNA can certainly not be assimilated to “moving pictures and audio” (the area MPEG is competent for) MPEG experts met with TC 276 Biotechnology to present their findings and propose a collaboration.

This move was positively received because TC 276 was indeed in need for such a standard but did not have the expertise to develop it. Therefore, MPEG and TC 276 engaged in a joint effort to refine the requirements of the project.

Then TC 276 entrusted the development of the standard (called MPEG-G) to MPEG on condition of regular reports to TC 276. Ballots on the standard at different phases of development were managed by MPEG, and the results were reported to TC 276.

Today the joint MPEG-TC 276 “venture” has produced 3 standards (File format, Compression, and API and Metadata), is finalising two standards (Reference software and Conformance) and has issued a Joint Call for Proposals for a 6th standard on “Genomic Annotation Representation”.

This is an excellent example of MPEG “entrepreneurship”. Some experts saw the opportunity to develop a DNA read compression standard using the MPEG “toolkit”. They “opened a gate” to communicate with the Biotechnology world and were lucky to find that Biotechnology was equally happy to “open a gate” on their side.

Collaboration with a non-ISO standards group in need of standards MPEG can develop

The MPEG-4 project, started in 1993 (!), has been the first consistent effort by MPEG to provide standards usable by the IT and mobile industry. The 3rd Generation Partnership Project (3GPP), so named because it started in December 1998, at the time of 3G (now we are at 5G and looking forward to 6G) is a very successful international endeavour providing standards for the entire protocol stack needed by the mobile industry (that largely includes the IT industry).

Quite a few MPEG experts attend 3GPP meetings. They are best placed to understand 3GPP’s early standardisation needs. Here I will mention two successful cases.

3GPP needed a file format for multimedia content and MPEG had developed the ISO Based Media File Format (ISOBMFF, aka MP4 File Format). MPEG liaised with 3GPP using its common members, understood the requirements and developed a specification that is essentially a restriction of ISOBMFF (ETSI TS 126 244).

More recently (end of 2010’s), 3GPP has initiated studies on adaptive streaming of time-dependent media. MPEG experts attending 3GPP saw the opportunity and convinced 3GPP that they should entrust to MPEG the development of the standard. MPEG developed requirements that were checked for consistency with 3GPP needs at 3GPP meetings by the common MPEG-3GPP experts. MPEG developed 3GPP-DASH standard and the experts attending both MPEG and 3GPP relayed the necessary information to 3GPP and checked that the choices made by MPEG were agreeable to 3GPP. The 3GPP-DASH specification is ETSI TS 126 247.

In the case of DASH, an industry forum (DASH-IF) was formed to handle the needs of industry members who cannot afford to join MPEG. Experts attending both MPEG and DASH-IF relay information in both directions. The information brought to MPEG has given and is still giving rise to amendments to the DASH standard supporting more functionalities.

DASH is again an excellent example of MPEG entrepreneurship. MPEG “opened gates” to DASH that are still very busy and connect to many more external “gates”, e.g. Digital Video Broadcasting) DVB, Hybrid Broadcast Broadband TV (HbbTV).

Collaboration with an ISO/IEC committee needing MPEG standards to enhance use of its standards

MPEG “opened gates” to JPEG to respond to its needs for “Systems” support to its standards.

The original JPEG image compression standards was widely used in the early days of digital video because it could use inexpensive VLSI chips implementing the relatively simple JPEG codec to store and transmit sequences of individual images (video frames). However, there was no specification for this “Motion JPEG”.

In the early 2000’s, JPEG 2000 appeared as the next generation image compression standard and JPEG needed a file format to store and transmit sequences of individually JPEG 2000 coded images. MPEG gladly adapted the ISOBMFF to make it able to carry sequences of JPEG 2000 and original JPEG images. The file format has allowed wider use of JPEG 2000, e.g. by the movie industry.

A related case is provided by the JPEG need to enable transport of two image compression formats – JPEG 2000 and JPEG XS – on the successful MPEG transport standard, MPEG-2 Transport Stream. For both case MPEG received requests with a first set of requirements. It analysed the requests, added other requirements and sent them back to JPEG. An occasional face-to-face meeting was needed to close the requirements and to provide suggestion for minor extensions to the JPEG standard.

MPEG developed and balloted the amendment to carry JPEG 2000 and JPEG XS on MPEG-2 Transport Stream.

Collaboration to develop a specific instance of an ISO/IEC committee’s general standard

MPEG has two instances of this form of collaboration: Internet of Media Things (IoMT) and Network Based Media Processing (NBMP). The former is about APIs for discovery of and interaction between “Media Things” (e.g. cameras, microphones, displays and loudspeakers) communicating according to the Internet of Things (IoT) paradigm. The latter is a set of APIs allowing a device (e.g. a handset) to get some processing on media done by a networked service.

In JTC 1 MPEG stands out for its standards because they offer interoperability between implementations as opposed to most other standards which are about frameworks and architectures. This does not mean that MPEG does not need architectures. It needs them but it makes no sense to develop its own architectures. Much better if its architectures are specific instances of general architectures. This is true of IoMT and NBMP.

SC 41 was in the process of developing a general architecture for Internet of Things (IoT). MPEG developed a draft architecture and had it validated by SC 41.

SC 42 has developed a general architecture for Big Media Data. MPEG is developing Network based Media Processing (BNMP), which can be seen as an instance of the general Big Media Data architecture. Work on aligning the architectures of the two development is progressing.

MPEG collaborates on a standard that is also of interest to another ISO/IEC committee

This is the case of the Mixed and Augmented Reality Reference Model that MPEG has jointly developed with SC 24. This happened because SC 24 needed a framework standard for Mixed and Augmented Reality, from the architectural viewpoint and MPEG had similar interests, but from the bit-level interoperability viewpoint. SC 24 and MPEG agreed on the requirements for the standard and established a joint group (in this case, a Joint Ad hoc Group) with terms of reference, two chairs (one for SC 24 and one for MPEG) and a timeline. Ballots were handled by the SC 24 secretariat and the Joint Ad hoc Group resolved the comments from both NBs.

Collaboration to enable MPEG to develop an extension of one of its standards that falls under another ISO/IEC committee’s scope

This case is exemplified by a scenario under which MPEG and JPEG have collaborated towards a new image coding standard that is based on an MPEG moving picture coding standard.

This happened because conventional video coding standards need to support “clean” switching between different channels in broadcast applications, and random access for other use cases. This allows a decoder to reconstruct certain pictures in a video sequence (intra pictures), independently from other pictures in that sequence.

MPEG wished to develop the High Efficiency Image Format (HEIF) by defining a special case of the ISOBMFF relative to the HEVC intra picture mode. In a face-to-face meeting this goal was agreed and HEIF is now a successful file format supporting many modalities of interest to users.

Conclusions

The scope of work of an ISO/IEC committee is certainly useful as a reference. However, the current trend toward more convergence and more complex systems that rely on multiple standards requires a more flexible “gate” approach exemplified above. A committee may “open gates” toward another committee and the two may committees agree on developing specific projects. This approach does not work in a “defence of territory” mode where collaborations are seen as limiting a committee’s freedom, but by seeing collaborations with other committees and groups as opportunities to develop standards with a larger field of use where the constituencies of both committees share the benefits.

The examples mentioned in this article are actual cases that show how the extent of the MPEG scope and the modalities of collaboration used have been made possible by the use of the “gate” approach to develop collaborative standards.

Posts in this thread