The MPEG trajectory is not the trajectory of an Information Technology (IT) group. Today software plays a key role in MPEG standard development. However, MPEG it is not an IT group. For MPEG, software is a tool to achieve the goal of producing excellent standards. But software remains a tool. Clearly, because MPEG assembles so many industries, with so many different agendas. There are MPEG members for which software is more than a tool.
In this article I will explore the relationship of MPEG with software and, in particular, Open Source Software.
In my early professional days I had the opportunity to be part of an old-type ICT standardisation, the European COST 211 project. A video codec specification (actually more than that, because it contained Systems aspects as well) was later submitted to and became a Recommendation of CCITT (Today ITU-T) with the acronym and title H.120 – Codecs For Videoconferencing Using Primary Digital Group Transmission.
The specification was developed on the basis of contributions received, discussed, possibly amended and then added to the specification. There was no immediate “verification” of the effectiveness of contributions adopted because that relied on hardware implementation of the specification and hardware is a different beast. Four countries (DE, FR, IT and UK) implemented the specification that was eventually confirmed by field trials using 2 Mbit/s satellite links where the 4 implementations were shown to interoperate.
MPEG-1 and MPEG-2
That happened in the years around 1980. Ten years later, MPEG started the development of the MPEG-1 Video and then the MPEG-2 Video standards using a different method.
MPEG assembled the first MPEG-1 Video Simulation Model (VM) at MPEG10 (1990/03). The VM was comparable with the H.120 evolving specification because it was a traditional textual description. At MPEG12 (1990/08), MPEG started complementing the text of the standard with pseudo C-code because people accustomed to write computer programs found it more natural to describe the operations performed by a codec with this language than using words.
In MPEG-1 and MPEG-2 times active participants developed and maintained their own simulation software. Some time later, however, it was decided to develop reference software, i.e. a software implementation of the MPEG-1 standard.
Seen with the eyes of a software developer, the process of standard development in MPEG-1 and MPEG-2 times was rather awkward, because the – temporally overlapping – sequence of steps was:
- Produce a textual description of the standard
- Translate the text to the individual software implementing the Simulation Model
- Optimise the software
- Translate the software back to text/pseudo C-code.
Reference Software and Conformance Testing
People of the early MPEG days used software – and quite intensely – because that was a tool that cut the time it would take to develop the specification by orders of magnitude while offering the opportunity to obtain a standard with better performance.
Another important development was the notion of conformance testing. Separation of the specification (the law) from determination of conformance (the tribunal) was a major MPEG innovation. The reference software could be used to test an encoder implementation for conformance by feeding the bitstream produced by the implemented encoder to the reference decoder. Especially produced conformance testing bitstreams could be used to test a decoder for conformance.
Conformance testing and its “tool” reference software is an essential add-on to the standard because it gives users the freedom to make their own implementation and enables the creation of ecosystems of interoperable implementations.
Open Source Software (OSS)
OSS is a very large and impactful world for which the software is not the tool to achieve a goal but the goal itself. Those adopting the Gnu’s Not Unix (GNU) General Public License (GPL) grant some basic rights to users of their software they call “Free”. The terms can be (roughly) summarised as
- Distribute copies of the software
- Receive the software or get it
- Change the software or use pieces of it in new programs
in exchange of a commitment of the user to
- Give another recipient all the rights acquired
- Make sure that recipients are able to receive or get the software
- Recipients must be made aware that a software is a modification.
Two additional issues should be borne in mind:
- There is no warranty for GNU license software
- Any patent required to operate the software must be licensed to everybody.
Development of the MPEG-4 Visual standard took another important turn, one that marked the convergence of the way telecom, broadcasting and consumer electronics on one side and information technology on the other side developed.
Unaware of the formalisation of the OSS rules that were already taking place in the general IT world, MPEG made the decision to develop the MPEG-4 reference software collaboratively because
- Better reference software would be obtained
- The scope of MPEG-4 was so large that probably no company could afford to develop the complete software implementation of the standard
- A software implementation made available to the industry would accelerate adoption of the standard
- A standard with two different forms of expression would have improved quality because the removal of an ambiguity from one form of expression would help clarify possible ambiguities in the other.
MPEG-4 Visual had only one person in charge of the Test Model. All new proposals were assessed and, if agreed, converted to Core Experiments. If at least two participants from two different institutions brought similarly convincing improvement results, the proposal would be accepted and added to the TM.
Therefore the MPEG-4 software was no longer just the tool to develop the standard, it became the tool to make products based on the standard, but not necessarily the only one. Therefore a reversal of priorities was required because the standard in textual form was still needed, but many users considered the standard expressed in a programming language as the real reference. This applied not just to those making software implementations, but often to those making more traditional hardware-based products and VLSI designs as well.
Therefore it was decided that the software version of the standard should have the same normative status as the textual part. This decision has been maintained in all subsequent MPEG standards.
While the previous approach where every participant had their own implementation of the TM did not raise the issue of “who owns the software?”, the new approach did. MPEG resolved that with the following rules labelled as “copyright disclaimer”:
- Whoever makes a proposal that is accepted must provide a software implementation and assign the copyright of the code to ISO
- ISO grants a license of the copyright of the code for products conforming to the standard
- Proponents are not required to release patents that are needed to exercise the code and users should not expect that the copyright release includes a patent licence.
More recently, MPEG has started using a modified version of the Berkeley Software Distribution (BSD), a licence originally used to distribute a Unix-like operating system. This licence, originally called “MXM licence” from the name of MPEG-M part 2 standard “MPEG Extensible Middleware (MXM)” simply says that code may be used as prescribed by the BSD licence with the usual disclaimer that patents are not released. This new licence is particularly interesting for software companies that do not want to have liabilities when using software not developed internally.
MPEG and OSS are close, but not quite so
Let me summarise the main elements of MPEG drivers to develop standards and software that implement them:
- MPEG develops the best standards satisfying identified requirements. Best standards must use technologies resulting from large R&D investments typically made by for-profit entities.
- MPEG uses a competitive and transparent process to acquire (Call for Proposals) and refine (Core Experiment) technologies. Today that process largely uses collaboratively developed software, with methodologies that resemble those of the OSS community
- Typically, an MPEG standard is available in two forms: The standard expressed in natural language (possibly with some pseudo C-code inside to improve clarity) and the standard expressed in computer code.
- Each form of the standard has the same normative value. If discrepancies are found, the group will decide which is the correct form and amend the one not considered correct.
- Reference Software and Conformance Testing are attached to MPEG standards. The former is used to test encoder implementations for conformance and the latter to test decoder implementations for conformance.
- The Reference Software of some MPEG standards is of extremely high quality and can be used in products. However, the natural language and computer code forms of the standard have the same status.
- MPEG standards are typically Option 2, i.e., essential patents may exist but those patents can be used at FRAND terms.
Who is better?
The question is not meaningful if we do not specify the context in which the standard or software is used.
I claim that only a standard that responds to the 7 drivers of the previous section can
- Embed state-of-the-art technology
- Be implemented by a variety of independent entities in different industries
- Allow for incremental evolutions in response to user requirements
- Stimulate the appearance of constantly improving implementations
- Enable precise assignment of responsibilities (e.g. for legal purposes).
I am sure that some will think differently on the subject of the previous section and I will certainly be willing to engage in a discussion.
I believe that Open Source Software is a great movement that has brought a lot to humankind. However, I do not think that it is adequate to create an environment that respond to the 7 drivers above.
Posts in this thread
- MPEG standards, MPEG software and Open Source Software
- The MPEG Metamorphoses
- National interests, international standards and MPEG
- Media, linked media and applications
- Standards and quality
- How to make standards adopted by industry
- MPEG status report (Jan 2020)
- MPEG, 5 years from now
- Finding the driver of future MPEG standards
- The true history of MPEG’s first steps
- Put MPEG on trial
- An action plan for the MPEG Future community
- Which company would dare to do it?
- The birth of an MPEG standard idea
- More MPEG Strengths, Weaknesses, Opportunities and Threats
- The MPEG Future Manifesto
- What is MPEG doing these days?
- MPEG is a big thing. Can it be bigger?
- MPEG: vision, execution,, results and a conclusion
- Who “decides” in MPEG?
- What is the difference between an image and a video frame?
- MPEG and JPEG are grown up
- Standards and collaboration
- The talents, MPEG and the master
- Standards and business models
- On the convergence of Video and 3D Graphics
- Developing standards while preparing the future
- No one is perfect, but some are more accomplished than others
- Einige Gespenster gehen um in der Welt – die Gespenster der Zauberlehrlinge
- Does success breed success?
- Dot the i’s and cross the t’s
- The MPEG frontier
- Tranquil 7+ days of hard work
- Hamlet in Gothenburg: one or two ad hoc groups?
- The Mule, Foundation and MPEG
- Can we improve MPEG standards’ success rate?
- Which future for MPEG?
- Why MPEG is part of ISO/IEC
- The discontinuity of digital technologies
- The impact of MPEG standards
- Still more to say about MPEG standards
- The MPEG work plan (March 2019)
- MPEG and ISO
- Data compression in MPEG
- More video with more features
- Matching technology supply with demand
- What would MPEG be without Systems?
- MPEG: what it did, is doing, will do
- The MPEG drive to immersive visual experiences
- There is more to say about MPEG standards
- Moving intelligence around
- More standards – more successes – more failures
- Thirty years of audio coding and counting
- Is there a logic in MPEG standards?
- Forty years of video coding and counting
- The MPEG ecosystem
- Why is MPEG successful?
- MPEG can also be green
- The life of an MPEG standard
- Genome is digital, and can be compressed
- Compression standards and quality go hand in hand
- Digging deeper in the MPEG work
- MPEG communicates
- How does MPEG actually work?
- Life inside MPEG
- Data Compression Technologies – A FAQ
- It worked twice and will work again
- Compression standards for the data industries
- 30 years of MPEG, and counting?
- The MPEG machine is ready to start (again)
- IP counting or revenue counting?
- Business model based ISO/IEC standards
- Can MPEG overcome its Video “crisis”?
- A crisis, the causes and a solution
- Compression – the technology for the digital age