Video coding has been one of the first standardisation areas addressed by MPAI (the first was Content-based Enhanced Audio Coding or MPAI-CAE). One MPAI Video Coding area is focused on the use of AI tools to improve the classic block-based hybrid coding framework and offer more efficient video compression solutions. This was motivated by the rough estimate that MPAI did from its survey on published AI-based video coding results leading to a potential improvement of up to 30%.

Figure 1 identifies 10 coding tools that can potentially be improved by AI tools.

Figure 1 – A classic block-based hybrid coding framework

MPAI decided to start from a high-performance “clean-sheet” traditional (i.e., data-processing-based) coding scheme and add AI-enabled improvements to it, rather than starting from a scheme overloaded by IP-laden compression technologies bringing dubious improvements. It selected the MPEG-5 Essential Video Coding (EVC) standard and is modifying it by enhancing/replacing existing video coding tools with AI tools. The EVC Baseline Profile has been selected because it employs 20+ years old technologies and has a compression performance close to that of HEVC.

MPAI calls this project MPAI AI-Enhanced Video Coding (MPAI-EVC). However, MPAI has made the deliberate decision not to initiate a standard project at this time, because it first wants to set up a unified platform and conduct experiments with the goal to ascertain that 25% coding performance improvement can be achieved. Therefore the name of the current activity within the project is “MPAI-EVC Evidence Project”.

Significant results have already been achieved by a growing group of participants that includes both MPAI and non-MPAI members. Note: MPAI has the policy of allowing non-members to participate in its preliminary pre-standardisation activities.

As soon as the Evidence Project will demonstrate that AI tools can improve the MPEG-5 EVC efficiency by at least 25%, MPAI will be in a position to initiate work on the MPAI-EVC standard. The functional requirements have already been developed and only need to be revised. Then, the framework licence will be developed by active principal members and the Call for Technology issued to acquire the technology needed to develop the MPAI-EVC standard..

MPAI-EVC covers the short-to-medium term video coding needs. If you want to know more. visit the MPAI-EVC web page or contact the MPAI Secretariat to participate in MPAI-EVC meetings.

MPAI has a second video coding project motivated by the consensus in the video coding research community that the so-called End-to-End (E2E) video coding schemes can yield significantly higher performance in the longer term.

MPAI is conducting the MPAI End-to-End Video Coding (MPAI-EEV) project in its role as a technical body whose mission is the provision of efficient and usable data coding standards, unconstrained by legacy IP. The notion of Framework Licence is at the basis of this work. The Basic Framework Licence for Collaborative Explorations is the interim MPAI-EEV Framework Licence. This means that contributions submitted to MPAI-EEV (formally, to the Requirements (EEV) group) shall be accompanied by this licence.

MPAI has selected OpenDVC as a starting point and is investigating the addition of novel motion compensation networks.

Figure 2 – The OpenDVC reference model with a motion compensation network

Clearly, MPAI-VVC targets the longoterm term video coding needs. If you want to know more visit the MPAI-EEV web page or contact the MPAI Secretariat to participate in MPAI-EEV meetings.