Established on 30 September 2020, MPAI spent the first 3 months giving itself a structure ensuring the execution of its mission “develop Artificial Intelligence (AI)-based data coding standards”.
Its first full year of operation – 2021 – has been engaging but rewarding:
- 5 Technical Specifications (TS) have been approved and released in the following domains:
- Human-machine communication.
- Audio enhancement.
- AI Framework
- Ecosystem Governance.
- The Company Performance Assessment TS was complemented by 3 additional specifications:
- Reference Software (RS). a conforming implementation of the TS,
- Conformance Testing (CT), to test that an implementation is technically correct and provides an adequate user experience
- Performance Assessment (PA), to assess implementation reliability and trustworthiness.
A goal can be declared as reached only if the next goal is known, and the purpose of this post is to disclose exactly that.
The AI Framework (AIF), depicted in Figure 1, is a cornerstone of the MPAI architecture.
Figure 1 – The AI Framework (AIF) Reference Model and its Components
- The AIF
- Is Operating System-independent.
- Has a local and distributed component-based Zero-Trust architecture.
- Can create AI Workflows (AIW) made of elementary units called AI Modules (AIM).
- Can access validated AIWs and AIMs by interfacing to the MPAI Store.
- Can execute in a range of computing environments: from MCUs to HPCs.
- Can interact with other AIFs operating in proximity.
- Supports Machine Learning functionalities.
- Its AIMs
- Encapsulate components to abstract them from the development environment.
- Call the Controller via standard interfaces.
- Can be AI-based or data processing-based.
- Can be in software or in hardware.
2022 MPAI Goal #1: AI Framework (MPAI-AIF)
MPAI has already developed 3 application oriented Technical Specifications: MPAI-CAE (Enhanced audio), MPAI-CUI (Company Performance Prediction) and MPAI-MMC (Multimodal human-machine conversation). It total there are 10 AIWs and some 20 AIMs (several of them are used in different AIWs).
An active MPAI generates an ecosystem with the following actors:
- MPAI develop standards.
- Implementers develop MPAI standard implementations
- Users access such implementations.
MPAI is all about facilitating a market of AI applications. Releasing standards enables a market but does not ensure that the market is functional. How can a user be sure that an implementation is secure, technically correct, unbiased? Note that by “user” we do not necessarily mean an end user, but also an app developer (i.e., AIW) who may need an AIM and does not have the resources or the competence to answer the 3 questions.
In its Governance of the MPAI Ecosystem TS, MPAI has envisaged two more players:
- Performance Assessors who assess that implementations are reliable and trustworthy.
- The MPAI Store where uploaded implementations are:
- Checked for security
- Tested for conformance
- Posted to the Store with a clear indication of level of performance.
Note that MPAI appoints Performance Assessors, and establishes and controls the MPAI Store, a not-for-profit commercial entity.
Figure 2 depicts the operation of the MPAI Ecosystem.
Figure 2 – The MPAI Ecosystem and its Governance
2022 MPAI Goal #2: Governance of the MPAI Ecosystem (MPAI-GME)
In 2020 MPAI has developed 3 application oriented TSs:
Compression and Understanding of Industrial Data (MPAI-CUI) with 1 use case.
Multimodal Conversation (MPAI-MMC) with 5 use cases.
Context-based Audio Enhancement (MPAI-CAE) with 4 use cases.
Figure 3 depicts the reference model of the Company Performance Prediction Use Case.
|AI-based Company Performance Prediction measures the performance of a Company by providing Default Probability, Organisational Model Index, and Business Discontinuity Probability of the Company within a given Prediction Horizon using the Company’s Governance, Financial and Risk data|
|Figure 3 – The Company Performance Prediction CUI-CPP) Reference Model|
MPAI-CUI includes the Reference Software (RS), Conformance Testing (CT) and Performance Assessment (PA) Specifications of the AI-based Company Performance Prediction (CPP).
2022 MPAI Goal #3: Compression and Understanding of Industrial Data (MPAI-CUI)
Multi-modal conversation (MPAI-MMC) uses AI to enable human-machine conversation emulating human-human conversation in completeness and intensity. It includes 5 Use Cases: Conversation with Emotion, Multimodal Question Answering, Unidirectional Speech Translation, Bidirectional Speech Translation and One-to-Many Unidirectional Speech Translation.
The figures below show the reference models of the MPAI-MMC Use Cases.
Currently, only the MPAI-MMC TS is available. Thereforethe
|2022 MPAI Goal #4 for Multimodal Conversation (MPAI-MMC)|
The figures below shows the reference models of the MPAI-CAE Use Cases. Note that an Implementation is supposed to run in the MPAI-specified AI Framework (MPAI-AIF).
|Emotion-Enhanced Speech (EES) enables a user to indicate a model utterance or an Emotion to obtain an emotionally charged version of a given utterance.|
In many use cases, emotional force can usefully be added to speech which by default would be neutral or emotionless,
|Figure 9 – Emotion Enhanced Speech|
|Audio Recording Preservation (ARP) Use Case enables a user to create of digital copies of a digitised audio of open-reel magnetic tapes suitable for long-term preservation and for correct play back of the digitised recording (restored, if necessary).|
|Figure 10 – Audio Recording Preservation|
|Speech Restoration System (SRS) enables a user to restore a Damaged Segment of an Audio Segment containing only speech from a single speaker. No filtering or signal processing is involved. Instead, replacements for the damaged vocal elements are synthesised using a speech model.|
|Figure 11 – Speech Restoration System|
|Enhanced Audioconference Experience (EAE) enables a user to improve the auditory quality of audioconference experience by processing speech signals recorded by microphone arrays and provide speech signals free from background noise and acoustics-related artefacts .|
|Figure 12 – Enhanced Audioconference Experience|
Currently, only the MPAI-CAE TS is available. Therefore
|MPAI Goal #5 in 2022 is further development of MPAI-CAE|
MPAI has 7 projects at different levels of development. For each of these a Goal is assigned.
|2022 MPAI Goal #6 in 2022 is development of MPAI-SPG|
|2022 MPAI Goal #7 for Connected Automotive Vehicles (MPAI-CAV)|
|2022 MPAI Goal #8 for Mixed-reality Collaborative Spaces (MPAI-MCS)|
|2022 MPAI Goal #9 for Integrative Genomic/Sensor Analysis (MPAI-GSA)|
|2022 MPAI Goal #10 for AI-Enhanced Video Coding (MPAI-EVC)|
|2022 MPAI Goal #11 for AI-based End-to-End Video Coding (MPAI-EEV)|
|2022 MPAI Goal #12 for Visual Object and Scene Description (MPAI-OSD)|