MPAI is about AI-enabled data coding across a very wide range of industries. Some industries are unable to negate dearly held traditions and make firm decisions, like MPEG standards did by specifying technology up to the last bit. MPAI must take this reluctance as a requirement for some of its standards.

How can we then make standards that are technology agnostic?

A standard should respond to a need and digging inside that need is a prerequisite for a meaningful standard. Most standards can be technology-neutral as a first step. Then there may be another phase where precise technologies are specified.

Let’s take the case of the XR Venue (MPAI-XRV) project where word Venue is used as a synonym for either Real or Virtual Environments. MPAI-XRV addresses a multiplicity of use cases enabled by Extended Reality (XR) – the combination of Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) technologies – enhanced by Artificial Intelligence (AI) technologies.

Live Theatrical Performance is one of the MPAI-XRV projects with a reference model depicted in Figure 1.

Figure 1 – Live Theatrical Performance (XRV-LTP)

XRV-LTP can acquire data from the Real Environment (real world) and from parallel Virtual Environments (metaverses). The data it can acquire includes Audio, Visual, MoCap, LiDAR, Biometrics, and more. Should it specify single formats for Audio, Volumetric, MoCap, and LiDAR? There is no way industry would follow that.

Another example is the MPAI-MMM (metaverse) project where the data types involved are manifold with new data types expected to come to the fore in the next year. Likewise, for the MPAI-AIH (health) project where the data types are many and with many formats each.

MPAI has devised the Data Types, Formats and Attributes (MPAI-TFA) standard, a growing collection of data types for each of which the Sub-Types, Formats and Attributes are specified. For instance, using MPAI-TFA, it is possible to indicate that an instance of the Visual Data Type:

• Sub-Type

• Uses the Rec. ITU-R BT.2020 colour space
• Has an alpha channel of 0.75
• Has a brightness of 1000 Nits
• Was 4:2:2 subsampled
• Is represented with 10 bits/pixel
• Format
  • Is time sampled at 50 Hz
  • Is space sampled at 100 dpi
  • Is a static 3D visual object with a bounding box
  • Has the OBJ format

• Attribute
  • Has real attributes (captured from the real world)
  • Uses Dublin Core metadata
  • Contains objects with IDs
  • There is a human expressing happiness
  • The visual object was captured by a device with an ID
  • The device had a Position and Orientation in the capture space

The example above illustrates capabilities of the MPAI-TFA standard and is not an exhaustive list.

MPAI designates Qualifier as the collection of these data elements and it defines Data Object as a Data Type with its Qualifier.

Most MPAI standards now reference the MPAI-TFA standard. The table below lists the data types for which Qualifiers have been (normal font) or are in the process of being (italic font) specified. The specification is available online.