The NXP eIQ ML (edge intelligence machine learning) software environment provides tools to perform inference on embedded systems using neural network models. The software includes optimizations that leverage the hardware capabilities of the i.MX95 family for improved performance. Examples of applications that typically use neural network inference include object/pattern recognition, gesture control, voice processing, and sound monitoring.

eIQ includes support for the following inference engines:

All of them support CPU-based inference, while NPU acceleration is only available for some engines on supported platforms. See NXP’s i.MX Machine Learning User’s Guide for details.

Include eIQ packages in Digi Embedded Yocto

Edit your conf/local.conf file to include the eIQ package group in your Digi Embedded Yocto image:

conf/local.conf
IMAGE_INSTALL:append = " packagegroup-imx-ml"

This package group contains all of NXP’s eIQ packages compatible with the ConnectCore 95.

Including this package group increases the size of the rootfs image significantly. To minimize the increase in image size, select a subset of its packages depending on your needs. See the package group’s recipe for more information on the packages it contains.

TensorFlow

TensorFlow support

TensorFlow Lite is a set of tools that enables on-device machine learning by helping developers run their models on mobile, embedded, and edge devices. TensorFlow Lite supports computation on the following hardware units:

  • CPU Arm Cortex-A cores

  • NPU hardware acceleration using Neutron Delegate.

Neutron Delegate is an external delegate on i.MX95 Linux platforms. It enables the inference to be accelerated via the on-chip hardware accelerator. Neutron Delegate directly uses the hardware accelerator driver to fully utilize the capabilities of the accelerator.

TensorFlow example with CPU

The following example shows how to use TensorFlow Lite by performing the inference on the CPU.

# cd /usr/bin/tensorflow-lite-2.19.0/examples
# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt
INFO: Loaded model mobilenet_v1_1.0_224_quant.tflite
INFO: resolved reporter
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: invoked
INFO: average time: 16.583 ms
INFO: 0.764706: 653 military uniform
INFO: 0.121569: 907 Windsor tie
INFO: 0.0156863: 458 bow tie
INFO: 0.0117647: 466 bulletproof vest
INFO: 0.00784314: 835 suit

The output displays the time taken to process the sample, with an average time of 16.583 ms.

TensorFlow example with NPU

First convert the model for the Neutron NPU using NXP’s eIQ Toolkit:

$ neutron-converter --input mobilenet_v1_1.0_224_quant.tflite --target imx95 \
    --output mobilenet_v1_1.0_224_quant_neutron.tflite

Copy the generated mobilenet_v1_1.0_224_quant_neutron.tflite model back to the target.

Then, run the example by specifying the converted inference model (-m option) and the NPU library (external_delegate_path option):

# cd /usr/bin/tensorflow-lite-2.19.0/examples
# ./label_image -m mobilenet_v1_1.0_224_quant_neutron.tflite -i grace_hopper.bmp \
    -l labels.txt --external_delegate_path=/usr/lib/libneutron_delegate.so
INFO: Loaded model mobilenet_v1_1.0_224_quant_neutron.tflite
INFO: resolved reporter
INFO: EXTERNAL delegate created.
INFO: NeutronDelegate delegate: 1 nodes delegated out of 3 nodes with 1 partitions.
INFO: Neutron delegate version: v1.0.0-f24d08e5, zerocp enabled.
INFO: Applied EXTERNAL delegate.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: invoked
INFO: average time: 1.328 ms
INFO: 0.756863: 653 military uniform
INFO: 0.145098: 907 Windsor tie
INFO: 0.0156863: 458 bow tie
INFO: 0.0117647: 466 bulletproof vest
INFO: 0.00784314: 835 suit

The output displays the time taken to process the sample, with an average time of 1.328 ms.

More information

See NXP’s i.MX Machine Learning User’s Guide for more information on eIQ.