AI Navigator: How to Implement AI model (RA8)

Overview

This page briefly explains how to implement AI models converted using RUHMI Framework AI Compiler for MCU (hereafter, ruhmi) through Conversion Tool and how to run inference in your AI application.
To implement the converted AI model, follow the steps below:

  1. Copy RUHMI output files into the src directory
  2. Implement the AI Model and Inference
  3. Build & Run
Note This guide is based on the following references and summarizes the necessary steps for implementation on AI Navigator.
Please refer to them for details for each software.

1. Copy RUHMI output files into the src directory

RUHMI outputs the following files including the converted AI models.

Ethos
CPU
RUHMI output files are as below.
Please copy these files into your directory, excluding hal_entry.c, model_io_data.h, and model_io_data.c, as they are provided only for reference or validation purposes.

hal_entry.c is an example code, and runs the inference and compares the result with the dummy in model_io_data and actual output.
*If you want to run this hal_entry.c as is, you need to prepare the files for RTT Viewer. Go to this page to download the SEGGER_RTT folder.


.
├── ethosu_common.h
├── hal_entry.c     # HAL entry example
├── model.c     # AI Model file
├── model.h
├── model_io_data.c # dummy AI Model I/O data file
├── model_io_data.h
├── sub_0000_command_stream.c   # Ethos-U55 subgraph generated C source code 
├── sub_0000_command_stream.h
├── sub_0000_invoke.c
├── sub_0000_invoke.h
├── sub_0000_io_data.c
├── sub_0000_io_data.h
├── sub_0000_model_data.c
├── sub_0000_model_data.h
├── sub_0000_tensors.c
├── sub_0000_tensors.h
└── ...

RUHMI output files are as below.
Please copy these files into your directory, excluding model_io_data.h and model_io_data.c, as they are provided only for reference or validation purposes. 


.
├── compute_sub_0000.c  # CPU subgraph generated C source code, including inference process.
├── compute_sub_0000.h
├── ...
├── kernel_library_int.c    # kernel library if CPU subgraphs are present 
├── kernel_library_int.h
├── kernel_library_utils.c
├── kernel_library_utils.h
├── model_io_data.c  # dummy AI Model I/O data file (*)
└── model_io_data.h

(*) Some models may not generate model_io_data.h and model_io_data.c during conversion.


2. Implement the AI Model and Inference

The following explains how to implement AI models and the inference process based on an AI application generated by RUHMI.

Click the button that matches your use case.

Ethos
CPU
Follow the steps below to implement AI inference using Ethos-U.
  1. Initialize Memory
  2. Initialize Ethos-U
  3. Add AI inference
  4. Stop Ethos-U
Reference： Runtime API - CPU + Ethos-U deployment

i. Initialize Memory

Initialize the memory used to store data such as the model, if required by your system configuration.
To initialize and configure SDRAM, follow the steps below:
  1. Open configuration.xml in your project.
  2. Select the BSP tab and open Properties.
  3. In Properties, go to RA8P1 Family > SDRAM > SDRAM Support, and set the value to Enable to generate the project contents.

ii. Initialize Ethos-U (NPU)

Initialize and start Ethos-U using RM_ETHOSU_Open() as shown below. 
/* Ethos-U Initialization */
fsp_err_t status = FSP_SUCCESS;
status = RM_ETHOSU_Open(&g_rm_ethosu0_ctrl, &g_rm_ethosu0_cfg);
if (status != FSP_SUCCESS) {
    /* Error Handling */
}

Tips If RM_ETHOSU_Open() fails, check the heap size setting.
Click "FSP Configuration" and select "BSP" tab. Then, in RA Common, enter an appropriate value in the "Heap size (bytes)" field.

iii. Add AI inference

You need to prepare the input data in the model input buffer first, and then run the inference.
The inference process is executed by RunModel(), which is defined in model.h.
#include "model.h"

    ...
/* Set the input model to the pointer */
memcpy(GetModelInputPtr_input_0(), model_input0, model_input_SIZE0);  
/* Run inference */
RunModel();
  • GetModelInputPtr_input_0: Model input pointer
  • model_input0: Model input buffer
  • model_input_SIZE0: Model input size

iv. Stop Ethos-U

After completing the entire inference process, stop Ethos-U with RM_ETHOSU_Close(). 
/* Stop Ethos */
status = RM_ETHOSU_Close(&g_rm_ethosu0_ctrl);
if (status != FSP_SUCCESS) {
    /* Error Handling */
}
Follow the steps below to implement AI inference using CPU only.
  1. Allocate the I/O buffers
  2. Allocate the intermediate buffer
  3. Run inference
Reference: Runtime API - CPU only deployment

i. Allocate the I/O buffers

Open compute_sub_0000.h and check the size of the input and output buffers.
/* File: compute_sub_0000.h */
void compute_sub_0000(
    /* buffer for intermediate results */
    uint8_t* main_storage, /* should provide at least <intermediate_buffers_size> bytes of storage */

    /* inputs */
    const int8_t <input_name>[XXX],
    /* outputs */
     int8_t <output_name>[YYY]
   );
Define the input and output buffers of the same size in your AI application code. 
/* Input buffer */
static int8_t input_buffer[XXX];
/* Output buffer */
static int8_t output_buffer[YYY];

ii. Allocate the intermediate buffer

Reopen compute_sub_0000.h and check the size of the intermediate buffer.
/* File: compute_sub_0000.h */
enum BufferSize_sub_0000 {
  kBufferSize_sub_0000 = <intermediate_buffers_size>
};
Define the intermediate buffer of the same size in your AI application code. 
#include "compute_sub_0000.h"

...
/* Intermediate buffer */
static uint8_t compute_buffer[kBufferSize_sub_0000];

iii. Run inference

Run the inference with compute_sub_0000(). 
#include "compute_sub_0000.h"


/* Input buffer */
static int8_t input_buffer[XXX];
/* Output buffer */
static int8_t output_buffer[YYY];
/* Intermediate buffer */
static uint8_t compute_buffer[kBufferSize_sub_0000];

/* Run inference */
compute_sub_0000(compute_buffer, input_buffer, output_buffer);


3. Build & Run

Click “Build” in the AI Application view of AI Navigator to build your project.
Then, click “Run on the Board” in the AI Navigator menu and click “Run the AI App”.

Wrapping up

That’s all for the explanation of how to implement the converted AI models into your project.
For more information, please refer to RUHMI GitHub.