AI is today’s new technological buzz word. Some fear AI. Some see a bright future in AI. However, AI is not a new concept. Back in 1956, Robby the Robot was piloting a high-speed land vehicle and brewing bourbon on the Forbidden Planet for Cook, a crew member of the United Planets Cruiser C-57D. Nine years later, the phrase “Danger, Will Robinson!” was frequently announced by Robot Model B-9 to Will and Dr. Smith who were at that time lost in space on the Jupiter 2. Ten years following Robby’s adventures, in 1966, Captain Kirk and Spock traversed the universe at Warp speed in an AI-based starship known as Enterprise. A few years later in 1987, the Enterprise would be piloted by a humanoid robot named Data.

And now, 68 years after Robby brewed his first bottle of Genuine Ancient Rocket Bourbon for Cook (see below), there is a Raspberry Pi 5 equipped with an M.2 HAT+ sitting on my bench. HAT is Raspberry Pi speak for “Hardware Attached on Top.” The M.2 HAT+ that my AI-enhanced Raspberry Pi 5 is wearing is loaded with a Hailo-8L Entry-Level M.2 AI Accelerator Module.

The combination of the M.2 HAT+ and the Hailo-8L Entry-Level M.2 AI Accelerator Module is officially known as the Raspberry Pi AI Kit, which comes preassembled with the Hailo-8L module mounted on a heatsink and plugged into the M.2 HAT+. Everything needed to mechanically and electrically attach the Raspberry Pi AI Kit to a host Raspberry Pi 5 is also included with the AI Kit. The Raspberry Pi 5 and its AI HAT combine to create a potent little AI edge device.

Related:How to Craft a Raspberry Pi 5 BLE Central Device

What can you do with the Raspberry Pi AI Kit?

The Raspberry Pi AI Kit’s Hailo-8L module is automatically detected by the current Raspberry Pi OS. To further reduce complexity, the Raspberry Pi camera applications have been integrated into the Hailo AI framework. The AI-enhanced Raspberry Pi’s suite of camera applications can be used for object detection, semantic and instance segmentation, pose estimation, and facial recognition. The AI Kit can also be used in non-camera applications that involve analyzing still images and videos. In either case, the neural network applications run entirely on the Hailo-8L.

Putting the Raspberry Pi AI Kit in action

A simple terminal command (sudo apt install hailo-all) is all that is needed to install all of the Hailo dependencies on the Raspberry Pi 5. The Hailo dependencies include the Hailo kernel device driver and firmware, HailoRT, Halio TAPPAS, and the rpicam-apps Hailo post-processing software demo stages.

HailoRT is a tool used to load a binary HEF (Hailo Executable Format) file created by the DFC (Dataflow Compiler) onto the Hailo device and provides library services necessary to run the associated inference. The DFC runs on an X86-based machine and can be used to retrain the multiple AI demos and examples provided for the Raspberry Pi 5. The DFC can also be used to compile custom models that can be run on the Hailo AI accelerator. Hailo TAPPAS (Template APPlications And Solutions), as its name implies, is an application demonstration tool.

Related:How to Control the Raspberry Pi 5's GPIO Using an Android Phone

The rpicam-apps suite of camera applications shares a common built-in post-processing framework that allows them to pass images from the Raspberry Pi 5 camera system to various stages. Stages support custom image processing and image-analysis routines. JSON files that contain references to the built-in post-processing stages are used to pass selected stages to the application. The Raspberry Pi AI Kit installation instructions call for cloning the rpicam-apps repository. The assets folder within the cloned rpicam-apps folder contains a number of aforementioned JSON files. 

Case study: Detecting human presence with the Raspberry Pi AI Kit

In the case of the Raspberry Pi 5, to run a camera-based AI inference, we simply open the Raspberry Pi 5 camera system with the rpicam-hello application, specify some options, and pass the desired JSON file to the post-process-file option. I have posted an example Raspberry Pi 5 AI application that uses a Camera Module 3 Wide attached to the Raspberry Pi 5 to detect the presence of a person and transmits an alert to a remote TCP/IP client when a person is sensed. The example Raspberry Pi 5 AI application could be used as the basis of an industrial safety AI app that detects humans in a construction environment. The AI application could be used to identify humans positioned in front of, to the side of, and behind construction vehicles. AI has an advantage in this construction scenario as the AI-driven cameras replace the eyes of multiple human observers, and the AI app can track and report the location of the construction workers in real time. You can download the application from the EDTP Electronics website. The Raspberry Pi 5 AI app consists of two parts. A TCP/IP server runs in a Python virtual machine while the AI application runs in the background. Instructions as to how to setup and run the TCP/IP server and AI background application are included in the download package.

Related:How to Craft a Raspberry Pi 5 BLE Central Device

The AI portion of the application uses rpicam-hello to pass the hailo_yolov8_inference.json file to the post processing network. Options specified in the rpicam-hello command force a line of text beginning with Object: (reference Figure 1) to be generated for each object that is detected. The application reads one line of text at a time. If the line of text includes the word person, the line of text is stored in a variable called current_line. The contents of the current_line variable are copied to a variable called previous_line. After each line is read, the current_line variable is compared to the previous_line variable. If the variable contents do not match, the most recent contents of the current_line variable are updated in a file called person_detected.txt.

Figure 1. Post-processing output generated by the hailo_yolov8_inference. FRED EADY

The Python server application opens the person_detected.txt file, reads the line of text that represents the contents of the current_line variable, and verifies that the word person exists in the line of text. The line of text gleaned by the server application from the person_detected.txt file is also scanned to obtain the probability ranking of the object detected being a person. As you can see in Figure 1, the probability ranking is contained within a set of parentheses. The Python server application assembles a message that informs the connected client that a person has indeed been detected and includes the probability ranking as a percentage. In Figure 2, the Hercules SETUP utility is acting as a TCP/IP client that has made a connection to the Raspberry Pi 5 TCP/IP server application. The TCP/IP server updates the Hercules SETUP utility client every 100 milliseconds.

Figure 2: Messages received from the Raspberry Pi 5 Python-based TCP/IP server. FRED EADY

Future use for the Raspberry Pi AI Kit

The AI application is by no means limited to detecting a person. The camera attached to the Raspberry Pi 5 can also see such things as the chair the person is sitting in, the keyboard, and mouse the person may be using and the mug of water sitting on the desk.

Welcome to the realm of AI by way of the Raspberry Pi 5!