diff --git a/_docs/tutorials/demos/edgeimpulse/ei_logo.png b/_docs/tutorials/demos/edgeimpulse/ei_logo.png new file mode 100644 index 000000000..35667c0f6 Binary files /dev/null and b/_docs/tutorials/demos/edgeimpulse/ei_logo.png differ diff --git a/_docs/tutorials/demos/edgeimpulse/index.md b/_docs/tutorials/demos/edgeimpulse/index.md new file mode 100644 index 000000000..bbbdc9355 --- /dev/null +++ b/_docs/tutorials/demos/edgeimpulse/index.md @@ -0,0 +1,41 @@ +--- +title: Edge Impulse Demo +layout: docs_noheader +permalink: /docs/tutorials/demos/edgeimpulse/ +redirect_from: + - /edgeimpulse/ +--- + + + +![Edge Impulse](ei_logo.png) + +--- + +This example demonstrates how to run an image classification neural network (built using [Edge Impulse](https://www.edgeimpulse.com/)) on the [Arduino Portenta H7](https://store.arduino.cc/products/portenta-h7) and publish the classification results using micro-ROS. The tutorial also shows how to add custom message types to micro-ROS and ROS 2. The motivation behind this demo is to offer a way to add AI functionality to ROS 2 robots without bogging down the main computational unit with additional neural nets. This is especially helpful when the offloading of computational tasks to a companion laptop (for instance) is not an option. + +With this approach the MCU publishes inference results using a custom message type `EIResult` (Edge Impulse result), which is made up of multiple `EIClassification` items. The tutorial associated with this demo explains how to go about adding these custom message types. A single classification looks like this: + +``` +'dog': 0.75 +``` + +It contains a single label (class) and its probability. A full result looks like this: + +``` +'dog': 0.75, +'cat': 0.25 +``` + +It contains all labels and their probabilities (together summing to 1). The size of the result messages depends on how many labels (classes) your image model is trained on (the example above has two). The code in this example will automatically allocate the necessary memory to support an arbitrary number of labels depending on your model. + + + +By running neural networks on MCUs and publishing their inferences using micro-ROS, it becomes straightforward to experiment with a "distributed" approach to AI in robotics, where the central computer is concerned only with the results of the neural networks and not with sensor data ingest nor calculation. + +![Traditional vs distributed approach](traditional_vs_distributed.png) + +--- + + +To run this demo yourself, check out the video walk through and instructions given [here](https://github.com/avielbr/micro_ros_ei). diff --git a/_docs/tutorials/demos/edgeimpulse/traditional_vs_distributed.png b/_docs/tutorials/demos/edgeimpulse/traditional_vs_distributed.png new file mode 100644 index 000000000..4d1bcba86 Binary files /dev/null and b/_docs/tutorials/demos/edgeimpulse/traditional_vs_distributed.png differ diff --git a/_docs/tutorials/demos/overview/index.md b/_docs/tutorials/demos/overview/index.md index e5a475769..8dd2f9ff5 100644 --- a/_docs/tutorials/demos/overview/index.md +++ b/_docs/tutorials/demos/overview/index.md @@ -21,6 +21,7 @@ Choose yourself: | [**DemoBox Power and Boot Time**](../demobox_demo/) | | NuttX | Olimex LTD STM32-E407, Raspberry Pi 4 | | [**Thumper, a six-wheeled robot**](../thumper_demo/) | | NuttX | Wild Thumper 6WD, Olimex LTD STM32-E407 | | [**MoveIt 2 Demo**](../moveit2_demo/) | | Zephyr | STM32L4 Discovery kit IoT | +| [**Edge Impulse AI demo**](../edgeimpulse/) | | Arduino | Arduino Portenta H7, Vision shield | | [**Combined Demos**](../combined_demos/) | -- | -- |
*As a matter of course, in addition to the hardware detailed above, you'll need a computer/laptop, various cables, and further auxiliary equipment.
@@ -28,3 +29,4 @@ Choose yourself: **WIP:** The demos may also interface with the FIWARE Context Broker, which is the core of the [FIWARE](https://www.fiware.org/) open source initiative for context data management. Learn more at [**Interfacing with FIWARE Context Broker**](../fiware_demo/). **WIP:** The demos can be combined in several ways to demonstrate further functionalities. Go to [**Combined Demos**](../combined_demos/) to learn more about these combinations and how to launch them. + diff --git a/_docs/tutorials/demos/overview/portenta.png b/_docs/tutorials/demos/overview/portenta.png new file mode 100644 index 000000000..e1589369c Binary files /dev/null and b/_docs/tutorials/demos/overview/portenta.png differ