Minority report: from fiction to (almost) reality!
Gesture-based device control can bring many benefits, providing either a better user experience or supporting touchless applications for sanitary reasons. For demonstration purposes, we have created 3 classes to distinguish several hand poses, but the model can be trained with any gestures providing a wide range of new features to the final user.
NanoEdge AI Studio supports the Time-of-Flight sensor, but this application can be addressed with other sensors, such as radar and more.
Approach
– We used a Time-of-Flight sensor rather than a camera for smaller signals, simpler information.
– We set the detection distance to 20 cm to reduce the influence of the background. Optional: binarizing the distance measured.
– We took 10 measures (frequency: 15Hz) and for each measure, we predicted a class. We then chose the class that appeared the most often.
– (Concatenating measures to create a longer signal is performed to study the evolution of a movement. Here, our goal was to classify a sign. No temporality is needed).
– We created a dataset with 3,000 records per class (rock, paper, scissors), avoiding empty measurement (no motion).
– Finally, we created an ‘N-Class classification’ model (3 classes) in NanoEdge AI Studio and tested it live on a NUCLEO_F401RE
Sensor
Time-of-Flight sensor: VL53L5
Data
3 classes of data: Rock, paper, scissors
Signal length: 64, successive 8×8 matrixes
Data rate: 15 Hz
Results
3 classes classification:
99.37% accuracy, 0.6 KB RAM, 192.2 KB Flash
