The handwriting recognition demo can identify characters, including letters and numbers. The typical use case for this demo is a smartwatch that allows users to compose specific commands (for ex. chat, call, mail, etc.) or to type text messages quickly. It can also be used as a way to interact with smart appliances.
Approach
The handwriting recognition demo recognizes letters and numbers written on the small touch screen (smartwatch) available on the STM32L562E MCU discovery kit.
- The touch screen is captured as an image to be classified by the neural network (NN)
- Each character or letter is recognized as a composition of specific commands
- The demo runs on the STM32L562E discovery kit with an NN inference time for each character
The model can be re-trained thanks to STM32 model zoo.
Sensor
The demo uses the touch screen as an input for the user.
The model zoo allows users to test the camera module bundle (reference: B-CAMS-OMV).
Data
Dataset:
In STM32 model zoo, the model is trained on a subset version of the EMNIST dataset.
In this experiment, only the ten-digit classes [0;9] and the capital letters of the alphabet [A-Z] were kept from the MatLab version of the EMNIST ByClass dataset.
For the demo, the dataset was enriched with images captured from the touch screen on the ST board.
Data format:
The dataset is made of:
- uppercase letters from A to Z
- digits from 0 to 9
The dataset contains 28 x 28 pixels of grayscale images organized in 36 balanced classes.
Results
Model: ST MNIST
Input size: 28x28x1
Memory footprint:
Float model:
38 Kbytes of flash memory for weights
30 Kbytes of RAM for activations
Quantized model:
10 Kbytes of flash memory for weights
14 Kbytes of RAM for activations
Accuracy:
Float model: 93.48%
Quantized model: 93.39%
Performance on STM32L562E @ 110 MHz
Float model:
Inference time: 83 ms
Frame rate: 12 fps
Quantized model:
Inference time: 29 ms
Frame rate: 34 fps
