Build a Raspberry Pi Digit Recognizer with TensorFlow & OpenCV

Discover how to train a deep convolutional neural network to transcribe handwritten digits and deploy it on a Raspberry Pi using the Pi Camera, OpenCV, and TensorFlow.

Overview

In this project we’ll walk through the entire AI pipeline: from data preparation and model training on a laptop, to real‑time digit recognition on a Pi camera. The solution relies on Scikit‑Learn and OpenCV 3.3 for image manipulation, and Keras (with TensorFlow backend) for the deep learning component.

We use the MNIST dataset – 60,000 training and 10,000 test images of 28×28‑pixel grayscale digits. Images captured by the Pi camera are pre‑processed to match this format before being fed into the trained network.

The network architecture is a standard convolutional stack followed by a fully‑connected output layer with ten neurons, each representing one digit (0–9).

Once the model is trained, the weights are transferred to the Raspberry Pi. A lightweight Python script captures a frame, applies grayscale conversion, thresholding, resizing, inversion, and then performs inference.

Hardware Setup

For the demonstration I mounted the Pi Camera on a makeshift holder – a simple, hand‑held rig that kept the camera focused on a single digit. In practice, a fixed mount or a small enclosure can improve stability.

Software Installation

Create a Python virtual environment on the Pi and install the required libraries. The commands below assume you have a working Raspberry Pi OS and Python 3.7+.

source ~/.profile
workon cv
python PiCameraApp.py --picamera 1

Install TensorFlow

pip install tensorflow

Install Keras

pip install keras

Install OpenCV 3.3

OpenCV must be compiled from source to enable full optimizations on the Pi. A step‑by‑step guide is available on PyImageSearch.

https://www.pyimagesearch.com/2017/09/04/raspbian-stretch-install-opencv-3-python-on-your-raspberry-pi/

Finally, install the PiCamera library with NumPy optimizations:

pip install "picamera[array]"

Model Training

Run the training script on a laptop (preferably with a GPU). The script defines the CNN in Keras, compiles it, trains it on the MNIST data, validates performance, and then saves the weights to model_weights.h5.

python Train_MNIST.py

If you have an NVIDIA GPU, install the GPU‑enabled TensorFlow and CUDA drivers to accelerate training. Without a GPU the process will still complete, though it will take longer.

After training, transfer model_weights.h5 to the Pi via scp or WinSCP.

Real‑Time Digit Recognition

Launch the recognition script on the Pi. The program captures a frame when you press t and terminates with q. Accuracy depends largely on lighting, angle, and handwriting clarity. Typical results achieve >95% accuracy on clean, high‑contrast digits.

Image Pre‑Processing Pipeline

• Capture RGB frame as a NumPy array.
• Convert to grayscale (drop two color channels).
• Scale pixel values to 8‑bit unsigned integers (0‑255).
• Apply Otsu’s thresholding to obtain a clean black‑and‑white image.
• Resize to 28×28 pixels – the MNIST input shape.
• Invert colors so digits are white on black, matching the training data.
• Flatten the image and feed it into the loaded model.

The model outputs a probability vector of length ten. The index with the highest probability is the predicted digit.

Key Takeaways

This example demonstrates a complete end‑to‑end workflow: data preparation, model training, deployment, and inference on a single board computer. The code is available on GitHub for anyone who wants to experiment or extend the system.

References

Source: AI Digit Recognition with PiCamera