Autonomous Donkey Car: TensorFlow MobileNetV2 AI for Garbage Collection

Build a self‑driving RC car that detects and collects trash using TensorFlow’s SSD MobileNetV2 on a Raspberry Pi 3 and Pi Camera.

Project Overview

This guide demonstrates how to transform a standard RC car (Raspberry Pi + Pi Camera) into an autonomous garbage‑collector. The solution relies on two lightweight neural networks: one for real‑time object detection and another for steering and throttle control. Because the Pi 3’s limited RAM and CPU can run only one model at a time, we alternate between detection and driving in a time‑sliced fashion.

The core idea is to train a model to recognize trash bins and then drive the car toward them, picking up waste autonomously.

Identifying the car and laptop components

The project has two main phases:

1. Object Detection – A modest convolutional network processes the Pi Camera feed using TensorFlow and OpenCV.
2. Behavioral Cloning – The car learns to navigate autonomously by mimicking human steering and throttle inputs. Additional sensors (ultrasonic, GPS, 6‑DOF IMU) enrich the training data and telemetry.

Introduction

MobileNetV2, released in 2019, is engineered for embedded devices. The Raspberry Pi 3’s CPU can comfortably run MobileNetV2 SSD, unlike heavier YOLO variants that demand excessive RAM.

Attempting to run VGG16 (≈350 MB weights) failed on the Pi, underscoring the need for slimmer models. Transfer learning with a pre‑trained MobileNetV2 SSD provides high accuracy with a minimal footprint.

Hardware Modifications

The base vehicle is a Magnet RC car, which we retrofit into a DonkeyCar chassis. Key changes include:

• Disassembly of the top cover to access the receiver, ESC, and servo.
• Installation of two 3D‑printed adapters to replace the original clips.
• Mounting the Raspberry Pi and a dedicated servo controller on a wooden plate.
• Connecting the servo controller to the car’s steering (channel 0) and throttle (channel 1) via I²C.
• Adding a 6‑DOF MPU6050 IMU, a GPS module, and an HC‑SR04 ultrasonic sensor (requires a 5 V level shifter).

The finished vehicle now includes:

• Monocular wide‑angle Pi Camera
• Servo controller for steering
• 6‑DOF IMU for acceleration and gyroscopic data
• GPS for outdoor positioning
• Ultrasonic distance sensor for obstacle avoidance

All sensor data are timestamped and logged via the tub framework to augment training datasets.

Software Setup

The software stack is minimal yet powerful: TensorFlow, OpenCV, and a few Python utilities. Below is a step‑by‑step installation guide.

1. Install TensorFlow, OpenCV, and Dependencies

pip install tensorflow[pi]
pip install matplotlib raspberry
sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev
sudo apt-get install qt4-dev-tools
pip3 install opencv-python

TensorFlow requires its own model format; we therefore use the official TensorFlow models repository and set the PYTHONPATH accordingly:

git clone --recurse-submodules https://github.com/tensorflow/models.git
export PYTHONPATH=$PYTHONPATH:/home/pi/tensorflow1/models/research:/home/pi/tensorflow1/models/research/slim

2. Install Protocol Buffers Compiler

ProtoBuf is needed to load the MobileNetV2 SSD weights.

sudo apt-get install autoconf automake libtool curl
wget https://github.com/google/protobuf/releases/download/v3.5.1/protobuf-all-3.5.1.tar.gz
tar -zxvf protobuf-all-3.5.1.tar.gz
cd protobuf-3.5.1
./configure
make
make check
sudo make install
cd python_export
python3 setup.py build --cpp_implementation
python3 setup.py test --cpp_implementations
sudo python3 setup.py install --cpp_implementation
export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp
export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION_VERSION=3
sudo ldconfig
protoc --version

3. Integrate IMU and GPS

Add the MPU6050 to the meta.json and manage.py scripts to capture acceleration and gyroscope data:

{"inputs": ["cam/image_array", "user/angle", "user/throttle", "user/mode", "imu/acl_x", "imu/acl_y", "imu/acl_z", "imu/gyr_x", "imu/gyr_y", "imu/gyr_z"], "types": ["image_array", "float", "float", "str", "float", "float", "float", "float", "float", "float"]}

Install the required Python libraries:

sudo apt-get install python3-smbus
pip install mpu6050-raspberrypi
pip install RPi.GPIO

4. Final Steps

After installing all dependencies, deactivate any virtual environment and reboot the Pi:

deactivate
sudo shutdown -h now

The car is now ready to run the SSD MobileNetV2 for object detection and the behavioral cloning model for autonomous driving.

Conclusion

By combining lightweight neural networks with robust sensor integration, a Raspberry Pi 3 can reliably detect and collect garbage bins, paving the way for scalable, low‑cost autonomous waste collection solutions.