MrRobot: Ubuntu Mobile‑Controlled Rapiro Robot with Voice, Touch & Shake Interaction

MrRobot is an Ubuntu Mobile application that brings the Rapiro robot to life through intuitive voice, touch, and motion controls. Developed in just 28 hours during the Ubuntu Shenzhen Hackathon, the project showcases how mobile devices can interface seamlessly with robotics using open‑source hardware and software.

All source code, Arduino firmware, and 3‑D printable parts are freely available, enabling hobbyists and professionals alike to assemble their own Rapiro and extend the system.

What You’ll Learn

1. Deploying a Qt‑based Ubuntu Mobile app from the SDK to a device.
2. Integrating wit.ai’s voice‑and‑text recognition API to create a hands‑free command interface.
3. Setting up a Raspberry Pi server that listens for WebSocket messages from the mobile app.
4. Designing a user‑friendly UI with touch, shake, and voice triggers.

Bill of Materials (BOM)

1. Ubuntu Mobile phone or tablet.
2. Raspberry Pi with a Wi‑Fi dongle.
3. Rapiro robot kit.
4. Optional music box for audio playback.

Required Software

1. Ubuntu Qt SDK (for app development).
2. Raspbian OS on the Raspberry Pi.
3. Arduino IDE (if you wish to build the robot from scratch).
4. wit.ai API key for voice processing.

Project Overview

The app offers three interactive modalities:

• Touch – Directly tap the on‑screen controls to command the robot.
• Shake – The device’s accelerometer detects motion gestures to trigger robot actions.
• Voice – A dedicated button starts audio recording; once released, the speech is sent to wit.ai for natural‑language interpretation and sent to the robot via WebSocket.

Step 1: Build the Mobile App

Follow the detailed instructions on the project’s GitHub page to install the Ubuntu Qt SDK, configure the project, and deploy the application to your Ubuntu Mobile device.

Step 2: Deploy a WebSocket Server on Raspberry Pi

Connect the Pi to your local Wi‑Fi network using a dongle, then clone the server repository. Start the server, and configure it to launch automatically at boot by adding the command to /etc/rc.local.

Step 3: Assemble the Hardware

Carefully disassemble the Rapiro head, mount the Raspberry Pi onto the dedicated plate, and connect the audio cable and Wi‑Fi dongle. The Pi now serves as the bridge between the mobile app and the robot’s Arduino board.

Get the Code

All code and 3‑D files are available on GitHub: https://github.com/eminors/MrRobot.

Enjoy building and customizing your own voice‑controlled robot experience!