Aya: Award‑Winning Interactive Robot Personal Assistant Powered by AWS

Aya is an award‑winning interactive robot that uses AWS Rekognition to recognize faces and identify objects, delivering natural voice responses via Amazon Polly.

Story

Inspiration

Seeing Amazon’s Alexa and Google Home sparked our vision of a truly human‑like personal assistant. Aya is our proof‑of‑concept robot that blends human appearance with advanced AI.

What It Does

Aya stores a user’s photo to recognize them and can identify the object in front of her, communicating the result through spoken language.

How We Built It

Building Aya required a complex embedded system: servo controls ran on separate threads from audio codecs to enable simultaneous speech and jaw movement. We 3‑d printed the chassis—an open‑source InMoov design—over 26 hours during Hack the North 2017. The system relies on AWS Rekognition for face comparison, Amazon Polly for speech synthesis, and a parallel thread for real‑time servo actuation.

Challenges

Hardware obstacles ranged from melting wires and burnt breadboards to servo overheating and tolerance issues. Synchronizing servo control with speech demanded careful multithreading. On the software side, choosing the most relevant label from Rekognition’s confidence array proved tricky; we explored k‑means clustering but time constraints limited full implementation. Completing all of this within a 36‑hour hackathon, with minimal rest, pushed us to stay agile and focused.

Accomplishments We’re Proud Of

We successfully integrated the servo system with the software stack and maintained excellent team collaboration throughout the event, resulting in a polished final product.

What’s Next for Aya

Future work includes applying TensorFlow to cluster Rekognition outputs and using pyAudioAnalysis to synchronize audio waveforms with servo motion more precisely.