Augmented Interactive Mirror

An augmented Interactive Mirror based on AI technology to develop a multi-modal second self interface.

The use of digital technology within homes is increasing, while the latest technological and digital breakthroughs allow significant advancement in this field. This progress creates many opportunities by designing an augmented reality interface for a mirror. We will see how to integrate this interface to an augmented mirror fully. Its use has to be natural and serve as a basis for developing platforms such as sports training, sign language, or medical monitoring of drug intake.

Context

The Augmented Reality Mirror project challenges students to design and build an interactive mirror system that overlays digital content onto a real-world reflection. Such mirrors combine computer vision, real-time tracking, and augmented reality (AR) rendering to create immersive, interactive experiences.

One compelling application is learning the sign language, where the mirror can detect the user’s hand and body movements and provide visual feedback, corrections, or instructions in real time. This allows learners to practice gestures, receive guidance, and track progress without needing a human instructor, making learning both playful and effective.

The project emphasizes hands-on design, real-time interactivity, and user experience, challenging students to integrate physical display systems with software, sensors, and AR interfaces. It provides an opportunity to explore human-computer interaction, gesture recognition, and immersive feedback in a tangible and educational context.

Playful learning is central to this project: immediate feedback, self-paced experimentation, and interactive guidance encourage engagement, motivation, and skill development. Students will gain experience in creating systems that respond dynamically to user actions while providing meaningful and instructive visual feedback.

General Overview

The Augmented Reality Mirror system consists of:

A mirror or reflective display surface
A camera or depth sensor to track user movements and gestures
An embedded computing unit to process input and generate AR overlays
Visual feedback that is superimposed on the user’s reflection
Optional audio feedback to reinforce instructions or corrections

Applications include:

Language learning, e.g., practicing sign language gestures
Interactive fitness or rehabilitation exercises
Artistic or creative installations where users interact with virtual elements
Educational tutorials with guided motion feedback

The system should allow real-time interaction, tracking multiple gestures or users, and provide engaging, clear feedback.

Technical Specification Schematics

Students are expected to design and implement a complete AR mirror system, including:

Structure & Display Layer

Mirror or reflective panel suitable for overlaying digital content
Display or projection system integrated with the mirror surface

Sensing & Interaction Layer

Depth cameras, RGB cameras, or motion sensors to detect hand/body gestures
Calibration tools to map gestures accurately onto the AR overlay

Processing & Feedback Layer

Embedded computing platform capable of real-time image processing and AR rendering
Gesture recognition algorithms for detecting and interpreting user actions
Visual overlays for guidance, corrections, and feedback
Optional audio cues synchronized with gestures

Constraints & Requirements

Low-latency processing for smooth interaction
Accurate gesture recognition for effective learning
Robust operation under different lighting conditions
Safe and ergonomic setup for users

Students should produce schematics and diagrams showing the mirror structure, sensor placement, data processing pipeline, and feedback system.

Developed Skills

Real-time computer vision and gesture recognition
AR rendering and overlay design
Embedded system programming and optimization
Integration of sensors and displays
Testing, calibration, and system tuning

Provided Materials

Mirror or reflective panels
Cameras or depth sensors (e.g., Kinect, Intel RealSense)
Embedded computing platforms (e.g., Raspberry Pi, Jetson, or PC)
Display system or projector
Software tools for gesture recognition and AR rendering

Students are expected to adapt their designs to the available materials and justify any additional components.

Pedagogical Goals

Develop students’ skills in interactive system design and AR technology
Explore gesture-based interfaces for learning and engagement
Encourage playful, hands-on experimentation with immersive technology
Build competence in real-time processing, user feedback, and system integration
Strengthen teamwork, autonomy, and communication skills