Raju
Chess-playing robot arm that combines computer vision, machine learning, and precision robotics to enable a mechanical arm to play chess intelligently.
Skills
Robotics, Computer Vision, Machine Learning, Chess Algorithms, Python, Motion Control, Reinforcement Learning, Hackathon Development
The Raju project was developed during the Betaworks Hackathon in New York, the world's largest robotics hackathon, which our team won with this innovative chess-playing robot.
Our challenge was to transform a generic robot arm with basic movement capabilities into a system that could play chess - a game that requires significant intelligence and precision. The solution involved a comprehensive five-step approach:
1. Board State Recognition: Initially, we attempted to use Google's Gemini model for chess board recognition, but encountered accuracy issues. We pivoted to a specialized trained computer vision model that could reliably identify chess pieces and their positions, creating an accurate digital representation of the board.
2. Move Calculation: With the digital board state established, we implemented a chess engine based on Stockfish to determine the optimal next move for the robot.
3. Coordinate Translation: The system needed to convert algebraic chess notation (e.g., E2 to E4) into physical coordinates within the robot's visual field. This spatial mapping was critical for precise piece movement.
4. Visual Guidance System: We developed an innovative overlay system that superimposed a blue circle on the starting position and a red circle on the target position within the camera feed. This visual guidance mechanism became central to our training approach.
5. Robot Training: Rather than programming explicit movement patterns, we trained the robot to recognize and respond to the visual indicators - always moving pieces from the blue circle to the red circle. This approach allowed the robot to develop a generalizable policy that worked regardless of the specific chess move required.
What made this project particularly impressive was completing it within the tight timeframe of a hackathon while addressing complex technical challenges across multiple domains - computer vision, machine learning, robotics, and game strategy. The final demonstration showed a robot that could convincingly play chess without any human intervention beyond the initial setup.
Robot arm executing a chess move
Visualization of the chess move planning system
Hackathon team with the Raju robot
Computer vision system detecting chess board state