Problem
Enable Segway Loomo robots to navigate crowded, dynamic indoor environments; most academic navigation systems never leave simulation.
Built
A modular ROS pipeline that runs on a server and drives the Loomo over a wireless socket link, one package per job:
- Perception: person detection (YOLOv5), a single-object tracker that holds on to the person being followed (Stark), and their 2D and 3D keypoints.
- Estimation: one node turns the IMU stream the Loomo sends into the robot’s state, relative to where it started; another builds a map and estimates the state from all its sensors.
- Prediction: where the followed person is heading.
- Planning and control: a collision-free path, followed with model predictive control.
It keeps the base of C. Conejo Bernal’s earlier pipeline (2020–21); I redid its architecture and its perception block in 2022–23, continuing the VITA Lab research line. The perception models and the ROS 1 workspace are pinned to the 2023 stack; the modern successor of the perception is the real-time vision pipeline.
Why hard
Real-time performance across several computationally intensive perception modules running simultaneously, validated among actual pedestrians, not just in simulation.
What happened
Validated end-to-end on physical robots (Segway Loomo, EPFL VITA Lab research platform); the code is public on GitHub (github.com/thrmnn/ROS_Autonomous_Driving).