How it works
The Simulation Pipeline Explained
This simulation pipeline is a powerful tool that combines physics simulation, 3D visualization, and computer vision to create a realistic training environment for your robot. Here's a breakdown of how it works:
The Three Parts
Maple Sim: This acts as the brain of the simulation. It's a headless program (no graphics) that simulates the robot, game objects, and other robots based on physics principles. It even incorporates realistic localization using sensor data. Maple Sim communicates with the other parts through a network connection.
AdvantageScope: This software provides a 3D view of the simulated environment. You can see your robot and other elements move around in real time. AdvantageScope also offers various features like graphs, charts, and joystick visualization for analysis. However, the base version cannot process camera data.
The Alt Codebase: This is your custom code that handles object detection and localization using computer vision techniques. Here's what happens within this codebase:
AI Model (YoloV5): This identifies objects in the camera feed (simulated in AdvantageScope with the forked version).
Detection Labeller (DeepSort): Tracks objects and assigns unique labels for better identification.
Localizer: Estimates the depth and direction of detected objects based on camera information.
Reference Frame Transformer: Adjusts object locations based on camera position and robot orientation.
Kalman Filter (UKF): Refines object location estimates using environmental information.
Probability Map: Creates a dynamic map showing the likelihood of objects being present at different locations.
Path Planning: Uses the probability map and A* algorithm to plan a route to a target object while considering static and moving obstacles.
Route Adjustment: The planned route adapts to account for the movement of other robots.
Communication
Maple Sim sends information about the simulated environment to AdvantageScope via network tables.
The forked version of AdvantageScope utilizes MJPEG streaming to send simulated camera feeds to the Alt Codebase.
The Alt Codebase processes the camera data, generates robot control commands based on the planned route, and sends them back to the simulation (presumably to Maple Sim).
The processed results (object locations, probability maps, etc.) are displayed in AdvantageScope for visualization and analysis.
Benefits
Realistic Training Environment: This simulation allows you to practice your robot's behavior before deploying it in the real world.
Fine-tuning: By testing different scenarios and configurations in the simulation, you can optimize your robot's object detection, localization, and path-planning algorithms.
Safety: The simulation allows you to experiment freely without risking damage to your robot or other equipment.
Visualization Tools: AdvantageScope provides valuable insights into your robot's performance through visual representations and data analysis.
Last updated