LIDAR for object recognition and tracking

One of the projects that I worked on with two other electrical engineering students was to use a RPLidar to detect objects in a room and track them.

We connected an ESP8266 to the RPLidar via UART and used it as a wifi-serial bridge so that all commands and processing could be performed on the PC side of things. The PC generated a point cloud using Python’s Plotly, which we then used to visualise the objects being detected and tracked.

I wrote the networking code, the control algorithm, and the final algorithm we used to track objects which based object tracking off the movement of the centroid of an object. If this centroid was moving radially away from the Lidar, at an angle in an arc around the Lidar, or in the z direction, then a black dot was placed in the point cloud to indicate the object was moving. Effectively just polar coordinates!

Here are some photos of the Lidar and the point clouds that were generated. The different colours in the point clouds were used to indicate different objects.

The RPLidar, and the stepper motor driver (red),
The RPLidar, and the stepper motor driver (red),
The lidar, stepper motor driver and ESP8266 (black module on bottom right).
The lidar, stepper motor driver and ESP8266 (black module on bottom right).
Here we tested if the lighting conditions affected the range of the lidar.
Here we tested if the lighting conditions affected the range of the lidar.
This test was used to visualise the point cloud showing the table, and the two whiteboards.
This test was used to visualise the point cloud showing the table, and the two whiteboards.
The lidar mounted on a tripod to get a better angle of observation for the room.
The lidar mounted on a tripod to get a better angle of observation for the room.
One of the initial point clouds (no object detection), showing the ceiling and a person.
One of the initial point clouds (no object detection), showing the ceiling and a person.
A milk crate!
A milk crate!
A person. The seven large dots include the centroid (centre) and the extremities of the object.
A person. The seven large dots include the centroid (centre) and the extremities of the object.
These images how a person being detected and then as they move a black dot is placed at the centroid to indicate movement!
These images how a person being detected and then as they move a black dot is placed at the centroid to indicate movement!