In the autonomous driving environment perception system, how to obtain high-precision real-time road condition data is the key to determining the driving safety of the autonomous driving system.

In the autonomous driving environment perception system, how to obtain high-precision real-time road condition data is the key to determining the driving safety of the autonomous driving system .

Among the two current mainstream perception technology paths, "pure vision" and "high-precision map + lidar ", since the 3D point cloud traffic data collected by lidar is more dense, accurate and has three-dimensional geographical information, 3D Perception algorithms based on point cloud data have become the core technology path for some autonomous driving companies, and have created a large-scale need for 3D point cloud data annotation.

1. What is 3D point cloud semantic segmentation

The annotation processing of 3D point cloud data can be roughly divided into three categories according to the processing methods and the use of annotation tools:

1) Pure point cloud

Annotation tools used for pure point clouds Mainly based on 3D three-dimensional frame, the object to be annotated is marked in the form of 3D three-dimensional frame, and the corresponding attribute information label is attached. The specific form is as follows:

Image source: Manfu Technology SEED Data Service Platform

2) Fusion annotation

Fusion annotation The 3D annotation tools used are still mainly 3D three-dimensional frames, but in addition to 3D point cloud data, 2D annotation tools also need to be used to annotate the 2D images corresponding to the point cloud data. The annotated objects in 3D point cloud data are different from 2D The annotation objects in the image correspond one to one, and the specific form is as follows:

Image source: Manfu Technology SEED data service platform

3) 3D point cloud semantic segmentation

is similar to the 2D image semantic segmentation annotation logic. 3D point cloud semantic segmentation also has different The objects to be annotated are color segmented and assigned semantic labels. The difference is that 3D point cloud semantic segmentation requires assigning a category label to each pixel in the 3D point cloud, such as vehicles, pedestrians, roads, buildings, etc. The specific form is as follows:

(Picture source: Manfu Technology SEED Data Service Platform)

Compared with the first two annotation methods, when more detailed spatial information is required (such as automatic parking), the three-dimensional box annotation scene such as articulated For objects with variable shapes such as buses, 3D point cloud semantic segmentation annotation can provide more accurate feedback on traffic data. Therefore, 3D point cloud semantic segmentation annotation plays an increasingly important role in autonomous driving perception algorithms.

2. Common annotation rules and annotation objects

Different algorithms and different application scenarios have different annotation requirements and rules for point cloud semantic segmentation. Common relatively simple autonomous driving 3D point cloud semantic segmentation annotation objects include:

1. Pedestrians : Usually refers to people who are not riding vehicles, but includes people who ride scooters, balance bikes .

2. Cyclists: usually include people riding motorcycles, bicycles, tricycles, electric vehicles, etc.

3. Vehicles: Vehicles are the most common objects in 3D point cloud semantic segmentation and annotation scenarios, and have many subcategories. The division of sub-objects is based on the specific vehicle model, such as cars, trucks, buses, ambulances, etc., and also based on categories, such as passenger cars, trucks, engineering vehicles, etc.

Take the category as an example, the details are as follows:

1) Micro cars : such as scooters, various types of tricycles, etc.;

2) Small cars: such as cars, SUVs, etc.;

3) Passenger cars: such as large and medium buses, double-decker buses and other manned vehicles;

4) trucks: cargo vehicles such as vans;

5) engineering vehicles: sprinklers, fire trucks, garbage trucks, etc.;

6) other vehicles.

4. Pavement related: Various objects related to the road surface can be specifically subdivided into drivable areas, sidewalks, lane lines, etc.

5. Buildings: such as buildings, tunnel decks, bridge piers, viaducts, etc.

6. Plants: such as grass, trees, etc.

7. Pavement objects: Pavement objects are also a type with many sub-objects. Common ones include:

1) Guardrails: such as traffic isolation barriers, traffic guardrails, etc.;

2) Fences: temporary ones such as Traffic , water horse, , etc.;

3) Traffic markings: such as traffic lights, signs, poles, etc.;

4) Obstacles: such as traffic cones, warning signs, anti-collision barrels, stone piers, etc.;

5) other objects.

8. Noise point clouds: various types of noise point clouds such as water mist, smoke, car exhaust, rainwater, etc.

and above are various common annotation objects for 3D point cloud semantic segmentation. However, there are huge differences in different scenarios, specific object division rules and types of objects to be annotated, and there is no unified standard.

In terms of more specific annotation rules, the differences are also huge.

Common labeling rules such as the starting range of an object usually stipulate that if the number of point clouds exceeds The concrete annotation examples of

are as follows:

Before annotation (Picture source: Manfu Technology SEED Data Service Platform)

After annotation (Picture source: Manfu Technology SEED Data Service Platform)

As autonomous driving commercial mass production enters a new stage, The scenes on open road sections are complex and changeable, requiring autonomous driving algorithms to have deeper scene accumulation and iteration capabilities to meet the needs of perception algorithms and driving safety.

3D point cloud semantic segmentation provides more refined scene information feedback, which has natural advantages in helping algorithms build more realistic scene models and improving perceived safety. Today, as autonomous driving moves towards large-scale commercial applications, it is bound to play a more important role. important role.