Continental-NTU is a joint corporate laboratory between Nanyang Technological University of Singapore (NTU) and Continental that focuses on developing key technologies in autonomous robotics, navigation, artificial intelligence (AI), cybersecurity, smart materials, sensing, communication, and cloud technologies for future urban mobility applications.
Continental-NTU aims to support public research into computer vision and the development of
autonomous navigation in urban environment by making our dataset public. The Continental-NTU
dataset consists of data that was collected using a prototype food delivery robotic platform
that was developed by Continental. The robotic platform is equipped with a 16 lines 3-D LiDAR,
two front facing RGB-D cameras, a rear facing RGB-D camera, an inertial measurement unit, and a
global positioning system.
The data is collected from planned routes within Nanyang Technological University of Singapore
(NTU) to include different urban settings, such as campus and residential areas, and in
different environment lighting. We would make an effort to include different type of pedestrians
pavement to allow for robust models for identification and classification.
Note that the LiDAR sensor is tilted slightly forward (towards the ground), and the x-axis is slightly off-center.
*Transforms for the frames are located in the tf_static folder from the dataset
Sensors information to be inserted....
One of the advantages of using this dataset is that it resemble the dataset structure from two popular
public datasets (KITTI and Nuscene datasets).
The raw data files are named sequentially, with their corresponding timestamp located in
timestamp.txt
As for data types that are neither image or pointcloud, they are stored as .json files with
information that are from ROS std_msgs/sensor_msgs.
| Topic Name | Description | File Type |
|---|---|---|
| tf_static | Transform information from base_link to target frame | .json |
| fixposition_navsatfix | GPS information output by VRTK2 in the form of NavSatFix msg type | .json |
| (fl/fr/rear)_cam_color_camera_info | Intrinsic parameters of the camera in the form of CameraInfo msg type | .json |
| (fl/fr/rear)_cam_color_compressed | RGB image output from the camera | .jpg |
| (fl/fr/rear)_cam_aligned_depth_to_color_image_raw | Depth image that is aligned to the RGB image | .png |
| imu_data | Inertial measurement output in the form of Imu msg type | .json |
| fixposition_odometry | Odometry of the FP_VRTK link with Earth's center as origin in the form of Odometry msg type | .json |
| rslidar_points | Pointcloud output from the LiDAR sensor | .pcd |
Alternatively, there are also .json files, that were exported out from MongoDB, located in the database folder which contains information that links the raw data together.
Information about the log from which the data was recorded
log {
“log_name”: <str> – – Name of the log recorded.
“log_token”: <str> – – Unique record identifier.
“date”: <str> – – Date: (DD—MM—YYYY).
“location”: <str> – – Acronym of the recorded route.
“time_of_day”: <str> – – Environment lightning of which the run was recorded (day, evening, night).
“category”: <str> – – The type of environment of which the run was recorded.
}Information about sensor used (LiDAR/camera/imu). All extrinsic parameters are given with respect to the base_link.
sensor_info {
“sensor_name”: <str> – – Given name of the sensor.
“log_token”: <str> – – Foreign key pointing to log category.
“sensor_info_token”: <str> – – Unique record identifier.
“translation”: <float> [3] – – Coordinate system origin in meters: x, y, z.
“rotation”: <float> [4] – – Coordinate system orientation as quaternion: x, y, z, w.
“camera_intrinsic”: <float> [9] – – 3 x 3 Intrinsic parameters of camera as 1‐D list. Empty for non‐camera.
“camera_distortion”: <float> [5] – – Camera distortion parameters (k1, k2, t1, t2, k3). Empty for non‐camera.
“camera_projection”: <float> [12] – – 3 x 4 Camera projection matrix. Empty for non‐camera.
}Localization output from VRTK2 in latitude, longitude and altitude format.
gps_data {
“log_token”: <str> – – Foreign key pointing to log category.
“gps_data_token”: <str> – – Unique record identifier.
“timestamp”: <float> – – Unix time stamp in seconds.
“latitude”: <float> – – Latitude [degrees°] | Positive ‐ north of equator | negative ‐ south of equator.
“longitude”: <float> – – Longitude [degrees°] | Positive ‐ east of prime meridian | negative ‐ west of prime meridian.
“altitude”: <float> – – Altitude [m]. Positive is above the WGS 84 ellipsoid.
“status”: <int> – – Determine fix augmentation based on fix type and last time differential corrections received | No fix position (‐1) | unaugmented fix (0) | satellite‐based augmentation (1) | ground‐based augmentation (2)
“service”: <int> – – Bits defining which Global Navigation Satellite System signals were used by the receiver | SERVICE_GPS=1 | SERVICE_GLONASS=2 | SERVICE_COMPASS=4 (includes BeiDuo) | SERVICE_GALILEO=8
“position_covariance”: <float> – – Position covariance [m^2] defined relative to a tangential plane through the reported position. The components are East, North, and Up (ENU), in row‐major order.
“next”: <str> – – Foreign key pointing to the next gps reading. Empty if end of sequence.
“prev”: <str> – – Foreign key pointing to the previous gps reading. Empty if start of sequence.
}Odometry of the FP_VRTK link with Earth's center as origin.
odom {
“log_token”: <str> – – Foreign key pointing to log category.
“odom_token”: <str> – – Unique record identifier.
“timestamp”: <float> – – Unix time stamp in seconds.
“translation”: <float> – – Coordinate system origin in meters: x, y, z.
“rotation”: <float> – – Coordinate system orientation as quaternion: x, y, z, w.
“next”: <str> – – Foreign key pointing to the next odom reading. Empty if end of sequence.
“prev”: <str> – – Foreign key pointing to the previous odom reading. Empty if start of sequence.
}Odometry of the base_link achieved using LiDAR-based localization on a pre-built map.
odom {
“log_token”: <str> – – Foreign key pointing to log category.
“sensor_info_token”: <str> – – Foreign key pointing to sensor_info category.
“imu_data_token”: <str> – – Unique record identifier.
“timestamp”: <float> – – Unix time stamp in seconds.
“orientation”: <float> [4] – – Coordinate system orientation as quaternion: x, y, z, w.
“orientation_covariance”: <float> [9] – – 3 x 3 Orientation covariance matrix as 1‐D List about x, y, z axes.
“angular_velocity”: <float> [3] – – Rotational velocity [rad/s].
“angular_velocity_covariance”: <float> [9] – – 3 x 3 Angular velocity covariance matrix as 1‐D List about x, y, z axes.
“linear_acceleration”: <float> [3] – – Linear acceleration [m/s^2].
“linear_acceleration_covariance”: <float> [9] – – 3 x 3 Linear Acceleration covariance matrix as 1‐D List about x, y, z axes.
“next”: <str> – – Foreign key pointing to the next imu reading. Empty if end of sequence.
“prev”: <str> – – Foreign key pointing to the previous imu reading. Empty if start of sequence.
}Information on sensor data, either image for camera or pointcloud for LiDAR.
sensor_data {
“log_token”: <str> – – Foreign key pointing to log category.
“sensor_info_token”: <str> – – Foreign key pointing to sensor_info category.
“sensor_data_token”: <str> – – Unique record identifier.
“timestamp”: <float> – – Unix time stamp in seconds.
“filename”: <str> – – Relative path to the image/pointcloud file depending on the type of sensor.
“fileformat”: <str> – – Extension of the file. | RGB image (.jpg) | Depth image (.png) | pointcloud (.pcd)
“height”: <int> – – Image height in pixels. Empty for non‐image file.
“width”: <int> – – Image width in pixels. Empty for non‐image file.
“next”: <str> – – Foreign key pointing to the next reading from the same sensor. Empty if end of sequence.
“prev”: <str> – – Foreign key pointing to the previous reading from the same sensor. Empty if start of sequence.
}| Log_name | Date | Category | Time | Download | ||
|---|---|---|---|---|---|---|
| RTP_SBS_EMB_QUAD_19042023 | 19/04/2023 | Campus | Lunch | Raw Data | Rosbag | |
| Log_name | Date | Category | Time | Download | ||
|---|---|---|---|---|---|---|
| CanB_RTP_18042023_lunch | 18/04/2023 | Campus | Lunch | Raw Data | Rosbag | |
| Log_name | Date | Category | Time | Download | ||
|---|---|---|---|---|---|---|
| CanB_H4_H5_Can1_20042023 | 20/04/2023 | Residential | Lunch | Raw Data | Rosbag | |
| Log_name | Date | Category | Time | Download | ||
|---|---|---|---|---|---|---|
| Can2_SRC_NYH_H9_25042023 | 25/04/2023 | Residential | Lunch | Raw Data | Rosbag | |
| Log_name | Date | Category | Time | Download | ||
|---|---|---|---|---|---|---|
| Can9_H8_NH_25042023 | 25/04/2023 | Residential | Lunch | Raw Data | Rosbag | |
| Log_name | Date | Category | Time | Download | ||
|---|---|---|---|---|---|---|
| NH_GH1_GH2_CRS_20042023 | 20/04/2023 | Residential | Dinner | Raw Data | Rosbag | |
| Log_name | Date | Category | Time | Download | ||
|---|---|---|---|---|---|---|
| CRS_H13_H14_20042023 | 20/04/2023 | Residential | Dinner | Raw Data | Rosbag | |
| Dataset Version | Last Updated | Download |
|---|---|---|
| Version 1 (v.1) | 25/04/2023 | Link |
| Dataset Version | Last Updated | Download |
|---|---|---|
| Version 1 (v.1) | 10/04/2023 | Link |
.accordion-flush class. This is the first item's accordion
body..accordion-flush class. This is the second item's accordion
body. Let's imagine this being filled with some actual content..accordion-flush class. This is the third item's accordion
body. Nothing more exciting happening here in terms of content, but just filling up the
space to make it look, at least at first glance, a bit more representative of how this would
look in a real-world application.