Skip to content

docs: clarify dataset split instructions (train/val/test vs trainval) #427

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dhruvildarji wants to merge 1 commit into
base: master
Choose a base branch
from
Open

docs: clarify dataset split instructions (train/val/test vs trainval) #427

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
123 changes: 72 additions & 51 deletions docs/dataset_setup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,66 +7,87 @@ After logging in you will see multiple archives.
For the devkit to work you will need to download *all* archives.
Please unpack the archives to the `~/nuplan/dataset` folder.

## Dataset splits: train, val, test vs. trainval

The download page provides separate archives for **train**, **val**, and **test** splits.
However, the devkit expects all `.db` files from the **train**, **val**, and **test** archives
to be placed together in a single **`trainval`** directory:

```
~/nuplan/dataset/nuplan-v1.1/trainval/
```

You do **not** need to create separate `train/`, `val/`, or `test/` directories. Simply unpack
all downloaded split archives into the `trainval` folder. The devkit handles the logical
splitting into train, val, and test internally using the log splitter configuration
(see `nuplan/planning/script/config/common/splitter/nuplan.yaml`), which maps individual
log filenames to their respective splits.

In short:
- **Download** the train, val, and test archives separately.
- **Merge** all `.db` files into the single `trainval/` directory shown in the hierarchy below.
- The devkit's configuration will determine which logs belong to which split at runtime.

## Filesystem hierarchy
The final hierarchy should look as follows (depending on the splits downloaded above):
The final hierarchy should look as follows:
```angular2html
~/nuplan
├── exp
│   └── ${USER}
│    ├── cache
│ └── ${USER}
│ ├── cache
│ │ └── <cached_tokens>
│    └── exp
│ └── exp
│ └── my_nuplan_experiment
└── dataset
├── maps
│   ├── nuplan-maps-v1.0.json
│   ├── sg-one-north
│   │   └── 9.17.1964
│   │   └── map.gpkg
│   ├── us-ma-boston
│   │   └── 9.12.1817
│   │   └── map.gpkg
│   ├── us-nv-las-vegas-strip
│   │   └── 9.15.1915
│   │   └── map.gpkg
│   └── us-pa-pittsburgh-hazelwood
│   └── 9.17.1937
│   └── map.gpkg
│ ├── nuplan-maps-v1.0.json
│ ├── sg-one-north
│ │ └── 9.17.1964
│ │ └── map.gpkg
│ ├── us-ma-boston
│ │ └── 9.12.1817
│ │ └── map.gpkg
│ ├── us-nv-las-vegas-strip
│ │ └── 9.15.1915
│ │ └── map.gpkg
│ └── us-pa-pittsburgh-hazelwood
│ └── 9.17.1937
│ └── map.gpkg
└── nuplan-v1.1
   ├── splits
   │ ├── mini
   │ │ ├── 2021.05.12.22.00.38_veh-35_01008_01518.db
   │ │ ├── 2021.06.09.17.23.18_veh-38_00773_01140.db
   │ │ ├── ...
   │ │ └── 2021.10.11.08.31.07_veh-50_01750_01948.db
   │ └── trainval
   │ ├── 2021.05.12.22.00.38_veh-35_01008_01518.db
   │ ├── 2021.06.09.17.23.18_veh-38_00773_01140.db
   │ ├── ...
   │ └── 2021.10.11.08.31.07_veh-50_01750_01948.db
   └── sensor_blobs
   ├── 2021.05.12.22.00.38_veh-35_01008_01518
   │ ├── CAM_F0
   │ │ ├── c082c104b7ac5a71.jpg
   │ │ ├── af380db4b4ca5d63.jpg
   │ │ ├── ...
   │ │ └── 2270fccfb44858b3.jpg
   │ ├── CAM_B0
   │ ├── CAM_L0
   │ ├── CAM_L1
   │ ├── CAM_L2
   │ ├── CAM_R0
   │ ├── CAM_R1
   │ ├── CAM_R2
   │ └──MergedPointCloud
   │ ├── 03fafcf2c0865668.pcd
   │ ├── 5aee37ce29665f1b.pcd
   │ ├── ...
   │ └── 5fe65ef6a97f5caf.pcd
   │
   ├── 2021.06.09.17.23.18_veh-38_00773_01140
   ├── ...
   └── 2021.10.11.08.31.07_veh-50_01750_01948
├── splits
│ ├── mini
│ │ ├── 2021.05.12.22.00.38_veh-35_01008_01518.db
│ │ ├── 2021.06.09.17.23.18_veh-38_00773_01140.db
│ │ ├── ...
│ │ └── 2021.10.11.08.31.07_veh-50_01750_01948.db
│ └── trainval ← place ALL .db files from train, val, and test here
│ ├── 2021.05.12.22.00.38_veh-35_01008_01518.db
│ ├── 2021.06.09.17.23.18_veh-38_00773_01140.db
│ ├── ...
│ └── 2021.10.11.08.31.07_veh-50_01750_01948.db
└── sensor_blobs
├── 2021.05.12.22.00.38_veh-35_01008_01518
│ ├── CAM_F0
│ │ ├── c082c104b7ac5a71.jpg
│ │ ├── af380db4b4ca5d63.jpg
│ │ ├── ...
│ │ └── 2270fccfb44858b3.jpg
│ ├── CAM_B0
│ ├── CAM_L0
│ ├── CAM_L1
│ ├── CAM_L2
│ ├── CAM_R0
│ ├── CAM_R1
│ ├── CAM_R2
│ └──MergedPointCloud
│ ├── 03fafcf2c0865668.pcd
│ ├── 5aee37ce29665f1b.pcd
│ ├── ...
│ └── 5fe65ef6a97f5caf.pcd
│
├── 2021.06.09.17.23.18_veh-38_00773_01140
├── ...
└── 2021.10.11.08.31.07_veh-50_01750_01948
```

If you want to use another folder, you can set the corresponding [environment variable](https://github.com/motional/nuplan-devkit/blob/master/docs/installation.md).