#

# ODM Benchmarks

This repository is a part of the ODM project and is dedicated to benchmark sample datasets found in the ODMData repository and a few other thrid party sources, to better understand the behavioural characterists of ODM.

OpenDroneMap benchmarking data is summarized in the following pages:

- View benchmark data organized by dataset (Markdown)
- View benchmark data organized by ODM version (Markdown)
- View benchmark raw data (CSV)

# Datasets for Benchmarking

Preferred datasets for benchmarking at this time are:

- Brighton Beach (18 photos)
- Toledo (87 photos)
- Wietrznia (225 photos)
- Shitan (493 photos)

These and many other sample datasets are indexed at the ODMData repository.

# How to Contribute Benchmarks

This section provides instructions for contributing to the ODM Benchmarks project.

- Review Previous Benchmarks – You can have a look at previous benchmarks which is found in the `benchmarks.csv` file, located in the `data` directory to understand various things such as for example, figuring out which dataset has been benchmarked lesser than in comparision to other datasets.
- Select a desired Dataset – Choose one of the datasets of your choice which is listed in the table seen above. Links to download the dataset is provided in the same.
- Run the Dataset – Create a new task in ODM and process the images.
- Verify the results – Ensure that the task completed successfully by viewing the maps and models generated
- Submit Your Results – Open the `benchmarks.csv` file and look at the header for each of the column and note the desired information required before submitting and edit the csv file. Submissions are accepted Via a Pull Request or you can choose to post your results on the ODM community forum. A table is given below to explain the structure of the CSV.

Attribute | Explanation | Example
--- | --- | ---
ID | Benchmark Number | 1
DATASET | Dataset Name | Toledo
PROCESSING_TIME | Time taken to Process The dataset | 1h 9m
PROCESSING_SUCCESS | Confirm If process was successful | Y
ERROR_TYPE | Mention error if any occured | –
RAM_SIZE | Size of RAM allocated | 16 GB
RAM_CLOCK_SPEED | RAM Frequency of the Machine | 2133 MT/s
CPU_TYPE | Make and model of the CPU | Intel i5
CPU_CLOCK_SPEED | Clock Speed of the CPU | 2.3 Ghz
CPU_NUM_CORES | Number of Cores of the CPU | 4
STORAGE_TYPE | Storage Type of the system. HDD/SSD/NVME | SSD
OS | Operating System of the Machine | Ubuntu 18.04
VM_TYPE | Virtual Machine on which ODM is running on | Docker
ODM_VERSION | Version of ODM Used to Benchmark | 1.3.1
ODM_CLUSTER | Declaration of usage of Cluster ODM | N
CONFIG_NAME | Mention of the Configuration in which the dataset is being processed | Default
CONFIG_RESIZE | Mention of image size if images are being resized | 2048 px
CONFIG_OTHER | Other Configurations made that are worth mentioning | –
TEST_DATE | Date of Test | 2020-03-07
TEST_BY | Information of the individual who tests the data | Corey Snipes
INCLUDE_IN_SUMMARY | Check if Data has been included in summary | Y
NOTES | Additional Notes | –

# Queries and Contributions

- Questions and Doubts – Any queries you may have can be posted in the ODM community forum.
- Contributing new Benchmark Data – If you have benchmark data to share, See this page for details on contributing.

# License

MIT License