Here we show you how to submit dockerized software to a TIRA task.
The idea here is that you pack your software in a docker image and upload this image to TIRA’s registry. TIRA will then run the image on the test dataset and return the evaluation scores.
Create a docker image with your software and its execution environment. Your software must be executable with a single command. This command must take at least 2 parameters: A path to the input dataset and a path to the directory where the results are written to.
Test if the image works and is TIRA-ready using the tira python library.
Register for a task and navigate to the submission page, as described in this guide
Select the tab
Select the tab
Upload Image. This tab shows you the name and login data for your team’s registry.
Tag your local image as per the instructions and upload it to your team’s registry.
Switch to the tab
Add Containerand click on Refresh Images.
Select the image you want to run in the dropdown menu under Docker Image.
Under Command, enter the full command that TIRA should run in your container. Please use full paths. Use TIRA’s variables
$outputDirin your command to refer to these dynamic paths.
A new tab was created with a random name for your configuration from step 4. Switch to this tab.
Select the dataset you want to run the container on.
Select the required resources (try not to oversize). Click on
The container is now run, its progress is shown in the queue at the top of the submission page, where you can inspect its state.
Once the pipeline finishes, the results will be added to a table below the software’s tab.
You can inspect the software output and evaluation results in the table below the software tab.
For private datasets (during a running shared task), these are blinded by default. A task organizer will inspect them and notify you about the results.
In the software tab, click on
Editif your submission was successful.
Give your software a proper name and description. If you publish your submission, please add the doi to the publication.