Here we show you how to submit dockerized software to a TIRA task.
The idea here is that you pack your software in a docker image and upload this image to TIRA’s registry. TIRA will then run the image on the test dataset and return the evaluation scores.
Prepare the image on your side
-
Create a docker image with your software and its execution environment. Your software must be executable with a single command. This command must take at least 2 parameters: A path to the input dataset and a path to the directory where the results are written to.
-
Test if the image works and is TIRA-ready using the tira python library.
Upload and execute the image
-
Register for a task and navigate to the submission page, as described in this guide
-
Select the tab
Docker Submission
-
Select the tab
Upload Image
. This tab shows you the name and login data for your team’s registry.
Tag your local image as per the instructions and upload it to your team’s registry. -
Switch to the tab
Add Container
and click on Refresh Images.
Select the image you want to run in the dropdown menu under Docker Image.
Under Command, enter the full command that TIRA should run in your container. Please use full paths. Use TIRA’s variables$inputDataset
and$outputDir
in your command to refer to these dynamic paths.
Click onAdd Container
. -
A new tab was created with a random name for your configuration from step 4. Switch to this tab.
-
Select the dataset you want to run the container on.
Select the required resources (try not to oversize). Click onRun Container
. -
The container is now run, its progress is shown in the queue at the top of the submission page, where you can inspect its state.
Once the pipeline finishes, the results will be added to a table below the software’s tab.
Evaluate and add provenance information
-
You can inspect the software output and evaluation results in the table below the software tab.
For private datasets (during a running shared task), these are blinded by default. A task organizer will inspect them and notify you about the results. -
In the software tab, click on
Edit
if your submission was successful.
Give your software a proper name and description. If you publish your submission, please add the doi to the publication.