How to make a software submission with virtual machines

Getting Started
1

If you do not have access to the test data, you can make a software submission. This means you install your software on one of our virtual machines (VM) and tell us how to execute your software. We will then run your software on the test data and return your scores.

Here I show you how to make a software submission:

  1. Develop your software on your own machine. If it works with the test data, head over to tira.io

  2. Find the appropriate task in the list of task (or search for it)

  3. Click on SUBMIT to go to your software page

    1-select task
    1-select task928×509 253 KB

  4. On your software page:

    • Click on CONNECTION INFO and use the data to access you VM.
    • Install your software in the VM.
    • Click on + ADD SOFTWARE and switch to the new tab.
      2-add software
      2-add software930×435 63.8 KB

  5. Specify the path to your software and the input parameters. You can check the info button to learn about the parameters of your tasks. You may need to adapt your software to accept variable information, such as paths, via the command line.

  6. From the dropdown, select the dataset you want to run your software on.

  7. Click on RUN to execute your command.

    3-save software
    3-save software925×508 77.7 KB

  8. TIRA will now execute your software. You can not log in to the VM during this process. Only one execution can run at a given time.
    a. The VM is going to be sandboxed. This will cut the Network.
    b. The software command executed as provided
    c. The VM is being unsandboxed. This will reset the VM to the state before the run and return network access.
    d. The new result (a RUN) is created.

    4-sandboxing
    4-sandboxing929×377 59.7 KB

  9. If the execution was successful, you can now see your new RUN in the table below.

    • Click on INSPECT to see the output and potential errors. On secret test data, an organizer of your task may need to reveal the results first.
    • Click on EVALUATE to get the scores of your results.
      5-evaluate
      5-evaluate941×99 15.4 KB

  10. The scores of your answers should appear below your upload as an evaluation run. Know Bug: You may need to reload the page again to see the evaluation run.

    • Click on INSPECT to see the output of the evaluator. On secret test data, an organizer of your task may need to reveal the results first. You can see organizer feedback there.
      6-inspect
      6-inspect939×106 25.1 KB

Please also consider the participation guidelines provided by your task organizer.

2

why i get NULL parameter，my commend is sh ./train.sh -i $inputDir -o $outputDir
but error suggest that it except one argument??? the issue make me so crazy,i use getpots get the input and output variable ,but why i get empty parms???

3

Yes, these parameters are replaced by TIRA when executing the software with the path to the dataset and the output folder.

4

1653293928(1)
1653293928(1)798×114 3.14 KB

a6a25a499bd6edf4cc31142e63e5f29
Dear kolyada:
I can run my program and get the answer.jsol when I used the fictional testdata in finalsheel/ssh, but I don’t know why I cann’t run normally in TIRA. help me solve this problem please,thank you very much.