Hi organizers.
Our team can’t be able to get a virtual machine for style change detection task. Our team filled the form given by PAN to request a VM but, when we access the tira.io website and want to make a submission clicking the submit button, the page says ‘Oops! That page doesn’t exist or is private.’
We have not receive any mail from our request yet neither.
As the dataset has the test dataset, we want to know if it’s possible to give you just the predictions outputed by our model, or we need to submit our software to TIRA?
We’ll hand out VMs for PAN in the very near future (via mail). Rest assured that we’ll also extend the early bird deadline to give all of you enough time for submission.
We’re also working on a new feature to allow you to directly upload runs when the test data is public. That’s still a bit experimental though, that’s why you get that error. We still encourage you to use VMs for a software submission for reasons of reproducibility.
Style Change Detection has three tasks, and the answer file will have three folders. I wanna know how to upload all answers to evaluate the scores at once. I try it but it can only upload a json file. Failed to upload even a folder.
There’s a secret feature for this: You can zip the folders and upload the zip. Tira will unzip it. Please make sure that there are no extra folders in the zip, but the output is in the zip’s root.
Hi, Matti
We want to know whether the results are on the leaderboards or not, if we only upload json files which are the predictions outputted by our model.
my question is also related to the test set in this task (PAN23 Multi-Author Writing Style Analysis).
The submission can be done by submitting the prediction files (.josn)
However, the test set is missing (it is not publicly available).
What is the possible way to submit the prediction of the test set?
A submission via run-file upload is only possible if the organizers of the task release the test dataset. If they have not done so (as your comment suggests), then you have no other choice than to do a software submission.
Doing a software submission comes with some overhead (i.e. dockerizing your predictor) but it greatly improves the reproducibility of your research contribution, for example, your software can be considered as a baseline for the coming years. We also provide several guides and examples o help you along.
