My last run did generate the output but shows error on the evaluator.py
[0 more lines] python3: can't open file '/builds/code-research/tira/multi-author-writing-style-analysis-2024/evaluator/evaluator.py': [Errno 2] No such file or directory
What is going wrong with the evaluator ? Please help
Dear Riya,
Thanks for reaching out!
There is indeed a small error in the deployment of the multi-author evaluator, @nikolay.kolyada is currently taking care of this and will report back when it is resolved.
Best regards,
Maik
I ran my code on the test data set as well, just to check if it succeeds or not. It did run but it does not show any output. It is just spinning (trying to load) but does not succeed in retrieving the output.
Is it down ?
Hi, no, it is not down, it might just take a while to get scheduled in the cluster.
Best regards,
Maik
It (the test run) had completed its run but did not show the output or other details of the run. Just shows the progress bar on click of ‘Review’ of the run.
Validation run on the other hand shows all the details of the run including the output.
@maik_froebe what is the problem with the review of the run on the test dataset ? What cant I see the output of the run. I want to make sure that the software runs on the test dataset without any errors and that it generates the desired output.
Please help.
Dear Riya,
Results on the test set are usually blinded until after the deadline is closed.
The organizers review the outputs and contact the participants in case there is a problem (usually within one day).
I am not 100% sure how the expected output of the task looks like, but we now fixed the evaluator and I did re-run the evaluator so I can report back in a moment.
Best regards,
Maik
Dear @riyahsanjesh,
There was indeed an error of the software on the test dataset, I unblinded your submission so that you can see the stacktrace and fix the problem.
Best regards,
Maik
I fixed the issue and ran on both validation and test dataset. Could you please verify if the last test dataset run went fine ?
Hi, I just reviewed the new runs. Your latest run on the validation set was successful.
Thanks. can you please review the test data set run as well. It doesnt show me if that was successful or not.
Validation data set run is visible to me and I am able to verify the same.
Thanks,
Riyah
Dear Riyah,
I reviewed the last run on the test dataset and everything looks good!
We noticed that the test dataset was not complete (some problem files in the easy section of the test set were missing), so we added the complete test set as a new version.
I already started your submissions on the new complete test set, and will review them later today.
Best regards,
Maik
