What happend on evaluate?

when i click the evaluate button,my file righlt output the preddcit results,but it give the info

Hey Hao Long Ma,

The evaluator is up and running, could you please try again?

I tried it,but why there is no score,24 teams only one correctly submited ,I feel puzzled about the black box test

Which task are you participating?



I checked it and the original run that you created evaluations for got some errors. I unblinded them, so you can see the error output now.

there is no std error,and my output size is 2k,
I’ve been suffering from this problem for days

Both your software runs 2022-05-20-06-52-51 and 2022-05-20-13-46-55 contain the similar error:

usage: main.py [-h] [--input INPUT] [--output OUTPUT] [--model MODEL]
main.py: error: argument --input: expected one argument

I found the train file Pass null parameter will give the error and its outputsie is 0k,but test not give the error,its ouput size is 2.5k,after i click the test set evaluate button,the ouput size is 2k

I use the same commend get diffrenent results please see the output size


the input command supposed to have the following format:

sh ./train.sh -i $inputDataset -o $outputDir

thanks,i found the inputdatasetDir include en,why not inputdatasetDir= inputdatasetDir+"/en".thought i get file nums,lines,outputsize,but why i click the evaluate button,there is still no score

Hey, I believe you supposed to write the resulting xml files into a subfolder like output/en/*.xml.

I’ve followed the steps you said. But I still can’t get this score help!!!

There is the error in your run 2022-05-20-16-30-08:

Traceback (most recent call last):
  File "main.py", line 75, in <module>
  File "main.py", line 71, in main
  File "main.py", line 53, in generate
    with open(input_file_path,"r",encoding="utf-8") as inputFile,open(out_file_path,"w") as ouputFile:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/longma22/2022-05-20-16-30-08/output/en/fafd37354efa9cf8948482619e1cc0f.xml'

Can you see it in the stderr output?

still like this

I unblinded your last runs. It looks good now!

does this means I successfully submited?but the leaderboard not appeard my score?

The task organizers should review all the submissions and then they will publish them.

Thank you for your patient help and have a good day