Dear all,
This is the information regarding the run file for Image Retrieval for arguments: Touché at CLEF 2023 - Image Retrieval for Arguments
The submission format adapts the standard TREC format. Each line corresponds to an image retrieved for some topic and stance at a certain rank, making a run file 1000 lines long (50 topics, 2 stances, 10 ranks). Each line contains the following fields, separated by single whitespaces:
The topic number (51 to 100).
The stance (“PRO” or “CON”).
The image’s ID (corresponds to the name of the image’s directory in the collection; always 17 characters long and starts with “I”).
The rank (1 to 10 in increasing order per topic and stance). Not used in this year’s evaluation.
A score (integer or floating point; non-increasing per topic and stance). Not used in this year’s evaluation.
A tag that identifies your group and the method you used to produce the run.
It explicitly states the runfile has to be 1000 lines long. What if we do not have exactly 10 images per stance but only 5? should we add dummy ids or is it okay for the file to be shorter?
Thank you so much!
Kind regards,
Sarah Bachinger