Dear Contest Organizers,
I would like to ask two following questions:
1. How many submissions can we choose for final evaluation?
2. The rule says that: "The total number of submissions for any single team is limited to 100 solutions.". In the submission page, I see that we can delete a submission after it is evaluated and returns the Leaderboard score. So are deleted submissions counted in 100 solutions?
Thank you for organizing interesting challenge!
Dear Tu,
regarding your first question - only one solution can be chosen for the final evaluation. This particular solution should also be shortly described in the competition report.
Regarding the limit in the number of submissions - we set this limit in order to discourage people from treating the preliminary evaluation system as a tool for fine-tuning their solutions. Since only 10% of data is used in the preliminary evaluation, testing multiple solutions may result in overfitting to that particular set. However, after a longer consideration we are thinking about removing this limit from the competition rules - participants should make responsible choices :-)
However, we still want to have a limit for the number of solutions from a single team, which are stored in our system. Users may delete solutions which they consider irrelevant.
Best,
Andrzej Janusz
