First of all, a huge thank you to everyone who took part in this challenge. We were totally blown away by the number of teams that made such high-quality submissions.
Leaderboard Release
We’re going to do our very best to have the leaderboard out in the next ~24 hours. Each entry on the leaderboard will contain:
- The team’s Rank
- The team members’ names (“First Author et al.” if there are more than two)
- The affiliation of the first author on the manuscript
- The kidney Dice
- The tumor Dice
- The composite Dice
- A link to download the manuscript
Statistical Significance
The bootstrap will be used to test whether each pair of submissions is statistically significantly different. This will not have an effect on awards, but it will be made public.
Manuscripts
Manuscripts are currently being reviewed and classified into four categories. You will be notified of your category by email before the leaderboard is public. The categories are
- Accept
- Minor Revisions
- Major Revisions
- Reject
4s are those in gross violation of the requirements, (e.g. files with less than a page of content). Submissions associated with a 4 will not appear on the leaderboard. Authors of these submissions were repeatedly warned of this and failed to make changes. We MAY consider amended 4s provided by email, but only if this is done VERY soon.
3s are those that clearly put effort into describing their methods, but they still require a nontrivial amount of work before we will can publish them. The authors of 3s must make the specified changes before August 5, otherwise their associated submissions will be removed from the leaderboard. Please send these as an attachment to a response to your notification email.
2s are sufficient for publication in their current form, but it would be nice if some minor changes were made.
1s are great as is, and changes are neither required nor suggested.
Most submissions are 1s and 2s.
Overlapping Submissions
As we review the manuscripts, we are thoroughly checking for overlapping manuscripts and author lists. If a team is found to have submitted under more than one user and thus received more than two approximate scores, the users will be combined into one team and all submissions made after the third score was received will be discarded. We will notify affected users as we discover them. This is not an accusation of cheating – we understand that miscommunication between team members can happen – but we must do this to ensure that the competition is fair.
Thank You!
Once again, thank you all so much for taking part in this challenge. The objective comparison of methods is not as common as it should be in medical image computing, and we really appreciate your willingness to test your methods out in the open.
