Of course, everyone hopes that they win, but regardless of performance, every team who made a submission to this challenge has helped to move our collective understanding of this problem forward, and you all deserve recognition for that.
There will be lots of time to do a deep exploration of the leaderboard, but at first glance, I notice a few things:
Based on our internal baselines we expected this, but it is an interesting phenomenon. KiTS23 featured a more difficult problem with the addition of the nephrogenic contrast phase, and this certainly could have played a role. It’s also possible that there was simply a tougher “draw” in test set cases this year. We’ve known for a while that cases vary significantly in their difficulty, and a greater fraction of difficult cases could bring down overall performance substantially. Rest assured that we will explore this further.
This is actually identical to KiTS21, but once again a large decrease from the 100 submissions to KiTS19. I think that this could be because in 2019 there were no widely-accepted baselines for kidney tumor segmentation. Now that there are, it is perhaps less exciting and more daunting to approach a problem that has become so well studied. I’d be happy to hear others’ perspectives on this.
I really feel that with every KiTS challenge, the quality of the papers becomes more and more impressive. I believe that these papers are very important because they enable such unique apples-to-apples meta analyses that are otherwise really difficult in our field. The value of a leaderboard truly resides in these manuscripts.