Rxivist logo

Critical Assessment of Metagenome Interpretation - the second round of challenges

By Fernando Meyer, Adrian Fritz, Zhi-Luo Deng, David Koslicki, Alexey Gurevich, Gary Robertson, Jie Zhu, Huijue Jia, Axel Kola, Antoine Limasset, Mikhail Kolmogorov, Rob Egan, Gail L. Rosen, Alexandru Cristian, Zhengqiao Zhao, Melissa A. Gray, Jakob Nissen, Georg Zeller, Lucas Paoli, Hans-Joachim Ruscheweyh, Alessio Milanese, Shinichi Sunagawa, Mohammed Alser, Lizhen Shi, Bo Chen, Zhengyang Wang, Ziye Wang, Pingqin Huang, Ronghui You, Fengzhu Sun, Shanfeng Zhu, Denis Bertrand, Chengxuan Tong, Chenhao Li, Niranjan Nagarajan, Bernhard Renard, Fabio Malcher-Miranda, Vitor C. Piro, Rayan C Chikhi, Riccardo Vicedomini, Zhong Wang, Ashleigh Thomas, Julien Tremblay, Piotr W Dabrowski, Sergey Nurk, Tue S. Jorgensen, Nils P. Willassen, Terje Klemetsen, Espen M. Robertsen, Evangelos Georganas, Katherine Yelick, C. Titus Brown, Claire Lemaitre, Pierre Peterlongo, Camille Marchet, Pierre Marijon, Luiz C. Irber, Jaqueline J. Brito, Nathan LaPierre, Eleazar Eskin, Simon Rasmussen, Soren J. Sorensen, Lars H. Hansen, Mirko Trajkovski, Silas D. Kieser, Daniel R Mende, Serghei Mangul, Varuni Sarwal, Vanessa R. Marcelino, Aaron E Darling, Francesco Beghini, Nicola Segata, Andrew Warren, Jacob S. Porter, Evan R Rees, Jason Kwan, Philip T.L.C. Clausen, Jan Buchmann, Knut Reinert, Enrico Seiler, Anton Korobeynikov, Dmitry Antipov, Dmitry Meleshko, Aydin Buluc, Steven Hofmeyr, Leonid Oliker, Eugene Goltsman, Gherman Uritskiy, Simona Radutoiu, Petra Gastmeier, Friederike Maechler, Ariane Khaledi, Susanne Haussler, Nathiana Smit, Till Strowig, Ruben Garrido-Oter, Fantin Mesny, Stephane Hacquard, Paul Schulze-Lefert, Andreas Bremges, Alexander Sczyrba, Alice C. McHardy

Posted 12 Jul 2021
bioRxiv DOI: 10.1101/2021.07.12.451567

Evaluating metagenomic software is key for optimizing metagenome interpretation and focus of the community-driven initiative for the Critical Assessment of Metagenome Interpretation (CAMI). In its second challenge, CAMI engaged the community to assess their methods on realistic and complex metagenomic datasets with long and short reads, created from ~1,700 novel and known microbial genomes, as well as ~600 novel plasmids and viruses. Altogether 5,002 results by 76 program versions were analyzed, representing a 22x increase in results. Substantial improvements were seen in metagenome assembly, some due to using long-read data. The presence of related strains still was challenging for assembly and genome binning, as was assembly quality for the latter. Taxon profilers demonstrated a marked maturation, with taxon profilers and binners excelling at higher bacterial taxonomic ranks, but underperforming for viruses and archaea. Assessment of clinical pathogen detection techniques revealed a need to improve reproducibility. Analysis of program runtimes and memory usage identified highly efficient programs, including some top performers with other metrics. The CAMI II results identify current challenges, but also guide researchers in selecting methods for specific analyses.

Download data

  • Downloaded 2,897 times
  • Download rankings, all-time:
    • Site-wide: 7,256
    • In bioinformatics: 711
  • Year to date:
    • Site-wide: 2,951
  • Since beginning of last month:
    • Site-wide: 11,642

Altmetric data

Downloads over time

Distribution of downloads per paper, site-wide