Rxivist logo

Rxivist combines preprints from bioRxiv with data from Twitter to help you find the papers being discussed in your field. Currently indexing 70,872 bioRxiv papers from 309,253 authors.

Addressing dereplication crisis: Taxonomy-free reduction of massive genome collections using embeddings of protein content

By A. Viehweger, Martin Hölzer, C. Brandt

Posted 25 Nov 2019
bioRxiv DOI: 10.1101/855262

Many recent microbial genome collections curate hundreds of thousands of genomes. This volume complicates many genomic analyses such as taxon assignment because the associated computational burden is substantial. However, the number of representatives of each species is highly skewed towards human pathogens and model organisms. Thus many genomes contain little additional information and could be removed. We created a frugal dereplication method that can reduce massive genome collections based on genome sequence alone, without the need for manual curation nor taxonomic information. We recently created a genome representation for bacteria and archaea called "nanotext". This method embeds each genome in a low-dimensional vector of numbers. Extending nanotext, our proposed algorithm called "thinspace" uses these vectors to group and dereplicate similar genomes. We dereplicated the Genome Taxonomy Database (GTDB) from about 150 thousand genomes to less than 22 thousand. The resulting collection increases the percent of classified reads in a metagenomic dataset by a factor of 5 compared to NCBI RefSeq and performs equal to both a larger as well as a manually curated GTDB subset. With thinspace, massive genome collections can be dereplicated on regular hardware, without affecting downstream results. It is released under a BSD-3 license (github.com/phiweger/thinspace).

Download data

  • Downloaded 283 times
  • Download rankings, all-time:
    • Site-wide: 41,041 out of 70,881
    • In bioinformatics: 4,856 out of 6,940
  • Year to date:
    • Site-wide: 4,532 out of 70,881
  • Since beginning of last month:
    • Site-wide: 1,382 out of 70,881

Altmetric data

Downloads over time

Distribution of downloads per paper, site-wide


Sign up for the Rxivist weekly newsletter! (Click here for more details.)