Rxivist logo

Rxivist combines preprints from bioRxiv with data from Twitter to help you find the papers being discussed in your field. Currently indexing 73,782 bioRxiv papers from 321,061 authors.

Separable Fully Connected Layers Improve Deep Learning Models For Genomics

By Amr Alexandari, Avanti Shrikumar, Anshul Kundaje

Posted 05 Jun 2017
bioRxiv DOI: 10.1101/146431

Convolutional neural networks are rapidly gaining popularity in regulatory genomics. Typically, these networks have a stack of convolutional and pooling layers, followed by one or more fully connected layers. In genomics, the same positional patterns are often present across multiple convolutional channels. Therefore, in current state-of-the-art networks, there exists significant redundancy in the representations learned by standard fully connected layers. We present a new separable fully connected layer that learns a weights tensor that is the outer product of positional weights and cross-channel weights, thereby allowing the same positional patterns to be applied across multiple convolutional channels. Decomposing positional and cross-channel weights further enables us to readily impose biologically-inspired constraints on positional weights, such as symmetry. We also propose a novel regularizer and constraint that act on curvature in the positional weights. Using experiments on simulated and in vivo datasets, we show that networks that incorporate our separable fully connected layer outperform conventional models with analogous architectures and the same number of parameters. Additionally, our networks are more robust to hyperparameter tuning, have more informative gradients, and produce importance scores that are more consistent with known biology than conventional deep neural networks. Availability: Implementation: https://github.com/kundajelab/keras/tree/keras_1. A gist illustrating model setup is at: goo.gl/gYooaa.

Download data

  • Downloaded 2,872 times
  • Download rankings, all-time:
    • Site-wide: 1,435 out of 73,766
    • In bioinformatics: 283 out of 7,172
  • Year to date:
    • Site-wide: 14,800 out of 73,766
  • Since beginning of last month:
    • Site-wide: 14,800 out of 73,766

Altmetric data

Downloads over time

Distribution of downloads per paper, site-wide


Sign up for the Rxivist weekly newsletter! (Click here for more details.)