Rxivist logo

Rxivist combines preprints from bioRxiv with data from Twitter to help you find the papers being discussed in your field. Currently indexing 62,198 bioRxiv papers from 276,130 authors.

Separable Fully Connected Layers Improve Deep Learning Models For Genomics

By Amr Mohamed Alexandari, Avanti Shrikumar, Anshul Kundaje

Posted 05 Jun 2017
bioRxiv DOI: 10.1101/146431

Convolutional neural networks are rapidly gaining popularity in regulatory genomics. Typically, these networks have a stack of convolutional and pooling layers, followed by one or more fully connected layers. In genomics, the same positional patterns are often present across multiple convolutional channels. Therefore, in current state-of-the-art networks, there exists significant redundancy in the representations learned by standard fully connected layers. We present a new separable fully connected layer that learns a weights tensor that is the outer product of positional weights and cross-channel weights, thereby allowing the same positional patterns to be applied across multiple convolutional channels. Decomposing positional and cross-channel weights further enables us to readily impose biologically-inspired constraints on positional weights, such as symmetry. We also propose a novel regularizer and constraint that act on curvature in the positional weights. Using experiments on simulated and in vivo datasets, we show that networks that incorporate our separable fully connected layer outperform conventional models with analogous architectures and the same number of parameters. Additionally, our networks are more robust to hyperparameter tuning, have more informative gradients, and produce importance scores that are more consistent with known biology than conventional deep neural networks. Availability: Implementation: https://github.com/kundajelab/keras/tree/keras_1. A gist illustrating model setup is at: goo.gl/gYooaa.

Download data

  • Downloaded 2,762 times
  • Download rankings, all-time:
    • Site-wide: 1,243 out of 62,198
    • In bioinformatics: 257 out of 6,210
  • Year to date:
    • Site-wide: 12,918 out of 62,198
  • Since beginning of last month:
    • Site-wide: None out of 62,198

Altmetric data

Downloads over time

Distribution of downloads per paper, site-wide

Sign up for the Rxivist weekly newsletter! (Click here for more details.)