Neural Network Compression by Low Rank Approximation

This is a very technical topic, which I would be interested to explore. It involves:

  • neural networks
  • low rank matrix approximation

Here the idea is to speed up neural network inference and maybe even training by approximating fully connected layers (i.e. matrices) by low-rank approximations of them.

WARNING: This is again a very mathematical topic.
References to be collected: