Author: zaspel

This topic combines prior knowledge on kernel ridge regression with neural networks. The following content will be considered:

• (deep) neural networks
• kernel ridge regression
• radial basis function networks

The objective would be to study the relationship of the predictive power of kernel ridge regression and radial basis function networks based on given data from quantum chemistry or other relevant science application.

A few first links:

• https://www.hindawi.com/journals/isrn/2012/324194/
• https://link.springer.com/chapter/10.1007/978-3-7908-1826-0_1
• https://www.jmlr.org/proceedings/papers/v51/que16-supp.pdf
• https://link.springer.com/article/10.1134/S0965542517010079

This is a very technical topic, which I would be interested to explore. It involves:

• neural networks
• low rank matrix approximation

Here the idea is to speed up neural network inference and maybe even training by approximating fully connected layers (i.e. matrices) by low-rank approximations of them.

WARNING: This is again a very mathematical topic.
References to be collected:

• https://www.sciencedirect.com/science/article/pii/S0925231220302253

The topic of this project is the efficient training of Machine Learning by Kernel Ridge Regression.

Relevant content will be:

• Kernel Ridge Regression
• iterative solvers for linear systems
• matrix approximation techniques:
  • low rank approximation (SVD, ACA, …)
  • Askit
  • Hierarchical Matrices

Application data should be large-scale and science-related. Maybe the first starting point would be data from quantum chemistry that I have access to.
The beauty of this project would be to further develop and analyze the impact of non-exact solvers for linear systems on the quality of the prediction of Kernel Ridge Regression. This is highly research relevant.

WARNING: Some flavor of this topic (e.g. hierarchical matrices) requires a profound mathematical background.

Some first links:

• http://www.cs.nyu.edu/~mohri/pub/kst.pdf
• http://dept.stat.lsa.umich.edu/~jizhu/pubs/Mukherjee-SADM11.pdf
• https://arxiv.org/abs/1410.0260
• https://www.sciencedirect.com/science/article/abs/pii/S0955799702001522
• http://jmlr.org/papers/volume18/15-025/15-025.pdf