ИСТИНА

This lecture is devoted to the advances in the application of neural networks with deep architecture in chemoinformatics (see major reviews [1-4]). It begins with the consideration of classical “shallow” neural networks and their main drawback - the unsolvability of the problem of vanishing gradients. The four new methodologies that have led to the solution of this problem are discussed: (1) unsupervised pretraining (e.g., RBM), (2) new activation functions (e.g., ReLU), (3) new regularization techniques (e.g., dropout), (4) new optimization algorithms (e.g., Adam). Thanks to this, it became possible to train neural networks with multiple hidden layers of neurons - deep neural networks. Deep learning can be defined as the application of artificial neural networks with multiple hidden layers that form multiple levels of representations corresponding to different levels of abstraction (see [5-7]). This definition is discussed in the lecture in the context of chemoinformatics applications. The importance of representation learning is underlined. Several benchmarking studies comparing deep vs shallow learning for building QSAR models are also considered. After that, several important methodologies rooted in deep learning are discussed in the context of their application in chemoinformatics: (1) energy-based neural networks [8], (2) autoencoders (e.g., variational autoencoder), (3) convolutional neural networks (1D-, 2D-, 3D-, atomic, graph-based, etc), (4) recurrent neural networks (e.g., LSTM), (5) generative adversarial networks (GANs), (6) deep reinforcement learning. Where possible, the connection with earlier studies (e.g., [9-11]) is discussed. At the end of the lecture, two breakthrough technologies are discussed in their application to chemoinformatics: generative modeling and natural language processing.