Basic concepts

• Definition of “learning”
• Continuous and discrete data
• Data imputation
• Model evaluation
• Cross-validation
• Hyper-parameters

Introduction to Supervised Learning

• Decision tree
• k-nearest neighbors (kNN)
• Ensembles
• Random forest
• Gradient boosting (XGBoost)

Introduction to Unsupervised Learning

• Principal component analysis
• k-means and k-medoids clustering

Neural Network (NN) Essentials

• Universal approximation theorem
• Perceptron
• Activation functions
• Multi-layer perceptron (MLP)
• NNs in supervised learning
• NNs in unsupervised learning

Regularization

• Ridge regression
• LASSO (Least Absolute Shrinkage and Selection Operator)

Neural Networks in Low Data Regime

• Transfer learning
• Self-supervision
• Data augmentation

Time Series Modelling

• Discrete-time NNs (RNN, LSTM, GRU)
• Neural ordinary differential equations (NeuralODEs)
• Universal differential equations (UDEs)
• Scientific machine learning (SciML) for pharmacometrics

Explainability

• SHAPELY (SHapley Additive exPlanations)
• LIME (Local Interpretable Model-agnostic Explanations)

Generative models

• Probabilistic principal component analysis
• Nonlinear mixed effects (NLME) models as generative models
• Variational autoencoder (VAE) as NLME + amortized learning
• Dimension reduction and clustering using generative models
• Normalizing flows

Conditional Generative Models

• Conditional VAE
• Scientific NLME models as conditional generative models
• Generative neural ODEs
• DeepNLME

Model Fitting and Evaluation

• Understanding the marginal likelihood
• Information criteria vs cross-validation

Generative adversarial networks (GANs)

• Adversarial learning and Jensen-Shannon divergence
• Optimal transport and Wasserstein GAN