Hands on AI II

Module 1: Machine Learning Fundamentals & Neural Networks

Data Representation

• Tabular Data: Organized in rows (samples) and columns (features). Described by a feature vector $x = (x_1, \dots, x_d)$ and a label $y$. A full dataset of $n$ samples is represented by a feature matrix $X$ and label vector $y$:
  $$X = \begin{bmatrix} x_{11} & \dots & x_{1d} \\ \dots & \dots & \dots \\ x_{n1} & \dots & x_{nd} \end{bmatrix}, y = \begin{bmatrix} y_1 \\ \dots \\ y_n \end{bmatrix}$$
• Images: Represented in 3 dimensions: Height, Width, and Channels (e.g., RGB).
• Sequences: Ordered elements $a_1, a_2, \dots, a_n$ useful for time series, molecule design, or text.

Supervised vs. Unsupervised Learning

• Unsupervised Learning: Learns structure without target labels (e.g., k-Means, Hierarchical Clustering, DBSCAN, Gaussian Mixture Models).
• Supervised Learning: Learns mapping from inputs to target labels (Classification for discrete labels, Regression for numerical labels).

Empirical Risk Minimization (ERM) & Loss

The generalization error (risk) over a joint probability distribution $p(x, y)$ is:

$$R(g(.;w)) = \int_X \int L(y, g(x;w)) \cdot p(x, y) dy dx$$

In practice, we minimize the empirical risk on our dataset of size $n$:

$$R_E(g(.;w), (X, y)) = \frac{1}{n} \sum_{i=1}^n L(y_i, g(x_i;w))$$

Linear and Logistic Regression

• Linear Regression: Predicts numerical values. Uses Mean-Squared Error (MSE) loss:
  $$L = \frac{1}{n} \sum_{i=1}^n (y_i - g(x_i;w))^2$$
• Logistic Regression: Applies the sigmoid/logistic function to a linear transformation for binary classification, yielding probabilities $y \in$:
  $$\sigma(z) = \frac{1}{1 + e^{-z}}$$
  where $z = b + w \cdot x$.

Softmax Function
Generalization of sigmoid for $K$ multi-class classification. The probability of $x$ belonging to class $i$ is:

$$p(y = i | x) = \sigma(z)_i = \frac{e^{z_i}}{\sum_{j=1}^K e^{z_j}}$$

Neural Networks & Convolutional Neural Networks (CNNs)

• Feed-Forward Networks: Map inputs through hidden layers using weight matrices $W$ and bias vectors $b$ combined with a non-linear activation function $f$:
  $$h(x) = f(b + Wx)$$
• CNNs: Utilize receptive fields and shared weight matrices (kernels). Reusing kernels across image positions provides translation invariance and reduces parameters. Layers include convolutions, non-linearities, and pooling (e.g., Average Pooling, Max Pooling).

Module 2: Network Optimization & Vanishing Gradients

Gradient Descent
Iterative method to minimize the loss function $L(.; \theta)$ by taking steps proportional to the negative gradient, with learning rate $\eta$:

$$\theta_{n+1} = \theta_n - \eta \frac{\partial L}{\partial \theta}$$

Regularization
Technique to prevent overfitting.

• L1 Regularization:
  $$L = L + \lambda \sum_i |w_i|$$
• L2 Regularization (Weight Decay):
  $$L = L + \lambda \sum_i w_i^2$$

Backpropagation and the Chain Rule
For nested functions $y = f_1(f_2(\dots f_n(x) \dots))$, the derivative is:

$$\frac{\partial y}{\partial x} = \frac{\partial f_1}{\partial f_2} \frac{\partial f_2}{\partial f_3} \dots \frac{\partial f_{n-1}}{\partial f_n} \frac{\partial f_n}{\partial x}$$

Derivatives of Activation Functions:

• Sigmoid Derivative:
  $$\sigma'(x) = \sigma(x) \cdot (1 - \sigma(x))$$
  Since $\sigma'(x) \leq 0.25$, multiplying many gradients in deep networks shrinks the gradient toward zero (Vanishing Gradient Problem).
• ReLU Derivative:
  $$ReLU(x) = \begin{cases} x & \text{if } x > 0 \\ 0 & \text{if } x \leq 0 \end{cases}, \quad ReLU'(x) = \begin{cases} 1 & \text{if } x > 0 \\ 0 & \text{if } x \leq 0 \end{cases}$$

Module 3: Recurrent Neural Networks (RNNs) & LSTMs

Standard RNNs
Designed for sequence data $s = (x_1, \dots, x_T)$. The hidden state $h_t$ at timestep $t$ concatenates the previous output $h_{t-1}$ with current input $x_t$:

$$h_t = W \cdot \begin{bmatrix} h_{t-1} \\ x_t \end{bmatrix} + b$$

Trained via Back Propagation Through Time (BPTT). Suffers heavily from vanishing/exploding gradients due to sequential unrolling.

Long Short-Term Memory (LSTM)
Mitigates vanishing gradients using a Constant Error Carousel (CEC) and gates:

• Forget Gate: Decides what to erase from $c_{t-1}$.
  $$f_t = \sigma(W_f x_t + R_f h_{t-1} + b_f)$$
• Input Gate: Decides what to write into $c_t$.
  $$i_t = \sigma(W_i x_t + R_i h_{t-1} + b_i)$$
• Output Gate: Decides what to read from $c_t$ to form $h_t$.
  $$o_t = \sigma(W_o x_t + R_o h_{t-1} + b_o)$$
• Cell Candidate:
  $$\tilde{c}_t = \tanh(W_c x_t + R_c h_{t-1} + b_c)$$
• Cell State Update: (Identity activation, preserving gradients)
  $$c_t = i_t \odot \tilde{c}_t + f_t \odot c_{t-1}$$
• Hidden State Output:
  $$h_t = o_t \odot \tanh(c_t)$$

Module 4: Natural Language Processing & Transformers

Language Modeling
The probability of the next word $x^{(t+1)}$ given previous words is modeled as:

$$P(x^{(t+1)} | x^{(t)}, x^{(t-1)}, \dots, x^{(1)})$$

• Negative Log Likelihood Loss:
  $$L^{(t)} = -\log \hat{y}_{x^{(t+1)}}^{(t)} = -\log P(x^{(t+1)} | x^{(t)}, \dots, x^{(1)})$$
  Overall loss over sequence of length $L$:
  $$L = \frac{1}{L} \sum_{t=1}^L L^{(t)}$$
• Perplexity:
  $$perplexity = \exp(L)$$

Transformers and Attention
Instead of recurrent sequential steps, transformers use self-attention to globally weigh token relevance, enabling full parallelization:

$$Attention(Q, K, V) = softmax(\frac{QK^T}{\sqrt{d_k}})V$$

Module 5: Machine Learning in Drug Discovery

• Quantitative Structure-Activity Relationship (QSAR): Uses ML to predict bioactivity based on numerical representations of molecular structure (e.g., SMILES strings, MOL/SDF graphs).
• Generative Models: Autoencoders (VAE), GANs, and RL are used to explore the chemical space of $10^{60}$ potential molecules to perform inverse molecular design.
• Protein Folding: Mapping amino acid sequences to 3D conformations (e.g., AlphaFold).

Module 6: Reinforcement Learning

Markov Decision Process (MDP)
Defined by States ( $S$ ), Actions ( $A$ ), Transition probabilities ( $P_a(s, s')$ ), and Rewards ( $R_a(s, s')$ ). The agent learns a policy $a = \pi(s)$ to maximize cumulative reward.

Q-Learning
Approximates the Q-Value Function $Q(s, a)$, the expected reward of taking action $a$ in state $s$ and acting optimally thereafter. Update rule:

$$Q(s_t, a_t) \leftarrow (1 - \alpha) \cdot Q(s_t, a_t) + \alpha \cdot (r_t + \gamma \max_{a_{t+1}} Q(s_{t+1}, a_{t+1}))$$