Curriculum Matrix Lecture Notes

Python II & Machine Learning Pipelines

Module 1: Advanced Python Constructs

Python offers functional programming paradigms, higher-order functions, and concurrency patterns that expand beyond basic script execution.

Lambda Functions

Lambda functions originate from Alonzo Church's λ\lambda-calculus and allow the definition of anonymous functions in Python. They follow the syntax lambda argument(s) : expression, meaning they can take multiple arguments but are strictly limited to evaluating a single expression. Lambdas are frequently utilized as arguments for higher-order functions like map(), filter(), and reduce(), or as the key argument in sorting operations to specify custom sorting criteria. For instance, sorting a list of objects by a specific attribute can be achieved via min(students, key=lambda s: s.age).

Callables

Callables are any Python objects that can be invoked like a function, including standard functions, methods, and class constructors. Additionally, instances of classes become callable if the class explicitly implements the __call__ magic method. This structure is advantageous when a function needs to retain an internal state or configuration without passing the state repeatedly as function arguments.

Decorators

Decorators provide syntactic sugar for nested function definitions, allowing the modification of a function's behaviour without changing its core code. When using @decorator above a function my_func(), it is computationally equivalent to my_func = decorator(my_func). Inside a decorator, a wrapper(*args, **kwargs) function is typically defined to execute operations before and after the original function is called. Common built-in decorators include @staticmethod, @classmethod, and the functools module's @cache (for returning cached values) and @wraps (for preserving docstrings and metadata).

Asynchronous Execution

Asynchronous functions enable concurrent execution within a single CPU process by pausing operations during I/O delays (like network requests or database interactions). Coroutines are declared using the async def syntax and can be paused explicitly using the await keyword to yield control back to the event loop. The asyncio.run(main()) method initializes the event loop, while asyncio.create_task() schedules a coroutine, and asyncio.gather() executes multiple tasks concurrently.

Module 2: Data Manipulation and Analysis

Pandas Core Data Structures

Pandas utilizes two primary data structures: the Series (a 1D ordered array with labels/indices) and the DataFrame (a 2D tabular structure containing rows and columns). Unlike NumPy arrays, DataFrames support different data types across different columns, though internally each column functions as a single-type Series. Indexing is achieved using df.loc[...] for label-based bounds (which includes the upper boundary) and df.iloc[...] for standard integer-based index bounds.

Tidy Data and Reshaping

Solid statistical analysis relies on "Tidy Data" principles: each variable forms a column, each observation forms a row, and each cell contains a single value. When data spans multiple columns for a single variable (a "wide" format), it can be reshaped into a "long" format using df.melt(). Conversely, df.pivot() or df.pivot_table() reorganizes long data back into a wide view.

Table Joins and Operations

Merging tables in Pandas works similarly to SQL joins via the merge() function.

  • Inner Join: Matches only rows with intersecting keys in both tables.
  • Left/Right Join: Retains all rows from the primary table, filling missing matches with NaN.
  • Outer Join: Combines all records from both tables, utilizing NaN for any missing values across the merge.
    Pandas also supports mathematical broadcasting across DataFrames, where methods like add() combine objects based on matching indices and columns, filling misaligned structures with NaN.

Module 3: Interactive Data Visualization

Matplotlib Foundations

Matplotlib structures visualizations using a Figure (the overarching window containing interaction tools) and Axes (the actual plotting regions). Image files produced or manipulated by Python can take several forms: raster-based formats using lossy compression (JPEG) or lossless compression (PNG), and vector-based graphics (SVG) which rely on mathematical coordinates rather than pixel arrays to prevent resolution loss.

Shiny Web Applications

Shiny transitions analytical scripts into interactive web applications without requiring raw HTML/CSS. It operates on reactive programming principles, utilizing a dependency graph that automatically updates user interfaces when input values are modified.
Inputs are captured via ui module functions (e.g., ui.input_slider()), and corresponding outputs are wrapped in reactive context decorators like @render.text or @render.plot. Shiny offers @reactive.calc to compute input-dependent values for use in other functions without redundant code execution, and @reactive.effect for triggering side effects.

Module 4: Neural Computation and PyTorch Optimization

Tensors and Network Architecture

PyTorch utilizes Tensors, which mirror NumPy arrays but support parallel processing on GPU VRAM and automatic differentiation. Neural architectures are built by inheriting from torch.nn.Module. Deep learning structures include Fully-Connected Feed-Forward Neural Networks (FFNNs), Convolutional Neural Networks (CNNs) which slide weight kernels over spatial dimensions, and Recurrent Neural Networks (RNNs) that process sequential data by utilizing hidden states from previous inputs.

Autograd and Gradient Descent

During training, neural networks aim to find weights θ\theta that minimize a loss function L(y^,y)L(\hat{y}, y), where y^\hat{y} is the network output g(x;θ)g(x; \theta) and yy is the true target. Because analytic global minimums cannot be directly solved, iterative gradient-based optimization is employed. The network updates weights by moving in the direction of steepest descent according to the equation:

θn+1=θn−η∂L∂θ\theta_{n+1} = \theta_n - \eta \frac{\partial L}{\partial \theta}

where η\eta represents the learning rate and ∂L∂θ\frac{\partial L}{\partial \theta} is the gradient of the loss with respect to the weights.
Gradients are automatically tracked via PyTorch's computational graph and computed using the loss.backward() method, after which an optimizer (like Stochastic Gradient Descent or Adam) applies the update via optimizer.step().

Module 5: Model Evaluation Frameworks

The Confusion Matrix

For binary classification, predictive performance is mapped on a Confusion Matrix tracking True Positives (TP), True Negatives (TN), False Positives (FP), and False Negatives (FN). Actual Positives are denoted as PP and Actual Negatives as NN.

Classification Metrics

Derived from the Confusion Matrix, the core rates are defined mathematically:

  • True Positive Rate (TPR / Sensitivity / Recall):

    TPR=TPTP+FN=TPPTPR = \frac{TP}{TP+FN} = \frac{TP}{P}

  • True Negative Rate (TNR / Specificity):

    TNR=TNTN+FP=TNNTNR = \frac{TN}{TN+FP} = \frac{TN}{N}

  • False Negative Rate (FNR / Miss Rate):

    FNR=FNFN+TP=FNP=1−TPRFNR = \frac{FN}{FN+TP} = \frac{FN}{P} = 1 - TPR

  • False Positive Rate (FPR / Fall-out):

    FPR=FPFP+TN=FPN=1−TNRFPR = \frac{FP}{FP+TN} = \frac{FP}{N} = 1 - TNR

Overall correctness is evaluated using Accuracy (ACC) and Balanced Accuracy (BACC) for imbalanced sets:

ACC=TP+TNTP+TN+FP+FN=TP+TNP+NACC = \frac{TP+TN}{TP+TN+FP+FN} = \frac{TP+TN}{P+N}

BACC=TPR+TNR2BACC = \frac{TPR+TNR}{2}

For multi-class setups involving nn classes, the metrics adjust to:

ACCmulti=∑TPi∑PiACC_{multi} = \frac{\sum TP_i}{\sum P_i}

BACCmulti=1n∑TPiPiBACC_{multi} = \frac{1}{n} \sum \frac{TP_i}{P_i}

Positive Predictive Value (PPV or Precision) evaluates the accuracy of positive predictions:

PPV=TPTP+FPPPV = \frac{TP}{TP+FP}

The Harmonic mean of Precision and Recall generates the F-score:

Fscore=2⋅PPV⋅TPRPPV+TPR=2TP2TP+FP+FNF_{score} = 2 \cdot \frac{PPV \cdot TPR}{PPV+TPR} = \frac{2TP}{2TP+FP+FN}

This can be weighted using the β\beta parameter to dictate the importance of Recall over Precision:

Fβ−score=(1+β2)⋅PPV⋅TPRβ2⋅PPV+TPRF_{\beta-score} = (1+\beta^2) \cdot \frac{PPV \cdot TPR}{\beta^2 \cdot PPV+TPR}

Regression Metrics

Continuous numerical prediction models require distinct error evaluations against nn samples, where yiy_i is the target, y^i\hat{y}_i is the prediction, and yˉi\bar{y}_i is the mean of the training target:

  • Mean Absolute Error (MAE):
    MAE=∑i=1n∣yi−y^i∣nMAE = \frac{\sum_{i=1}^n |y_i - \hat{y}_i|}{n}
  • Mean Squared Error (MSE):
    MSE=∑i=1n(yi−y^i)2nMSE = \frac{\sum_{i=1}^n (y_i - \hat{y}_i)^2}{n}
  • Relative Absolute Error:
    ∑i=1n∣yi−y^i∣∑i=1n∣yi−yˉi∣\frac{\sum_{i=1}^n |y_i - \hat{y}_i|}{\sum_{i=1}^n |y_i - \bar{y}_i|}
  • Relative Squared Error:
    ∑i=1n(yi−y^i)2∑i=1n(yi−yˉi)2\frac{\sum_{i=1}^n (y_i - \hat{y}_i)^2}{\sum_{i=1}^n (y_i - \bar{y}_i)^2}