Master Study Guide: Visual Data Science Fundamentals

Module 1: Data Foundations and Structures

1.1 The Data Science Process

The foundational pipeline for visual data science begins with real-world data acquisition and proceeds through several distinct stages:

1. Raw Data Collection
2. Data Processing
3. Clean Data Generation
4. Exploratory Data Analysis (EDA)
5. Machine Learning Algorithms & Statistical Modelling
6. Communication and Visualization of Insights.

1.2 Data Pre-processing and Reduction

Raw data often contains noise and missing values, requiring cleaning steps such as rescaling, weighing, and normalization. Because visualizing large datasets is computationally and perceptually challenging, data reduction strategies are employed:

• Filtering: Eliminating items or attributes based on thresholds or user-driven navigation.
• Sampling: Taking random subsets or every $n$ -th item.
• Aggregation: Representing groups of items using metrics like average, min/max, or sum (e.g., binning or categorical clustering).
• Projection: Reducing the dimensionality of the dataset rather than the number of items.

1.3 Data and Dataset Types

Data is structurally interpreted through primitives:

• Items: Discrete individual entities (e.g., a person or device).
• Attributes: Measured or logged properties (variables/dimensions).
• Links: Relationships between items.
• Positions: Spatial data coordinates.
• Grids: Sampling strategies for continuous data.

These combine to form complete dataset types: Tables, Networks (Graphs/Trees), Fields (Continuous grids like scalar or tensor fields), and Geometry (Spatial/Geospatial data).

1.4 Attribute Semantics

Attributes describe items and dictate how they can be mathematically manipulated or visualized:

• Categorical (Nominal): Operations are limited to comparing equality; they have no inherent order (e.g., product category).
• Ordered (Ordinal): Sorting is defined, determining greater/less than, but arithmetic is not supported (e.g., T-shirt sizes).
• Quantitative (Numeric): Supports full arithmetic operations (e.g., length, weight).

Ordering direction defines the scale's mapping:

• Sequential: Ordered series, indexed by the natural numbers $\mathbb{N}$ or mapped to the positive reals $\mathbb{R}^+$.
• Diverging: Two sequential series meeting at a common zero point, indexed by integers $\mathbb{Z}$ or mapped to reals $\mathbb{R}$.
• Cyclic: Looping sequences capturing periodicity, mathematically mapped to angles $0$ to $2\pi$.

Module 2: Visual Encoding and Perception

2.1 Marks and Channels

Visual encoding translates abstract data into graphical representations using Marks and Channels.

• Marks: Geometric primitives (Points, Lines, Areas) that depict items or links.
• Channels: Visual variables that control the appearance of marks (e.g., position, colour hue, saturation, size).

2.2 Expressiveness and Effectiveness

The Expressiveness Principle mandates that the visual channel type must correctly match the data attribute type.

• Magnitude Channels: Used for ordered/quantitative attributes (e.g., position on an aligned scale, length).
• Identity Channels: Used for categorical attributes (e.g., spatial region, colour hue, shape).

2.3 Perception vs. Cognition

• Perception is the reflexive, unconscious identification of sensory information processed in the visual cortex.
• Cognition refers to the active processing of information, applying knowledge, and drawing relationships between objects.
• Gestalt Principles: Patterns of human perception, such as Continuity (aligned objects appear as a continuous group) and Connection (connected objects are grouped together, overriding most other principles).

Module 3: View Interaction and Dashboards

3.1 Dealing with Large Datasets

When the number of data points exceeds the available pixels ( $n_{points} > n_{pixels}$ ), interaction and partitioning are required.

• Temporal Partitioning: Changing the view over time through navigation (Pan, Zoom, Rotate).
• Spatial Partitioning: Faceting data into multiple coordinated views (Juxtaposition, Superposition).
• Visual Information Seeking Mantra: "Overview first, zoom and filter, then details-on-demand".

3.2 Coordinated Multiple Views (Dashboards)

Views can share encodings, data subsets, or navigation controls.

• Small Multiples: Same visual encoding, distinct (none-overlapping) subsets of data.
• Overview/Detail: Same or multiform encoding showing a subset of data focused within the context of all data.

Module 4: Visual Data Analysis and Machine Learning

4.1 Supervised vs. Unsupervised Learning

• Supervised Learning: Relies on training samples to classify or predict unseen data (e.g., k-NN classification, regression modelling).
• Unsupervised Learning: Extracts structure from data without prior knowledge (e.g., Cluster analysis, Dimensionality Reduction).

4.2 Dimensionality Reduction (DR)

Dimensionality reduction transforms high-dimensional data into a lower-dimensional space (1D, 2D, or 3D) to reveal hidden structures and clusters.

Mathematical Formulation:
Consider a dataset represented as a matrix $X$ of size $m \times n$, where $m$ represents rows (data points) and $n$ represents columns (features). Dimensionality reduction maps $X$ to a new matrix $X'$ of size $m \times d$, where $d < n$.

If $X$ is an $n \times n$ square matrix and we set $d=2$, the transformation yields a tall, rectangular matrix:

Each $n$ -dimensional vector is effectively embedded as a $d$ -dimensional coordinate vector suitable for spatial mapping (e.g., PCA, t-SNE, UMAP).

Module 5: Collaboration, Storytelling, and Evaluation

5.1 Computer-Supported Cooperative Work (CSCW)

The CSCW matrix categorizes collaborative tools into four quadrants based on synchronization:

1. Same Time / Same Place: Face-to-face interactions (e.g., shared tables, wall displays).
2. Same Time / Different Place: Remote synchronous interactions (e.g., video conferencing, shared screens).
3. Different Time / Same Place: Continuous local tasks (e.g., team rooms).
4. Different Time / Different Place: Asynchronous communication (e.g., version control, wikis).

5.2 Data-Driven Storytelling

Storytelling structures data into a cohesive narrative (Introduction, Problem, Climax, Resolution, Conclusion) to provide context and engagement.

• Martini Glass Structure: Begins with an author-driven narrative and eventually opens up into an exploratory interface.
• Interactive Slideshow: Splits the narrative into scenes with limited mid-way interaction.
• Drill-Down Story: Allows the reader to select annotated paths.

5.3 Evaluation

Visual Data Analysis is highly difficult to evaluate because analytical insights are non-atomic and lack straightforward baselines (e.g., counting "insights per minute" is invalid).

• Case Studies: Serve to demonstrate the efficacy of a tool by presenting detailed usage scenarios where domain experts discover valid, meaningful patterns in real datasets.