Data Manipulation Essentials

Week	Topic	Before the lecture	Slides	Practice	Assignment	Additional reading
1	The foundation under data analysis What one row means, what one column holds, N/O/Q/T, grain		Slides	Repository		PDA ch. 1 — Preliminaries (web) PDA ch. 5, 5.1–5.3 — Getting Started with pandas (web) PPDW ch. 1 — Introduction to Data Wrangling and Data Quality (O’Reilly) PPDW ch. 3 — Understanding Data Quality (O’Reilly)
2	DataFrames and profiling Indexing, filtering, sorting, derived columns, type conversion, duplicates, a systematic profiling method	Notebooks	Slides	Hub		PDA 5.2 — Essential Functionality (web) PPDW ch. 6 — Assessing Data Quality (O’Reilly)
3	Aggregation and grouping Split-apply-combine, aggregation traps, average-of-averages, non-additive measures	Notebook	Slides	Hub		PDA ch. 10 — Data Aggregation and Group Operations (web)
4	Text and missing values String methods, regular expressions, the rows a rule does not fit, NaN behaviour and strategy	Notebooks	Slides	TBD	A1 released	PDA 7.1 — Handling Missing Data (web) PDA 7.4 — String Manipulation (web) PPDW ch. 7 — Cleaning, Transforming, and Augmenting Data (O’Reilly)
5	Tidy data and reshaping Wide and long, time as grain, resampling, categorical types. Week 5 is the mid-term	Notebooks	Mid-term	TBD	A1 due	PDA 7.5 — Categorical Data (web) PDA 8.3 — Reshaping and Pivoting (web) PDA 11.2 — Time Series Basics (web) PDA 11.6 — Resampling and Frequency Conversion (web)
6	Data formats CSV, JSON, Excel, Parquet; nested JSON; reading from an API	Notebooks	Slides	TBD	A2 released A2 due	PDA ch. 6 — Data Loading, Storage, and File Formats (web) PPDW ch. 4 — Working with File-Based and Feed-Based Data in Python (O’Reilly) PPDW ch. 5 — Accessing Web-Based Data (O’Reilly)
7	Joining tables Inner, left, right and outer joins, rows lost or added in a join, join validation, keys and identity	Notebook	Slides	TBD	A3 released A3 due	PDA ch. 7 — Data Cleaning and Preparation (web) PDA 8.1 — Hierarchical Indexing (web) PDA 8.2 — Combining and Merging Datasets (web) PPDW ch. 6 — Assessing Data Quality (O’Reilly) PPDW ch. 7 — Cleaning, Transforming, and Augmenting Data (O’Reilly) PPDW ch. 8 — Structuring and Refactoring Your Code (O’Reilly)
8	Synthesis and review Where these skills go next, exam preparation. The final exam is in this week, after the practice sessions	TBD	TBD	TBD		PDA ch. 13 — Data Analysis Examples (web) PPDW ch. 9 — Introduction to Data Analysis (O’Reilly)

PDA is Python for Data Analysis, 3rd edition by Wes McKinney, free to read on the web. PPDW is Practical Python Data Wrangling and Data Quality by Susan E. McGregor, on O’Reilly.

The final exam date is on the timetable. Deadlines for practices and assignments are on Moodle.