Schedule

Please note that this schedule is subject to change.

Calendar

Date	Topic	Readings	Assignments
1/18	Introduction
1/23	Python	McKinney, Chs. 1 & 2
1/25	Python	McKinney, Chs. 3	A1
1/30	Relational Databases
2/01	Relational Databases
2/06	Structured Data	McKinney, Chs. 4	A2
2/08	Data and Pandas	McKinney, Chs. 5 & 6
2/13	Pandas and DuckDB
2/15	Data Wrangling	Kandel et al.
2/20	Data Wrangling	Jin et al.
2/22	Data Transformation	Yan & He	A3
2/27	Test 1
3/01	Data Cleaning	Rekatsinas et al.
3/06	Data Integration	Stonebraker & Ilyas
3/08	Data Fusion	Dong et al.
3/13	No Class
3/15	No Class
3/20	Scalable Databases	Gessert et al.
3/22	Scalable Databases	Pavlo & Aslett	A4
3/27	Scalable Dataframes	Petersohn et al.
3/29	Scalable Dataframes	Jindal et al.
4/03	Time Series Data	Pelkonen et al.
4/05	Graph Data	Sahu et al.
4/10	Test 2
4/12	Databases and Visualization	Moritz et al.	A5
4/17	Spatial Data	Eldawy et al.
4/19	Data Curation	Wilkinson et al.
4/24	Provenance	Chapman et al.
4/26	Reproducibility	Collberg & Proebsting
5/01	Databases and Machine Learning	Kraska et al.
5/03	Review
5/10	Final Exam (8-9:50am)

Lectures

(01/18) Introduction

• Slides

(01/23) Python

• Slides
• Reading:
  • Python for Data Analysis, Chs. 1 & 2
• Notebook: Download, View

(01/25) Python

• Slides
• Reading:
  • Python for Data Analysis, Ch. 3
• Notebook: Download, View

(01/30) Databases

• Slides
• References:
  • Database Systems Concepts, Silberschatz, Korth, and Sudarshan
• Notebook: Download, View

(02/01) Relational Databases

• Slides
• References:
  • Database Systems Concepts, Silberschatz, Korth, and Sudarshan

(02/06) Structured Data

• Slides
• Reading:
  • Python for Data Analysis, Ch. 4
• Notebook: Download, View

(02/08) Data and Pandas

• Slides
• Reading:
  • Python for Data Analysis, Chs. 5 & 6

(02/13) Pandas and DuckDB

• Slides
• Notebook: Download, View

(02/15) Data Wrangling

• Slides
• Reading:
  • Wrangler: Interactive Visual Specification of Data Transformation Scripts, S. Kandel et al., 2011
• References:
  • Original Wrangler Demo
  • Potter’s wheel: An interactive data cleaning system, V. Raman and J. M. Hellerstein, 2001.
  • Proactive Wrangling: Mixed-Initiative End-User Programming of Data Transformation Scripts, P. J. Guo et al., 2011
  • Predictive Interaction for Data Transformation, J. Heer et al., 2015
  • Trifacta Wrangler
  • Self-Service Data Preparation: Research to Practice, J. M. Hellerstein et al., 2018

(02/20) Data Wrangling

• Slides
• Reading:
  • Auto-Transform: Learning-to-Transform by Patterns
• References:
  • Foofah: Transforming Data By Example, Z. Jin et al., 2017
  • Spreadsheet Table Transformations from Examples, W. R. Harris and S. Gulwani, 2011.
  • Transform-Data-by-Example (TDE): An Extensible Search Engine for Data Transformations, Y. He et al., 2018

(02/22) Data Transformation

• Slides
• Reading:
  • Auto-Suggest: Learning-to-Recommend Data Preparation Steps Using Data Science Notebooks, C. Yan and Y. He, 2020
• References:
  • Wrex: A Unifed Programming-by-Example Interaction for Synthesizing Readable Code for Data Scientists
  • Spreadsheet Data Manipulation Using Examples, S. Gulwani et al., 2012
  • Tidy Data, H. Wickham, 2014
  • Component-Based Synthesis of Table Consolidation and Transformation Tasks from Examples, Y. Feng et al., 2017
  • Python for Data Analysis, Ch. 8

(03/01) Data Cleaning

• Slides
• Reading:
  • HoloClean: Holistic Data Repairs with Probabilistic Inference, T. Rekatsinas et al., 2017
• References:
  • SampleClean: Fast and Reliable Analytics on Dirty Data, S. Krishnan et al., 2015
  • Data Cleaning: Overview and Emerging Challenges, X. Chu et al., 2016
  • Python for Data Analysis, Ch. 7

(03/06) Data Integration

• Slides
• Reading:
  • Data Integration: The Current Status and the Way Forward, M. Stonebraker and I. Ilyas, 2018
• References:
  • Principles of Data Integration, AH Doan et al., 2012
  • Data Fusion—Resolving Data Conflicts in Integration, X. L. Dong and F. Naumann
  • Data Integration and Machine Learning: A Natural Synergy, X. L. Dong and T. Rekatsinas, 2018

(03/08) Data Fusion

• Slides
• Reading:
  • Integrating Conflicting Data: The Role of Source Dependence, Dong et al., 2009
• References:
  • Data Fusion, J. Bleiholder and F. Naumann, 2008
  • Entity Resolution, L. Getoor and A. Machanavajjhala, 2012
  • Record Linkage, P. Christen, 2019
  • Data Fusion—Resolving Data Conflicts in Integration, X. L. Dong and F. Naumann
  • Slides: Integrating Conflicting Data:The Role of Source Dependence, X. L. Dong, 2009

(03/20) Scalable Databases

• Slides
• Reading:
  • NoSQL Database Systems: A Survey and Decision Guidance, F. Gessert et al., 2017.
• References:
  • Cassandra - A Decentralized Structured Storage System, A. Lakshman and P. Malik, 2009.
  • Introduction to Apache Cassandra, R. Stupp
  • Introduction to Cassandra, G. Atil
  • A Survey on NoSQL Stores, A. Davoudian et al., 2020

(03/22) Scalable Databases

• Slides
• Reading:
  • What’s Really New with NewSQL?, A. Pavlo and M. Aslett, 2016
• References:
  • NewSQL, A. Pavlo, 2012.
  • The Official Ten-Year Retrospective of NewSQL, A. Pavlo, 2021.
  • Spanner: Google’s Globally-Distributed Database, J. C. Corbett et al., 2012.
  • F1: A Distributed SQL Database That Scales, J. Shute et al., 2013.
  • Spanner, TrueTime & The CAP Theorem, E. Brewer, 2017.
  • A Critique of the CAP Theorem, M. Kleppmann, 2015
  • Is Scalable OLTP in the Cloud a Solved Problem?, T. Ziegler et al., 2022

(03/27) Scalable Dataframes

• Slides
• Reading:
  • Towards Scalable Dataframe Systems, D. Petersohn et al., 2020
• References:
  • Scaling up your pandas workflows with Modin, D. Petersohn, 2022.
  • Spark SQL: Relational Data Processing in Spark, M. Armbrust et al., 2015
  • Flexible Rule-Based Decomposition and Metadata Independence in Modin: A Parallel Dataframe System, D. Petersohn et al., 2022
  • Beyond pandas: The great Python dataframe showdown, J. L. C. Rodríguez, 2022
  • Polars
  • Modin
  • Vaex
  • Dask

(03/29) Scalable Dataframes

• Slides
• Reading:
  • Magpie: Python at Speed and Scale using Cloud Backends, A. Jindal et al., 2021
• References:
  • ConnectorX: accelerating data loading from databases to dataframes, X. Wang et al., 2022
  • Magpie CIDR Slides, A. Jindal, 2021

(04/03) Time Series Data

• Slides
• Reading:
  • Gorilla: a fast, scalable, in-memory time series database, T. Pelkonen et al., 2015
• References:
  • Time Series Data Encoding for Efficient Storage: A Comparative Analysis in Apache IoTDB, J. Xiao et al., 2021
  • Time Series Management Systems: A Survey, S. K. Jensen et al., 2019

(04/05) Graph Data

• Slides
• Reading:
  • The Ubiquity of Large Graphs and Surprising Challenges of Graph Processing, S. Sahu et al., 2017
• References:
  • Introduction to Neo4j and Graph Databases, M. D. Allen, 2019.
  • Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph Queries (preprint), M. Besta et al., 2022.
  • Survey of Graph Database Models, R. Angles and C. Gutierrez, 2008.
  • An Introduction to Graph Data Management, R. Angles and C. Gutierrez, 2017
  • Graph Databases, D. Lembo and R. Rosati, 2015
  • Introduction to Graph Databases, M. De Marzi, 2012
  • The Future Is Big Graphs: A Community View on Graph Processing Systems, S. Sakr et al., 2021
  • The (sorry) State of Graph Database Systems, P. Boncz, 2022

(04/12) Databases and Visualization

• Slides
• Reading:
  • Falcon: Balancing Interactive Latency and Resolution Sensitivity for Scalable Linked Visualizations, D. Moritz et al., 2019
• References:
  • imMens: Real-time Visual Querying of Big Data, Z. Liu et al., 2013
  • A Structured Review of Data Management Technology for Interactive Visualization and Analysis, L. Battle and C. Scheidegger, 2021
  • Scalable Visualization, J. Heer, 2020

(04/17) Spatial Data

• Slides
• Reading:
  • Beast: Scalable Exploratory Analytics on Spatio-temporal Data, A. Eldawy et al., 2021
• References:
  • Big Spatial Data Management, A. Eldawy, 2020
  • Data Cubes, J. Han, M. Kamber, and J. Pei, 2011.
  • Nanocubes for Real-Time Exploration of Spatiotemporal Datasets, L. Lins et al., 2013.
  • TopKube: A Rank-Aware Data Cube for Real-Time Exploration of Spatiotemporal Datasets, F. Miranda et al., 2017.
  • Dynamic prefetching of data tiles for interactive visualization, L. Battle et al., 2016.

(04/19) Data Curation

• Slides
• Reading:
  • The FAIR Guiding Principles for scientific data management and stewardship, M. D. Wilkinson et al., 2016.
• References:
  • Big Data, Little Data, No Data: Scholarship in the Networked World, (eBook available through NIU Library) C. L. Borgman, 2015.
  • Research Data Infrastructure: A Problem of Governance, C. L. Borgman, 2022.

(04/24) Provenance

• Slides
• Reading:
  • Capturing and Querying Fine-grained Provenance of Preprocessing Pipelines in Data Science, A. Chapman et al., 2020
• References:
  • Provenance for Computational Tasks: A Survey, J. Freire et al., 2008
  • Provenance in Databases: Why, How, and Where, J. Cheney et al., 2007
  • Provenance in Databases, A. Amarilli, 2019.
  • Capturing and querying fine-grained provenance of preprocessing pipelines in data science, P. Missier, 2023.
• Notebook: Download, View

(04/26) Reproducibility

• Slides
• Reading:
  • Repeatability in computer systems research, C. Collberg and T. A. Proebsting, 2016.
• References:
  • Repeatability and Benefaction in Computer Systems Research, C. Collberg et al., 2015.
  • Reproducible Research in Computational Science, R. D. Peng, 2011.
  • Ten Simple Rules for Reproducible Computational Research, G. K. Sandve et al., 2013.
  • Computational Reproducibility: State-of-the-Art, Challenges, and Database Research Opportunities, J. Freire et al., 2012.
  • A Large-scale Study about Quality and Reproducibility of Jupyter Notebooks, J. F. Pimentel, 2019.

(05/01) Databases and Machine Learning

• Slides
• Reading:
  • SageDB: A Learned Database System, T. Kraska et al., 2019
• References:
  • Learning and Self-Designing Data Structures (SIGMOD Tutorial), S. Idreos and T. Kraska: Part 1 and Part 2
  • Benchmarking Learned Indexes, R. Marcus et al., 2021
  • Tsunami: A Learned Multi-dimensional Index for Correlated Data and Skewed Workloads, J. Ding et al., 2021
  • Bao: Making Learned Query Optimization Practical, R. Marcus et al., 2021

(05/03) Review

• Slides