Data Science Education in the AI Era

Data Science in the AI Era: How AI changes the workplace

Eric Van Dusen
October 2026
github.com/ericvd-ucb

Data Science as a framework for teaching and learning in the AI era

A new generation of AI tools is changing how people work with data and code. Data Science curricula can help students adapt to these changes.

Transferable skills

Computational skills that carry over to new tools and APIs.

Authentic practice

Experience with the workflows professionals use.

Domain expertise

Interdisciplinary knowledge that guides which problems to solve.

Continued learning

A foundation for teaching yourself what comes next.

Inference Insight Computational Thinking

Learning a new Python package

Read the docs
Supply data
Configure
Run
Interpret

In Econ 148, students meet new data and ML packages every few weeks and learn to use them productively.
- NumPy → statsmodels → scikit-learn
AI tools expose similar Python interfaces and APIs, and the same pattern applies.
- PyTorch, TensorFlow, the OpenAI API

Moving through the Data Science major is a progression of learning new Python packages.

One Econ 148 lecture, nine packages

pandas: data frames
matplotlib: plotting
numpy: arrays and math
yfinance: market data
statsmodels: ARIMA
xgboost: gradient boosting
prophet: trend and seasonality
Kronos: a financial foundation model
OpenRouter: an LLM inference API

Each package has its own documentation and arguments, but the workflow is the same.
Calling a foundation model or an LLM API looks much like fitting ARIMA.

Econ 148 notebook: “Can the Model Beat the Market?”

From the Data Science lifecycle to MLOps

Data Science lifecycle

Collect data
Prepare
Model
Evaluate
Iterate

AI Harness Engineering

Parse data into a tokenized format
Data pipelines
Inference pipelines
Parse and manage outputs
Update and Iterate

Data Science curricula can build on this foundation by introducing MLOps and AI operations.

Debugging and evaluating AI workflows

Currently debugging is a big part of using AI tool. Problems can come from any part of a workflow, not only the code.

Data quality
Joins
Transformations
Code
Models
Interpretation

Things break and you have to problem solve to fix them.
Check assumptions, inspect intermediate results, question unexpected outputs.
AI-generated code and model outputs still require validation.
Test driven development and reproducibility.

Data engineering as a foundation for ML/AI

AI applications: RAG, embeddings, enterprise tools
Models
Analysis
Pipelines and transformations
SQL and databases
Organizational data

Many Data Science graduates begin professional work with SQL, databases and data transformations.

AI may automate some query writing. Understanding an organization’s data and keeping pipelines reliable remain essential.

Metacognition 1: Notebook-based instruction

Code, narrative, data, visualizations and results in one environment
Students start with real questions and working examples, then modify them
Programming and math concepts arrive in the context of a substantive problem

Computational and Inferential Thinking, the open Data 8 textbook

inferentialthinking.com

Is notebook-based instruction a change in how we teach computing?

Metacognition 2: Why are we learning this way?

Jupyter, NumPy, Python APIs and cloud environments are not only classroom conveniences.

They are part of the scientific and professional computing ecosystem.

Students learn to explore, troubleshoot, adapt and build with the same tools professionals use.
Students can adapt to new technologies as they emerge.
Faculty need to keep evolving the curriculum to reflect changes in the professional ecosystem.

Metacognition 3: Teaching through the open-source stack

Data Science courses teach with open-source tools, so students learn the same stack used in research and industry.

Origins

IPython and Jupyter grew out of scientific computing and open research communities.

An open stack

Python, NumPy, pandas and Jupyter are open-source projects maintained by communities.

Reproducibility

Notebooks make methods and outputs shareable. Reproducing them also depends on environments, dependencies, data access and execution order.

Fernando Pérez and IPython

From Medellín; studied at Universidad de Antioquia
PhD in Physics at CU Boulder, where he took time from his dissertation to write IPython
Professor of Statistics at UC Berkeley
Now leading work to bring Jupyter to new fields
- GIS, biology and health

github.com/fperez

Everyone can participate in open source

Open-source projects depend on participants, and many contributions do not require writing code.

Code
Documentation
Bug reports
Tutorials
Answering questions
Supporting new users

I have learned a lot by participating in Project Jupyter. My outreach to the education community has made me feel like a member of the Jupyter community.

The current journey of learning about LLM and AI and there are incredible communities around open source tools. It makes it exciting to learn.

Juanita Gómez: a path through open source

Undergraduate in pure mathematics at Universidad Javeriana, Colombia
Core developer for the Spyder IDE at Quansight
Community manager for the Scientific Python Project
PhD candidate in Computer Science at UC Santa Cruz
Open Source Sustainability Program Manager at NVIDIA

github.com/juanis2112

GitHub Open Source and AI

GitHub hosts much of the world’s collaborative software development and open-source work.
GitHub Copilot and related tools bring AI assistance into the places where software is built.
AI is being added to established collaborative workflows rather than replacing them.
Econ 148 course materials and assignments use GitHub.

GitHub by the numbers

180M+

Developers on GitHub; 36M joined in 2025

395M

Public and open-source repositories, up 19% in a year

4.3M+

AI-related repositories, nearly double in under two years

50M

GitHub Copilot users, July 2026

Entrance tunnel at GitHub Universe with the Octocat at the end Eric Van Dusen in front of the Octocat sculpture at GitHub Universe, San Francisco

GitHub Universe, San Francisco

Latin America added about 3.2 million developers in 2025, and GitHub names Colombia among its stand-out markets.

GitHub Octoverse 2025; Microsoft FY2026 Q4 earnings call, July 29, 2026

Databricks: Big Data infrastructure

AMPLab
Apache Spark
Databricks
Enterprise AI

Databricks grew out of Berkeley’s AMPLab and the Apache Spark project.
Its early work focused on large-scale data processing and analytics.
Enterprise AI applications create new demand for platforms that connect an organization’s data to models.

The Berkeley connection shows the continuity between data engineering and AI.

Databricks in 2026

$190B

Valuation after a $5B round led by Coatue, August 2026

$7B+

Revenue run-rate, growing more than 80% year over year

20,000+

Organizations worldwide use the platform

70%

Of the Fortune 500 are customers

The AI revolution did not make big data obsolete. It made big data infrastructure even more important.

Databricks press release, August 13, 2026

Alpaca Python API trading

Alpaca built an API-first brokerage for trading.
Python-based trading tools used to require a fairly specialized set of skills.
AI coding assistants make it easier for more people to experiment with APIs and build applications.
Understanding finance, data, testing and risk still matters when code is easier to generate.

Alpaca in 2026

$1.15B

Series D valuation, January 2026

9M+

Brokerage accounts powered through partners, not unique users

~4×

Growth in monthly active API users over six months, to July 2026

300+

Partner organizations in more than 40 countries

Alpaca attributes much of its recent API growth to AI agents connecting to its trading infrastructure.

Alpaca press releases, Business Wire, January 14 and July 16, 2026

Interdisciplinarity! - Built into the Data Science major

Berkeley built disciplinary expertise into the Data Science major through the Domain Emphasis requirement.

Economics
Biology
Physics
Public policy

Students connect computational skills with substantive knowledge in a field.
AI is becoming relevant to research and practice across disciplines.

Domain expertise as a comparative advantage

AI assistants are making coding and prototyping more accessible.
Identifying worthwhile problems requires knowing a field, its institutions and its constraints.
Domain expertise helps decide which data matter and whether results are credible.

Students who combine domain knowledge, Data Science and AI fluency can build distinctive applications.

Ethics and social context

Data and AI systems operate within social and institutional contexts.

Data and models reflect choices about measurement, representation and outcomes.
Privacy, fairness, bias, accountability and social impact belong within technical education.
AI raises further questions about transparency and concentrated technological power.
These questions apply in industry, science, government and nonprofit work.

Professional responsibility in AI

Data Science graduates can help shape how AI systems are evaluated and governed.

Evaluate performance, reliability and limitations
Test products for bias and unequal impacts
Promote documentation, reproducibility and auditing
Help build professional norms and safeguards
Recognize when to raise concerns or challenge decisions

What should a professional do when they understand a system well enough to see that it may cause harm?

Continuing to learn throughout your career

AI frameworks and professional tools will continue to change.
Students already practice reading documentation, trying examples, debugging and adapting workflows.
My recent side project learning small open-weight language models followed the same process.
The next important package or workflow may not exist yet.

You have learned how to teach yourself new computational tools. Keep Learning.

ericvd@berkeley.edu · github.com/ericvd-ucb

Building a GitHub portfolio

A portfolio documents what you built and how you thought about it.

Pinned projects, each with a short description
A record of contributions over time
Work in other people’s projects and organizations

Pinned repositories and contribution graph on Juanita Gómez's GitHub profile

github.com/juanis2112

Build a GitHub portfolio and keep improving it. A well-explained project tells an employer more than code alone.