The Empirical Economist’s Toolkit

Research with AI Agents · Universidad Francisco Marroquín · Fall 2026

Author

Leonel A. Regalado Cardoso

About this workshop

One empirical project, from the first idea to the final talk, built with AI agents. Each part follows one step in the life of a paper, and everything you produce goes into one GitHub repository that grows every week.

Part	What we do	What you leave with	Status
1	Tools and literature	A repository and a verified reading list	Available
2	Data	A dataset anyone can rebuild	Available
3	Estimation	Working code you understand for the main methods	Available
4	Evaluation	A referee report on your own work	Coming
5	Presentation	A talk and a public portfolio	Coming

The AI does the labor. The economist does the judgment.

How to use this page. Each part has the material, the links you need, and the deliverable. The page grows as the workshop goes: new parts are added after each class. Use the contents on the left to jump between sections.

Running example. Throughout the workshop I rebuild one of my own papers from scratch: Powell and Regalado Cardoso (2025), “Emigration and Origin Country Economic Institutions,” Public Choice. The question: does emigration change the economic institutions of the country people leave? You follow the same steps with your own question.

Before you start: setup

Install these before Part 1. If something fails, write to me before class.

Tool	What it is for	Where to get it
Claude Code	The AI agent we use in every part	docs.claude.com/en/docs/claude-code
GitHub account	Where your project lives	github.com
GitHub CLI	Lets the agent create and update your repository. After installing, run `gh auth login` once	cli.github.com
Wispr Flow	Dictation: speak your prompts instead of typing them. Any dictation tool works, including the one built into your computer	wisprflow.ai
Python 3	To run the code the agent writes	python.org

Python packages used in the workshop:

pip install pandas openpyxl requests pyyaml country_converter python-dotenv

Installing a skill

Each part starts with the skills you need for it. To install one:

Click Download next to the skill. You get a .zip file.
Open Claude Code and say: “Install the skill in ~/Downloads/<name>.zip into my skills folder.” The agent unzips it into ~/.claude/skills/.

Or do it yourself in the terminal (Mac):
```
mkdir -p ~/.claude/skills
unzip ~/Downloads/litsearch.zip -d ~/.claude/skills/
```
On Windows the folder is C:\Users\<you>\.claude\skills\.
Close and reopen Claude Code. Type / and the skill appears in the list.

A skill is only text and small scripts. Open the SKILL.md inside before you install it: you should always know what instructions your agent is following.

Using ChatGPT or another tool instead of Claude Code? You can follow along: the ideas are the same, with more copying and pasting.

Part 1: Foundations and Finding the Literature

In this part: the three tools we use all workshop (voice, GitHub, and an AI agent), and the first research task: finding the literature without inventing it.

Download the skills for this part

Click a skill to see what it does. Installation steps are in the setup section.

/litsearch: search the literature Download litsearch.zip

Searches four academic databases at once (OpenAlex, Semantic Scholar, Crossref, arXiv) and returns real papers with DOIs, ranked by relevance and journal quality.

What it does

Turns your question into several short searches and merges the results.
Snowballing: from one paper, finds what it cites and who cites it.
Saves every search as a numbered reading list in literature/.
On request: a clean .bib file for LaTeX, and the open-access PDFs (never goes around a paywall).
Ranks journals with journal_tiers.md, a list you can edit.

Ask it, for example

“Find the main empirical papers on emigration and economic institutions.”
“Who cites Powell and Regalado Cardoso (2025)?”
“Export papers 1, 4, and 7 as BibTeX.”

Needs: Python 3 with requests. Optional: set LITSEARCH_EMAIL to your email for faster responses from OpenAlex and Crossref.

Built by a colleague with Claude; shared with permission.

/paper-facts: what did each paper do? Download paper-facts.zip

Turns a reading list into one evidence table: data, method, and main result of each paper, and where each row came from.

What it does

Reads the paper itself when you have the PDF (with page numbers), and the abstract when you do not.
Writes "not stated" instead of guessing, and lists papers it could not find a source for.
Saves the table as literature/facts_<topic>.md.

Ask it, for example

“Run /paper-facts on my litsearch list.”
“Make an evidence table for the PDFs in literature/pdfs.”

Needs: Python 3 with requests.

Written for this workshop.

How I actually work

I start by creating a folder. Every project is one folder, and every folder is a GitHub repository.
The first two files I create are a working log and a CLAUDE.md.
The CLAUDE.md holds my rules. One of them: the agents timestamp every change they make in the working log.
I talk to the computer. Most of my prompts are dictated, not typed.
An AI agent works inside that folder: it reads my files, writes code, and runs it.
When I repeat a task, I turn it into a skill: searching the literature, merging datasets, translating Stata to Python, writing a referee report.
I read everything it writes. I open the code myself. If I cannot explain it, it does not go in the paper.

Voice: dictate your prompts

People give longer and clearer instructions when they speak than when they type.

Typed

find papers on emigration and institutions

Dictated

I am studying whether emigration changes economic institutions in the countries people leave. Find the main empirical papers. I care most about cross-country panel studies and about the channels: remittances, diaspora, return migration. Prioritize top field journals, and tell me which papers everyone else cites.

The agent can only use the context you give it. Speaking makes giving context cheap.

Hold a key, speak, release. The text appears wherever the cursor is.
Say everything you would tell a research assistant: the question, what you already know, what you want back, and what you do not want.

GitHub: one project, one repository

A repository is a project folder with a memory: every saved version, who changed what, and why.

Why it matters for research

You can undo anything, including what the AI did.
Your project is backed up and shareable.
Someone else can rerun your work.
It is your portfolio.

The three steps you will repeat

Edit: change files in the folder.
Commit: save a snapshot with a one-line message.
Push: send the snapshot to GitHub.

You do not need to memorize git commands. Ask the agent to commit and push, and read what it says.

Commit before you let the agent make big changes. It is your undo button.

Claude Code: a chatbot and an agent are different tools

	Chatbot	Agent
Where it works	In a browser window	Inside your project folder
What it sees	What you paste	Your files, data, and code
What it does	Writes text	Reads, writes, and runs things
Who moves the work	You copy and paste	It edits the files directly
What you must check	The answer	The answer and what it changed

More power means more to verify. That is why we use GitHub from day one.

`CLAUDE.md`: the instructions it always reads

A plain text file in the project folder. The agent reads it every time you start it in that folder.

## About this project
- Research question: Does emigration change economic institutions at origin?
- Data: country panel, 1980 to 2010
- Method: OLS with region and year effects

## Rules
- Never cite a paper from memory. Every reference has a DOI I can open.
- After every merge, report rows before and after and the match rate.
- Ask before you delete or overwrite a file.

When you catch yourself repeating an instruction, put it in CLAUDE.md.

Skills: good practice, packaged

A skill is a set of written instructions, sometimes with a script, that the agent follows for one kind of task.

You call it by name: /litsearch, /merge-check, /review-paper.
It makes the agent do the task the same careful way every time.
Anyone can write one. They are text files, and people share them on GitHub.

Part	Skill	What it does	Written by
1	`/litsearch`	Searches four academic databases at once	A colleague, who built it with Claude
1	`/paper-facts`	Summarizes data, method, and result of each paper	Written for this workshop
2	`/merge-check`	Checks a merged dataset against its raw sources	Written for this workshop
4	`/referee2`	Audits and replicates your code	Scott Cunningham
4	`/review-paper`	Simulated peer review	Pedro Sant’Anna (our standalone version)

Finding the literature

Why chatbots invent citations

A language model writes the most plausible next words. A reference is a very predictable pattern: authors, year, title, journal.

It can write a reference that looks right and does not exist.
Real authors, a real journal, a believable title, and a wrong or invented paper.
It sounds equally confident either way.

The fix is not a better prompt. It is a different source. A search tool asks a database of real publications and returns records, and every record has a DOI: a permanent link you can open.

The model should find and organize papers. It should never be the source of them.

`/litsearch`

One request searches four academic databases at once: OpenAlex, Semantic Scholar, Crossref, and arXiv.

Topic search. The agent turns your question into several short searches and merges them, ranked by relevance and journal quality.
Snowballing. Start from one paper you trust: what does it cite, and who cites it?
Journal tiers. journal_tiers.md is a list of journals you edit yourself. It tells the tool which venues you trust.
A file every time. Each search saves a numbered reading list in literature/.
When you ask. A clean .bib file for LaTeX, and the PDFs of the papers that are open access.

Each result is one line with a link you can open:

[1] Title of the paper (link to DOI) · Authors, Journal [tier · h-index] · year · citations

Short searches work best: two or three words per search, then snowball from the best paper you find. A long sentence as a single search returns almost nothing.

`/paper-facts`: what did each paper do?

It takes the reading list and writes one table: data, method, main result, and where each row came from.

Paper	Data	Method	Main result	Source
Powell and Regalado Cardoso (2025)	132 origin countries, 1980 to 2010, 752 observations	Two-way fixed effects; shift-share instrument	Larger emigrant stocks go with later gains in economic freedom	paper, pp. 7-11
Clark et al. (2015)	not stated	not stated	No evidence of negative, some evidence of positive impacts on institutions	abstract

It reads the paper when you have the PDF, and the abstract when you do not.
It never fills a cell from memory. “not stated” tells you which paper to open next.

The table is a map of what to read. It does not replace reading.

The rule for this part

If a paper goes into your references, you have to read it. Your eyes on the paper itself, not on an AI summary of it.

The AI helps you find papers. Reading them is your job.

AI detectors

Tools such as Pangram estimate how much of a text was written by AI.

Your professors, journals, and employers can run your text through one.
Asking the AI to sound like you does not make the text yours.
Detectors also make mistakes. A score starts a conversation; it does not settle one.

Let the AI search, code, and check. Write your own sentences.

Practice and deliverable

Start in an empty folder, and say it: “Set up this folder as my research project. Create a GitHub repository, a README, a CLAUDE.md, an AI log, and a literature folder.”

	What you do	What it leaves
1	Open Claude Code in an empty folder; ask it to set up the project	Your repository, on GitHub
2	Tell it your research question	`README.md`
3	Tell it about your project and your rules	`CLAUDE.md`, `ai_log.md`
4	Search with `/litsearch`, then snowball from the best paper	`literature/litsearch_<topic>.md`
5	Ask for a `.gitignore` that keeps `literature/pdfs/` off GitHub, then download the open-access papers	`literature/pdfs/`
6	Summarize the papers with `/paper-facts`	`literature/facts_<topic>.md`
7	Pick the most interesting papers and read them	What you will use in your literature review
8	Log what you did, then commit and push	`ai_log.md`

Deliverable 1. The /paper-facts table for your research question: every paper from your search, with its data, method, and main result.

Save it in your repository (literature/facts_<topic>.md) and push it to GitHub.

Part 2: Getting Data and Building Datasets

In this part: where data comes from, how to download it, what an API is, how to scrape a website when there is nothing to download, and how to build a merged dataset that anyone can rebuild and that you can prove is correct.

Download the skills for this part

Click a skill to see what it does. Installation steps are in the setup section.

/merge-check: is my merged dataset right? Download merge-check.zip

Compares a merged dataset against the raw files it was built from, value by value, and draws a random sample for you to check by hand.

What it does

Checks every value of every merged variable against its raw source: match, mismatch, lost, or invented.
Recomputes derived variables (squares, ten-year changes) from their formulas.
Finds duplicated keys, countries dropped in the merge, and names that did not convert to codes.
Checks a source's country codes against its country names.
Writes a report in checks/ and tells you which three values to confirm yourself in the raw file.

Ask it, for example

“Run /merge-check on clean/panel.csv.”
“Check my merge against the files in raw/.”

Needs: Python 3 with pandas, pyyaml, openpyxl, and country_converter.

Written for this workshop.

The paper we rebuild

Throughout this part, we go through the whole process of getting the data and building the dataset for one paper: Powell and Regalado Cardoso (2025), “Emigration and Origin Country Economic Institutions,” Public Choice.

The paper asks whether emigration changes the economic institutions of the country people leave. To answer it, it needs one row per country and year with:

how many people have left each country (emigrant stocks to 20 OECD countries);
the country’s economic freedom score, and how it changes over the next ten years;
controls: inequality, civil liberties, and education;
the country’s world region.

Each of those comes from a different source. Putting them together, correctly, is the work of this part.

How I built this dataset in 2024

The paper’s dataset has 132 countries, seven years (1980 to 2010, every five years), and 752 observations. Here is how I built it the first time:

I downloaded six sources and combined them by hand, in Excel.
I saved new versions as I went: emigration data.xls, emigration fixed.xls, dataset emigration 1980-2010.xlsx, and finally datanew.dta. The paper’s folder ended up with about 130 files.
Country names did not match across sources, so my code has a long block of fixes:

replace country = "Bolivia" if country == "Bolivia (Plurinational State of)"
replace country = "Vietnam" if country == "Viet Nam"
replace country = "Congo, Dem. Rep." if country == "D.R. of the Congo"
replace country = "Iran, Islamic Rep." if country == "Iran (Islamic Republic of)"
replace country = "Türkiye" if country == "Turkey"
* ... and many more

Halfway through I changed my main data source (from UN migrant stocks to the IAB Brain Drain data), and the old files stayed in the folder.

It worked, and the paper was published. But it took days of coding by hand and hunting down errors one by one. If I started from scratch today, getting all the data and doing all the merging would be much easier: the agent writes the code, and I spend my time checking the result instead of fixing country names. This part shows how.

Data basics

Three ideas behind every merge. Most merge errors come from getting one of them wrong.

1. The unit of observation. What one row represents. In the running example: one country in one year (a country-year).

2. The key. The columns that identify a row. Here: country code and year. No two rows can have the same key.

3. Long and wide. The same data can be stored two ways. Merges need long format.

Wide: one column per year

country	1990	1995	2000
GTM	6.13	7.07	6.82
HND	5.95	6.02	6.65

Long: one row per country-year

country	year	EFW
GTM	1990	6.13
GTM	1995	7.07
…	…	…

EFW scores from the paper’s dataset (2023 report).

Country codes, not names. “Bolivia,” “Bolivia (Plurinational State of),” and “Bolivia, Plurinational State of” are the same country in three sources. The ISO3 code (BOL) is the same everywhere. Convert names to codes first, then merge on codes. In Python, the package country_converter does the conversion and tells you which names it could not match.

Four ways data arrives

	How it works	Example in our data
API	A web address that returns data instead of a webpage. A script asks, the server answers with numbers	World Bank, Our World in Data
Direct file link	A permanent link to a file. A script can download it, but it is the whole file	IAB Brain Drain, Penn World Table
Manual download	You click through a website, fill in a form, and save the file	Fraser Institute (the site blocks scripts)
Web scraping	No file and no API: the data is only on webpages, so code reads the pages one by one	The REMHI massacre database (below)

Not everything can be automated. Everything can be documented: whatever the route, the data log says where each file came from and when.

What an API is

An API (application programming interface) is a door to a database that a program can knock on. You send a request, which is just a web address with your question written in it, and you get data back.

Click this link. It asks the World Bank for Guatemala’s population in 2010:

https://api.worldbank.org/v2/country/GTM/indicator/SP.POP.TOTL?format=json&date=2010

Read the address piece by piece:

Piece	Meaning
`api.worldbank.org/v2`	The World Bank’s API
`country/GTM`	Guatemala (ISO3 code)
`indicator/SP.POP.TOTL`	Total population (the indicator’s code)
`format=json&date=2010`	Return it as JSON, for 2010

Change GTM to HND and you get Honduras. Change the indicator and you get another variable. That is all an API is. In Python:

import requests
import pandas as pd

url = "https://api.worldbank.org/v2/country/GTM;HND;SLV/indicator/SP.POP.TOTL"
params = {"format": "json", "date": "1980:2010", "per_page": 500}

response = requests.get(url, params=params)   # send the request
data = response.json()[1]                     # element 0 is metadata, element 1 the data

df = pd.DataFrame({
    "iso3":  [row["countryiso3code"] for row in data],
    "year":  [int(row["date"]) for row in data],
    "pop":   [row["value"] for row in data],
})
print(df.head())

You do not need to write this yourself. You need to be able to read it, and to ask the agent: “Download total population for Central America from the World Bank API, 1980 to 2010, and save it in raw/.”

API keys

Some APIs ask you to identify yourself with a key: a long password tied to your account. FRED, the database of the Federal Reserve Bank of St. Louis, is the most common one in economics. Getting a key is free: create an account at fred.stlouisfed.org and request a key under API Keys.

A key is a password. Treat it like one:

Put it in a file called .env in your project folder:
```
FRED_API_KEY=your_key_here
```
Add .env to your .gitignore, so it never goes to GitHub.

Read it from the code, never write it in the code:

import os
from dotenv import load_dotenv
from fredapi import Fred          # pip install fredapi

load_dotenv()                     # reads .env
fred = Fred(api_key=os.environ["FRED_API_KEY"])
rate = fred.get_series("FEDFUNDS")    # US federal funds rate

Agents paste keys into scripts if you let them. Put the rule in your CLAUDE.md: “API keys live in .env. Never write a key in code, in a log, or in a commit.” If a key ever reaches GitHub, delete it on the provider’s website and create a new one. Deleting the file from the repository is not enough: the history keeps it.

Web scraping: when there is nothing to download

Sometimes the data exists only as webpages: one page per record, no file, no API. Scraping means writing code that visits those pages, saves them, and pulls the information out of them into a table.

Example: the REMHI massacre database

The REMHI project (Guatemala: Nunca Más, ODHAG 1998) documented the massacres of the Guatemalan civil war from thousands of witness testimonies. Its database is public at remhi.org.gt/bd, but only as webpages: one page per massacre, one per testimony, one per victim. There is no download button. I asked ODHAG for the data through their formal channel and never got an answer.

So I scraped it, with Claude Code, over a weekend. The result is a dataset of 530 massacre records (482 with full detail), with locations, dates, victim counts, and responsible forces, and an interactive map on my website.

The project has the same structure you are building today, one script per step:

Script	What it does
`01_scrape.py`	Visits about 10,000 pages and saves each one, untouched, in `data/raw/`
`02_parse.py`	Reads the saved pages and pulls the information into tables
`03_clean.py`	Repairs broken characters, validates dates, reconciles victim counts. Every fix is flagged
`04_geocode.py`	The site has no coordinates: matches place names to a gazetteer of Guatemalan places
`05_build.py`	Assembles the final tables and checks them against the codebook
`06_build_map.py`	Builds the map

Step 1: look before you scrape

Before any code, the agent and I explored the site and wrote down what we found:

Every page is reachable with a simple address. Massacre number 31 is ver_masacre2.php?cual=31; massacres are numbered from 1 to about 844.
The list pages accept a hidden option (&no_pager=1) that returns the entire table in one page instead of 20 rows at a time.
The site’s robots.txt (the file where a website says what bots may visit) has no restrictions.
There are no coordinates anywhere, so locations would have to be matched to a map separately.
The data is dirty: different testimonies give different death tolls for the same massacre (60 to 200 for Chacalté), one date says 1920, and accents were lost in the site’s own database.

This exploration decided the whole design. Ask the agent to do it first, and to write the findings in your project log.

Step 2: scrape politely

A website is someone else’s server. A script that requests thousands of pages as fast as it can looks like an attack and can take a small site down. My scraper follows these rules:

Identify yourself. Every request says who is asking and gives an email.
Go slowly. About two pages per second. The full harvest took 40 to 50 minutes.
Save the raw page, untouched. If the parsing has a bug, you fix the parser, not download everything again.
Keep a manifest: every address, the file it was saved to, the response, and the time.
Resume, do not restart. Pages already on disk are skipped.

The core of it, simplified:

import time
import requests

BASE = "https://www.remhi.org.gt/bd/"
session = requests.Session()
session.headers["User-Agent"] = (
    "research project on Guatemalan massacres; contact: your_email@ufm.edu"
)

for i in range(1, 845):                          # massacre pages 1 to 844
    dest = f"data/raw/masacres/masacre_{i:04d}.html"
    response = session.get(f"{BASE}ver_masacre2.php?cual={i}", timeout=90)
    with open(dest, "wb") as f:
        f.write(response.content)                # save the page exactly as received
    time.sleep(0.5)                              # be polite: wait between requests

Step 3: parse the saved pages

Parsing means finding the information inside the page. A page is HTML: text wrapped in tags. The package BeautifulSoup reads it so you can ask for what you need, for example every link to a testimony on a massacre page:

import re
from bs4 import BeautifulSoup

html = open("data/raw/masacres/masacre_0031.html", "rb").read().decode("latin-1")
soup = BeautifulSoup(html, "html.parser")

testimonies = soup.find_all("a", href=re.compile("ver_testimonio"))
print(len(testimonies))      # massacre 31 (Vegas de Santo Domingo, Rabinal): 183 links

Run on the real saved page, that prints the same text the site shows, including the damage: the heading “Ubicación” comes out as Ubicaciï¿½n, because the accent was already lost in REMHI’s own database. Real data looks like this. The cleaning script repairs those words one by one and flags every repair.

Before you scrape anything

Is there an API or a download? Use it. Scraping is the last option.
Read the terms of use and robots.txt. If the site forbids scraping, do not scrape it.
Ask. Write to the owner first. Sometimes they send you the file.
Is it personal data? Names, addresses, and health records carry responsibilities. The REMHI database names victims; that data is public for the purpose of memory, and it should be used with respect, not for anything else.
Scrape politely, keep the raw pages, and log everything.

Agents are very good at scraping: they read the page structure, write the parser, and fix it when it breaks. That is exactly why the rules above matter. The agent will go as fast as you let it.

The data we need

Source	What we take	Where to download	File
IAB Brain Drain Data	Emigrant stocks to 20 OECD countries, by education, 1980 to 2010	iab.de/en/daten/iab-brain-drain	`iabbd_8010_v1_emigration.dta`
Economic Freedom of the World (Fraser Institute)	EFW index, 1970 to latest	efotw.org/economic-freedom/dataset	Excel, full dataset
Freedom in the World (Freedom House)	Civil liberties rating	freedomhouse.org/report/freedom-world	“Country and Territory Ratings and Statuses,” Excel
World Inequality Database	Gini coefficient, pre-tax income	wid.world/data	Excel or CSV
Penn World Table 11.0	Human capital index (`hc`)	rug.nl/ggdc/productivity/pwt	Stata (`.dta`) or Excel
UN M49	World regions	unstats.un.org/unsd/methodology/m49/overview	CSV or Excel

Save every file, untouched, in raw/, and its codebook or methodology document in docs/. Write down where it came from, the version (report year), and the date you downloaded it.

We use the current releases, so the numbers will not match the paper exactly. That is expected, and below you will see why.

Where to find data

	Source	What is there
Global	World Bank Open Data	Development indicators for every country; has an API
	Our World in Data	Thousands of series with documented sources; every chart downloads as CSV
	Penn World Table	GDP, productivity, capital, human capital
	FRED	Macroeconomic and financial series; API with a free key
	IPUMS International	Census microdata from over 100 countries, including Guatemala; free registration
	Quality of Government (QoG)	One file that already merges hundreds of country-level variables (EFW, Freedom House, governance indicators). A huge shortcut for country-level projects
Latin America	CEPALSTAT	ECLAC statistics for the region; has an API
	LAPOP	AmericasBarometer opinion surveys
	Latinobarómetro	Opinion surveys, 18 countries, since 1995
Guatemala	Banco de Guatemala	Exchange rates, remittances, national accounts
	INE Guatemala	Household surveys (ENCOVI, ENEI), census

Read the documentation first

Every dataset comes with a codebook or a methodology document that says what each variable means and how it was built. Read it before you use a variable, and give it to the agent too. Save the documents in a docs/ folder and tell the agent to look up every variable there before it uses it. An agent that can read the codebook does not have to guess.

In the running example, the IAB data comes in two files:

Counts (iabbd_8010_v1): the number of emigrants from each origin country living in each of the 20 destination countries, by education and gender.
Rates (iabbd_8010_v1_emigration): emigration rates that the IAB has already computed.

The paper uses the IAB’s own rates; we do not compute them ourselves. The IAB methodology report defines them as:

\[ \text{Emigration rate} = \frac{\text{emigrants living in the 20 OECD countries}}{\text{residents} + \text{emigrants}} \]

with residents estimated from Barro and Lee’s education data. The rates by education divide emigrants with a given education by everyone (residents and emigrants) with that same education. So the rate is not “emigrants divided by population.” None of that is in the spreadsheet: it is only in the methodology report.

If you ask an agent for “the emigration rate” without the document, it will make up a definition, and it will sound right.

Give the agent the codebook. Do not let it choose a definition.

Rules for building a dataset

my-project/
├── docs/         codebooks and methodology documents of each source
├── raw/          downloaded files, never edited
├── build/        the notebook that reads raw/ and writes clean/
├── clean/        the merged dataset (can always be rebuilt)
├── checks/       merge checks and their reports
├── codebook.md   what each variable is, its source, its unit
└── data_log.md   where each file came from, when, and what was dropped and why

raw/ is never edited. Not even to fix a typo. Fixes go in the build notebook, where they are visible.
One build notebook turns raw/ into clean/. Run it again from the top, get the same dataset.
Merge on codes, not names. Convert names to ISO3 first, and print the names that did not convert.
Every merge reports its match rate: rows before, rows after, and how many found a match.
Pin the version of every source: report year and download date, in the data log.
Check the result against the raw files (see /merge-check below).

Put rules 3 and 4 in your CLAUDE.md, so the agent follows them without being asked.

Silent errors

A merge can run without a single error message and still be wrong. These are real examples from the running example’s data.

1. Names that do not match

The same country is spelled differently in each source: “Bolivia (Plurinational State of),” “Viet Nam,” “D.R. of the Congo.” A merge on names silently drops every country that is spelled differently. The fix: convert to codes, and print the list of names that did not convert.

2. The source’s own codes are wrong

The IAB data comes with ISO3 codes. Ten of them are wrong or outdated:

Country in IAB	IAB code	Correct ISO3 code
Guinea	`GNB`	`GIN`
Guinea-Bissau	`GIN`	`GNB`
Romania	`ROM`	`ROU`
Singapore	`SIN`	`SGP`
Congo, Dem. Rep.	`ZAR`	`COD`
United Arab Emirates	`UAE`	`ARE`

Guinea and Guinea-Bissau are swapped. Merge on IAB’s codes and Guinea-Bissau gets Guinea’s emigration data, with no error message. The fix: when a source has both codes and names, check that they agree.

3. Rows lost in a merge

When a key does not match, the merge does not fail: the row just ends up with a missing value. If one file says “Czech Republic” and another says “Czechia,” every Czech country-year silently loses that variable. You notice only if you count. The fix: count rows and missing values before and after every merge, and ask why they changed.

4. Frequency changes

The EFW index is published every five years until 2000 and every year after. A “change over the last year” means something different before and after 2000. The fix: know the frequency of every source before you combine them.

5. Data versions

Sources revise their past numbers. Albania’s EFW score for 1990 is 4.68 in the 2023 report the paper used, and 5.22 in the 2024 report. Neither is a mistake: the Fraser Institute revises past scores when it updates the index. The fix: record the version, and do not mix versions of the same source.

6. Changes that need data you do not have

The outcome in the paper is the change in EFW over the next ten years. For 2010, that needs EFW in 2020, which is outside the panel. If you compute it inside the merged panel only, the last years silently become missing. The fix: compute changes from the full source, before you cut the years.

Checking the merge: `/merge-check`

The skill compares the merged dataset against the raw files, value by value, and then gives you a random sample to check yourself.

What it checks:

Structure: duplicated keys, missing keys.
Every value of every variable, against its raw source. Each row gets a status:

Status	Meaning	Usual cause
match	Same value in both
mismatch	Both have a value, and they differ	Wrong row, wrong formula, or a different data version
lost	The raw file has a value; the merged data does not	A name that did not match
invented	The merged data has a value the raw file does not support	Wrong country, wrong year, or a wrong code

Derived variables (squares, ten-year changes), recomputed from their formulas.
Codes against names, when a source has both.
Countries dropped in the merge, so you can confirm you meant to drop them.

The hand check. The report ends with a random sample: the merged value next to the raw value, for the same rows every time you run it. Open the raw file yourself and confirm three of them. The script does the full check; you do the spot check. Both are required.

Live demo: rebuilding the dataset

I ask Claude Code, by voice, for a Jupyter notebook that builds the dataset from the files in raw/, one step per section. Then I open the notebook in VS Code and run it cell by cell, so we can see what happens at each step: which country names were changed, what each merge kept and dropped, and what the data looks like before and after.

The request, roughly as I will dictate it:

Build a Jupyter notebook in build/ that creates the emigration panel from the files in raw/. One section per step, with a short explanation before each cell. After every step, print what changed: which country names were converted or fixed, how many rows each merge matched and which ones it did not, and the number of rows and missing values. Use the definitions in the paper, and look up every variable in the codebooks in docs/ before you use it. Do not hide any step inside a function I cannot see.

What the notebook goes through:

Describe the sources. For each file in raw/, using its codebook in docs/: unit, years, key, and the variables we need.
Codes. Convert every country name to ISO3, and print a table of the names that were changed and the ones that did not convert.
Merge. One source at a time, printing the match rate and the countries that did not match after each merge.
Build the variables from the paper’s definitions: emigration stock, ten-year change in EFW, controls.
Explore. Summary statistics, missing values by variable and year, and a first look at the relationship (EFW change against emigration stock), including the outliers.
Check. Run /merge-check, read the report, open the raw file to confirm three values.
Document. The agent writes the codebook and the data log; I read them, then commit and push.

A notebook is a good way to read what the agent did: every step shows its output right below the code. If you cannot follow a cell, ask the agent to explain it before you run the next one.

Once all the files are downloaded, the same pipeline can download them itself: I show the API version at the end, and then the REMHI scraper and its manifest.

Homework and deliverable

Build a dataset with the same steps we followed in class. Choose one of two options:

Your own idea. Build the dataset for your own research question: find the sources, download them, and merge them.
Rebuild a paper’s data. Pick a published empirical paper (one from your Part 1 search works well), read its data section, and try to get the same data the authors used: the same sources, variables, countries or units, and years. Then compare your dataset with the paper’s descriptive statistics table. Where your numbers differ, explain why.

Deliverable 2. Your dataset, built from raw files, for your own idea or for the paper you chose. In your repository:

raw/ with your downloaded files (or, if a source does not allow redistribution, instructions to download it);
a build notebook in build/ that turns raw/ into your dataset in clean/, run from top to bottom with its outputs saved;
codebook.md: every variable, its source, its unit;
data_log.md: where each file came from, the version, the download date, and what was dropped and why;
the /merge-check report, with every FAIL and WARN explained;
if you rebuilt a paper: the reference to the paper, and a short comparison of your descriptive statistics with theirs;
your ai_log.md, updated.

Part 3: Econometrics and Estimation

In this part: how to go from a dataset to estimates you can trust, starting from code you understand.

From idea to estimation

Where ideas come from is a topic for another day. This part starts once you have one. From there, I follow the same steps every time:

Write the estimation plan. What do you want to test, and how are you going to test it?
Choose the method. The design you wrote down decides the method, not the other way around.
Look at the data. Descriptive statistics and figures before any regression.
Write the code and estimate. Starting from code you already understand.
Run the robustness checks. Does the main result survive reasonable changes?

AI helps in steps 3 to 5, a lot. Steps 1 and 2 are yours.

Step 1: the estimation plan

The estimation plan is the design of your study, written down before you write any code. It answers two questions:

What do you want to test? One sentence, with an outcome and a variable of interest. Does a larger emigrant population improve the economic institutions of the origin country?
How are you going to test it? What you would compare, in which data, and what result would support or contradict your idea.

Think of the experiment you would run if you could. Ideally, we would randomly send some countries’ citizens abroad and watch what happens to their institutions. We can’t, so the plan says what we do instead, and why it gets us close.

For the emigration paper, the plan looked like this:

Outcome	Change in economic freedom over the next ten years (EFW index, Fraser Institute)
Variable of interest	Emigrant stock as a percent of the population (IAB Brain Drain data)
Unit and period	Origin countries, 1980 to 2010, every five years
Comparison	Countries in the same region and the same year, with more and fewer emigrants
Equation	Change in EFW on emigration, initial EFW, region and year fixed effects, controls
Why ten years	Institutions change slowly; a short horizon would mostly capture noise
Extensions	Does the effect depend on how much emigration there is (a square term) and on initial institutions (an interaction)?

Write it in one page. Show it to your advisor or a coauthor before estimating anything. It is much cheaper to fix a design on paper than after a month of code.

Step 2: the method follows from the design

Once the plan says what you compare, the method is mostly decided:

Your design	Method	In the code library
Compare units with more and less of something, holding constant what they share	OLS with fixed effects	OLS with fixed effects
Something affects the variable of interest but has no other path to the outcome	Instrumental variables	IV / 2SLS
A policy starts at one date for some units and not for others	Difference-in-differences	DiD and event study
The policy starts at different dates in different units	Staggered DiD	Callaway–Sant’Anna
Only one (or a few) units are treated	Synthetic control	Synthetic control
Treated and untreated units differ in observable ways	Matching, entropy balancing	Matching
Treatment depends on a score crossing a cutoff	Regression discontinuity	RD
The outcome is how long until something happens (a business closes)	Survival analysis	Kaplan–Meier and Cox

Each method comes with an assumption that makes it work (parallel trends, exclusion, no manipulation at the cutoff). Write that assumption in the plan, and what in your data supports it. If you can’t defend the assumption, change the design, not the story.

Choosing the method is your job, not the AI’s. Once you have chosen it, /find-replication finds published code that uses it.

Step 3: look at the data first

Before any regression, I look at the variables: summary statistics, distributions, and a few figures. The goal is to see what is going on before the model tells you.

Summary statistics for every variable in the plan: mean, range, missing values. Is anything impossible (a negative rate, a share above 100)?
Distributions. Is the variable of interest very skewed? A few countries with huge values can drive a whole regression.
The raw relationship. A scatter plot of the outcome against the variable of interest, or the outcome over time for treated and untreated units. If the regression later says something the figure doesn’t show, find out why.
The sample. How many units and periods, and which ones drop out when you add each control.

This is where AI saves the most time: “Make summary statistics and a histogram for each variable in my estimation plan, and a scatter plot of the outcome against the variable of interest, labeling outliers.” You read the output; it writes the code.

Step 4: write the code and estimate

Most of the estimation code I write today is code I already have. I take a notebook that I wrote, or a replication package I understand, and ask the AI to adapt it to the new data. That is the rule of this part:

Start from code you understand. The AI adapts it. You read every line of what it writes.

I do all of this in Jupyter notebooks, not plain scripts. A notebook lets you write a short description above every step: what this cell does, why, and what you expect to see. The output (a table, a figure) appears right below the code. When you come back in three months, or when someone else reads it, the notebook explains itself. Ask the agent to keep that structure: one step per cell, with a note above it.

Where the reference code comes from:

Your own previous projects. The best reference, because you already know what every line does.
The code library below. One notebook per method, each one checked against a published result.
A published paper’s replication package, found with /find-replication.

Then estimate the main specification exactly as written in the plan. If you change something, write down what and why.

Step 5: robustness checks

A robustness check asks: does the main result survive reasonable changes to how I measured, sampled, and estimated? It is not a search for a better result. It is a test of whether the one you have depends on a choice you made.

The usual kinds:

Kind	Question	Example (emigration paper)
Measurement	Does it depend on how I measured the variables?	Emigration of high-skilled vs. low-skilled people; a different institutions index
Sample	Does it depend on which units are in?	Drop very small countries; drop one region at a time
Specification	Does it depend on the controls or fixed effects?	Add or remove controls; country fixed effects
Timing	Does it depend on the horizon?	Five-year instead of ten-year changes
Inference	Does the significance depend on the standard errors?	Classical vs. robust vs. clustered by country

One rule: report all of them, including the ones where the result gets weaker. That is information, not failure.

This is where AI is most useful

Once the main notebook works, each robustness check is a small edit: change a variable, drop some rows, add a control. That used to take an afternoon of copying and editing code. Now it is one prompt:

Here is my main estimation notebook (estimate.ipynb). Write robustness.ipynb that reruns the main specification (1) with high-skilled emigration instead of total emigration, (2) without countries under one million people, (3) with five-year instead of ten-year changes, and (4) with standard errors clustered by country. Do not change the main specification. Before each check, add a short note saying what changes and why. Put all results in one table next to the main result, with the number of observations in each column.

Then read every cell: each check should change only what it says it changes.

The danger: specification searching

Because checks are now cheap, it is easy to run a hundred specifications and report the ones that look good. That is specification searching (or p-hacking), and it produces results that don’t hold up. What protects you:

Report everything. Every check you ran goes in the paper or the appendix, not only the ones that look good.
Git. Every notebook and result is committed, so the history shows what you ran and when.
The log. Keep a file (analysis_log.md) with every specification you ran and why. Ask the agent to update it each time it runs something.

Download the skills for this part

Click a skill to see what it does. Installation steps are in the setup section.

/find-replication: reference code for your method Download find-replication.zip

For a method you have already chosen, finds published papers that use it on data like yours, locates their replication packages, and explains the code line by line in light of the paper. It never chooses or suggests the method.

What it does

Asks for the method if you did not name it. Choosing it is your decision.
Profiles your dataset (panel structure, time spacing, treatment timing) to prefer papers with similar data.
Searches for papers with that method and checks Harvard Dataverse, Zenodo, openICPSR, and GitHub for their packages.
Gives you a shortlist of 3 to 5, with every link opened and checked.
Explains the chosen package in replication_search/explained.md: the equation term by term, which script makes which table, every option of the estimation, a Python version next to the original, and what changes for your data.

Ask it, for example

“/find-replication: Callaway-Sant'Anna, for my data in clean/panel.csv.”
“Find a paper with synthetic control on a single country, with its code.”
“Explain the replication code of this paper: doi 10.1162/rest_a_01544.”

Needs: Python 3 with pandas. No API keys. Packages from openICPSR (AEA journals) need a free login, so you download those yourself.

Written for this workshop.

Finding reference code: `/find-replication`

The rule of this part is to start from code you understand. If the method you chose is not in the code library, the best reference is the replication package of a published paper that used it on data like yours. /find-replication finds it. You tell it the method; it never chooses or suggests one.

You name the method. “Callaway–Sant’Anna”, “synthetic control”, “a Cox model”. If you don’t, it asks.
It profiles your data. How many units and periods, how far apart the periods are, whether the panel is balanced, and when each unit is treated. It uses this only to prefer papers whose data look like yours.
Papers and packages. It searches for papers with that method and checks Harvard Dataverse, Zenodo, openICPSR, and GitHub for their replication packages. You get a shortlist of three to five, with every link checked.
The code, explained with its paper. For the package you pick, it maps each script to the table it produces and each line of the estimation to the paper’s equation, explains every option, writes the Python version next to the original, and lists what has to change for your data.

Code library

Working code for the methods we use in the workshop. Each method has a notebook you can download and run, commented line by line, with the Stata equivalent next to the Python. Each one uses a classic published example and checks its numbers against the published result before doing anything else. The notebooks read their data directly from the links given below, and each download includes the one line you need to install its packages.

All examples come from published papers and textbooks, which are cited in each entry and at the top of each notebook: Wooldridge’s Introductory Econometrics, Cunningham’s Causal Inference: The Mixtape, Huntington-Klein’s The Effect, and the original articles. The datasets are distributed through the Rdatasets archive and the packages’ own repositories.

How to use it. Do not ask the agent to “run a regression.” Give it code you understand as a reference:

Here is a notebook that estimates an OLS model with fixed effects (ols_fixed_effects.ipynb). Adapt it to my data in clean/, with my outcome and my variables. Keep the same structure and the same checks. Do not change the type of standard errors without asking me.

Then read every cell of what it writes, as you did with the reference.

OLS with fixed effects

When to use it. You want the association between a variable and an outcome, holding constant everything that is common to a group (a region, a year) through fixed effects. It is the starting point of most empirical papers.

The key assumption. Once you control for the variables and the fixed effects, nothing left out of the model is related to both the explanatory variable and the outcome. OLS gives you an association; it is causal only if that assumption is believable.

The example. Table 3 of Powell and Regalado Cardoso (2025): the ten-year change in economic freedom on the emigrant stock, with region and year fixed effects. The notebook translates the paper’s Stata code to Python and checks that every number matches the published table.

import pandas as pd
import statsmodels.formula.api as smf

df = pd.read_csv("emigration_table3.csv")

# Rows with all controls (models with controls use fewer observations)
sample = df.dropna(subset=["invgini", "civil", "hc"])

# C(region) and C(year) create the fixed effects; I(stock**2) is the square
formula = ("EFWdelta10 ~ stock + I(stock**2) + stock:EFW + EFW"
           " + invgini + civil + hc + C(region) + C(year)")

result = smf.ols(formula, data=sample).fit(
    cov_type="cluster",                                   # standard errors clustered...
    cov_kwds={"groups": pd.factorize(sample["country"])[0]})  # ...by country

print(result.summary())

import delimited emigration_table3.csv, clear
encode region, gen(r)          // region as a numeric code, for i.r
encode country, gen(cid)       // country as a numeric code, for clustering

* c.stock##c.stock adds stock and its square; c.EFW##c.stock adds EFW and stock x EFW
reg EFWdelta10 c.stock##c.stock c.EFW##c.stock invgini civil hc i.r i.year, vce(cluster cid)

What the notebook checks

Your sample. Models with controls drop rows with missing values (752 rows become 682). The notebook makes this explicit.
The published numbers. All four columns of Table 3 are reproduced exactly: coefficients, standard errors, observations, and R-squared. Only after this check do you use the code on new data.
The standard errors. Classical, robust, and clustered by country give the same coefficients and different standard errors. Choose before you look at the results.

Where AI-written code goes wrong with this method

Year or region entered as a number instead of as categories (one coefficient instead of one dummy per year).
A square or an interaction written by hand in the wrong place (stock*EFW in a formula also adds stock and EFW; stock:EFW adds only the product).
A column whose name clashes with a function: in the original dataset, a variable named C breaks the formula function C().
Default standard errors that are not the ones you meant.

Download the notebook and data (ols_fixed_effects.zip)

Instrumental variables (2SLS)

When to use it. Your variable of interest is chosen by the units themselves (education, migration, an investment), so it is related to things you don’t observe. You have an instrument: something that moves the variable of interest and has no other path to the outcome.

The key assumption. Relevance (the instrument predicts the variable; testable with the first stage) and exclusion (it affects the outcome only through that variable; not testable, you must argue it).

The example. Returns to education for working women, with the parents’ education as instruments (Mroz data; Wooldridge, Introductory Econometrics, Examples 15.1, 15.5, 15.7 and 15.8). OLS gives 11% per year of education; 2SLS gives about 6%. Data: mroz.csv.

import pandas as pd
from linearmodels.iv import IV2SLS

df = pd.read_csv("https://vincentarelbundock.github.io/Rdatasets/csv/wooldridge/mroz.csv")
s = df.dropna(subset=["lwage"])                      # working women only

# Endogenous variable and its instruments go in brackets; exogenous controls outside
iv = IV2SLS.from_formula("lwage ~ 1 + exper + expersq + [educ ~ motheduc + fatheduc]",
                         data=s).fit(cov_type="robust")
print(iv.summary)
print(iv.first_stage)                                # always report the first stage

import delimited "https://vincentarelbundock.github.io/Rdatasets/csv/wooldridge/mroz.csv", clear
ivregress 2sls lwage exper expersq (educ = motheduc fatheduc), vce(robust)
estat firststage        // relevance: first-stage F
estat endogenous        // is educ endogenous?
estat overid            // do the two instruments agree?

What the notebook checks

The published numbers. OLS, IV and 2SLS estimates and standard errors match Wooldridge exactly, and so do the endogeneity test (t = 1.67) and the overidentification test (n·R² = 0.378).
The first stage. The instruments’ F-statistic is 55: strong.
The two-stages-by-hand mistake. Running the second stage with OLS on fitted values gives the right coefficient and the wrong standard error.

Where AI-written code goes wrong

Running the two stages by hand: wrong standard errors.
Leaving exogenous controls out of the first stage.
Reporting 2SLS without the first stage, or treating the overidentification test as proof that the instruments are valid.

Download the notebook (iv_2sls.zip)

Difference-in-differences and event study

When to use it. A policy starts at one date for some units and not for others. Compare the change in the treated units with the change in the untreated ones over the same period.

The key assumption. Parallel trends: without the policy, treated and untreated units would have moved in parallel. The event study checks it before the policy.

The example. In 2011 California made organ donor registration an “active choice” at the DMV (Kessler and Roth, 2014; Huntington-Klein, The Effect, Chapter 18). Registration fell by 2.2 percentage points. Data: organ_donations.csv.

import pandas as pd
import pyfixest as pf                   # syntax like R's fixest and Stata's reghdfe

df = pd.read_csv("https://vincentarelbundock.github.io/Rdatasets/csv/causaldata/organ_donations.csv")
df["California"] = (df["State"] == "California").astype(int)
df["Treated"] = df["California"] * (df["Quarter_Num"] >= 4)     # California, after the policy

# Two-way fixed effects: state and quarter, clustered by state
did = pf.feols("Rate ~ Treated | State + Quarter", data=df, vcov={"CRV1": "State"})

# Event study: one coefficient per quarter, relative to the last one before the policy
es = pf.feols("Rate ~ i(Quarter_Num, California, ref=3) | State + Quarter_Num",
              data=df, vcov={"CRV1": "State"})
pf.etable([did, es])

import delimited "https://vincentarelbundock.github.io/Rdatasets/csv/causaldata/organ_donations.csv", clear
gen california = state == "California"
gen treated = california & quarter_num >= 4
reghdfe rate treated, absorb(state quarter) vce(cluster state)
reghdfe rate california##ib3.quarter_num, absorb(state quarter_num) vce(cluster state)

What the notebook checks

The published numbers. −0.022 (0.006) with 162 observations, as in The Effect, Table 18.2; and the same number computed by hand from four means.
Parallel trends before the policy. The event-study figure.
Inference with one treated unit. A placebo test that treats each other state as if it were California: p ≈ 0.19, against 0.001 with clustered standard errors.

Where AI-written code goes wrong

Treated coded wrong (it must be 1 only for treated units after treatment). Crosstab it first.
Clustering at the wrong level, or trusting clustered errors with one or very few treated units.
Using two-way fixed effects when units are treated at different dates (see the next method).

Download the notebook (did_event_study.zip)

Staggered DiD: Callaway and Sant’Anna

When to use it. Units adopt the policy at different dates. Two-way fixed effects then compares newly treated units with already treated ones, which can bias the estimate badly when effects change over time.

The key assumption. Parallel trends between each group of adopters and the comparison group (never treated, or not yet treated), and no anticipation.

The example. Minimum wage increases and teen employment in U.S. counties, 2003–2007: the example in Callaway and Sant’Anna’s did package (Journal of Econometrics, 2021). Data: mpdta.csv.

import pandas as pd
from csdid.att_gt import ATTgt

df = pd.read_csv("https://raw.githubusercontent.com/d2cml-ai/csdid/main/data/mpdta.csv")

# first.treat = year first treated (0 = never treated)
cs = ATTgt(yname="lemp", gname="first.treat", idname="countyreal", tname="year",
           xformla="lemp~1", data=df, control_group="nevertreated").fit(est_method="dr")
cs.summ_attgt()               # one ATT for each group and year
cs.aggte(typec="dynamic")     # event-study aggregation

import delimited "https://raw.githubusercontent.com/d2cml-ai/csdid/main/data/mpdta.csv", clear
csdid lemp, ivar(countyreal) time(year) gvar(first_treat) method(dripw)
estat event        // event study
estat simple       // one overall ATT

What the notebook checks

The published numbers. All 12 group-time ATTs and the event-study estimates match the did vignette (overall −0.077).
Against TWFE. The TWFE coefficient (−0.037) next to the Callaway–Sant’Anna aggregations.
Robustness. Not-yet-treated units as controls, and a covariate.

Where AI-written code goes wrong

Never-treated units coded as missing or 9999 instead of 0, which silently drops them or makes them a cohort.
Calling TWFE “staggered DiD.”
Bootstrap standard errors without a fixed seed (results don’t replicate).

Download the notebook (callaway_santanna.zip)

Synthetic control

When to use it. One unit (or a few) is treated: a country, a state, a city. Build a comparison as a weighted average of untreated units that tracks the treated unit before the policy.

The key assumption. If the synthetic unit matches the treated one for many years before the policy, it would have kept matching it without the policy.

The example. California’s 1988 tobacco tax and anti-smoking program, Proposition 99 (Abadie, Diamond and Hainmueller, JASA 2010). Cigarette sales fell by about 26 packs per person by 2000. Data: smoking_data.csv.

import pandas as pd
from pysyncon import Dataprep, Synth

URL = ("https://raw.githubusercontent.com/OscarEngelbrektson/SyntheticControlMethods/"
       "master/examples/datasets/smoking_data.csv")
df = pd.read_csv(URL)
df["year"] = df["year"].astype(int)
donors = [s for s in df["state"].unique() if s != "California"]

prep = Dataprep(foo=df, predictors=["lnincome", "age15to24", "retprice"], predictors_op="mean",
                time_predictors_prior=range(1980, 1989),
                special_predictors=[("beer", range(1984, 1989), "mean"), ("cigsale", [1975], "mean"),
                                    ("cigsale", [1980], "mean"), ("cigsale", [1988], "mean")],
                dependent="cigsale", unit_variable="state", time_variable="year",
                treatment_identifier="California", controls_identifier=donors,
                time_optimize_ssr=range(1970, 1989))
synth = Synth()
synth.fit(dataprep=prep, optim_method="Powell", optim_initial="ols")
synth.weights()               # which states make up synthetic California
synth.path_plot(); synth.gaps_plot()

ssc install synth, replace all
sysuse smoking, clear
tsset state year
synth cigsale beer(1984(1)1988) lnincome retprice age15to24 cigsale(1988) cigsale(1980) cigsale(1975), ///
      trunit(3) trperiod(1989) xperiod(1980(1)1988) nested fig

What the notebook checks

The published results. The same five donor states as the paper (Utah, Nevada, Colorado, Montana, Connecticut), a similar pre-1989 fit, and a gap of about −25 packs in 2000. The simple outcome-matching version gives −19.5, against −19.6 in Arkhangelsky et al. (2021).
The optimizer. Different optimizers give different weights; the notebook keeps the best pre-period fit.
Inference. Placebo tests for all 39 states. California ranks 3rd (p ≈ 0.08) in this version and 1st in the paper: how unusual an estimate looks depends on how the placebos are built.

Where AI-written code goes wrong

Trusting one run of the optimizer.
Reporting the effect without the pre-period fit.
Reporting a package’s standard error instead of placebo inference.

Download the notebook (synthetic_control.zip)

Matching and entropy balancing

When to use it. Treated and untreated units differ in observable characteristics, and you believe that, once you compare like with like on those characteristics, treatment is as good as random.

The key assumption. Selection on observables: nothing unobserved drives both treatment and the outcome. Matching does not fix selection on unobservables.

The example. The NSW job training program (LaLonde, AER 1986; Dehejia and Wahba, JASA 1999). The experiment says the program raised 1978 earnings by $1,794. Replace the experimental controls with survey (CPS) respondents and see which methods recover that number. Data: nsw_mixtape.csv and cps_mixtape.csv.

import pandas as pd
import statsmodels.formula.api as smf
from scipy.spatial import cKDTree

# df: the 185 trainees plus the CPS controls (see the notebook)
ps = smf.logit("treat ~ age + I(age**2) + educ + I(educ**2) + marr + nodegree"
               " + black + hisp + re74 + re75 + u74 + u75", data=df).fit()
df["ps"] = ps.predict(df)

t, c = df[df.treat == 1], df[df.treat == 0]
_, idx = cKDTree(c[["ps"]].values).query(t[["ps"]].values, k=5)    # 5 nearest controls
att = (t["re78"].values - c["re78"].values[idx].mean(axis=1)).mean()

teffects psmatch (re78) (treat age c.age#c.age educ c.educ#c.educ marr nodegree ///
         black hisp re74 re75 u74 u75, logit), atet nneighbor(5)

ssc install ebalance
ebalance treat age educ black hisp marr nodegree re74 re75 u74 u75, targets(2)
reg re78 treat [pw=_webal], robust

What the notebook checks

The benchmark. The experimental estimate, $1,794 (633), as in Dehejia and Wahba.
Balance. Standardized differences before and after; entropy balancing makes them exactly zero.
All estimators against the experiment. Raw difference −$8,498; OLS $1,184; matching on 1 or 5 neighbors; entropy balancing $1,402. None hits $1,794 exactly, and they disagree with each other: that is LaLonde’s lesson.

Where AI-written code goes wrong

No balance table.
Ignoring overlap (treated units with no comparable controls).
Bootstrapping nearest-neighbor matching (not valid; use teffects).
Matching on variables measured after treatment.

Download the notebook (matching.zip)

Survival analysis: Kaplan–Meier and Cox

When to use it. The outcome is how long until something happens: a business closes, an immigrant opens a firm, a worker finds a job. Many units haven’t experienced the event when the data end (censoring), so OLS on the duration is wrong.

The key assumption. Censoring is unrelated to the risk of the event (given the covariates). For Cox: proportional hazards, meaning each covariate multiplies the risk by the same factor at every point in time.

The example. Re-arrest of 432 men released from prison, half randomly given financial aid (Rossi, Berk and Lenihan, 1980; Fox and Weisberg’s appendix on Cox regression). Aid lowered the risk of re-arrest by about 32%. Data: Rossi.csv.

from lifelines import KaplanMeierFitter, CoxPHFitter

# df: week (duration), arrest (1 = event, 0 = censored), covariates coded 0/1 (see the notebook)
for value in [0, 1]:
    g = df[df["fin"] == value]
    KaplanMeierFitter().fit(g["week"], g["arrest"], label=f"fin = {value}").plot_survival_function()

cox = CoxPHFitter().fit(df, duration_col="week", event_col="arrest")     # Efron ties, as in R
cox.print_summary()           # exp(coef) = hazard ratios
cox.check_assumptions(df)     # proportional hazards

stset week, failure(arrest)
sts graph, by(fin)
sts test fin
stcox i.fin age i.race i.wexp i.mar i.paro prio, efron
estat phtest, detail

What the notebook checks

The published numbers. All seven Cox coefficients and standard errors match Fox and Weisberg exactly, and so does the likelihood ratio test (33.3).
Proportional hazards. Age fails the test, as in their appendix; the notebook stratifies by age group.

Where AI-written code goes wrong

Dropping censored observations, or treating them as events.
Different handling of ties across software (Stata’s default is Breslow; R and Python use Efron).
Reading Cox coefficients as effects on time instead of on the risk.

Download the notebook (survival.zip)

Regression discontinuity

When to use it. Treatment depends on a score crossing a cutoff: a vote share above 50%, a test score above a threshold, an age limit. Units just above and just below the cutoff are comparable.

The key assumption. Continuity: without treatment, the outcome would change smoothly at the cutoff. It fails if units can sort precisely around it. Check the density of the score and placebo cutoffs.

The example. Close U.S. House elections, 1948–1990 (Lee, Moretti and Butler, QJE 2004; Cunningham, Mixtape, Chapter 6). Barely electing a Democrat makes the district’s representative vote about 20 points more liberal in the next Congress. Data: close_elections_lmb.csv.

import pandas as pd
from rdrobust import rdrobust
from rddensity import rddensity

df = pd.read_csv("https://vincentarelbundock.github.io/Rdatasets/csv/causaldata/close_elections_lmb.csv")
df = df.dropna(subset=["score", "lagdemvoteshare"])
x = df["lagdemvoteshare"] - 0.5                      # center the running variable at the cutoff

rd = rdrobust(y=df["score"], x=x, cluster=df["id"])  # local linear, optimal bandwidth
print(rd)                                            # report the robust confidence interval
print(rddensity(X=x.drop_duplicates()).test)         # no jump in the density of the score

import delimited "https://vincentarelbundock.github.io/Rdatasets/csv/causaldata/close_elections_lmb.csv", clear
ssc install rdrobust
rdrobust score lagdemvoteshare, c(0.5) vce(cluster id)
rdplot score lagdemvoteshare, c(0.5)
rddensity lagdemvoteshare, c(0.5)

What the notebook checks

The published numbers. Table I of Lee, Moretti and Butler: same 915 observations, estimates within 0.1 of the published ones (21.3 vs. 21.2). The notebook explains why it is close but not exact.
The modern estimate. rdrobust gives 18.6 with a robust 95% interval of [13.5, 22.9].
Validity. Bandwidth sensitivity, the density test (p = 0.35), and placebo cutoffs.

Where AI-written code goes wrong

High-order polynomials over the whole range instead of local linear regression.
Choosing the bandwidth by hand to get a result.
Skipping the density and placebo tests.

Download the notebook (regression_discontinuity.zip)

Resources

Skills used in the workshop

/litsearch and /paper-facts (Part 1), /merge-check (Part 2): download them at the start of each part.
Scott Cunningham, MixtapeTools (/referee2): github.com/scunning1975/MixtapeTools
Pedro Sant’Anna, Claude Code academic workflow: github.com/pedrohcgs/claude-code-my-workflow

The running example

Powell, B., and Regalado Cardoso, L. (2025). Emigration and Origin Country Economic Institutions. Public Choice. doi.org/10.1007/s11127-025-01336-8

About this workshop

Before you start: setup

Installing a skill

Part 1: Foundations and Finding the Literature

Download the skills for this part

How I actually work

Voice: dictate your prompts

GitHub: one project, one repository

Claude Code: a chatbot and an agent are different tools

CLAUDE.md: the instructions it always reads

Skills: good practice, packaged

Finding the literature

Why chatbots invent citations

/litsearch

/paper-facts: what did each paper do?

The rule for this part

AI detectors

Practice and deliverable

Part 2: Getting Data and Building Datasets

Download the skills for this part

The paper we rebuild

How I built this dataset in 2024

Data basics

Four ways data arrives

What an API is

API keys

Web scraping: when there is nothing to download

Example: the REMHI massacre database

Step 1: look before you scrape

Step 2: scrape politely

Step 3: parse the saved pages

Before you scrape anything

The data we need

Where to find data

Read the documentation first

Rules for building a dataset

Silent errors

1. Names that do not match

2. The source’s own codes are wrong

3. Rows lost in a merge

4. Frequency changes

5. Data versions

6. Changes that need data you do not have

Checking the merge: /merge-check

Live demo: rebuilding the dataset

Homework and deliverable

Part 3: Econometrics and Estimation

From idea to estimation

Step 1: the estimation plan

Step 2: the method follows from the design

Step 3: look at the data first

Step 4: write the code and estimate

Step 5: robustness checks

This is where AI is most useful

The danger: specification searching

Download the skills for this part

Finding reference code: /find-replication

Code library

Resources

`CLAUDE.md`: the instructions it always reads

`/litsearch`

`/paper-facts`: what did each paper do?

Checking the merge: `/merge-check`

Finding reference code: `/find-replication`