Topic 3 Study Designs Overview
Pre-class work
Videos/slides
Checkpoint questions: Link to Moodle checkpoint
- We are analyzing survey data collected by the Minnesota Department of Health to understand the effect of vaping during adolescence on adulthood respiratory problems. What study design best describes this?
- Randomized experiment
- Instrumental variables design
- Regression discontinuity
- Interrupted time series
- General observational study
- A state program only has the budget to fund a social services program in the 10 poorest counties in the state. We want to study the effect of this program on county-level social and economic indicators. What study design best describes this?
- Randomized experiment
- Instrumental variables design
- Regression discontinuity
- Interrupted time series
- General observational study
- A hospital randomly puts up flyers throughout public hallways reminding people to get their flu shot. We want to study the effect of actually taking this year’s flu shot on risk of getting the flu. What study design best describes this?
- Randomized experiment
- Instrumental variables design
- Regression discontinuity
- Interrupted time series
- General observational study
- In at most two sentences, describe a general strategy that quasi-experimental studies use to try to achieve conditional exchangeability.
Learning Goals
- DESI1: Explain how randomized experiments relate to exchangeability.
- DESI2: Explain how quasi-experimental and general observational studies relate to exchangeability.
- DESI3: Compare the strengths and weaknesses of different study designs for answering a research question.
Exercises
Exercise 1
In a randomized experiment, participants are randomly assigned to receive either a new medication (\(A = 1\)) or the standard medication (\(A = 0\)) for cholesterol control. (Randomization is performed overall for all participants - not within subgroups.) The average cholesterol level in the new medication group is 130 mg/dL, and the average cholesterol level in the standard medication group is 140 mg/dL. Information on individuals’ family history of heart disease is also collected (\(Z = 1\) for a family history and \(Z = 0\) for no family history).
What can be said about the potential outcome data in the table below? What would be an estimate of the average causal effect \(E[Y^{a=1}-Y^{a=0}]\)?
| \(n\) | \(Z\) | \(A\) | \(Y^{a=1}\) | \(Y^{a=0}\) |
|---|---|---|---|---|
| 50 | 1 | 1 | ||
| 50 | 1 | 0 | ||
| 50 | 0 | 1 | ||
| 50 | 0 | 0 |
Exercise 2
A common use of regression discontinuity designs in policy research occurs when funding for a program is limited, and areas are ranked by need to determine whether they will receive the benefits of the program. Only areas with the greatest need receive program benefits.
Explain how this type of design is attempting to create exchangeability. Do you think that this design is effective in that goal? Explain.
Exercise 3
Consider the use of an advertisement to promote a health product. The ultimate goal is to understand the causal effect of the product on an outcome. How might advertisements be released to capitalize on an instrumental variables design?
Explain how this type of design is attempting to create exchangeability. Do you think that this design is effective in that goal? Explain.
Exercise 4
Of the three quasi-experimental study designs we’ve discussed (regression discontinuity, instrumental variables, and interrupted time series), which do you think is most effective in reaching exchangeability? Which do you think is least effective? Explain your thoughts.