A Brief Introduction to Multi-Stage Trials for Developing Dynamic Treatment Regimens

class: center, middle, inverse, title-slide

# A Brief Introduction to Multi-Stage Trials for Developing Dynamic Treatment Regimens
### Nicholas J. Seewald, Ph.D.
### 22 October 2021

Working Group on Clinical Research, 
University of Rochester Medical Center

---

class: inverse, center, middle

# Follow along!

Slides are available at [urmc-smarts-2021.netlify.app](urmc-smarts-2021.netlify.app)

Materials are partially adapted from workshops given by the [d3lab](https://d3lab.isr.umich.edu) at the University of Michigan Institute for Social Research and from <a name=cite-seewaldSequentialMultipleAssignment2021></a>[Seewald, Hackworth, and Almirall (2021)](https://doi.org/10.1007/978-3-319-52677-5_280-1).

---
# What is a dynamic treatment regimen?

A **dynamic treatment regimen (DTR)** is

- an *intervention* design that 
- adapts the type, timing, intensity, or dose of treatment over time
- according to a patient's specific and changing needs.

In practice, a dynamic treatment regimen is a sequence of decision rules that can be used to guide how treatment can be adapted and readapted to an individual.

--
.center[
### Sounds a lot like clinical practice!
]

--
**MANY other names**: adaptive intervention, adaptive treatment strategy, individualized treatment rule, treatment algorithm, individualized intervention...

---
class: inverse, center, middle

# A dynamic treatment regimen is an *intervention design*, NOT an experimental design.

We'll talk about experiments soon!

---
# 5 components of a DTR

Dynamic treatment regimens consist of

1. Decision points
1. Tailoring variable(s)
1. Intervention options
1. Decision rule(s)
1. Proximal and distal outcomes

---
# Example DTR: Alcohol Use Disorder

- **Naltrexone** is known to diminish pleasurable effects of alcohol <a name=cite-oslinTargetingTreatmentsAlcohol2006></a>([Oslin, Berrettini, and O'Brien, 2006](https://doi.org/10/fgcfbk))

- Response to naltrexone is heterogeneous (adherence, biology, social support, etc.) <a name=cite-nahum-shaniSMARTDataAnalysis2017></a>([Nahum-Shani, Ertefaie, Lu, Lynch, McKay, Oslin, and Almirall, 2017](https://doi.org/10/ghpb9n))

- Heterogeneity necessitates a supportive intervention alongside naltrexone
  - **Medical management**: face-to-face clinical support + adherence monitoring
  - **Combined behavioral intervention**: targets patient's motivation for change, emphasizes social support <a name=cite-longabaughOriginsIssuesOptions2005></a>([Longabaugh, Zweben, Locastro, and Miller, 2005](https://doi.org/10/ghpb9f))

---
# Example DTR: Alcohol Use Disorder

![](images/PP_ClinTri_fig1.png)

--
1. Decision points
1. Tailoring variable(s)
1. Intervention options
1. Decision rule(s)
1. Proximal and distal outcomes

---
# Example DTR: Decision Points

A **decision point** is a time at which treatment might be adapted to the individual.

![:scale 90%](images/decision_points.png)
- **Decision point 1:** Treatment outset -- decide how to initiate treatment
- **Decision point 2:** Adjust treatment after 8 weeks

---
# Example DTR: Tailoring Variables

A **tailoring variable** is used to individualize treatment at each decision point. Can be static (age, baseline risk, etc.) or dynamic (adherence to treatment, disease severity, etc.)

![](images/tailoring_variable.png)

---
# Example DTR: Tailoring Variables

.center[![:scale 80%](images/tailoring_variable.png)]

Here, the tailoring variable is the **number of self-reported heavy drinking days following treatment initiation**.

- `\(\geq\)` 2 heavy drinking days `\(\rightarrow\)` _non-responder_
- `\(<\)` 2 heavy drinking days `\(\rightarrow\)` _responder_

---
# Some Notes on Tailoring Variables

Note that the tailoring variable is _pre-specified_ and _well-defined_.

- Tailoring variables **are part of the intervention**!
  - Should be based on _practical_, _ethical_, or _scientific_ considerations.
  
--

.center[
### We'll come back to how to choose a tailoring variable.
]

---
# Example DTR: Intervention Options

- **Intervention options** at each decision point might be aspects of treatment: type, intensity, dose, delivery method, timing, etc.

- Here, intervention options are combinations of _naltrexone_, _medical management_, and _combined behavioral intervention_

.center[
  ![](images/intervention_options.png)
]

---
# Example DTR: Intervention Options

- In subsequent stages, could be _adaptation strategies_ like "augment", "intensify" or "stay the course".

.center[
  ![](images/intervention_options_strat.png)
]

---
# Example DTR: Decision Rules

.center[
  ![:scale 90%](images/decision_rule_init.png)
]

A **decision rule** recommends an _intervention option_ for individuals at each decision point, possibly based on prior information (i.e., a tailoring variable)

---
# Example DTR: Decision Rules

.center[
  ![:scale 90%](images/decision_rule.png)
]

A **decision rule** recommends an _intervention option_ for individuals at each decision point, possibly based on prior information (i.e., a tailoring variable)

---
# Example DTR: Proximal & Distal Outcomes

A dynamic treatment regimen's design should be guided by both short-term (**proximal**) and long-term (**distal**) outcomes

- **Distal outcomes** are the long-term goals of the DTR
  - Long-term, example DTR should reduce _likelihood of alcohol use relapse_
  
- **Proximal outcomes** are the near-term goals of the DTR; perhaps a mechanism by which we can achieve the distal outcome.
  - Short-term, we want to lower risk of relapse by reducing the _proportion of drinking and heavy-drinking days over 8 weeks after adaptation_ <a name=cite-leiSMARTDesignBuilding2012></a>([Lei, Nahum-Shani, Lynch, Oslin, and Murphy, 2012](https://doi.org/10.1146/annurev-clinpsy-032511-143152))
  
---
# Scientific Questions about DTRs

Often unanswered questions about how to sequence and adapt interventions! Typically related to
  - relative effectiveness of different DTRs
  - relative effectiveness of different intervention options
  - how intervention options work with/against each other

###Common Questions
1. Which treatment option should the dynamic treatment regimen begin with?
1. How should we modify treatment for initial non-responders?
1. How should we modify treatment for initial responders?
1. How do we _define_ response/non-response?
1. How should we time decision points?

---
# Alcohol Use Disorder: Scientific Questions

- How do we define non-response to naltrexone in the context of a dynamic treatment regimen?
  - Unclear how many heavy drinking days indicate a need to augment treatment

- What treatments should be offered following the initial intervention (naltrexone + medical management)?
  - More intense support for non-responders?
  - Do responders need a supportive intervention alongside naltrexone?

## Let's now discuss an experimental design which can help answer these questions

---
class: inverse, middle, center

# Sequential, Multiple-Assignment Randomized Trials (SMARTs)

![](https://c.tenor.com/J2V8sWAs-igAAAAC/geek-cat-studious.gif)

---
# SMARTs

A **sequential, multiple-assignment randomized trial** is _one type_ of randomized trial design that can be used to answer questions at multiple stages of the development of a high-quality dynamic treatment regimen.

The key feature of a SMART is that some (or all) participants are randomized _more than once_.

---
# Example: The ExTENd Study (PI: D. Oslin)

.center[
  ![:scale 70%](images/ExTENd.png)
]

---
# 8 DTRs "embedded" in ExTENd

.center[
  ![:scale 70%](images/ExTENd_embedded_dtr1.png)
]

---
# 8 DTRs "embedded" in ExTENd

.center[
  ![:scale 70%](images/ExTENd_embedded_dtr2.png)
]
---
# 8 DTRs "embedded" in ExTENd

.center[
  ![:scale 70%](images/ExTENd_embedded_dtr3.png)
]

---
# 8 DTRs "embedded" in ExTENd

.center[
  ![:scale 70%](images/ExTENd_embedded_dtr4.png)
]

---
# 8 DTRs "embedded" in ExTENd

.center[
  ![:scale 70%](images/ExTENd_dtrs.gif)
]

---
# How do SMARTs inform DTR development?

Randomizations in a SMART correspond to open scientific questions about constructing a dynamic treatment regimen!

In ExTENd,

- First randomization seeks to answer a question about definition of non-response

- Second randomization in responders investigates whether lower-intensity medical management ("telehealth") is needed as maintenance therapy

- Second randomization in non-responders looks at the effect of naltrexone in the context of combined behavioral intervention + medical management

---
# Do I need a SMART?

SMARTs are designed to answer questions about the development of high-quality dynamic treatment regimes.

Consider a SMART if...

1. You want to develop a DTR
1. There are open questions preventing the construction of an effective DTR
1. There are open questions at _multiple decision points_ within a DTR

If any of these aren't true, you don't need a SMART!

---
# Do I need a SMART?
## What if we know what to do for responders?

.center[
  ![:scale 60%](images/ExTENd_hypothetical_noResp.png)
]

---
# Do I need a SMART?
## What if we know what to do for responders?

.center[
  ![:scale 70%](images/ExTENd_hypothetical_noResp_newTrial.png)
]

---
# Do I need a SMART?
## What if we know what to do initially?

.center[
  ![:scale 60%](images/ExTENd_hypothetical_init.png)
]

---
# Do I need a SMART?
## What if we know what to do initially?

.center[
  ![:scale 80%](images/ExTENd_hypothetical_init_newTrial.png)
]

---
# Design Considerations: Tailoring Variables

Tailoring variables are often used to **restrict randomization** (recommend different intervention options to different subgroups of participants)

Tailoring variables should be **well-justified**: remember, they're part of the DTRs!

You do **not** need to restrict randomization!

---
# Design Considerations: Tailoring Variables

If you restrict randomization, do so based on ethical, scientific, and/or practical considerations.

- **Ethical:** Treatment options should be appropriate for the subset of participants

- **Scientific:** Options should be based on already-established empirical evidence

- **Practical:** You might save more intense or costly options for those who need it most (e.g.)

### Ultimately, _keep it simple_.

---
# Design Considerations: Aims

Focus on a few scientific aims about developing a high-quality DTR.

- **Primary aim:** Choose sample size based on this aim

- **Secondary aims:** Leverage rich data on treatment sequences to further inform a DTR

---
# Design Considerations: Primary Aims
## Compare initial intervention options

.center[
  ![:scale 60%](images/ExTENd_primary_stg1.png)
]

---
# Design Considerations: Primary Aims
## Compare initial intervention options
.pull-left[
.center[
![](images/ExTENd_primary_stg1.png)
]
]

.pull-right[
Hypothetical hypothesis might be:
> A stringent definition of non-response in the context of a dynamic treatment regimen will lead to fewer heavy drinking days, on average, than a lenient definition of non-response.
]

---
# Design Considerations: Primary Aims
## Compare initial intervention options
.pull-left[
.center[
![](images/ExTENd_primary_stg1.png)
]
]

.pull-right[
- Analysis is just a two-group comparison of subgroups A-D vs. subgroups E-H
- Sample size requirements are the same as for a two-arm RCT.
]

---
# Design Considerations: Primary Aims
### Compare second-stage options among (non-)responders

.center[
  ![:scale 60%](images/ExTENd_primary_stg2r.png)
]

---
# Design Considerations: Primary Aims
### Compare second-stage options among (non-)responders

.pull-left[
.center[
  ![scale:60%](images/ExTENd_primary_stg2r.png)
]
]
.pull-right[
Hypothetical hypothesis:
> Individuals who "respond" to naltrexone and medical management benefit more from naltrexone along with a telehealth intervention than naltrexone alone, on average, in terms of proportion of heavy drinking days after 8 weeks.

]

---
# Design Considerations: Primary Aimss
## Compare second-stage options among (non-)responders

.pull-left[
.center[
  ![scale:60%](images/ExTENd_primary_stg2r.png)
]
]
.pull-right[
- Analysis is a two-group comparison of subgroups A & E vs. subgroups B & F
- Sample size is as for a two-group comparison, upweighted by response rate.
]

---
# Design Considerations: Primary Aims
## Compare embedded dynamic treatment regimens

.center[
  ![:scale 60%](images/ExTENd_primary_dtr.png)
]

---
# Design Considerations: Primary Aims
## Compare embedded dynamic treatment regimens

.pull-left[
.center[
  ![](images/ExTENd_primary_dtr.png)
]
]

.pull-right[
  Hypothetical hypothesis:
  > The red DTR will lead to fewer heavy drinking days after 8 weeks, on average, as compared to the blue DTR.
]

---
# Design Considerations: Primary Aims
## Compare embedded dynamic treatment regimens

.pull-left[
.center[
  ![](images/ExTENd_primary_dtr.png)
]
]

.pull-right[
- Analysis is *slightly* more complicated, but still 
- Sample size formulae available (see additional resources)
]

---
# Sample Size Resources
## for comparing dynamic treatment regimens

- **Continuous Outcomes:** <a name=cite-oettingStatisticalMethodologySMART2011></a><a name=cite-ogbagaberDesignSequentiallyRandomized2016></a>[Oetting, Levy, Weiss, and Murphy (2011)](#bib-oettingStatisticalMethodologySMART2011); [Ogbagaber, Karp, and Wahed (2016)](https://doi.org/10.1002/sim.6747)
- **Binary Outcomes:** <a name=cite-kidwellDesignAnalysisConsiderations2018></a>[Kidwell, Seewald, Tran, Kasari, and Almirall (2018)](https://doi.org/10.1080/02664763.2017.1386773)
- **Survival Outcomes:** <a name=cite-fengSampleSizeTwostage2009></a><a name=cite-liSampleSizeFormulae2011></a>[Feng and Wahed (2009)](https://doi.org/10.1002/sim.3593); [Li and Murphy (2011)](https://doi.org/10.1093/biomet/asr019)
- **Cluster-Randomized:** <a name=cite-necampComparingClusterlevelDynamic2017></a>[NeCamp, Kilbourne, and Almirall (2017)](https://doi.org/10.1177/0962280217708654)
- **Longitudinal:** <a name=cite-seewaldSampleSizeConsiderations2020></a>[Seewald, Kidwell, Nahum-Shani, Wu, McKay, and Almirall (2020)](https://doi.org/10/gf85ss)
- **Finding the best embedded DTR:** <a name=cite-artmanPowerAnalysisSMART2018></a>[Artman, Nahum-Shani, Wu, Mckay, and Ertefaie (2018)](https://doi.org/10/ggth75)

---
# Additional Reading

Materials from 3-day workshop on SMARTs and DTRs: <https://d3lab.isr.umich.edu/workshop/getting-smart-about-adaptive-interventions-in-education-2019/>

More in-depth, full book on DTRs:

Kosorok, M.R., and E.E.M. Moodie, eds. 2015. Adaptive Treatment Strategies in Practice: Planning Trials and Analyzing Data for Personalized Medicine. Philadelphia, PA: Society for Industrial and Applied Mathematics. https://doi.org/10.1137/1.9781611974188.

Overview of many different SMART in the field:

Lei, H., I. Nahum-Shani, K. Lynch, D. Oslin, and S.A. Murphy. “A ‘SMART’ Design for Building Individualized Treatment Sequences.” Annual Review of Clinical Psychology 8, no. 1 (2012): 21–48. https://doi.org/10.1146/annurev-clinpsy-032511-143152.

---
# Additional Reading

A nice introduction to SMARTs and DTRs in behavioral science:

Nahum-Shani, I., M. Qian, D. Almirall, W.E. Pelham, B. Gnagy, G.A. Fabiano, J.G. Waxmonsky, J. Yu, and S.A. Murphy. “Experimental Design and Primary Data Analysis Methods for Comparing Adaptive Interventions.” Psychological Methods 17, no. 4 (2012): 457–77. https://doi.org/10.1037/a0029372.

An introduction to "Q-learning" for building more "deeply tailored" DTRs:

Nahum-Shani, I., M. Qian, D. Almirall, W.E. Pelham, B. Gnagy, G.A. Fabiano, J.G. Waxmonsky, J. Yu, and S.A. Murphy. 2012. “Q-Learning: A Data Analysis Method for Constructing Adaptive Interventions.” Psychological Methods 17 (4): 478–94. https://doi.org/10.1037/a0029373.

---
# References

<a name=bib-artmanPowerAnalysisSMART2018></a>[Artman, W. J. et
al.](#cite-artmanPowerAnalysisSMART2018) (2018). "Power Analysis in a
SMART Design: Sample Size Estimation for Determining the Best Embedded
Dynamic Treatment Regime". In: _Biostatistics_. DOI:
[10/ggth75](https://doi.org/10%2Fggth75).

<a name=bib-fengSampleSizeTwostage2009></a>[Feng, W. et
al.](#cite-fengSampleSizeTwostage2009) (2009). "Sample Size for
Two-Stage Studies with Maintenance Therapy". In: _Statistics in
Medicine_ 28.15, pp. 2028-2041. ISSN: 0277-6715. DOI:
[10.1002/sim.3593](https://doi.org/10.1002%2Fsim.3593).

<a name=bib-kidwellDesignAnalysisConsiderations2018></a>[Kidwell, K. M.
et al.](#cite-kidwellDesignAnalysisConsiderations2018) (2018). "Design
and Analysis Considerations for Comparing Dynamic Treatment Regimens
with Binary Outcomes from Sequential Multiple Assignment Randomized
Trials". In: _Journal of Applied Statistics_ 45.9, pp. 1628-1651. ISSN:
0266-4763, 1360-0532. DOI:
[10.1080/02664763.2017.1386773](https://doi.org/10.1080%2F02664763.2017.1386773).

<a name=bib-leiSMARTDesignBuilding2012></a>[Lei, H. et
al.](#cite-leiSMARTDesignBuilding2012) (2012). "A "SMART" Design for
Building Individualized Treatment Sequences". In: _Annual Review of
Clinical Psychology_ 8.1, pp. 21-48. ISSN: 1548-5943, 1548-5951. DOI:
[10.1146/annurev-clinpsy-032511-143152](https://doi.org/10.1146%2Fannurev-clinpsy-032511-143152).

---
# References

<a name=bib-liSampleSizeFormulae2011></a>[Li, Z. et
al.](#cite-liSampleSizeFormulae2011) (2011). "Sample Size Formulae for
Two-Stage Randomized Trials with Survival Outcomes". In: _Biometrika_
98.3, pp. 503-518. ISSN: 00063444. DOI:
[10.1093/biomet/asr019](https://doi.org/10.1093%2Fbiomet%2Fasr019).

<a name=bib-longabaughOriginsIssuesOptions2005></a>[Longabaugh, R. et
al.](#cite-longabaughOriginsIssuesOptions2005) (2005). "Origins, Issues
and Options in the Development of the Combined Behavioral
Intervention." In: _J. Stud. Alcohol Suppl._, pp. 179-187. ISSN:
0363-468X. DOI: [10/ghpb9f](https://doi.org/10%2Fghpb9f).

<a name=bib-nahum-shaniSMARTDataAnalysis2017></a>[Nahum-Shani, I. et
al.](#cite-nahum-shaniSMARTDataAnalysis2017) (2017). "A SMART Data
Analysis Method for Constructing Adaptive Treatment Strategies for
Substance Use Disorders". In: _Addiction_ 112.5, pp. 901-909. ISSN:
1360-0443. DOI: [10/ghpb9n](https://doi.org/10%2Fghpb9n).

---
# References

<a name=bib-necampComparingClusterlevelDynamic2017></a>[NeCamp, T. et
al.](#cite-necampComparingClusterlevelDynamic2017) (2017). "Comparing
Cluster-Level Dynamic Treatment Regimens Using Sequential, Multiple
Assignment, Randomized Trials: Regression Estimation and Sample Size
Considerations". In: _Statistical Methods in Medical Research_ 26.4,
pp. 1572-1589. ISSN: 0962-2802, 1477-0334. DOI:
[10.1177/0962280217708654](https://doi.org/10.1177%2F0962280217708654).

<a name=bib-oettingStatisticalMethodologySMART2011></a>[Oetting, A. I.
et al.](#cite-oettingStatisticalMethodologySMART2011) (2011).
"Statistical Methodology for a SMART Design in the Development of
Adaptive Treatment Strategies". In: _Causality and Psychopathology:
Finding the Determinants of Disorders and Their Cures_. Ed. by P. E.
Shrout, K. M. Keyes and K. Ornstein. New York: Oxford University Press,
pp. 179-205. ISBN: 978-0-19-975464-9.

<a name=bib-ogbagaberDesignSequentiallyRandomized2016></a>[Ogbagaber,
S. B. et al.](#cite-ogbagaberDesignSequentiallyRandomized2016) (2016).
"Design of Sequentially Randomized Trials for Testing Adaptive
Treatment Strategies". In: _Statistics in Medicine_ 35.6, pp. 840-858.
DOI: [10.1002/sim.6747](https://doi.org/10.1002%2Fsim.6747).
---
# References

<a name=bib-oslinTargetingTreatmentsAlcohol2006></a>[Oslin, D. W. et
al.](#cite-oslinTargetingTreatmentsAlcohol2006) (2006). "Targeting
Treatments for Alcohol Dependence: The Pharmacogenetics of Naltrexone".
In: _Addiction Biology_ 11.3-4, pp. 397-403. ISSN: 1369-1600. DOI:
[10/fgcfbk](https://doi.org/10%2Ffgcfbk).

<a name=bib-seewaldSequentialMultipleAssignment2021></a>[Seewald, N. J.
et al.](#cite-seewaldSequentialMultipleAssignment2021) (2021).
"Sequential, Multiple Assignment, Randomized Trials (SMART)". In:
_Principles and Practice of Clinical Trials_. Ed. by S. Piantadosi and
C. L. Meinert. Cham: Springer International Publishing, pp. 1-19. ISBN:
978-3-319-52677-5. DOI:
[10.1007/978-3-319-52677-5_280-1](https://doi.org/10.1007%2F978-3-319-52677-5_280-1).

<a name=bib-seewaldSampleSizeConsiderations2020></a>[Seewald, N. J. et
al.](#cite-seewaldSampleSizeConsiderations2020) (2020). "Sample Size
Considerations for Comparing Dynamic Treatment Regimens in a Sequential
Multiple-Assignment Randomized Trial with a Continuous Longitudinal
Outcome". In: _Stat Methods Med Res_ 29.7, pp. 1891-1912. ISSN:
0962-2802. DOI: [10/gf85ss](https://doi.org/10%2Fgf85ss).

---
class: center, inverse, middle
# Reach out with questions (or ideas)!