D.A.R.E. keepin’ it REAL Elementary School Curriculum is Evidence-based, Successful and Effective

An Evaluation of the “D.A.R.E.: keepin’ it REAL” Elementary School Program

Executive Summary

January 2022

Introduction

In 2018 D.A.R.E. America entered into a contract with UNC Greensboro (UNCG) and Prevention Strategies to complete an independent evaluation of the “D.A.R.E.: keepin’ it REAL” elementary school program. The goal of the project was to provide an independent assessment of the impact of D.A.R.E. on key psychosocial and behavioral outcomes among elementary school children. In addition, the evaluation assessed aspects important to understanding quality of delivery and fidelity.

The following is an Executive Summary of the results of the evaluation. The research team is preparing an academic manuscript for publication in the Journal of Primary Prevention. The academic publication will provide a detailed description of the evaluation purpose, protocol, and findings that support D.A.R.E. as an evidence-based, effective program.

Program

The “D.A.R.E. keepin’ it REAL” elementary school program consists of ten 45-minute lessons that also include take-home family talk activities. The curriculum, designed based on the Socio-Emotional Learning Theory (SEL), identifies fundamental, basic skills and developmental processes needed for healthy development including: (1) self-awareness and management, (2) responsible decision making, (3) understanding others, (4) relationship and communication skills, and (5) handling responsibilities and challenges.

Key Findings

The “D.A.R.E.: keepin’ it REAL” elementary school program is delivered by certified D.A.R.E. officers with high fidelity and their delivery is engaging to students. It is effective and successful in the long-term reduction of drinking alcohol, getting drunk, smoking cigarettes, and vaping.

Highlights of the evaluation include:

• Year 1 Elementary School- There were statistically significant reductions in the prevalence of drinking alcohol, getting drunk, smoking cigarettes, and vaping among students who received D.A.R.E. compared to the control cases.
• D.A.R.E. Officers performed very well in the classroom based on both teacher and student ratings. Teachers and students rated D.A.R.E. Officers extremely high on both program fidelity (9.4 out of 10 – delivering the program as intended) and engagement (9.32 out of 10 – actively engaging students during program delivery).
• The teachers’ ratings of officer implementation were highly correlated with students’ ratings of personal engagement (r = .813; p < .0001).
• D.A.R.E. Officer instruction was directly related to positive program outcomes. That is, there were statistically significant correlations between student engagement and student scores at posttest on all variables assessed – increased student engagement was related to positive outcomes on all variables.
• The responsiveness of the D.A.R.E. Officer to the students was also predictive of scores at post-test for variables like decision-making, intentions to avoid drug use, and beliefs about peer norm use.

Drug Use Findings

Results reported here are based on analyses performed using Virtual Controls (see Retention section below). At pretest, there were no instances of marijuana use. As cohort ages, the likelihood of marijuana use increases. At posttest, there were no new cases of marijuana use identified. Therefore, it can be concluded that the program was entirely successful in preventing marijuana use.

D.A.R.E. results compare pretest to posttest and pretest to follow-up changes for elementary school analyses for past 30-day drinking alcohol, getting drunk, smoking cigarettes, using marijuana, and vaping. There is typically very little change in prevalence between pretest and immediate posttest, which is typically 10 to 12 weeks following the pretest. From pretest-to-posttest, there was a slight reduction in self-reported drunkenness, smoking cigarettes, and vaping.

A follow-up survey was administered approximately 4-months following program implementation. In this case there were statistically significant reductions in the prevalence of drinking alcohol, getting drunk, smoking cigarettes, and vaping among students who received D.A.R.E. compared to the virtual control cases.

Fidelity Findings

Classroom teachers in 45 elementary school classrooms in 9 states in which D.A.R.E. Officers implemented D.A.R.E. rated fidelity. For each lesson, teachers completed a structured survey instrument that asked them to note which lesson activities were completed, how well lesson objectives had been achieved, how energetic and prepared the officer was, how attentive students were, how many students were engaged in the lesson, how many asked and answered questions, and if there were student discipline problems. An analysis of teachers’ data revealed that there were three underlying constructs: (1) teachers’ ratings of student responsiveness, (2) teachers’ ratings of officer implementation and (3) teachers’ ratings of the number of activities delivered.

Students in classrooms also answered questions about their participation. They rated the officer’s teaching, their enjoyment of the program, and how often they shared their personal opinions and paid attention. They rated how much they liked the officer, if the officer paid attention to them, and if the officer knew their name. They assessed how much the program helped them think about what was important. An analysis of students’ data revealed two constructs: (1) students’ ratings of engagement and (2) students’ sharing their opinion and being known by the officer.

D.A.R.E. Officers performed very well in the classroom based on both teacher and student ratings. Teachers and students rated D.A.R.E. Officers very high on both program fidelity (9.4 out of 10 – delivering the program as intended) and engagement (9.32 out of 10 – actively engaging students during program delivery). The teachers’ ratings of officer implementation were highly correlated with students’ ratings of personal engagement (r = .813; p < .0001).

D.A.R.E. Officer instruction was directly related to positive program outcomes. That is, there were statistically significant correlations between student engagement and student scores at posttest on all variables assessed – increased student engagement was related to positive outcomes on all variables. The responsiveness of the D.A.R.E. Officer to the students was also predictive of scores at posttest for variables like decision-making, intentions to avoid drug use, and beliefs about peer norm use.

Design

The design of the evaluation included recruiting two cohorts of 5th grade students (i.e., treatment and control). We recruited elementary treatment schools (47 schools in 9 states; 176 classrooms; and 3,266 students) in which D.A.R.E. Officers delivered the program. Control schools (12 schools; 27 classrooms; 368 students), in which no D.A.R.E. program was delivered, were also recruited. All participating students (treatment and control) were assessed at pretest, immediately after the program was delivered to the treatment students (posttest), and then with a 4-month follow-up survey. In addition to student surveys, teachers in whose rooms the program was delivered were asked to complete fidelity assessments for the D.A.R.E. Officers.

Student surveys assessed demographics (age, gender, and race/ethnicity), key psychosocial variables targeted by the program (beliefs about harmful consequences, bullying self-efficacy, decision making skills, intentions/commitment, lifestyle incongruence, normative beliefs, perceived parental attitudes/attentiveness, and peer pressure refusal skills) and drug use behaviors (past 30-day alcohol, drunkenness, smoking, vaping, and marijuana). At the immediate posttest, the student survey also included questions regarding their engagement in the D.A.R.E. program.

Evaluation Protocol

We obtained classroom rosters for participating 5th grade classrooms and participating students were assigned a unique identification number that was used for the entirety of the evaluation. Individual schools and classrooms were also assigned identification numbers for tracking and analyses purposes. Schools were given the option of participating in the evaluation online (via a Qualtrics survey link) or with paper surveys. Prior to the first D.A.R.E. lesson, teachers were provided with paper surveys or classroom survey links for their students. At the 8th lesson, teachers were contacted again and provided with paper copies of the posttest, or the Qualtrics link. Teachers were encouraged to send back fidelity observation forms at the conclusion of the last D.A.R.E. lesson. Schools were able to implement D.A.R.E. lessons in either the fall or spring semester of the 2019-2020 academic year. Schools that began lessons in the spring of 2020, were significantly affected by the COVID-19 pandemic requiring completion of online surveys. For a detailed recollection of COVID-19 protocol and how that impacted the 4-month follow-up survey for the spring 2020 students, see below.

Retention

The coronavirus directly impacted the evaluation. Only 31% of elementary students provided posttest surveys. At the second follow-up 38% of elementary students provided survey data. However, in a separate project funded by the National Institutes of Health (NIH) (Grant ID: 5R44AA024657), our research team developed an alternative technology, Virtual Controls, that allowed us to use students’ psychosocial scores at pretest to create an alternative way for estimating how a control group would have performed. Due to the low number of control students recruited and retained for this evaluation, we relied on the Virtual Controls tool to make treatment/control comparisons.

Academic references for Virtual Controls include:

Hansen, W.B., Derzon, J.H., & Reese, E.L. (2014). A Synthetic Comparator Approach to Local Evaluation of School-Based Substance Use Prevention Programming. 37(2), 258-282. https://doi.org/10.1177/0163278712464772.

Hansen, W.B., Chen, S.H., Santiago, S., & Edward, H. (2018). An algorithm for creating virtual controls using integrated and harmonized longitudinal data. Evaluation & The Health Professions. 41(2), 183-215.

*** A brief description of this method can be found at https://vimeo.com/486993156