UURC Research - Educator Impact

To evaluate the impact of the current project on improving educators' ability to become more effective and efficient at helping struggling readers meet grade level expectations, criterion-based assessment, content knowledge, and self-efficacy measures are administered.

2003 - 2016 Educator Tier I & Tier II Criterion-Referenced Assessment:

To ensure project integrity and evaluate educator efficacy, each educator is observed 6-12 times (depending on the model) during a practicum year. For six of those observations, UURC staff use a checklist of required teaching behaviors (see Appendix A for Tier I observation forms) to provide feedback to each educator:

  • completion of sufficient sessions
  • maintenance of satisfactory records
  • satisfactory execution of instruction/intervention
  • attendance at trainings

Educators volunteer to participate in the project. Since 2003, 1,476 educators have participated in the current project and 1,412 of those educators have successfully completed a UURC practicum, earning certification. Thus, over that time, a total of 64 educators failed to meet one or more criteria described above (e.g., sufficient number sessions completed with students). Those educators did not earn certification.

2007 - 2017 Educator Tier II Content Knowledge Assessment:

To evaluate growth in reading content knowledge, educators complete baseline and post-practicum assessments. Pre-post measures are identical and are composed of multiple choice and short answer items that focus on instructional content. Sample question:

  1. Phonemic Awareness develops in the following order:
  1. initial sound, medial sound, final sound
  2. final sound, medial sound, initial sound
  3. medial sound, initial sound, final sound
  4. initial sound, final sound, medial sound
  5. none of the above
  6. any of the above

Complete Tier II Content Knowledge measures are available upon request.

2007 - 2017 Tier II Educator Content Knowledge Assessment - Baseline & Posttest Results

The average percent correct scored by participants for baseline and post-practicum knowledge assessments provides evidence of participants' marked increased knowledge within practica. The baseline scores for Higher StepsSM provide evidence of participants' increased knowledge across multiple practica (i.e., most Higher StepsSM participants have completed a prior practicum).

 Baseline AveragePost-Practicum Average
Intervention for At-Risk Beginners (Early StepsSM)35.45%63%
Intervention for Struggling Readers (Next StepsSM)34.7%67.2%
Intervention for Advanced Strugglers (Higher StepsSM)52.2%71.1%

2007 - 2017 Tier II Educator Self-Efficacy Assessment - Baseline & Posttest Results:

Evidence for educators' self-efficacy for reading intervention can be found in a comparison of baseline and post-practicum questionnaire data. Educators were asked to score themselves on a number of statements related to reading intervention. The trend in these self-report data show marked improvement. Samples of these data follow; results for the entire post-practicum questionnaire are available upon request.

6. (Before participating in a UURC practicum)      [N = 694]
I am a very effective reading teacher for students who read below level
strongly
disagree
disagreemildly
disagree
mildly
agree
agreestrongly
agree
94410727724549
 
6. (After participating in a UURC practicum)      [n = 728]
I am very effective at providing intervention for a struggling reader in a one-on-one setting.
strongly
disagree
disagreemildly
disagree
mildly
agree
agreestrongly
agree
02150406269
 
After participating in an Early/Next /Higher StepsSM practicum...
Do you plan to use Early/Next/Higher StepsSM with individual students in the future?
No: 19Yes: 716(N = 735)
 
Have you transferred any Early/Next/Higher StepsSM techniques to other settings (e.g., small group, whole class, "outside the schoolday" tutoring)?
No: 164Yes: 571(N = 735)

2014 - 2017 Tier I Educator Self-Efficacy Assessment:

After participating in a UURC pilot Tier I Text practicum, educators completed a self-efficacy assessment to provide data related to their perceptions of this professional development experience. The summaries below provide data sets for Murray, Salt Lake City, and South Sanpete School District Tier I Text practica during 2016-17.

1. Our primary goal for this practicum was to help you become more effective and more efficient at using grade-level text with your students. Which statement best expresses your feelings:

I am much more effective and efficient at using text with my students.   63%
Why?   Sample responses:

  • At the beginning of the year, I was using worksheets to teach reading, now we're spending more time reading and focusing on the text.
  • I now have a plan and format to follow as I teach reading. I didn't have that before and was just "winging" it.
  • I got strong pre-reading skills. I know how to engage students before reading a text. I can give my students a big picture of the text and anticipate their questions.
  • As a first year teacher I have the FORI protocol in place the students are familiar with and are successful in completing each day. What they accomplish every day gives them new reading skills and makes them motivated to continue to read on their own.
  • I've never had good modeling for tier 1 reading instruction, only for small group/guide reading. It also helps me stay on track with reading and not get side-tracked.
  • I'm a first year teacher, so this practicum was incredibly helpful for me to build a foundation for literacy.
  • I am able to quickly get my students in the text. They understand the procedure and my questions are already organized.
  • I used to just follow the teacher manual and hope it was effective... I feel like using this model is a better way to use grade level text with all my students.

I am more effective and efficient at using text with my students.   30%
Why?   Sample responses:

  • I am able to help reading reach all my readers.
  • I have routines in place that allows my students to spend their cognitive work on just reading. Because they are familiar with the routines, they can concentrate on reading and comprehension.
  • I never really understood how to teach reading and comp before.
  • Students get more time reading the text. I'm able to monitor engagement more effectively. Percentage of student engagement is higher (90%).
  • I take more time to look at the text in depth before teaching the story, this has made my teaching of texts more effective.
  • I have never used text multiple times like this before. I can see now how beneficial this is, especially since everyone gets the opportunity to experience on grade text.

I am slightly more effective and efficient at using text with my students.   7%
Why?   No responses:

I am not more effective and efficient at using text with my students.   0%

2. How did the following UURC Tier I components contribute to your students' reading achievement? Please circle your answer for each component.

 Not at AllA LittleA Lot
using grade-level text0%22%78%
using instructional level text0%33%67%
FORI read-to0%22%78%
FORI echo-read0%48%52%
FORI partner read0%22%78%
consistent prompts (e.g., "Fingers on ___.")4%11%85%
a weekly schedule for text consumption0%19%81%
QtA queries for comprehension0%22%78%
planning w/grade level team26%41%33%
being observed & receiving feedback4%19%78%
developing an Organizer for text4%19%78%
developing the Major Understanding for text4%11%85%
miles on the page reading0%26%76%
< teacher talk, > kid talk0%26%76%

3. Prior to beginning FORI/QtA, approximately how many minutes of sustained reading did your students complete daily?

Average daily number of minutes = 22.5 (range = 20-40)

4. Since beginning FORI/QtA, has your students' amount of daily reading (circle one):

Increased = 64%Decreased = 0%Stayed the same = 36%

5. When you use FORI/QtA routines, how many minutes of sustained reading do your students complete daily?

Average daily number of minutes = 32.5 (range = 30-50)

6. Do you use FORI/QtA routines in teaching other content areas? If so, which areas?

Math, Science, Phonics. Some educators responded that they use the routines in all subject areas.