Who Gets Identified? The Consequences of Variability in Teacher Ratings and Combination Rules for Determining Eligibility for Gifted Services for Young Children
Abstract
This study illustrates the consequences of accounting for or ignoring teacher variability in student ratings in conjunction with combination rules when identifying students for gifted services in one rural primary school. Teachers (n = 16) rated 282 first-- and second grade students on creativity, motivation, mathematics, and science. Results indicated the most variability in how teachers used the science scale and the least variability in the mathematics scales. Further, teachers rated female students higher than male students in motivation, but not on any other scale. More students were identified if the top students were identified in each class versus the top students in each grade level, and largely, the students who were identified within their classrooms were not the same students who were identified within their grade level. And as expected, OR rules resulted in the highest number of students identified. Implications and recommendations are discussed.
Get full access to this article
View all access and purchase options for this article.
References
Anthony C. J, Styck K. M., Cooke E., Martel J. R., Frye K. E. (2022). Evaluating the impact of rater effects on behavior rating scale score validity and utility. School Psychology Review, 51(1), 25–39. https://doi.org/10.1080/2372966x.2020.1827681
Azano A. P., Callahan C. M., Brodersen A. V., Caughey M. (2017). Responding to the challenges of gifted education in rural communities. Global Education Review, 4(1), 62–77.
Beghetto R. A., Kaufman J. C., Baxter J. (2011). Answering the unexpected questions: Exploring the relationship between students' creative self-efficacy and teacher ratings of creativity. Psychology of Aesthetics, Creativity, and the Arts, 5(4), 342–349. https://doi.org/10.1037/a0022834
Bennett R. E., Gottesman R. L., Rock D. A., Cerullo F. (1993). Influence of behavior perceptions and gender on teachers’ judgments of students’ academic skill. Journal of Educational Psychology, 85(2), 347–356. https://doi.org/10.1037/0022-0663.85.2.347
Biber M., Kose Biber S., Ozyaprak M., Kartal E., Can T., Simsek I. (2021). Teacher nomination in identifying gifted and talented students: Evidence from Turkey. Thinking Skills and Creativity, 39, 100751. https://doi.org/10.1016/j.tsc.2020.100751
Carlana M. (2019). Implicit stereotypes: Evidence from teachers’ gender bias. The Quarterly Journal of Economics, 134(3), 1163–1224. https://doi.org/10.1093/qje/qjz008
Castejón J. L., Gilar R., Miñano P., González M. (2016). Latent class cluster analysis in exploring different profiles of gifted and talented students. Learning and Individual Differences, 50, 166–174. https://doi.org/10.1016/j.lindif.2016.08.003
Copur-Gencturk Y., Cimpian J. R., Lubienski S. T., Thacker I. (2020). Teachers’ bias against the mathematical ability of female, Black, and Hispanic students. Educational Researcher, 49(1), 30–43. https://doi.org/10.3102/0013189x19890577
Dicke A. L., Lüdtke O., Trautwein U., Nagy G., Nagy N. (2012). Judging students' achievement goal orientations: Are teacher ratings accurate? Learning and Individual Differences, 22(6), 844–849. https://doi.org/10.1016/j.lindif.2012.04.004
Evans-McCleon T. N. (in press) [Review of the Scales for Rating the behavioral Characteristics of superior students Third Edition]. In: Carlson J. F., Geisinger K. F., Jonson J. L. (Eds.), The twentieth mental measurements.
Farah Y. N., Chandler K. L. (2018). Structured observation instruments assessing instructional practices with gifted and talented students: A review of the literature. Gifted Child Quarterly, 62(3), 276–288. https://doi.org/10.1177/0016986218758439
Gabriele A. J., Joram E., Park K. H. (2016). Elementary mathematics teachers’ judgment accuracy and calibration accuracy: Do they predict students’ mathematics achievement outcomes? Learning and Instruction, 45, 49–60. https://doi.org/10.1016/j.learninstruc.2016.06.008
Gralewski J., Karwowski M. (2013). Polite girls and creative boys? Students' gender moderates accuracy of teachers' ratings of creativity. The Journal of Creative Behavior, 47(4), 290–304. https://doi.org/10.1002/jocb.36
Gralewski J., Karwowski M. (2018). Are teachers' implicit theories of creativity related to the recognition of their students' creativity? The Journal of Creative Behavior, 52(2), 156–167. https://doi.org/10.1002/jocb.140
Gralewski J., Karwowski M. (2019). Are teachers’ ratings of students’ creativity related to students’ divergent thinking? A meta-analysis. Thinking Skills and Creativity, 33, 100583. https://doi.org/10.1016/j.tsc.2019.100583
Gridley B. E., Treloar J. H. (1984). The validity of the scales for rating the behavioral characteristics of superior students for the identification of gifted students. Journal of Psychoeducational Assessment, 2(1), 65–71. https://doi.org/10.1177/073428298400200108
Grissom J. A., Redding C., Bleiberg J. F. (2019). Money over merit? Socioeconomic gaps in receipt of gifted services. Harvard Educational Review, 89(3), 337–369. https://doi.org/10.17763/1943-5045-89.3.337
Herman J. L., Choi K. (2008). Formative assessment and the improvement of middle school science learning: The role of teacher accuracy. CRESST report 740. In National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
Hunter L. J., Bierman K. L., Hall C. M. (2018). Assessing noncognitive aspects of school readiness: The predictive validity of brief teacher rating scales of social–emotional competence and approaches to learning. Early Education and Development, 29(8), 1081–1094. https://doi.org/10.1080/10409289.2018.1495472
Jarosewich T., Pfeiffer S. I., Morris J. (2002). Identifying gifted students using teacher rating scales: A review of existing instruments. Journal of Psychoeducational Assessment, 20(4), 322–336. https://doi.org/10.1177/073428290202000401
Kaufman A. S., Kaufman N. L. (2014). KTEA-3: Kaufman Test of Educational Achievement: Form A. Pearson.
Kettler R. J., Albers C. A. (2013). Predictive validity of curriculum-based measurement and teacher ratings of academic achievement. Journal of School Psychology, 51(4), 499–515.
Kettler T., Bower J. (2017). Measuring creative capacity in gifted students: Comparing teacher ratings and student products. Gifted Child Quarterly, 61(4), 290–299. https://doi.org/10.1177/0016986217722617
Kolovou D., Naumann A., Hochweber J., Praetorius A. K. (2021). Content-specificity of teachers’ judgment accuracy regarding students’ academic achievement. Teaching and Teacher Education, 100, 103298. https://doi.org/10.1016/j.tate.2021.103298
Lakin J. M. (2018). Making the cut in gifted selection: Score combination rules and their impact on program diversity. Gifted Child Quarterly, 62(2), 210–219. https://doi.org/10.1177/0016986217752099
Lee H., Seward K., Gentry M. (2022). Equitable identification of underrepresented gifted students: The relationship between students’ academic achievement and a teacher-rating scale. Journal of Advanced Academics, 33(3), 400–432. https://doi.org/10.1177/1932202X221088816
Lohman D. F., Lakin J. (2021). Nonverbal test scores as one component of an identification system: Integrating ability, achievement, and teacher ratings Alternative assessments. In VanTassel-Baska J. (Ed.), Alternative assessments with gifted and talented students (pp. 41–66). Routledge. https://doi.org/10.4324/9781003232988-3
Lopata C., Donnelly J. P., Rodgers J. D., Thomeer M. L., Booth A. J. (2020). Reliability and validity of teacher ratings on the Adapted Skillstreaming Checklist for children with autism spectrum disorder. Autism: The International Journal of Research and Practice, 24(5), 1127–1137. https://doi.org/10.1177/1362361319894824
Mashburn A. J., Hamre B. K., Downer J. T., Pianta R. C. (2006). Teacher and classroom characteristics associated with teachers’ ratings of prekindergartners’ relationships and behaviors. Journal of Psychoeducational Assessment, 24(4), 367–380. https://doi.org/10.1177/0734282906290594
Masten W. G., Plata M., Wenglar K., Thedford J. (1999). Acculturation and teacher ratings of Hispanic and Anglo‐American students. Roeper Review, 22(1), 64–65. https://doi.org/10.1080/02783199909554001
McBee M. T., Makel M. C. (2019). The quantitative implications of definitions of giftedness. AERA Open, 5(1), 233285841983100. https://doi.org/10.1177/2332858419831007
McBee M. T., Peters S. J., Waterman C. (2014). Combining scores in multiple-criteria assessment systems: The impact of combination rule. Gifted Child Quarterly, 58(1), 69–89. https://doi.org/10.1177/0016986213513794
McMurrer J. (2008). Instructional time in elementary schools: A closer look at changes for specific subjects, a report in the series, from the capitol to the classroom: Year 5 of the no child left behind act. Center on Education Policy.
Miller E. M. (2009). The effect of training in gifted education on elementary classroom teachers’ theory-based reasoning about the concept of giftedness. Journal for the Education of the Gifted, 33(1), 65–105. https://doi.org/10.1177/016235320903300104
Mullet D. R., Willerson A., Lamb K. N., Kettler T. (2016). Examining teacher perceptions of creativity: A systematic review of the literature. Thinking Skills and Creativity, 21, 9–30. https://doi.org/10.1016/j.tsc.2016.05.001
Neumeister K. L. S., Adams C. M., Pierce R. L., Cassady J. C., Dixon F. A. (2007). Fourth-grade teachers’ perceptions of giftedness: Implications for identifying and serving diverse gifted students. Journal for the Education of the Gifted, 30(4), 479–499. https://doi.org/10.4219/jeg-2007-503
Nordness P. D., Lambert M. C., Geležinienė R., Kuhn M. (2019). Convergent and concurrent validity of the Lithuanian version of the Behavioral and Emotional Rating Scale–2 Teacher Rating Scale. Journal of International Special Needs Education, 22(1), 35–42. https://doi.org/10.9782/16-00049
Office of Civil Rights (n.d). 2017-18 State and National Estimations. https://ocrdata.ed.gov/estimations/2017-2018
Pereira N. (2021). Finding talent among elementary English learners: A validity study of the HOPE teacher rating scale. Gifted Child Quarterly, 65(2), 153–166. https://doi.org/10.1177/0016986220985942
Peters S. J., Gentry M. (2012). Group-specific norms and teacher-rating scales: Implications for underrepresentation. Journal of Advanced Academics, 23(2), 125–144. https://doi.org/10.1177/1932202x12438717
Peters S. J., Makel M. C., Rambo-Hernandez K. (2021). Local norms for gifted and talented student identification: Everything you need to know. Gifted Child Today, 44(2), 93–104. https://doi.org/10.1177/1076217520985181
Petersen J. (2013). Gender differences in identification of gifted youth and in gifted program participation: A meta-analysis. Contemporary Educational Psychology, 38(4), 342–348. https://doi.org/10.1016/j.cedpsych.2013.07.002
Peterson J. S., Margolin L. (1997). Naming gifted children: An example of unintended “reproduction”. Journal for the Education of the Gifted, 21(1), 82–100. https://doi.org/10.1177/016235329702100105
Pfeiffer S. I. (2002). Identifying gifted and talented students: Recurring issues and promising solutions. Journal of Applied School Psychology, 19(1), 31–50. https://doi.org/10.1300/j370v19n01_03
Plata M., Masten W. G., Trusty J. (1999). Teachers’ perception and nomination of fifth-grade Hispanic and Anglo students. Journal of Research & Development in Education, 32(2), 113–123.
Praetorius A. K., Koch T., Scheunpflug A., Zeinz H., Dresel M. (2017). Identifying determinants of teachers’ judgment (in) accuracy regarding students’ school-related motivations using a Bayesian cross-classified multi-level model. Learning and Instruction, 52, 148–160. https://doi.org/10.1016/j.learninstruc.2017.06.003
Reid E. E., Diperna J. C., Missall K., Volpe R. J. (2014). Reliability and structural validity of the teacher rating scales of early academic competence. Psychology in the Schools, 51(6), 535–553. https://doi.org/10.1002/pits.21769
Reis S. M., Sullivan E. E., Renzulli S. J. (2021). Characteristics of gifted learners: Varied, diverse, and complex Methods and materials for teaching the gifted. In Reis S., Sullivan E., Renzulli S. (Eds.), Methods and Materials for Teaching the Gifted (pp. 69–103). Routledge. https://doi.org/10.4324/9781003236603-5
Renzulli J. S. (2011). What makes giftedness? Re-Examining a definition. Phi Delta Kappan, 92(8), 81–88, 261. https://doi.org/10.1177/003172171109200821
Renzulli J. S., Reis S. M. (2018). The three-ring conception of giftedness: A developmental approach for promoting creative productivity in young people. In Pfeiffer S. I., Shaunessy-Dedrick E., Foley-Nicpon M. (Eds.), APA handbook of giftedness and talent (pp. 185–199). American Psychological Association. https://doi.org/10.1037/0000038-012
Renzulli J. S., Siegle D., Reis S. M., Gavin M. K., Reed R. E. S. (2010). An investigation of the reliability and factor structure of four new scales for rating the behavioral characteristics of superior students. Journal of Advanced Academics, 21(1), 84–108. https://doi.org/10.1177/1932202x0902100105
Renzulli J. S., Smith L. H., White A. J., Callahan C. M., Hartman R. K., Westberg K. L., Gavin M. K., Reis S. M., Siegle D., Sytsma R. E. (2013). Scales for Rating the Behavioral Characteristics of Superior Students. Psychology and education of the gifted. https://doi.org/10.1037/t24062-000
Ridgley L. M., DaVia Rubenstein L., Finch W. H. (2019). Issues and opportunities when using rating scales to identify creatively gifted students: Applying an IRT approach. Gifted and Talented International, 34(1-2), 6–18. https://doi.org/10.1080/15332276.2020.1722041
Riegle-Crumb C., Humphries M. (2012). Exploring bias in math teachers’ perceptions of students’ ability by gender and race/ethnicity. Gender & Society, 26(2), 290–322. https://doi.org/10.1177/0891243211434614
Roda A. (2017). Parenting in the age of high-stakes testing: Gifted and talented admissions and the meaning of parenthood. Teachers College Record, 119(8), 1–53. https://doi.org/10.1177/016146811711900804
Şahin F., Çetinkaya Ç. (2015). An investigation of the effectiveness and efficiency of classroom teachers in the identification of gifted students. Türk Üstün Zekâ ve Eğitim Dergisi, 5(2), 133–146.
Schrank F. A., Mather N., McGrew K. S. (2014). Woodcock-Johnson IV Tests of Achievement. Riverside.
Siegle D., Moore M., Mann R. L., Wilson H. E. (2010). Factors that influence in-service and preservice teachers’ nominations of students for gifted and talented programs. Journal for the Education of the Gifted, 33(3), 337–360. https://doi.org/10.1177/016235321003300303
Siegle D., Powell T. (2004). Exploring teacher biases when nominating students for gifted programs. Gifted Child Quarterly, 48(1), 21–29. https://doi.org/10.1177/001698620404800103
Skinner E. A., Belmont M. J. (1993). Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal of Educational Psychology, 85(4), 571–581. https://doi.org/10.1037/0022-0663.85.4.571
Smith R. L., Eklund K., Kilgus S. P. (2018). Concurrent validity and sensitivity to change of direct behavior rating single-item scales (DBR-SIS) within an elementary sample. School Psychology Quarterly, 33(1), 83–93. https://doi.org/10.1037/spq0000209
Splett J. W., Raborn A., Brann K., Smith-Millman M. K., Halliday C., Weist M. D. (2020). Between-teacher variance of students’ teacher-rated risk for emotional, behavioral, and adaptive functioning. Journal of School Psychology, 80, 37–53. https://doi.org/10.1016/j.jsp.2020.04.001
Stambaugh T. (2007). Next steps: An impetus for future directions in research, policy, and practice for low-income promising learners. In VanTassel-Baska J., Stambaugh T. (Eds.), Over- looked gems: A national perspective on low-income promising learners (pp. 83–88). National Association for Gifted Children.
Standage M., Duda J. L., Ntoumanis N. (2006). Students’ motivational processes and their relationship to teacher ratings in school physical education: A self-determination theory approach. Research Quarterly for Exercise and Sport, 77(1), 100–110. https://doi.org/10.1080/02701367.2006.10599336
Styck K. M., Anthony C. J., Sandilos L. E., DiPerna J. C. (2021). Examining rater effects on the classroom assessment scoring system. Child Development, 92(3), 976–993. https://doi.org/10.1111/cdev.13460
Sweet A. P., Guthrie J. T., Ng M. M. (1998). Teacher perceptions and student reading motivation. Journal of Educational Psychology, 90(2), 210–223. https://doi.org/10.1037/0022-0663.90.2.210
Szymanski T., Shaff T. (2013). Teacher perspectives regarding gifted diverse students. Gifted Children, 6(1), 1–27.
Urhahne D. (2011). Teachers’ judgments of elementary students’ ability, creativity and task commitment. Talent Development & Excellence, 3(2), 229–237.
Urhahne D., Chao S. H., Florineth M. L., Luttenberger S., Paechter M. (2011). Academic self‐concept, learning motivation, and test anxiety of the underestimated student. British Journal of Educational Psychology, 81(1), 161–177. https://doi.org/10.1348/000709910x504500
Urhahne D., Wijnia L. (2021). A review on the accuracy of teacher judgments. Educational Research Review, 32, 100374. https://doi.org/10.1016/j.edurev.2020.100374
VanTassel-Baska J. L. (2008). Epilogue: What do we know about identifying and assessing the learning of gifted stu- dents? In VanTassel-Baska J. L. (Ed.), Alternative assessment with gifted and talented students (pp. 309–319). Prufrock Press.
V Hoff E., Carlsson I. (2011). Teachers are not always right: Links between teacher ratings and students' creativity scores, self-images and self-ratings in school subjects. The Open Education Journal, 4(1), 120–129. https://doi.org/10.2174/1874920801104010120
Wechsler D. (2014). Wechsler Intelligence Scale for Children (5th ed.). Bloomington, MN: Pearson, Inc.
Woods-Groves S., Choi T., Balint-Langel K. (2021). Examining teachers' judgment of students' 21st century skills and academic and behavioral outcomes. Psychology in the Schools, 58(11), 2202–2224. https://doi.org/10.1002/pits.22585
Worrell F. C. (2009). Myth 4: A single test score or indicator tells us all we need to know about giftedness. Gifted Child Quarterly, 53(4), 242–244. https://doi.org/10.1177/0016986209346828
Worrell F. C., Erwin J. O. (2011). Best practices in identifying students for gifted and talented education programs. Journal of Applied School Psychology, 27(4), 319–340. https://doi.org/10.1080/15377903.2011.615817
Zhu M., Urhahne D. (2015). Teachers’ judgements of students’ foreign-language achievement. European Journal of Psychology of Education, 30(1), 21–39. https://doi.org/10.1007/s10212-014-0225-6
Biographies
Karen E. Rambo-Hernandez, PhD, is an associate professor at Texas A&M University in the School of Education and Human Development. Her research focuses on the assessment of educational interventions to improve STEM education, and access for all students—particularly high-achieving and underrepresented students—to high-quality education.
Carla Brigandi, PhD, is an associate professor at West Virginia University where she teaches courses in educational psychology, research methods, and gifted and special education. Her scholarship aims to improve educational opportunities for students with high academic ability, focusing particularly on rural education, evidence-based enrichment practices, and teacher professional learning.
Syahrul Amin is a Fulbright scholar and a third-year PhD student in curriculum and instruction, with an emphasis in science education at Texas A&M University. He graduated with an MS degree in the same program in 2020. He believes that diversity can empower science education.
Nancy K. Spillane, EdD, is a service professor at West Virginia University and a Master Teacher in the WVUteach program. Dr. Spillane’s current research endeavors to impact the quality, participation, and diversity of learning in STEM fields.
Cite
Cite
Cite
OR
Download to reference manager
If you have citation software installed, you can download citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: May 12, 2023
Issue published: June 2023
Keywords
Authors
Metrics and citations
Metrics
Journals metrics
This article was published in Journal for the Education of the Gifted.
View All Journal MetricsPublication usage*
Total views and downloads: 795
*Publication usage tracking started in December 2016
Altmetric
See the impact this article is making through the number of times it’s been read, and the Altmetric Score.
Learn more about the Altmetric Scores
Publications citing this one
Receive email alerts when this publication is cited
Web of Science: 6 view articles Opens in new tab
Crossref: 8
- Are Teacher Rating Scales of Gifted Student Behavior Just Measures of Personality?
- Cracking the Code: Enhancing Teacher Self-Efficacy and Student Engagement Through Professional Learning in Computer Science and Gifted Education
- State Definitions, Policies, and Practices of Gifted and Talented Identification: What Difference Does a Decade Make?
- Addressing Equity Challenges and Expanding Opportunities in Gifted Education for Rural Multilingual Learners
- Teacher Rating Scales Continue to be a Problematic Source of Identification: Evidence From Ratings of Primary Students in a Rural, Low Socioeconomic School
- Who Are We Measuring? Teacher Effects in Gifted and Talented Teacher Rating Scales
- What Happens After Nomination? Evaluating the Probability of Gifted Identification With the Torrance Test of Creative Thinking
- Exceptional Minds in Regular Classrooms: Teachers’ Lived Experiences Through a Phenomenological Lens
Figures and tables
Figures & Media
Tables
View Options
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
I am signed in as:
View my profileSign out
I can access personal subscriptions, purchases, paired institutional access and free tools such as favourite journals, email alerts and saved searches.
loading institutional access options
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
