Study Results for the Critical Race Framework Study:
The Official Website
Abstract
Race is one of the most common variables in public health surveillance and research. Yet, studies involving racial measures show poor conceptual clarity and inconsistent operational definitions. There does not exist a bias tool in the public health literature for structured qualitative evaluation in critical areas of critical appraisal – reliability, validity, internal validity, and external validity – for studies that use racial taxonomy. This study developed the Critical Race (CR) Framework to address a major gap in the literature.
The study involved three iterative phases to answer five research questions (RQs). Phase I was a pilot study of the CR Framework among public health faculty and doctoral students to assess measures of fit (RQ1) and to identify areas of improvement in training, instrumentation, and study design (RQ2). Study participants received training and performed a single article evaluation. Phase II was a national cross-sectional study of public health experts to assess perceptions of the revised training and tool to assess measures of fit (RQ1), to determine the influence of demographic and research factors on perceptions (RQ3), and to gather validity evidence on constructs (RQ4). In Phase III, three raters performed article evaluations to support reliability evidence (RQ4) and to determine the quality of health disparities and behavioral health research studies against the CR Framework (RQ5).
We assessed the reliability of study results and the CR Framework using non-differentiation analysis, thematic analysis, missingness analysis, user data, measures of internal consistency for adopted instruments, interrater agreement, and interrater reliability. Validity was assessed using content validity (CVI and k*), construct validity, and exploratory factor analyses (EFA).
The study recruited 30 highly skilled public health experts across its three phases as part of the final analytic sample. Phase I had poor reliability in which the results could not be confidently interpreted (RQ1) and indicated needed improvement in study design, training, and instrumentation (RQ2). Based on Phase II results, we met or exceeded acceptable thresholds for measures of fit – acceptability, appropriateness, feasibility, and satisfaction (RQ1).
Demographic or research factors were not associated with responses (RQ3). Interrater agreement was moderate to high among rater pairs (RQ4). Due to lack of confidence in significance testing, interrater reliability results were inconclusive. Overall data results showed excellent content validity. Based on EFA results, construct validity for reliability and validity items was poor to fair (RQ4). Data results were inconclusive on internal validity and external validity. The twenty studies used in critical appraisal showed low quality or no discussion when the Critical Race Framework was used (RQ5).
The CR Framework study developed a tool and training with quality evidence for implementation effectiveness, content validity, and interrater reliability to fill a major gap in the public health literature. It contributed an innovative theory-based tool and training to the literature. Future research should seek to study individual perceptions and practices that influence outcomes of CR Framework application and to reduce barriers to ensure that minimum sample sizes can be met for additional testing.
How to cite:
MLA - Williams, Christopher Menvell. The Critical Race Framework Study: Standardizing Critical Evaluation for Research Studies that Use Racial Taxonomy. Diss. University of Maryland, College Park, 2024.
APA - Williams, C. M. (2024). The Critical Race Framework Study: Standardizing Critical Evaluation for Research Studies that Use Racial Taxonomy (Doctoral dissertation, University of Maryland, College Park).
About Study Results
This dissertation study represents a significant and timely contribution to the field of public health research methodology. The development of the Critical Race Framework (CRF) addresses a crucial gap in the literature by providing a much-needed tool for critically appraising studies that use racial taxonomy.
The study's rigorous approach to developing and refining the CRF is particularly commendable. By employing a multi-phase design that incorporated expert feedback and iterative improvements, the researchers ensured that the final tool is both theoretically grounded and practically applicable. The emphasis on establishing reliability and validity through multiple methods, including content validity assessment, exploratory factor analysis, and interrater reliability testing, demonstrates a commitment to scientific rigor that is essential for any new research instrument.
One of the study's greatest strengths is its comprehensive approach to addressing the complex issues surrounding race in public health research. By focusing on four critical areas of appraisal - reliability, validity, internal validity, and external validity - the CRF provides researchers with a structured framework for evaluating the quality and appropriateness of racial variables in health studies. This holistic approach has the potential to significantly improve the methodological rigor of health disparities research and behavioral health studies.
The researchers' attention to implementation effectiveness is also noteworthy. By assessing measures of fit such as acceptability, appropriateness, feasibility, and satisfaction among public health experts, they have ensured that the CRF is not only theoretically sound but also practical and user-friendly. This focus on real-world applicability increases the likelihood that the tool will be widely adopted and utilized in the field.
Furthermore, the study's findings regarding the low quality or absence of discussion around racial variables in existing health disparities and behavioral health literature underscore the critical need for a tool like the CRF. By highlighting these deficiencies, the study makes a compelling case for the widespread adoption of more rigorous standards in race-based research.
In conclusion, this dissertation study represents a significant advancement in the field of public health research methodology. The Critical Race Framework developed through this work has the potential to transform how researchers conceptualize, collect, analyze, and interpret race-based data in health studies. By providing a standardized, theoretically-grounded, and practically-applicable tool for critical appraisal, this study makes a valuable contribution to improving the quality and validity of health disparities research. The implications of this work extend beyond academia, with the potential to positively impact public health policy, practice, and ultimately, health outcomes for diverse populations.
*Review was generated using AI (Perplexity)
Dedication
I dedicated this dissertation to the women leaders in my community of practice – Mrs. P. Bishop, Mrs. L. Brown, Commissioner R. Hamilton, Mrs. C. Spencer, and Mrs. D. Walker. Your community leadership and lifelong service in Washington, DC enriched my understanding of contemporary structural racism and health inequity reproduction. You provided an essential part of my professional identity and training. Together, we published our manuscript on elucidating and affecting the public health economy (C. Williams, Birungi, et al., 2022). The policies, poor agency performance, and societal attitudes that perpetuate particularized harm against vulnerable communities deserve heightened scrutiny in public health research, policy, and practice. The ostensible concordance with those to whom we targeted our advocacy and research compelled me, as a researcher, to directly confront major drawbacks in public health theory and practices. The current paradigm not only impedes scientific progress, but also obscures unique community experiences like yours that explain the persistence of vast health inequity and structural racism. Families in low-income housing warrant sustained focus in the discipline of public health.
ANC6D Commissioner Rhonda Hamilton led community opposition to the air permit application of Vulcan - a polluter in Southwest DC, long associated the District's legacy of environmental racism in the neighborhood. The meeting was held on June 26 at Southwest Library. In addition to dozens of community voices, Southwest Voice Editor-in-Chief and Public Health Liberation Founding Director Dr. Christopher Williams also testified.