The following summary was prepared as an overview of the reliability and validity of the Youth Risk Behavior Survey. The findings, however, are applicable to self-administered, school-based surveys. This summary is included here to address general issues related to reliability and validity of the Healthy Youth Survey. A more complete discussion is available from Lillian.Bensley@doh.wa.gov.
Executive Summary: Reliability and Validity of the Youth Risk Behavior Survey
This briefing paper summarizes the research evidence as to the reliability and validity of the Youth Risk Behavior Survey (YRBS). If a survey has high reliability, the same responses are given when a person completes the same survey on two separate occasions in a short time-frame. Reliability is necessary, but not sufficient, to establish validity. Validity refers to whether the instrument accurately measures what it is intended to measure. Self-report measures are subject to the limitations of memory, cognitive biases such as a desire to present oneself in a positive way, and other intentional and unintentional sources of inaccuracy. Validity can be measured by comparing results to another measure (e.g., comparing smoking self-reports to saliva cotinine levels) or by determining whether the variables measured by the instrument relate to other variables in predictable ways (e.g., testing whether a measure of intelligence is related to grades in school, because intelligence is expected to relate to progress in school).
Reliability
We identified a large-scale study of the test-retest reliability of the YRBS. In this study, 1679 students in ten schools in five states were administered the YRBS twice, fourteen days apart. Over 90% of the items had moderate or better levels of reliability (consistency over time). Four items had inadequate reliability. Three of these items measured low-prevalence behaviors (injection drug use, sexually transmitted disease, and cocaine use in past 30 days) and the fourth measured intentions to use cigarettes in the next year. None of the prevalence rates for behaviors measured by the survey changed significantly across the two administrations, and prevalence rates typically varied less than 2% between the two administrations.
Back to Top
Validity
We identified six studies which addressed interrelationships among YRBS data, and each of these studies provided evidence of the measure's validity. For example, studies of gender and age differences and of differences between in-school youth and dropouts provided results that were consistent with other research. Validity studies of a similar school-based survey were also generally positive.
Comparisons of school-based surveys to other methodologies suggest that more reporting of health risk behaviors is associated with greater privacy/anonymity and more objective (i.e., physiological) measures. Generally, telephone surveys and face-to-face interviews yield the lowest prevalence rates; school-based self-administered surveys yield intermediate rates, and biochemical indicators and indirect techniques designed to increase honesty yield the highest rates. For example, two out of three studies comparing adolescents' self-reported smoking behavior to saliva cotinine levels found that under-reporting, but not over-reporting on the survey was a problem. (The third study found the self-reports to be accurate.) More detail is provided in the report.
Washington State Data
Fewer than 3% of the students completing the 1999 YRBS reported using a fictitious drug. This suggests that relatively few students falsely reported having used drugs.
Data from the 1995 WSSAHB and national data from the 1994 Monitoring the Future study revealed almost identical lifetime prevalence rates of alcohol and cigarette usage. Lifetime usage of marijuana was higher in Washington adolescents compared to nationwide. Data from the 1999 YRBS will be compared to national results, when available.
Conclusions
This summary was prepared in August, 1997, and so research appearing after that time has not been included. The available evidence suggests adequate reliability of the YRBS. Studies of predictable relationships among the data (such as gender and age differences and differences between dropouts and in-school youth) and validity studies of a similar measure provide relatively consistent evidence for validity across a wide variety of behaviors.
Questions about validity which remain are based on differences between different methodologies. School-based self-administered surveys appear to yield higher prevalence than either telephone surveys or face-to-face interviews, but lower prevalence than biochemical indicators or methods which provide even greater anonymity. Biochemical indicators, which provide the most objective comparison data, and low self-reported use of a fictitious drug suggest that most self-reported health risk behaviors on the YRBS are probably valid, but that some under-reporting may occur. Because of these methodological differences, the best use of these data may be to measure trends over time within one methodology. Under-reporting of socially disapproved behaviors, particularly when the individual is more identifiable, is not unique to adolescents but has also been noted in adults.
Questions about Healthy Youth Survey:
Kevin Beck
360-236-3492
Toll free: 877-HYS-7111
Back to Top