SAMPLE DESIGN AND ANALYSIS GUIDELINES Sample Design The general structure of the NHANES III sample design is the same as that of the previous NHANES. Each of these surveys used a stratified, multi-stage probability design. The major design parameters of the two previous NHANES and the special Hispanic HANES, as well as NHANES III, have been previously summarized (Miller, 1973; McDowell, 1981; NCHS, 1985; NCHS, 1994). The NHANES III sample was designed to be self-weighting within a primary sampling unit (PSU) for subdomains (age, sex, and race-ethnic groups). While the sample was fairly close to self-weighting nationally for each of these subdomain groups, it was not representative of the total population, which includes institutionalized, non-civilian persons that were outside the scope of the survey. The NHANES III sample represented the total civilian, noninstitutionalized population, two months of age or over, in the 50 states and the District of Columbia of the United States. The first stage of the design consisted of selecting a sample of 81 PSU's that were mostly individual counties. In a few cases, adjacent counties were combined to keep PSU's above a minimum population size. The PSU's were stratified and selected with probability proportional to size (PPS). Thirteen large counties (strata) were chosen with certainty (probability of one). For operational reasons, these 13 certainty PSU's were divided into 21 survey locations. After the 13 certainty strata were designated, the remaining PSU's in the United States were grouped into 34 strata, and two PSU's were selected per stratum (68 survey locations). The selection was done with PPS and without replacement. The NHANES III sample therefore consists of 81 PSU's or 89 locations. The 89 locations were randomly divided into two groups, one for each phase. The first group consisted of 44 and the other of 45 locations. One set of PSU's was allocated to the first three-year survey period (1988-91) and the other set to the second three-year period (1991-94). Therefore, unbiased estimates (from the point of view of sample selection) of health and nutrition characteristics can be independently produced for both Phase 1 and Phase 2 as well as for both phases combined. For most of the sample, the second stage of the design consisted of area segments composed of city or suburban blocks, combinations of blocks, or other area segments in places where block statistics were not produced in the 1980 Census. In the first phase of NHANES III, the area segments were used only for a sample of persons who lived in housing units built before 1980. For units built in 1980 and later, the second stage consisted of sets of addresses selected from building permits issued in 1980 or later. These are referred to as "new construction segments." In the second phase, 1990 Census data and maps were used to define the area segments. Because the second phase followed within a few years of the 1990 Census, new construction did not account for a significant part of the sample, and the entire sample came from the area segments. The third stage of sample selection consisted of households and certain types of group quarters, such as dormitories. All households and eligible group quarters in the sample segments were listed, and a subsample was designated for screening to identify potential sample persons. The subsampling rates enabled production of a national, approximately equal-probability sample of households in most of the United States with higher rates for the geographic strata with high Mexican-American populations. Within each geographic stratum, there was a nearly equal-probability sample of households across all 89 stands. Persons within the sample of households or group quarters were the fourth stage of sample selection. All eligible members within a household were listed, and a subsample of individuals was selected based on sex, age, and race or ethnicity. The definitions of the sex, age, race or ethnic classes, subsampling rates, and designation of potential sample persons within screened households were developed to provide approximately self-weighting samples for each subdomain within geographic strata and at the same time to maximize the average number of sample persons per sample household. Previous NHANES indicated that this increased the overall participation rate. Although the exact sample sizes were not known until data collection was completed, estimates were made. Below is a summary of the sample sizes for the full six-year NHANES III at each stage of selection: Number of PSU's 81 Number of stands (survey locations) 89 Number of segments 2,144 Number of households screened 93,653 Number of households with sample persons 19,528 Number of designated sample persons 39,695 Number of interviewed sample persons 33,994 Number of MEC-examined sample persons 30,818 Number of home-examined sample persons 493 More detailed information on the sample design and weighting and estimation procedures for NHANES III can be found in the Plan and Operation of the Third National Health and Nutrition Examination Survey, 1988-94 (NCHS, 1994; U.S. DHHS, 1996) and in the Analytic and Reporting Guidelines: Third National Health and Nutrition Examination Survey (NHANES III), 1988-94 (U.S. DHHS, 1996). Analysis Guidelines Because of the complex survey design used in NHANES III, traditional methods of statistical analysis based on the assumption of a simple random sample are not applicable. Detailed descriptions of this issue and possible analytic methods for analyzing NHANES data have been described earlier (NCHS, 1985; Yetley, 1987; Landis, 1982; Delgado, 1990). Recent analytic and reporting guidelines that should be used for most NHANES III analyses and publications are contained in Analytic and Reporting Guidelines (U.S. DHHS, 1996). These recommendations differ slightly from those used by analysts for previous NHANES surveys. These suggested guidelines provide a framework to users for producing estimates that conform to the analytic design of the survey. All users are strongly urged to review these analytic and reporting guidelines before beginning any analyses of NHANES III data. It is important to remember that this set of statistical guidelines is not absolute. When conducting analyses, the analyst needs to use his/her subject matter knowledge (including methodological issues) as well as information about the survey design. The more one deviates from the original analytic categories defined in the sample design, the more important it is to evaluate the results carefully and to interpret the findings cautiously. In NHANES III, 89 survey locations were randomly divided into two sets or phases, the first consisting of 44 and the other of 45 locations. One set of PSU's was allocated to the first three-year survey period (1988-91) and the other set to the second three-year period (1991-94). Therefore, unbiased national estimates of health and nutrition characteristics can be independently produced for each phase as well as for both phases combined. Computation of national estimates from both phases combined (i.e., total NHANES III) is the preferred option; individual phase estimates may be highly variable. In addition, individual phase estimates are not statistically independent. It is also difficult to evaluate whether differences in individual phase estimates are real or due to methodological differences. That is, differences may be due to changes in sampling methods or data collection methodology over time. At this time, there is no valid statistical test for examining differences between Phase 1 and Phase 2. Therefore, although point estimates can be produced separately for each phase, no test is available to test whether those estimates are significantly different from each other. NHANES III is based on a complex, multi-stage probability sample design. Several aspects of the NHANES design must be taken into account in data analysis, including the sample weights and the complex survey design. Appropriate sample weights are needed to estimate prevalence, means, medians, and other statistics. Sample weights are used to produce correct population estimates because each sample person does not have the same probability of selection. The sample weights incorporate the differential probabilities of selection and include adjustments for noncoverage and nonresponse. A detailed discussion of nonresponse adjustments and issues related to survey coverage have been published (U.S. DHHS, 1996). With the large over sampling of young children, older persons, black persons, and Mexican-Americans in NHANES III, it is essential that the sample weights be used in all analyses. Otherwise, a misinterpretation of results is highly likely. Other aspects of the design that must be taken into account in data analyses are the strata and PSU pairings from the sample design. These pairings should be used to estimate variances and test for statistical significance. For weighted analyses, analysts can use special computer software packages that use an appropriate method for estimating variances for complex samples such as SUDAAN (Shah, 1995) and WesVarPC (Westat, 1996). Although initial exploratory analyses may be performed on unweighted data using standard statistical packages and assuming simple random sampling, final analyses should be done on weighted data using appropriate sample weights. A summary of the weighting methodology and the type of sample weights developed for NHANES III is included in Weighting and Estimation Methodology (U.S. DHHS, 1996). The purpose of weighting the sample data is to permit analysts to produce estimates of statistics that would have been obtained if the entire sampling frame (the United States) had been surveyed. Sample weights can be considered as measures of the number of persons the particular sample observation represents. Weighting takes into account several features of the survey: the specific probabilities of selection for the individual domains that were over sampled as well as nonresponse and differences between the sample and the total U.S. population. Differences between the sample and the population may arise due to sampling variability, differential under coverage in the survey among demographic groups, and possibly other types of response errors, such as differential response rates or misclassification errors. Sample weighting in NHANES III was used to: 1. Compensate for differential probabilities of selection among subgroups (i.e., age-sex-race-ethnicity subdomains where persons living in different geographic strata were sampled at different rates); 2. Reduce biases arising from the fact that nonrespondents may be different from those who participate; 3. Bring sample data up to the dimensions of the target population totals; 4. Compensate, to the extent possible, for inadequacies in the sampling frame (resulting from omissions of some housing units in the listing of area segments, omissions of persons with no fixed address, etc.); and 5. To reduce variances in the estimation procedure by using auxiliary information that is known with a high degree of accuracy. In NHANES III, the sample weighting was carried out in three stages. The first stage involved the computation of weights to compensate for unequal probabilities of selection (objective 1, above). The second stage adjusted for nonresponse (objective 2). The third stage used poststratification of the sample weights to Census Bureau estimates of the U.S. population to accomplish the third, fourth, and fifth objectives simultaneously. In NHANES III, several types of sample weights (see the sample weights table that follows) were computed for the interviewed and examined sample and are included in the NHANES III data file. Also, sample weights were computed separately for Phase 1 (1988-91), Phase 2 (1991-94), and total NHANES III (1988-94) to facilitate analysis of items collected only in Phase 1, only in Phase 2, and over six years of the survey. Three sets of pseudo strata and PSU pairings are provided to use with SUDAAN in variance estimation. Since NHANES III is based on a complex, multi-stage sample design, appropriate sample weights should be used in analyses to produce national estimates of prevalence and associated variances while accounting for unequal probability of selection of sample persons. For example, the final interview weight, WTPFQX6, should be used for analysis of the items or questions from the family or household questionnaires, and the final MEC examination weight, WTPFEX6, should be used for analysis of the questionnaires and measurements administered in the MEC. Furthermore, for a combined analysis of measurements from the MEC examinations and associated medical history questions from the household interview, the final MEC examination weight, WTPFEX6, should be used. We recommend using SUDAAN (Shah, 1995) to estimate statistics of interest and the associated variance. However, one can also use other published methods for variance estimation. Application of SUDAAN and alternative methods, such as the average design effect approach, balance repeated replication (BRR) methods, or jackknife methods for variance estimation, are discussed in Weighting and Estimation Methodology (U.S. DHHS, 1996). Appropriate Uses of the NHANES III Sample Weights Final interview weight, WTPFQX6 Use only in conjunction with the sample interviewed at home and with items collected during the household interview. Final examination (MEC only) weight, WTPFEX6 Use only in conjunction with the MEC-examined sample and with interview and examination items collected at the MEC. Final MEC+home examination weight, WTPFHX6 Use only in conjunction with the MEC+home-examined sample and with items collected at both the MEC and home. Final allergy weight, WTPFALG6 Use only in conjunction with the allergy subsample and with items collected as part of the allergy component of the exam. Final CNS weight, WTPFCNS6 Use only in conjunction with the CNS subsample and with items collected as part of the CNS component of the exam. Final morning examination (MEC only) subsample weight, WTPFSD6 Use only in conjunction with the MEC-examined persons assigned to the morning subsample and only with items collected in the MEC exam. Final afternoon/evening examination (MEC only) subsample weight, WTPFMD6 Use only in conjunction with the MEC-examined persons assigned to the afternoon/evening subsample and only with items collected in the MEC exam. Final morning examination (MEC+home) subsample weight, WTPFHSD6 Use only in conjunction with the MEC- and home-examined persons assigned to the morning subsample and with items collected during the MEC and home examinations. Final afternoon/evening examination (MEC+home) weight, WTPFHMD6 Use only in conjunction with the MEC- and home-examined persons assigned to the afternoon/evening subsample and with items collected during the MEC and home examinations.