Methods in survey research: evidence for the reliability of group administration vs personal interviews

The magazine publisher is the copyright holder of this article and it is reproduced with permission. Further reproduction of this article in violation of the copyright is prohibited.

    Selection of a method to collect survey data is dependent upon a number of factors, including available funds, how soon results are needed, and the potential for introducing bias. Personal interviews are advantageous when members of the target population are difficult to locate or recruit(FN1) and when their reading skills may be limited.(FN2) For example, when the health status and needs of the homeless have been assessed, public health researchers have commonly used personal interviews.(FN3,4) However, such interviews are often expensive and time consuming.(FN1)
    In a longitudinal study of formerly homeless persons living in transitional housing, we conducted personal interviews to collect baseline data but decided to collect follow-up data via the infrequently used "group administration" survey methodology. We report here our experiences and examine whether group administration produces reliable survey data.
    The Healthcare Empowerment Alliance for People Living in Transitional Housing (HEALTH) Project is a program designed to provide formerly homeless individuals currently living in transitional housing facilities with health care services and education. The program began offering on-site medical care to residents of 4 transitional housing facilities in September 1999. In evaluations of the project, preintervention data were collected via face-to-face interviews. As a result of scheduling problems and a number of other challenges, 15 weeks were required to collect data from 202 residents.
    Twenty-one staff members, medical students, and medical professionals conducted the interviews. Salary expenses, along with those related to training and traveling, exceeded our budget. In addition, to characterize the impact of the intervention at 6 months from the initiation of medical and health services, we needed to condense the data collection period for the follow-up survey. To do so, we opted for the group administration method. That is, residents were recruited to meet at specified locations (e.g., dining halls) at each site.
    Once residents were convened, each was given a copy of the questionnaire attached to a clipboard and asked to wait until the facilitator read aloud each item and response option before he or she marked an answer. During this process, transparencies of each page of the questionnaire were displayed on a screen via an overhead projector. A total of 209 residents participated in 19 group administration survey sessions (mean = 11 residents per group, range = 5-47) during a 4-week period.
    Although participants were encouraged to ask questions during this process, they may have been less likely to do so than during a personal interview. Moreover, in a group setting, we suspected that it would be easier for participants to become distracted and not sustain the same level of attention required in a face-to-face situation.
    To test whether the self-reported data collected in a group setting were any less reliable than data collected through personal interviews, we calculated Cronbach alpha coefficients for 3 validated scales: the SF-36 Physical Functioning Scale,(FN5) the SF-36 Mental Health Scale,(FN5) and the Visit-Specific Patient Satisfaction Scale of Ware and Hays.(FN6) As can be seen in Table 1, measures of internal consistency for these 3 scales remained constant, or slightly improved, with the group administration method.
    The 6-month follow-up survey required 176 fewer interviewer hours and 11 fewer weeks to obtain approximately the same number of questionnaires. Thus, the group administration survey methodology was less expensive and less time consuming. An alternative method could have been for residents to self-administer the questionnaire in a group setting. However, we believed that the minimal additional effort involved in facilitating the group administration was valuable given the assumed diverse levels of reading comprehension among the members of the target population. Subsequent analyses showed that high school was reported as the highest level of education among half (50.7%) of the survey participants at baseline.
    Of course, the group administration methodology requires that the target population be assembled in clusters representing manageable groups. Such instances might include surveys of residents of public housing, drug or alcohol treatment facilities, or homeless shelters. Our findings suggest that when researchers are faced with such populations, the group administration method saves time and money and introduces no more measurement error than personal interviews.
ADDED MATERIAL
    Fred Molitor, PhD
    Richard L. Kravitz, MD, MSPH
    Yue-yun To, BS
    Arlene Fink, PhD
    Fred Molitor, Richard L. Kravitz, and Yue-yun To are with the Center for Health Services Research in Primary Care, University of California, Davis. Arlene Fink is with the Department of Medicine and School of Public Health, University of California, Los Angeles.
    Requests for reprints should be sent to Fred Molitor, PhD, Center for Health Services Research in Primary Care, University of California, Davis, 4150 V St, Suite 2500, Patient Support Services Bldg, Sacramento, CA 95817 (e-mail: fmolitor@ucdavis.edu).
    This research letter was accepted October 22, 2000.

CONTRIBUTORS
    F. Molitor wrote the research letter. Y. To analyzed the data. All of the authors were involved in the design of the study and questionnaires, and all contributed to the writing of the letter.
    TABLE 1--Measures of Internal Consistency for 3 Scales Included in the HEALTH Project Questionnaires Administered via Personal Interviews and in Groups

                                                 Survey Methodology (Survey Period)
                                             Personal Interview          Group Administration
                                               (Baseline),(FNa) alpha    (6-Month Follow-Up),(FNb) alpha
SF-36 Physical Functioning Scale                   0.90                         0.91
SF-36 Mental Health Scale                          0.93                         0.93
Visit-Specific Patient Satisfaction Scale          0.87                         0.93

FOOTNOTES
a 202 (72%) residents were recruited for the baseline survey.
b 209 (75%) residents participated in the 6-month follow-up survey.

FOOTNOTES
1. Simon JL, Burstein P. Basic Research Methods in Social Science, 3rd ed. New York, NY: Random House Inc; 1985.
2. Fowler FJ Jr. Survey Research Methods. 2nd ed. Newbury Park. Calif: Sage Publications; 1993.
3. O'Toole TP, Gibbon JL, Hanusa BH, Finke MJ. Utilization of health care services among subgroups of urban homeless and housed poor. J Health Polit Policy Law. 1999;24:91-114.
4. Kleinman LC, Freeman H, Pearlman J, Gelberg L. Homing in on the homeless: assessing the physical health of homeless adults in Los Angeles County using an original method to obtain physical examination data in a survey. Health Serv Res. 1996;31:533-549.
5. Ware JE Jr, Snow KK, Kosinski M, Gandek B. SF-36 Health Survey: Manual and Interpretation Guide. Boston, Mass: Nimrod Press; 1993.
6. Ware JE Jr, Hays RD. Methods for measuring patient satisfaction with specific medical encounters. Med Care. 1988;26:393-402.