Abstract

Abstract

Introduction

Procedures

Results

Discussion

References

A content analysis was used to examine the general status of health education research and methodology applications of 336 articles published by two major health education journals from 1988 to 1997. It was found that most studies reported information about research type and methods, sampling, target population, response rate, data collection, instrumentation, data analysis, and study purposes. However, some articles failed to provide relevant information for readers to judge the quality of those studies.

Introduction

Abstract

Introduction

Procedures

Results

Discussion

References

Quality research has long been recognized as the backbone of any well developed applied discipline (Torabi & Ding, 1998). It is crucial to health education because of its relatively young age compared to other health-related disciplines. There is no doubt that health educators are making every effort in conducting sound research studies. Those efforts have been reflected by professional publications. However, what and how health education research has been conducted are the questions one might want to ask. To date, no studies have been designed to examine the quality and status of health education researches.

Many variables which should be found from any research articles have been correlated to the quality of a research study. Experts have identified many of those variables as follows: the significance of the research questions being examined, the representative of samples, sample size, response rate, the design and development of data collection instrument, statistical techniques used in data analysis, the interpretation of the result, and the report of certain important information (Baumgartner & Strong, 1994; Carver, 1996; Editorial, 1997; Lamp, Price, & Desmond, 1989; Torabi, 1986). Any well-designed studies that address these variables ensure the quality and credibility of the results. This study examined research articles published in two major refereed health education journals, the Journal of Health Education and the Journal of School Health, from 1988 to 1997.

The purpose of the study was to investigate the over all status of health education research as reflected in professional publications. Specifically, following research questions regarding the status and methodology applications were addressed:

• What types of research studies have been conducted in health education profession?
• What was the status of methodological application in health education research?
• Who had been studied?
• What had been studied?

Procedures

Abstract

Introduction

Procedures

Results

Discussion

References

Design of the Study

Content analysis was applied in this study. Content analysis is considered as one type of qualitative research methods (Patton, 1990), and has been practiced in health education research (Fetro & Drolet, 1991; Lamp, Price, & Desmond, 1989; Miner & Baker, 1994; Seibert & Drolet, 1993; Welle, Kittleson & Ogletree, 1995). In general, content analysis searches key information from raw data, defines a set of rules to group valuable information into comparable descriptor units,

The purpose of the study was to investigate the over all status of health education research as reflected in professional publications.

analyzes the units, and provides frequencies with which certain things, groups, or concepts are referenced (Krippendorf, 1980; P atton, 1990). It seeks to reveal patterns of meanings and makes valid inferences from test (Weber, 1985). For the purpose of this study, content analysis was considered to be the most proper approach due to the fact that most of the sound researches are published in peer referred professional journals. Content analysis is able to maximally use such existing information and generate valid results.

Article selection

Two journals, the Journal of School Health and the Journal of Health Education, were selected for their popularity and authority in the field. Articles published from 1988 to 1997 in these two journals were reviewed. Only those involved data collection and applied certain research methodologies were considered as research papers and were selected for this study.

Coding Instrument

A data coding instrument is a crucial component of content analysis (Krippendorff, 1980). It serves as defined criteria in sorting raw information and recording corresponding code for late analysis. Nineteen variables to be examined in this study were first identified, including the number of authors, affiliation and position of the first two authors, research type (i.e., survey study, experimental study, etc.), research methods (i.e., quantitative, qualitative, or the combination of both quantitative and qualitative methods), data source (1st hand or 2nd hand), data collection method (mail or phone), instrumentation, target population, number of subjects, response rate, sampling methods, statistical test, report of significance, report of power, report of confidence intervals, report of error, report of effect size, and the purpose of the study.

For the purpose of recording information, these variables were tabulated into a data-coding sheet which was used in the pilot test. The revised coding sheet was used in final data collection.

Coding Reliability

Inter-rater reliability was tested after research articles were identified. A total of 36 articles were randomly selected among the total research papers, and reviewed by the authors and a third party during the same time period. The codes of each variable recorded from both parties were compared to find coding disagreement. Holsti's formula was utilized in the calculation of inter-rate reliability (Holsti, 1969). The formula is:

Where, M = number of coding events on which two coders agree, N₁ = Total number of coding decisions by the first coder, and N₂ = Total number of coding decisions by the second coder. This formula establishes a percentage of agreement between two coders. In this study, there were 19 variables being examined which yield 19 coding events. Since 36 articles were re-coded to examine the inter-rater reliability, there were 19 x 36 = 684 coding events. Of those events, a total of 65 disagreements were found. The agreement is 619 out of the 684 coding events (684 - 65 = 619). Thus, the inter-rater reliability result was determined as: 2(619)/(684 + 684) = 0.905, or 90.5%.

Data Analysis

Data collected from all of the research articles defined by the authors were entered into Statistics Package for Social Science (SPSS) computer program. Descriptive statistics and chi-square test were used in data analysis. Significant test was defined at alpha .05 level.

A total of 336 articles published in two major health education journals, Journal of School Health (JOSH) and Journal of Health Education (JOHE) from 1988 to 1997 were defined as research papers. The distribution of those papers by the journals and by year is presented in Table 1. Both journals published a similar amount of research papers in each year (chi-square = 4.9, p = 0.8). The average number of annual publication for JOSH and JOHE is 18 and 16, respectively.

Over half of the papers (52.7%) were written by two to three authors, 19.6% by four authors, 16.7% by five or more authors, and 11% by one author. Most of the authors (80.1%) were university faculty members.

Regarding the types of researches, survey studies were used most often (67.6%), followed by experimental (17.0%), qualitative (3.0%), content analysis (3.3%), instrument development (4.2%), and secondary data analysis (3.9%). There is no statistical significant difference between two journals regarding the types of research used (chi-square = 9.4, p > 0.05).

Figure 1 displays the distribution of data collection methods. It can be seen that administered questionnaire was used by 43.8% of the total studies in data collection, followed by mailed questionnaire (23.2%), observation (5.7), phone interview (3.9%), face-to-face interview (3.0), and group interview (1.5%). It was noticed that a number of papers did not report data collection method (13.1%).

Based on the available information drawn from research articles, the methods of sample selection were presented in Figure 2. It reveals that convenient sampling, being the single most frequently used method, was employed by over one-third of the studies (36.9%). One-quarter of the articles reported the use of complex method (i.e., stratified sampling, cluster sampling, combined random and convenience sampling) or combined use of several sampling techniques in one study. Randomly sampling was also reported in 12.5% of the papers. Approximately 14% of the papers did not mention how samples were selected in those studies.

Among the 221 studies using inferential statistics, over half (54.3%) used one statistical test, 34.4% used two, 10.0% used three, and 1.4% used four or more statistical tests. P values were reported by nearly all of them (99.1%) as a result of significant test. As Figure 3 shows, chi-square test was the most widely used approach, reported by 43% of the papers, followed by analysis of variance (36.7%), t-test (23.5%), correlation (13.6%), regression (11.3%), non-parametric statistics tests (6.8%), ANCOVA (5.4%), logistic regression (5.0%), factor analysis (3.6%), and others including linear structure model (1.4%).

For studies involving inferential statistics in data analysis, report of effect size, statistics power, standard error, and/or confidence intervals was also investigated. Effect size was not reported by any of the studies examined. Confidence interval was found in 9 (4.1%) papers. Standard error and statistics power were expressed by 5 (2.3%) and 2 (0.9%) papers, respectively.

As to the subjects that have been studied, it was found that existing researches have not only focused on people but also other subjects such as health education curricula, or media. Student was the single most often studied population (58.9% of the 336 studies), followed by health educators or health education related subjects (14.9%), teachers (8.3%), and other populations (7.1%) ( Table 2). Parents, school administration personnel, and other subjects were also investigated in health education research. Among student population, middle and high school students were studied most often, followed by college students and elementary students. Approximately 11.6% studies focused on special student population, including minority groups and students with disabilities. It is worthy of noting that some reports (2.7%) did not give detailed information about study population. A few of researches studied day care center, patient, pharmacists, retailers, snacks and vending machines, and others. They are grouped into category "Others" in Table 2.

As to the frequencies of both main and sub categories of various topics covered by the studies, sex-related issues were studied most frequently (21.7%) including AIDS/HIV, STDs, Condom use, contraception, and so forth ( Table 3). Tobacco, alcohol, and/or other drug use was addressed in 15.8% of the articles, followed by nutrition (9.2%), professional preparation (7.7%), general health related issues (6.5%), safety issues (4.2%), physical exercise (3.9%), safety (3.9%), cancer prevention (3.6%), and health services (3.6%). A few studies covered death, policies, politics, health educator list-server, and conference components categorized as others in Table 3.

Table 3. Classification and Frequency of Contents/Topics Being Studied in Health Education Research (n=336)
Content/Topic		Frequency	Percent
Sexuality		73	21.7
AIDS/HIV/STDS		38	11.3
Sex education		22	6.5
Condom use		4	1.2
Contraception		4	1.2
Teen pregnancy		3	0.9
Rape		2	0.6
Alcohol, Tobacco, and Other Drug Abuse		53	15.8
Nutrition		31	9.2
Obesity		11	3.3
Food choice		8	2.4
Nutrition		8	2.4
Cholesterol		4	1.2
Safety		14	4.2
Safety		7	2.1
Gun/Fight		5	1.5
First Aid		2	0.6
Physical Exercise		13	3.9
Risk Behavior in General		10	3.0
Cancer prevention		12	3.6
Cancer		9	2.7
Sun exposure		3	0.9
Stress/Mental health		9	2.7
Stress		6	1.8
Mental Health		3	0.9
Heart Disease		4	1.2
Health Service		12	3.6
Health Education Program		10	3.0
Health Education Information		4	1.2
Theory		7	2.1
Moral Issue		2	0.6
Prof. Preparation		26	7.7
General Health		22	6.5
Other Topics		34	10.1
Total		336	100.0
Indicates breakdown-data for the general topic.

As to the reported purposes of each article, of the total of 336 research studies, 61% involved investigation activities, 14.3% were for evaluation purpose, 10.7% were searching for correlation/associations, 6.3% did comparisons, 3% were needs assessment, and 4.8% tested new instruments. The study objects of nearly half of the studies focused on knowledge, attitude, and/or behaviors (47.61%) with more studies focusing on attitude and behaviors. Characteristics or current health status of selected populations, health education programs, or certain issues was the objects for 20.2% of the studies. The rest were innovative intervention methods or program evaluation (12.5%), teaching method or curriculum (6.3%), source of information or media coverage (2.4%), and others (6%).

Articles published by the two major health education journals from 1988 to 1997 were reviewed. A total of 336 articles were identified as reports of research studies which involved data collection and certain research methodologies. Key information of those papers was collected using the coding instrument developed for this study. Most papers were written by two or three authors and most authors were university faculty members.

Students were studied most often. Many of the studies also involved teachers and parents. Sex related issues are the hottest topic being investigated. Most studies focused on attitude and behavior. Survey study was the most popular type of research. Due to the fact that the most frequently targeted population was student, the number one data collection method was administered questionnaire. Half of the studies had a sample size of 300 or more, and usually had a high response rate. Convenient and complex sampling methods were used most often.

Pure qualitative study is still rarely used in health education research practices. Majority of the researches employed quantitative method. Chi-square test was the most popular statistical technique used in data analysis, followed by ANOVA, t-test, correlation test, and regression test. However, few studies reported parameters such as effect size, statistics power, confidence intervals, and/or standard error in their reports. It was found from this study that fewer qualitative studies were published than quantitative studies. Some researches collected data qualitatively such as using group interviews with open-ended question format, and analyzed the data quantitatively. Although there are arguments about the combined use of qualitative and quantitative methods (Lincoln, 1985; Noblitt & Hare, 1988; Rosenberg, 1988), it is true that qualitative data provides in-depth information.

Although some inferential statistical tests are more sophisticated than the others are, any inferential statistical test can be the best one if used appropriately. It was found from this study that chi-square, ANOVA, and t-test were the essential techniques widely used in health educational research possibly due to their simplicity and capacity. However, advancing the use of inferential statistics should be advocated. This does not necessarily mean one should use more sophisticated tests, instead, it calls for explicit statements about the use of statistics and about the parameters of interests. Effect size, statistics power, confidence intervals, and standard error can provide information about the magnitude of a difference, practical significance of a test, and/or type I and II errors, and for future meta-analysis. They have been recommended to be included in research papers by a number of authors (Carver, 1996; Daniel, 1977; Gill, McNamara and Skinkle, 1980; Loftus, 1991; Serlin, 1996; Torabi, 1986).

How study sample is selected is the core of a quantitative study (Baumgartner & Strong, 1994). In health education research, convenient or complex sampling prevails although there are cases in which randomly sampling was practiced. This fact may not satisfy many of the statistical tests which function only when the assumption of randomization is met. However, it may reflect what the reality is in health education research. As Asher (1996) indicated, "Even in purely descriptive research, there are relatively few random samples from well-defined, major populations in education. We generally use sampling statistics in a subjunctive comparison like the following: 'if I had a random sample from a major population, what would those results mean, compared with the sample I do have?'"(p. 389). Again, the conflict between theory and reality is evident in health education research.

Recommendations

It was found from this study that most articles addressed their sample size as well as response rate, others failed to do so. If a study intends to generalize its conclusion to others, the sample size and response rate, the indicators of the representative of study population, should be reported. It is necessary for health education research to continue targeting student, teacher, and general populations. More studies should target those who may be influential to the health education profession, such as policy makers, politicians, law enforcement, police officers, or other social groups. In addition, qualitative researches should be encouraged. However, qualitative study, even combined with quantitative one, needs to be well designed in advance in terms of its data analysis. It may make no sense to statistically analyze qualitative data.

In order to improve methodological application in health education research, it is necessary for major health education journals to up-grade the standards for manuscript submission. Report of response rate, sample size, sample selection methodology, data collection instrument, certain statistics parameters, and/or any other necessary and valuable information might be legitimate requirements.

Note: This paper was presented in the 126th Annual American Public Health Association Meeting in Washington, D.C., November 1998

References

Abstract

Introduction

Procedures

Results

Discussion

References

Welle, H. M., Kittleson, M. J., & Ogletree, R. J. (1995). A content analysis of first year of the HEDIR list. Journal of Health Education, 26(6), 366-371.

---

[Reprint (PDF) Version of this article]

---

Copyright © 1999 by IEJHE.