The International Electronic Journal of Health Education

[Reprint (PDF) Version of this article]
IEJHE, Vol. 2(3), 101-110, July 1, 1999, Copyright 1999

Research Studies in Two Health Education Journals, 1988-1997: Targets and Methodologies

Kele Ding, MS. MD1; Ping Hu, PhD, MD2
1 Indiana University; 2 Florida Atlantic University

Corresponding author: Kele Ding, MS, MD; Department of Applied Health Science; Indiana University; Bloomington, IN 47405; fax: 812.855.4940; phone: 812.855.4905; e-mail: KDING@INDIANA.EDU.

Received January, 19, 1999; revised and accepted June 3, 1999.

Abstract
Abstract Introduction Procedures Results Discussion References

A content analysis was used to examine the general status of health education research and methodology applications of 336 articles published by two major health education journals from 1988 to 1997. It was found that most studies reported information about research type and methods, sampling, target population, response rate, data collection, instrumentation, data analysis, and study purposes. However, some articles failed to provide relevant information for readers to judge the quality of those studies.

Introduction
Abstract Introduction Procedures Results Discussion References

Quality research has long been recognized as the backbone of any well developed applied discipline (Torabi & Ding, 1998). It is crucial to health education because of its relatively young age compared to other health-related disciplines. There is no doubt that health educators are making every effort in conducting sound research studies. Those efforts have been reflected by professional publications. However, what and how health education research has been conducted are the questions one might want to ask. To date, no studies have been designed to examine the quality and status of health education researches.

Many variables which should be found from any research articles have been correlated to the quality of a research study. Experts have identified many of those variables as follows: the significance of the research questions being examined, the representative of samples, sample size, response rate, the design and development of data collection instrument, statistical techniques used in data analysis, the interpretation of the result, and the report of certain important information (Baumgartner & Strong, 1994; Carver, 1996; Editorial, 1997; Lamp, Price, & Desmond, 1989; Torabi, 1986). Any well-designed studies that address these variables ensure the quality and credibility of the results. This study examined research articles published in two major refereed health education journals, the Journal of Health Education and the Journal of School Health, from 1988 to 1997.

The purpose of the study was to investigate the over all status of health education research as reflected in professional publications. Specifically, following research questions regarding the status and methodology applications were addressed:

Procedures
Abstract Introduction Procedures Results Discussion References

Design of the Study

Content analysis was applied in this study. Content analysis is considered as one type of qualitative research methods (Patton, 1990), and has been practiced in health education research (Fetro & Drolet, 1991; Lamp, Price, & Desmond, 1989; Miner & Baker, 1994; Seibert & Drolet, 1993; Welle, Kittleson & Ogletree, 1995). In general, content analysis searches key information from raw data, defines a set of rules to group valuable information into comparable descriptor units,

The purpose of the study was to investigate the over all status of health education research as reflected in professional publications.


analyzes the units, and provides frequencies with which certain things, groups, or concepts are referenced (Krippendorf, 1980; P atton, 1990). It seeks to reveal patterns of meanings and makes valid inferences from test (Weber, 1985). For the purpose of this study, content analysis was considered to be the most proper approach due to the fact that most of the sound researches are published in peer referred professional journals. Content analysis is able to maximally use such existing information and generate valid results.

Article selection

Two journals, the Journal of School Health and the Journal of Health Education, were selected for their popularity and authority in the field. Articles published from 1988 to 1997 in these two journals were reviewed. Only those involved data collection and applied certain research methodologies were considered as research papers and were selected for this study.

Coding Instrument

A data coding instrument is a crucial component of content analysis (Krippendorff, 1980). It serves as defined criteria in sorting raw information and recording corresponding code for late analysis. Nineteen variables to be examined in this study were first identified, including the number of authors, affiliation and position of the first two authors, research type (i.e., survey study, experimental study, etc.), research methods (i.e., quantitative, qualitative, or the combination of both quantitative and qualitative methods), data source (1st hand or 2nd hand), data collection method (mail or phone), instrumentation, target population, number of subjects, response rate, sampling methods, statistical test, report of significance, report of power, report of confidence intervals, report of error, report of effect size, and the purpose of the study.

For the purpose of recording information, these variables were tabulated into a data-coding sheet which was used in the pilot test. The revised coding sheet was used in final data collection.

Coding Reliability

Inter-rater reliability was tested after research articles were identified. A total of 36 articles were randomly selected among the total research papers, and reviewed by the authors and a third party during the same time period. The codes of each variable recorded from both parties were compared to find coding disagreement. Holsti's formula was utilized in the calculation of inter-rate reliability (Holsti, 1969). The formula is:

Where, M = number of coding events on which two coders agree, N1 = Total number of coding decisions by the first coder, and N2 = Total number of coding decisions by the second coder. This formula establishes a percentage of agreement between two coders. In this study, there were 19 variables being examined which yield 19 coding events. Since 36 articles were re-coded to examine the inter-rater reliability, there were 19 x 36 = 684 coding events. Of those events, a total of 65 disagreements were found. The agreement is 619 out of the 684 coding events (684 - 65 = 619). Thus, the inter-rater reliability result was determined as: 2(619)/(684 + 684) = 0.905, or 90.5%.

Data Analysis

Data collected from all of the research articles defined by the authors were entered into Statistics Package for Social Science (SPSS) computer program. Descriptive statistics and chi-square test were used in data analysis. Significant test was defined at alpha .05 level.

Results
Abstract Introduction Procedures Results Discussion References

A total of 336 articles published in two major health education journals, Journal of School Health (JOSH) and Journal of Health Education (JOHE) from 1988 to 1997 were defined as research papers. The distribution of those papers by the journals and by year is presented in Table 1. Both journals published a similar amount of research papers in each year (chi-square = 4.9, p = 0.8). The average number of annual publication for JOSH and JOHE is 18 and 16, respectively.

Table 1. Number of Research Papers Published
by Journal and Year
JOURNAL Total
YEAR JOSH JOHE
1988 13 12 25
1989 20 15 35
1990 13 17 30
1991 18 13 31
1992 19 15 34
1993 22 17 39
1994 18 16 34
1995 18 17 35
1996 25 19 44
1997 11 18 29
Total 177  159  336 

Over half of the papers (52.7%) were written by two to three authors, 19.6% by four authors, 16.7% by five or more authors, and 11% by one author. Most of the authors (80.1%) were university faculty members.

Regarding the types of researches, survey studies were used most often (67.6%), followed by experimental (17.0%), qualitative (3.0%), content analysis (3.3%), instrument development (4.2%), and secondary data analysis (3.9%). There is no statistical significant difference between two journals regarding the types of research used (chi-square = 9.4, p > 0.05).

Figure 1 displays the distribution of data collection methods. It can be seen that administered questionnaire was used by 43.8% of the total studies in data collection, followed by mailed questionnaire (23.2%), observation (5.7), phone interview (3.9%), face-to-face interview (3.0), and group interview (1.5%). It was noticed that a number of papers did not report data collection method (13.1%).

Among the total studies, nearly half (47.6%) developed their own instruments, 28.3% adapted or modified an existing instrument, and 2.1% used an instrument that was partially developed and partially adapted from an existing instrument to collect data. Approximately 13.7% of the articles did not discuss data collection instrument.

It was found from this study that the sample size in health education research tended to be large. Of the total researches, one in five (20.2%) had a sample size of 1,000 to 9,999, nearly 15% had 500 to 999, 13.1% had 300 to 499, and one-quarter had 100 to 299 subjects. Extremely large sample size (10,000 or over) is found in 2.4% of the studies.

Based on the available information drawn from research articles, the methods of sample selection were presented in Figure 2. It reveals that convenient sampling, being the single most frequently used method, was employed by over one-third of the studies (36.9%). One-quarter of the articles reported the use of complex method (i.e., stratified sampling, cluster sampling, combined random and convenience sampling) or combined use of several sampling techniques in one study. Randomly sampling was also reported in 12.5% of the papers. Approximately 14% of the papers did not mention how samples were selected in those studies.

The response rates of a total of 227 survey studies were analyzed. It was found that three in five (60.8%) did not report response rate or provide information with which response rate could be determined (such as the number of samples surveyed and the number of surveys completed and returned). Over half (52.8%) of the studies of which the response rate information was provided obtained a response rate of 70% or higher, nearly one-third (32.6%) between 50% to 70%, and 15% at 50% or below. In general, studies involving student subjects received higher response rates than those involving other populations.

Data analysis methods were described by nearly all of the 336 research papers with 26.2% using descriptive statistics, 65.8% employing descriptive and inferential statistics, and the rest studies involving qualitative, test construction, and other non-application studies (8%).

Among the 221 studies using inferential statistics, over half (54.3%) used one statistical test, 34.4% used two, 10.0% used three, and 1.4% used four or more statistical tests. P values were reported by nearly all of them (99.1%) as a result of significant test. As Figure 3 shows, chi-square test was the most widely used approach, reported by 43% of the papers, followed by analysis of variance (36.7%), t-test (23.5%), correlation (13.6%), regression (11.3%), non-parametric statistics tests (6.8%), ANCOVA (5.4%), logistic regression (5.0%), factor analysis (3.6%), and others including linear structure model (1.4%).

For studies involving inferential statistics in data analysis, report of effect size, statistics power, standard error, and/or confidence intervals was also investigated. Effect size was not reported by any of the studies examined. Confidence interval was found in 9 (4.1%) papers. Standard error and statistics power were expressed by 5 (2.3%) and 2 (0.9%) papers, respectively.

As to the subjects that have been studied, it was found that existing researches have not only focused on people but also other subjects such as health education curricula, or media. Student was the single most often studied population (58.9% of the 336 studies), followed by health educators or health education related subjects (14.9%), teachers (8.3%), and other populations (7.1%) ( Table 2). Parents, school administration personnel, and other subjects were also investigated in health education research. Among student population, middle and high school students were studied most often, followed by college students and elementary students. Approximately 11.6% studies focused on special student population, including minority groups and students with disabilities. It is worthy of noting that some reports (2.7%) did not give detailed information about study population. A few of researches studied day care center, patient, pharmacists, retailers, snacks and vending machines, and others. They are grouped into category "Others" in Table 2.

As to the frequencies of both main and sub categories of various topics covered by the studies, sex-related issues were studied most frequently (21.7%) including AIDS/HIV, STDs, Condom use, contraception, and so forth ( Table 3). Tobacco, alcohol, and/or other drug use was addressed in 15.8% of the articles, followed by nutrition (9.2%), professional preparation (7.7%), general health related issues (6.5%), safety issues (4.2%), physical exercise (3.9%), safety (3.9%), cancer prevention (3.6%), and health services (3.6%). A few studies covered death, policies, politics, health educator list-server, and conference components categorized as others in Table 3.

Table 2. Category and Frequency of Group Populations Being Studied in 
Health Education Research (n=336)
Study Population Frequency Percent
Students/Kids
198
58.9
 
Middle/High school student
78
23.2
 
Special student population (disabled, ethnical groups, etc.)
39
11.6
 
College student
36
10.7
 
Elementary students
24
7.1
 
High school students
12
3.6
 
Preschool children
5
1.5
 
Middle school student
4
1.2
Teachers
28
8.3
 
Middle/High school teachers
12
3.6
 
School teachers
9
2.7
 
Elementary teachers
7
2.1
Parents or parent & children
11
3.3
School administration (officers, principals, superintendents, etc).
5
1.5
Health related
50
14.9
 
Others, including journal, curricula, programs, literature, service, etc.
24
7.1
 
Health educators at any levels
22
6.5
 
Nurse
4
1.2
Other population
24
7.1
 
Adults
13
3.9
 
Employee/Employer
5
1.5
 
Elderly
4
1.2
 
General population
2
0.6
Media
2
0.6
Others
9
2.7
Not mentioned
9
2.7
Total
336
100.0 
Indicates breakdown-data for the group of population.

Table 3. Classification and Frequency of Contents/Topics Being Studied in Health Education Research (n=336) 

Content/Topic

Frequency

Percent

Sexuality

73

21.7

AIDS/HIV/STDS

38

11.3

Sex education

22

6.5

Condom use

4

1.2

Contraception

4

1.2

Teen pregnancy

3

0.9

Rape

2

0.6

Alcohol, Tobacco, and Other Drug Abuse

53

15.8

Nutrition

31

9.2

Obesity

11

3.3

Food choice

8

2.4

Nutrition

8

2.4

Cholesterol

4

1.2

Safety

14

4.2

Safety

7

2.1

Gun/Fight

5

1.5

First Aid

2

0.6

Physical Exercise

13

3.9

Risk Behavior in General

10

3.0

Cancer prevention

12

3.6

Cancer

9

2.7

Sun exposure

3

0.9

Stress/Mental health

9

2.7

Stress

6

1.8

Mental Health

3

0.9

Heart Disease

4

1.2

Health Service

12

3.6

Health Education Program

10

3.0

Health Education Information

4

1.2

Theory

7

2.1

Moral Issue

2

0.6

Prof. Preparation

26

7.7

General Health

22

6.5

Other Topics

34

10.1

Total

 

336

100.0

Indicates breakdown-data for the general topic.

As to the reported purposes of each article, of the total of 336 research studies, 61% involved investigation activities, 14.3% were for evaluation purpose, 10.7% were searching for correlation/associations, 6.3% did comparisons, 3% were needs assessment, and 4.8% tested new instruments. The study objects of nearly half of the studies focused on knowledge, attitude, and/or behaviors (47.61%) with more studies focusing on attitude and behaviors. Characteristics or current health status of selected populations, health education programs, or certain issues was the objects for 20.2% of the studies. The rest were innovative intervention methods or program evaluation (12.5%), teaching method or curriculum (6.3%), source of information or media coverage (2.4%), and others (6%).

Discussion
Abstract Introduction Procedures Results Discussion References

Articles published by the two major health education journals from 1988 to 1997 were reviewed. A total of 336 articles were identified as reports of research studies which involved data collection and certain research methodologies. Key information of those papers was collected using the coding instrument developed for this study. Most papers were written by two or three authors and most authors were university faculty members.

Students were studied most often. Many of the studies also involved teachers and parents. Sex related issues are the hottest topic being investigated. Most studies focused on attitude and behavior. Survey study was the most popular type of research. Due to the fact that the most frequently targeted population was student, the number one data collection method was administered questionnaire. Half of the studies had a sample size of 300 or more, and usually had a high response rate. Convenient and complex sampling methods were used most often.

Pure qualitative study is still rarely used in health education research practices. Majority of the researches employed quantitative method. Chi-square test was the most popular statistical technique used in data analysis, followed by ANOVA, t-test, correlation test, and regression test. However, few studies reported parameters such as effect size, statistics power, confidence intervals, and/or standard error in their reports. It was found from this study that fewer qualitative studies were published than quantitative studies. Some researches collected data qualitatively such as using group interviews with open-ended question format, and analyzed the data quantitatively. Although there are arguments about the combined use of qualitative and quantitative methods (Lincoln, 1985; Noblitt & Hare, 1988; Rosenberg, 1988), it is true that qualitative data provides in-depth information.


More studies should target those who may be influential to the health education profession, such as policy makers, politicians, law enforcement, police officers, or other social groups.


Although some inferential statistical tests are more sophisticated than the others are, any inferential statistical test can be the best one if used appropriately. It was found from this study that chi-square, ANOVA, and t-test were the essential techniques widely used in health educational research possibly due to their simplicity and capacity. However, advancing the use of inferential statistics should be advocated. This does not necessarily mean one should use more sophisticated tests, instead, it calls for explicit statements about the use of statistics and about the parameters of interests. Effect size, statistics power, confidence intervals, and standard error can provide information about the magnitude of a difference, practical significance of a test, and/or type I and II errors, and for future meta-analysis. They have been recommended to be included in research papers by a number of authors (Carver, 1996; Daniel, 1977; Gill, McNamara and Skinkle, 1980; Loftus, 1991; Serlin, 1996; Torabi, 1986).

How study sample is selected is the core of a quantitative study (Baumgartner & Strong, 1994). In health education research, convenient or complex sampling prevails although there are cases in which randomly sampling was practiced. This fact may not satisfy many of the statistical tests which function only when the assumption of randomization is met. However, it may reflect what the reality is in health education research. As Asher (1996) indicated, "Even in purely descriptive research, there are relatively few random samples from well-defined, major populations in education. We generally use sampling statistics in a subjunctive comparison like the following: 'if I had a random sample from a major population, what would those results mean, compared with the sample I do have?'"(p. 389). Again, the conflict between theory and reality is evident in health education research.

Recommendations

It was found from this study that most articles addressed their sample size as well as response rate, others failed to do so. If a study intends to generalize its conclusion to others, the sample size and response rate, the indicators of the representative of study population, should be reported. It is necessary for health education research to continue targeting student, teacher, and general populations. More studies should target those who may be influential to the health education profession, such as policy makers, politicians, law enforcement, police officers, or other social groups. In addition, qualitative researches should be encouraged. However, qualitative study, even combined with quantitative one, needs to be well designed in advance in terms of its data analysis. It may make no sense to statistically analyze qualitative data.

In order to improve methodological application in health education research, it is necessary for major health education journals to up-grade the standards for manuscript submission. Report of response rate, sample size, sample selection methodology, data collection instrument, certain statistics parameters, and/or any other necessary and valuable information might be legitimate requirements.

Footnotes

Note: This paper was presented in the 126th Annual American Public Health Association Meeting in Washington, D.C., November 1998

References
Abstract Introduction Procedures Results Discussion References

Asher, W. (1996). The role of statistics in research. Journal of Experimental Education, 61(4), 388-393.

Conducting and reading research in health and human performance. Dubuque, IA: Wm. C. Brown Communications, Inc.

Carver, R. P. (1996). The case against statistical significance testing, revisited. Journal of Experimental Education, 61(4), 287-292.

Daniel, W. W. (1977). Statistical significance versus practical significance. Science Education, 61(3), 423-427.

Fetro, J. V., & Drolet, J. C. (1991). State conferences for school worksite wellness: A content analysis of conference components, Journal of Health Education, 22(2), 81-84.

Journal of Industrial Teacher Education, 17(2), 5-19.

Holsti, O. R. (1969). Content analysis for the social sciences and humanities. Reading. MA: Addison Wesley Publication Company.

Krippendorf, K. (1980). Content analysis: An introduction to its methodology. Beverly Hills, CA: SAGE Publications .

Lamp, E., Price, J. H., and Desmond, S. M. (1989). Instrument validity and reliability in three health education journals, 1980-1987. Journal of School Health, 59(3), 105-108.

Naturalistic Inquiry. Newbury Park, CA: Sage .

Loftus, G. R. (1991). On the tyranny of hypothesis testing in the social sciences. Contemporary Psychology, 36, 102-105.

Miner, K. J., & Baker, J. A. (1994). Media coverage of suntanning and skin cancer: Mixed message of health and beauty. Journal of Health Education, 25(4), 235-237.

Noblitt, O. W., & Hare, R. D. (1988). Meta-Echnography: Synthesizing qualitative studies. Newbury Park, CA: Sage .

Patton, M. Q. (1990). Qualitative evaluation methods. Beverly Hills, CA: SAGE Publications.

Rosenberg, A. (1988). Philosophy of social science. Boulder, CO: Westview Press Inc.

Serlin, R. C. (1996). Confidence intervals and the scientific method: A case for Holm on the range. Journal of Experimental Education, 61(4), 350-360.

Shaver, J. P. (1996). What statistical significance testing is, and what it is not? Journal of Experimental Education, 61(4), 293-316.

Seibert, D., & Drolet, J. C. (1993). Death themes in literature for children ages 3-8. Journal of School Health, 63(2), 86-91.

Torabi, M. R. (1986). How to estimate practical significance in health education research. Journal of School Health, 56(6), 232-234.

Torabi, M. R., and Ding, K. (1998). Selected Critical Measurement and Statistical Issues in Health Education Evaluation and Research. The International Electronic Journal of Health Education, 1(1), 26-38.

Weber, P. R. (1985). Basic content analysis. Beverly Hills, CA: SAGE Publications.

Welle, H. M., Kittleson, M. J., & Ogletree, R. J. (1995). A content analysis of first year of the HEDIR list. Journal of Health Education, 26(6), 366-371.


[Reprint (PDF) Version of this article]



Copyright 1999 by IEJHE.