|The International Electronic Journal of Health Education|
Corresponding author: Lynn L. Miller, Department of Health
Woman's University, Denton, TX 76204-3499; 972.766.3676
(phone), 972.766.1724 (fax), email@example.com. Date revised and
approved: March 13, 1998
The purpose of this study was to develop a computer assisted instruction (CAI) module on computer literacy for undergraduate community health majors and conduct a formative evaluation. Subjects utilized a CAI module on computer literacy for health educators. The subjects provided feedback on the module through investigator observation and in interviews. Evaluation criteria for interactive multimedia courseware were used by the subjects in this study as a checklist for evaluating the CAI module during use and the interview process. Content analysis was conducted on data collected in the interviews. Notes and transcripts were coded to identify consistent comments, then further analyzed to determine patterns in the responses.
The relationships between the amount of time spent using the CAI module and previous computer experience or training and between topic selection and previous computer experience or training was determined. A significant difference was found between having previous computer training or not in terms of the amount of time spent using the module. There was a direct relationship between time spent using the module and the amount of topics chosen. A significant difference was found between having previous computer training or not in terms of the topics selected. In general, the participants perceived the CAI module positively. Some of the participants expressed negative perceptions about the mechanics of navigating in the program. The negative perceptions seemed to have no relationship to previous computer experience or training, time on task, or topics viewed.
To obtain a copy of this computer assisted instructional program, click here. Save it in a new folder/file. After it has been download, highlight the file, MILLER.EXE, and click twice to unpack it. Then, click "Part1.exe" to run the program.
Computer literacy and computer skills are becoming increasingly important for the effective practice of health education in our technology-based society. While many of the technological skill requirements will change over time, basic computer literacy will remain essential.
Computers are tools for processing information. They provide a mechanism for locating, capturing, organizing, and disseminating information (Sternin, 1996). As such, the computer is an effective tool for health educators. Professional preparation of health educators must address the need for awareness of the role technology will play in the practice of health education.
Health education preparation programs can meet this need in several ways. One of the more efficient ways is through the use of computer assisted instruction (CAI). CAI is the use of computers and multimedia technology for instruction in a way that promotes both student interest and motivation. The computer's multimedia capability to show graphics, run simulations, and play sound, video, and animation greatly strengthens the learning experience. Multimedia creates a complete multi-sensory learning program, allowing students to interact with the material and to learn according to their own needs, pace, and learning styles (Infopoint, 1996).
A computer literacy policy, usually met through a required single course, has been put into effect by most colleges and universities (Butler, 1997). Because technology continues to change rapidly, much of the information students learn may be obsolete by the time they graduate. Teaching computer basics and technology assessment, both hardware and software, will prepare students to adapt to and learn new technology.
Internet applications and online services are constantly evolving. The future use of this technology is difficult to predict. Many individuals cannot stay competitive in the workforce if they don't stay educated on technological changes (Butler, 1997). One type of distance learning employs the Internet to increase instructional effectiveness and opportunities.
While many health educators will evaluate existing health education software in order to determine if it meets their standards and needs (Butler, 1997), others may develop their own
Teaching computer basics and technology assessment, both hardware and software, will prepare students to adapt to and learn new technology.
Health educators often have difficulty finding appropriate CAI programs that meet their objectives, setting, and available hardware. Under such circumstances, health educators may choose to develop their own CAI programs. Yet the costs can still be very high, both in money and time. Many people have computer access at home and if the program could be distributed, this could be a major way to reach large numbers of people.
Due to the lack of research examining the use of CAIs in health education and because of the ever increasing need for computer skills in our technologically growing society, this study focused on the evaluation of a CAI module on computer literacy.
This was a formative evaluation of a CAI module on computer literacy as an instructional tool for undergraduate community health majors. Students enrolled in undergraduate Health Studies courses at Texas Woman's University (TWU) utilized a CAI module on computer literacy developed by the investigator. The students provided feedback on the module through investigator observation and interviews.
The CAI module on computer literacy for health educators was developed by the investigator using Authorware Star for Windows, Version 2.2.0 Academic. The investigator developed the storyboard, including text, instructional design, and possible graphics. Resources for the text included Computers Simplified from IDG Books (1995) and 1,001 Komputer Answers by Kim Komando (1995). In order to allow the participant to choose topics according to previous experience and interest, the branching tutorial design was used.
The next step of the CAI development was the actual creation of the module. The text was written and entered first. While the text was entered, the technical aspects and flow of the module were also addressed so that the investigator could move about the module while developing it. Graphics, colors, and fonts were chosen next. This was the most time-consuming aspect of the CAI module development. Appropriate graphics to accompany the text, colors, and fonts were selected in an attempt to make the CAI module more effective and attractive. Graphics were obtained from the Internet, and from various computer software programs (Coreldraw, PowerPoint, and Freelance). Multimedia (video, animation, and sound) were the last components added to the module. Video and sounds were obtained by downloading files from the Internet. After these components were added, tested, and evaluated, the module was packaged. Authorware software packages the CAI module so that it can be used on a computer, i.e. as a stand-alone, without the need for the Authorware software.
The CAI module consists of several computer topics: hardware, presentation software, word-processing and desktop publishing, the Internet, graphics, and CAIs.
The CAI module consists of several computer topics: hardware, presentation software, word-processing and desktop publishing, the Internet, graphics, and CAIs. The hardware section contains information about 11 components: RAM, keyboard/mouse, monitor/videocard, modem, printer, processor, hard drive, CD ROM, sound card, scanner, and microphone. Each component selection embodies a description and utility. The presentation software branch contains suggestions for health education uses and features to seek when purchasing presentation software. The word-processing and desktop publishing section compares the two types of software and their uses. The Internet segment consists of information about Internet basics, e-mail, discussion groups, resources, the world wide web, and selecting a service provider. The graphics component includes information about animation, clip art, morphing, video, and draw and paint programs. The CAI branch is made up of CAI formats, applications, evaluating health promotion software, ways of developing CAI, and guidelines for developing CAI.
The CAI module utilizes multimedia, including animation, sound, and video. The module follows a tutorial format, allowing the student to choose which pathway she wanted to pursue. Scenarios are presented with questions at the end of each pathway to assess the student's progress and to provide feedback on that progress.
The subjects for this study comprised a convenience sample of 28 students recruited as volunteers from those enrolled in undergraduate Department of Health Studies courses at Texas Woman's University in the spring semester of 1997. The first five students on the schedule were used as the pilot test group.
The remaining 23 students made up the test group. In order to use the CAI module, students accessed a computer and zip drive. Each participant used the CAI module and was observed during the process of using the CAI, with the overall time spent on and the pathways taken in the module recorded by the investigator.
All participants were asked to self-report their age, gender, and prior computer training and experience. In addition, they completed a brief checklist consisting of ten questions from the criteria developed by Barker and King (1993) that helped them to evaluate the CAI module during the viewing. Students were interviewed immediately afterwards in order to determine the student's perceptions about the CAI module. The interview questions consisted of the remaining questions developed by Barker and King (1993). At the end of this questioning, the students were given the opportunity to add any other comments that they wanted to make about the CAI module. Total participation by each student required no more than one hour. Responses were recorded on audiotape and notes were taken during the interviews by the investigator. The questionnaires were coded to assure anonymity. Audiotapes were then transcribed by the investigator. Transcripts were analyzed for recurrent responses and coded for data analysis.
Evaluation criteria developed by Barker and King (1993) for interactive multimedia courseware were used by the students in this study as a checklist for evaluating the CAI during use and as prompts during the interview process. The evaluation checklist included the following categories: engagement, interactivity, tailorability, appropriateness of multimedia mix, mode and style of interaction, quality of interaction, quality of end-user interfaces, learning styles, monitoring and assessment techniques, built-in intelligence, adequacy of ancillary learning support tools, outstanding strengths and attractive features, and outstanding limitations and weaknesses. One question from each of the ten categories comprised the checklist. The investigator added four demographic questions following the checklist. These questions included age, gender, computer experience, and computer training. Computer training and computer experience were differentiated because many individuals are self-taught. The remaining questions were used during the interview process.
Data were entered and compiled using the computer programs Biomedical Statistical Package (BMDP) and Statistical Signal Processing (SSP).
The data collection instrument collected demographic data with four questions regarding the gender of the participant, the age of the participant, previous computer experience, and previous computer training.
All 28 of the participants were female. The age of the participants in this study ranged from 20 to 46 years with a mean age of 28 years.
All 28 of the participants had previous computer experience. Twenty of the participants (71%) had some type of previous computer training. The types of training included a high school course, college level courses (Introduction to Computers at Texas Woman's University [TWU], Information Delivery Systems at TWU), and seminar classes at the TWU library. Eight participants (29%) indicated that they had no previous computer training.
The module's main menu offers two selections, or pathways. Within each pathway there is information about that topic. Each pathway may offer additional pathways or branches in which the user can select in order to obtain information on that subtopic. Table 1 lists the hardware topics viewed. Table 2 lists the software topics viewed. The topics viewed by the least number of participants were compatibility, masters, charting, slide sorter, and transitions. All 28 participants viewed the Internet topic.
All 28 of the participants indicated that the CAI tasks were at the right level and interesting. The participants were almost evenly split in response to their certainty as to how to proceed through the CAI.
The participants responded in a variety of ways when asked how they felt about the way in which information was presented to them. These responses included: interesting, good, logical and clear, informative, user friendly, simple, organized, unclear, wanted feedback when something had already been viewed, and not enough cultural diversity of graphics.
The participants were split when asked if they ever felt that they were making choices just for the sake of it. Yet, a majority of the participants felt that they could decide what they wanted to do. Several participants indicated that they wanted to have the option to interact more with the module.
A majority of the participants indicated that graphics, video, and sound quality were of appropriate size and quality. Comments in reference to the video and graphics included that they were too small if a group would be viewing, italics fonts were hard to read, and video played too early and distracted them from reading what was on the screen. A back button was requested by one participant in order to review a previous screen.
The participants responded positively to these items or indicated that the way they interacted made no difference to them.
|Table 1. Hardware Topics Selected for Viewing By Participants (N=28)|
The participants (N=28) were asked how they felt about the screen displays. There were a variety of responses. Some of the responses were as follows:
Seven participants responded that at times they were confused while utilizing the CAI module. One participant indicated that the hardware section was too technical. Others (n=6) reported that the instructions weren't clear enough and "went in circles."
A majority of the participants reported that they knew how to move from one part to another and that there was a regular way of moving about the CAI module.
The participants were asked for any additional comments. The comments are as follows:
A Pearson correlation coefficient was calculated to determine the relationship between time spent using the CAI module and the amount of topics chosen. There was a direct relationship between time spent using the module and the amount of topics chosen (r=0.813,p<.001). An independent t-test was calculated to determine differences between previous computer training with varying amount of times spent using the CAI module. A significant difference was found between having previous computer training or not in terms of the amount of time spent using the module, t(26) = 2.08, p < .05. A chi-square followed by a post hoc test run on SSP was calculated to determine differences between previous computer training with varying topic selections. A significant difference was found between having previous computer training or not in terms of the topics selected, c2 (61) = 624.22, p < .01.
|Table 2. Software Topics Selected For Viewing By Participants (N=28)|
|Word Processing/Desktop Publishing||19||67.9|
|Draw and Paint Programs||14||50..0|
|Computer Assisted Instruction||23||82.1|
|Drill and Practice||5||17.9|
|Ways to Develop CAI||15||53.6|
|Guidelines for Developing CAI||10||35.7|
|Delivery of Programs||11||39.3|
|Evaluating CAI Software||8||28.6|
|Selecting a Service||15||53.6|
Although studies have compared CAI to other more traditional methods and use of CAI for a variety of topics, the literature failed to provide documentation of students' perceptions of a CAI module on computer literacy. The results of this study provide a formative evaluation of a computer literacy CAI module provided by undergraduate community health majors.
Time on Task and Topic Selection
Time on task ranged from 5 to 32 minutes, with a mean of 18 minutes. There was a significant difference between the amount of time spent using the CAI module by those with previous computer training and those without. Those with previous training utilized the module for a mean of 18.15 minutes, while those without previous training utilized the module for a mean of 20.88 minutes.
As a component of the two main topics, hardware and software, the Internet was viewed the most. Perhaps this was due to all of the recent media focus about the Internet.
Topic selection was diverse and the amount of topics chosen seemed to be related to time on task. In general, the longer the time spent using the CAI module, the more topics that were selected. Software and hardware topics were equally chosen first. The software pathway or topic was selected the most, with several participants returning to this portion of the module for a second time. As a component of the two main topics, hardware and software, the Internet was viewed the most. Perhaps this was due to all of the recent media focus about the Internet. There appeared to be a relationship between topic selection and previous computer training, as indicated by the chi-square test, and the post hoc test.
Perception of CAI Module
The questionnaire's engagement category can be used to assess whether the product engages the user's interest, or involves the user because of factors which are especially motivating, enjoyable, or challenging. In general, the participants' perceptions were positive for this category. Some participants reported that the mechanics dealing with where they were in the program were confusing at times. Participants expressed this confusion regardless of previous computer experience or training. Those who reported confusion tended to repeat topics but did not necessarily spend more or less time on task.
In the interactivity category, the module was assessed on whether it offered both passive and active interactions with the user and whether it provided the means by which a high degree of user involvement could be achieved. The participants generally responded positively to this category. Some of the participants believed that they were making choices "just for the sake of it." There appeared to be no relationship between this belief and previous computer training and experience, topics viewed, or time spent on task.
Questions in the appropriateness of the multimedia mix category assessed whether various multimedia features worked well in relation to the educational aims of the module and in relation to one another. On the whole, participants perceived the multimedia mix positively. Several participants said that the video clips added to the learning experience but needed to play a little later, allowing the participants to finish reading the accompanying text. A majority of the participants indicated that having extra features would help them learn. The participants who said no to this question reported that the module was "fine the way it was." Several of the participants asked for clarification of this question; asking if the question meant that the multimedia feature helped them learn or whether additional features would help them learn. Once again, the perceptions of this category seemed to have no relationship with previous computer experience or training, time on task, or topics viewed. The participants who did not view all of the topics did not see or hear many of the multimedia features and, therefore, had no basis on which to respond to the question.
The mode and style of interaction category sought comment on the nature of the interface in terms of the mode of interaction and the style of interaction, such as, the way choices were made or options selected. All except one of the participants reacted positively to the items in this category. When one participant responded with no, she clarified that the mode or style of interaction made no difference in her perception of the CAI module.
The quality of interaction category assessed the quality of the participants' interactions with the system. The nature of the control that is given to the participant, and the ease of use of the module in terms of the help and support systems and their general level of accessibility were considered. The category also permitted possible comment on the capacity of the module to offer participants the opportunity to make real decisions about her route through the module, thereby enhancing a sense of ownership in relation to the learning process. Observation and interview responses indicated that not all participants knew how to proceed while using the module. There seemed to be no relationship between not knowing how to proceed and previous computer experience or training, time on task, and topics viewed. A few participants reported that the instructions were not clear or that more were needed. Navigation problems may have been due to several factors, including: (a) lack of proper instruction and guidance throughout the module; (b) inexperience of participants with this type of media and format; (c) more time was needed to adjust and learn how to navigate; and (d) previous computer experiences, training, and attitudes may have influenced the participant. Some believed that the hardware section was too technical. Again, there did not seem to be any relationship between these perceptions and the participants' previous computer experience and training, the topics viewed, and time on task.
The quality of end-user interfaces category addressed the issue of the type of interface that is presented to the participant and with which the participant will have to interact. Type of interface or features might include the use of color and graphics, windowing, the design of icons and the positioning of information. The question about knowing how to use the help system was not included in the instrument or interview because the CAI module did not have a help system. Participants who indicated they had problems with the mechanical aspects of the CAI reported negative perceptions about knowing how to move from one part to another and about using the icons and how to use them.
In the learning styles category, comments were made on the learning style chosen for the subject content under consideration, and the views held by the learner and designer about the conceptual learning model and the design aims of the product. In this category, participants could also comment on whether the learning material recognized their existing skills and encourages the transfer of these skills into the new learning situation. A majority of the participants considered the learning style of the CAI module to be a tool or a combination of a tool and another learning style. Most of the participants believed that they brought previously acquired skills to this module. No relationship between this belief and previous computer experience and training, time on task, or topics chosen was apparent.
The monitoring and assessment techniques category assessed whether the module monitored the progress of its users, offered formal assessment, or provided the means for self-assessment. This category also addressed whether the module had features that can utilize user information in order to provide support or enhance engagement. A majority of the participants believed that their progress was assessed but eight did not believe that there was adequate feedback. Twenty-seven of the participants did indicate that the module would help them set goals. One student reported that she wanted feedback when she had already viewed a topic, possibly by that topic button changing in some way. There appeared to be no relationship between these perceptions and previous computer experience and training, time on task, or topics chosen.
The last category addressed was built-in intelligence. This category was concerned with whether the module included features of an intelligent tutoring system, used artificial intelligence techniques with knowledge bases, used an expert system, or used monitored information to provide user support such as advisement strategies or prompting. The participants were divided on their perceptions about "getting extra help from the module because it seemed to remember what they did in the past." There seemed to be no relationship between this perception and previous computer experience or training, time on task, or topics chosen.
The investigator asked the participants for any additional comments or observations about the CAI module. Responses to this question were fairly consistent with responses to prior questions. A few participants asked where the module would be available for their future use. Others recommended adding the use of the module in the undergraduate community health curriculum, especially in the Information Delivery Systems course.
Questionnaire and Presence of Investigator
Two factors that may have contributed to the results of the present study were the questionnaire and the presence of the investigator. The questionnaire was developed by Barker and King (1993) to evaluate interactive multimedia courseware. The questions for non-expert users were applied but
Navigation problems may have been due to several factors, including: (a) lack of proper instruction and guidance throughout the module; (b) inexperience of participants with this type of media and format; (c) more time was needed to adjust and learn how to navigate; and (d) previous computer experiences, training, and attitudes may have influenced the participant.
The presence of the investigator, both during the observation and the interviewing processes, may have influenced the results. One participant specifically stated that she would have spent more time on the module if the investigator had not been present, while others indicated that they spent more time on task because the investigator was present. Many participants asked the investigator if they had viewed all of the topics. The investigator instructed the participants that questions could not be answered during the viewing of the module. Because of the investigator's response, participant frustration may have resulted, possibly leading to a shortened or lengthened time on task. Before the interview, the participants were instructed to answer the questions as honestly as possible. However, the participants may have answered the questions in an effort to appease the investigator.
Suggestions for the CAI Module
Based on the results in this study, recommendations can be made for the CAI module. The module should include more instruction throughout. More guidance may alleviate the confusion about the mechanics of navigating in the module. There was evidence during observation and the interviews that some of the participants had difficulty navigating in the CAI module. During the observation process, some participants hesitated or often repeated material. The participants often did not know how to proceed with the scenarios. The participants could make several selections and would return to the scenario after the appropriate feedback. A "Done" button was available for the participant to select once she was finished. Many of the participants used a menu button instead. During the interviewing process many of the participants reported being confused about navigating the module and suggested more instructions. One way the investigator could address this problem is through different levels of difficulty in the module. There could be a beginner's section which would contain more detailed instructions. But as the user advanced through the intermediate level and into the advanced level, there would be fewer instructions.
Feedback, such as highlighted buttons, may alleviate some of the confusion about the module's mechanics. During the interview process, one participant suggested just this idea. Menu buttons could change color after the topic has been viewed. This would help the participant realize that the topic was viewed and reduce unnecessary repetition. More quizzing or interactions should be included in order for the users to self-assess their progress. The most important untapped aspect of CAI is its capacity to generate interactive quizzes and to guide self-assessment, including remedial exercises, without embarrassment to the learner (Jaffe & Lynch, 1995). All goal-oriented learning needs feedback so that the student knows when the learning objective has been achieved.
A back arrow that allows participants to view previous screens would provide more individual interaction. During the interviewing process, a participant indicated that the menu buttons provided insufficient interaction and control. She wanted a button that would allow her to see the previous screen. The CAI module could be set up by levels, such as, beginners, intermediate, and advanced. This would allow for more appropriate individual interactions. For those participants who wanted more advanced features, such as practicing the use of drawing tools, different levels would allow for the inclusion of these types of interaction. Several participants tried clicking on a variety of graphics, wanting to interact and practice some of the tools described in the module. During the interviewing process, these same participants expressed a desire for more interaction in order to learn more about these specific applications. In order to accomplish this, more opportunities would need to be created. This would allow the user to click on graphics or text, allowing her to branch to an application.
When videos are involved, a play button should be added in order for the user to start the video after reading the text. Animation could be played in the same way. Several participants indicated during the interviewing process that they were distracted from reading and missed part of the videos and animations because they started playing before the participants were finished reading. A play button would also enable the user to play the video or animation as frequently as she wanted.
The overall appearance of the CAI module should be changed in order to give it a more professional appearance. Multicultural aspects of graphics should be added to the module. The investigator attempted to represent a variety of ethnic groups but because of the limitations of available software and graphics, this was not accomplished to the satisfaction of both the participants and the investigator. Time was a factor in the development of the module, not allowing for all the special touches needed for a professional look. For example, more multimedia could be added, including more sound and animation.
More computer topics should be added to the CAI module. Because of time restraints during module development, several topics were not included in the module. Additional topics would include databases, statistical programs, and spreadsheets.
Record-keeping capabilities would be added to the CAI module for future research purposes. This would involve tracking the user's pathways through the module, time spent on the module as a whole and on individual topics, and correct and incorrect responses to questions asked in the module. Once again, because of time constraints, this aspect was not included in the current module but would help alleviate problems associated with data collection via investigator observation.
Suggestions for Future Research
Based on this study, the following are some of the directions proposed for future research:
Barker, P., & King, T. (1993). Evaluating interactive multimedia courseware-a methodology. Computer Education, 21(4), 307-319.
Butler, J.T. (1997). Principles of health education and health promotion (2nd ed.). Englewood, CA: Morton Publishing Company.
Chodorow, S. (1996). Educators must take the electronic revolution seriously. Academic Medicine, 71(3), 221-226.
Gilbert, G.G., & Sawyer, R.G. (1995). Creating strategies for school and community health. Boston: Jones and Bartlett Publishers.
Glenn, J. (1996). A consumer-oriented model for evaluating computer-assisted instructional materials for medical education. Academic Medicine, 71(3), 251-255.
Gold, R. (1991). Microcomputer applications in health education. Dubuque, IA: Brown.
IDG's 3-D Visual Series. (1995). Computers simplified (2nd ed.). Foster City, CA: IDG Books Worldwide, Inc.
Infopoint (No date). Infopoint: Computer based training [Online]. Available HTTP: http://web20.mindlink.net/infopoint/cbt.html (1996, November 14).
Jaffe, C.C., & Lynch, P.J. (1995). Computer-aided instruction in radiology: Opportunities for more effective learning. American Journal of Radiology, 164(2) 463-467.
Komando, K. (1995). 1,1001 komputer answers. Foster City, CA: IDG Books Worldwide, Inc.
Miller, L.A. (1995) Computer assisted instruction [Online]. Available HTTP: http://www.n ova.edu/gtep/ SCI/SCI1651/CRS124/leemill.html [1996, November 14].
Sternin, E. (No date). CCCP report on computer literacy 1993/94 [Online]. Available HTTP: http://www.physics.broc ku.ca/doc/ccc p/cccp.html (1996, November 14).
Weil, M., & Rosen, L. (1990). The aetiology of computerphobia. Computers in Human Behavior, 6, 361-379.