Abstract

Abstract

Introduction

Asset-Based

Assessment

Evaluation

Discussion

References

Adolescent pregnancy is a significant public health problem. The adolescent pregnancy rate in the United States is higher than in many other industrialized nations. Reviews of programs designed to prevent teen pregnancy conclude that few effective interventions exist. This article describes a promising approach to teen pregnancy prevention currently underway in diverse neighborhoods in Oklahoma City. The project--Healthy, Empowered and Responsible Teens of Oklahoma City--is asset-based and focuses on positive youth development. The needs and assets assessment and the evaluation methodologies are described in detail. The methods are both qualitative and quantitative and include key informant interviews, focus groups, data mapping, observation, collection and content analysis of documents, case studies, a neighborhood youth and resident survey, community leader survey, and organizational network survey. Concepts measured in the evaluation include neighborhood mobilization and planning, implementation, institutionalization, sustainability, assets at multiple levels, risk behaviors associated with adolescent pregnancy, and teen birth rates.

Introduction

Abstract

Introduction

Asset-Based

Assessment

Evaluation

Discussion

References

Over one million girls become pregnant in the United States each year.^1-3 The rates of teen pregnancy and births in the United States continue to remain far above those of other industrialized nations. Compared to females who wait until age 20 to give birth, teenage childbearing results in poorer health outcomes, lower levels of school completion, an increased chance of welfare dependency, larger families and a greater likelihood of being a single parent.^3-5

During the preceding decades program providers and researchers have explored several different strategies to prevent adolescent pregnancy, from school-based programs serving pregnant and parenting students to sexuality education to programs that link with job training. Unfortunately, recent comprehensive reviews of adolescent pregnancy prevention programs and evaluations draw similar conclusions: after several decades of discussion, strategy development, and pilot programs, there exist few programs that have been well-evaluated and successful in reducing teen pregnancy.^6-8

One of the most thorough reviews of teen pregnancy program interventions and their evaluations identified four broad factors that consistently predict early parenthood: poverty, early school failure, early behavioral problems, and family problems and dysfunction.⁶ Moore and colleagues ⁶ note that only a small number of interventions address the four factors identified as predictors of early parenthood. Instead, they observe that interventions focus narrowly on one aspect of prevention, tend to be brief and superficial and happen too late to have any impact on high- risk populations. Moore and colleagues ⁹ also describe several points that need to be considered in designing effective adolescent pregnancy prevention programs, including the

...four broad factors that . . . predict early parenthood: poverty, early school failure, early behavioral problems, and family problems and dysfunction.

importance of working with families and communities to create successful programs and avoid destructive controversy, the need to recognize cultural diversity and that varied groups need varied degrees of intervention. They also encourage adolescent pregnancy prevention projects to recognize that sexual risk-taking is one of several forms of risk-taking, that non- voluntary sex plays a role in initiation of sexual activity and pregnancy, and that males partners of adolescent women may not be teenagers.

In another review of the literature, Philliber & Namerow⁸ comment that given the complexity of issues surrounding adolescent pregnancy, the program responses have been " too simple, too weak, too short, and overall, not up to the task of dealing with these complex behaviors and the societal trends surrounding them" (p. 3). They note several factors that only recently have started receiving attention, including the role of adult men, the link with other risk behaviors such as alcohol and drug use, the direct relationship with sexual abuse, and the greater number of teens growing up in poverty.

Recent reviews of evaluated programs suggest there may be general strategies that are more likely to be effective in addressing adolescent pregnancy and its antecedents. These include paying attention to cognitive skills and educational achievement, job preparation and opportunities, specific skills necessary to avoid pregnancy, and peer influences. Additional promising strategies include support from a responsible adult and involvement of community members in planning and carrying out interventions to enhance cultural relevance.⁸ Another report summarizing what works in adolescent pregnancy programs concludes: "there are no single or simple approaches that will markedly reduce adolescent pregnancy. [...] programs need to address both postponing sex and using contraception. In addition, particularly among high-risk groups, prevention initiatives should address such other factors as poverty, lack of opportunity, family dysfunction, and social disorganization more generally."⁷ (p. 13).

These conclusions highlight the need to address a greater number of factors and implement more broadly those programs with the greatest evidence for success. Kirby⁷ stresses that while giving greater attention to the broad array of risk factors that appear to impact adolescent pregnancy (e.g., poverty, lack of opportunity, and other aspects of social disorganization), we must rigorously evaluate approaches that show promise.

In recent years, youth development has emerged as a major prevention strategy. Youth development is a comprehensive approach that focuses on providing youth with skills that will help them succeed as adults. The Healthy Communities, Healthy Youth initiative from Search Institute¹⁰ provides one framework for a positive youth development approach. Search Institute has identified developmental assets that all young people need to grow into healthy adults. These assets are conceptualized as both internal and external to adolescents. Internal assets include concepts such as educational commitment, values, social competencies, and positive identity. External assets include concepts such as support, boundaries and expectations, and time use. These specific internal and external assets provide the foundation of the asset approach to youth development.

An asset-based approach to youth development acknowledges that adolescents are embedded in social contexts which greatly influence risk behavior, including sexual behavior. Social contexts exist at a variety of levels, including family, school, neighborhood and the larger community. In recent years, research has begun to explore the relationships among social context and a variety of adolescent risk behaviors. Resnick and colleagues¹¹, for example, examined the relationship between individual, family and school contexts and sexuality, violence, emotional health and substance abuse in adolescents.

Family Context

In Resnick and colleagues' study of context and adolescent risk behavior, five variables were used to operationalize family context: parent-family connectedness, parent- adolescent activities, parental presence at key times during the day, parental school expectations and family suicide attempts or completions.¹¹ Sexuality was operationalized through two behaviors: age at first intercourse and history or pregnancy. Several family-related factors were related to both measures of sexuality. Factors associated with delay of first intercourse included high levels of parent-family connectedness and parental disapproval of sexual activity and contraceptive use. Similarly, parental disapproval of adolescent contraceptive use and a larger number of shared activities with parents protected against history of pregnancy.

School Context

Resnick and colleagues used nine variables to measure the school context. These included school connectedness, student prejudice, attendance, dropout rates, type of school, classroom size, percentage of teachers with master's degrees, proportion of students who are college-bound and percentage of parents involved with a parent-teacher organization.¹¹ Although no school factors were associated with teen pregnancy per se, several factors were associated with delay of first intercourse, including connectedness to school, attending a parochial school and attending a school with high attendance rates.

Neighborhood Context

Coultron¹² summarized research on the relationship of socioeconomic characteristics of neighborhoods and children's well-being. She found that the presence of affluent and middle-class families promotes school achievement, cognitive development, and avoidance of teen pregnancy. Coultron also noted that neighborhood social organization has been shown to be related to teen pregnancy. As neighborhoods experience increases in turnover and female head of households, a community's internal control is diminished, resulting in decreases in friendship networks, participation in institutions, normative consensus and monitoring of the environment.

Other studies have examined neighborhood context and its relationship to health outcomes such as violence and low birth weight.^13-15 Sampson and colleagues¹³ conducted a study showing that collective efficacy, defined as social cohesion and informal social control, is associated with lower levels of violence. When controlling for differences in a wide variety of factors such as neighborhood composition, kinship and friendship ties, organizational participation, and neighborhood services among others, collective efficacy still predicted lower rates of violence. Roberts¹⁴ examined the relationship between neighborhood social environments and low birth weight. He examined neighborhood-level indicators while controlling for individual-level indicators. He concluded there are social influences beyond health-seeking behavior of mothers which influence health, factors such as resource distribution and neighborhood and family dynamics that promote health.

The remainder of this article describes a teen pregnancy prevention project that uses a building assets model and details the quantitative and qualitative approaches used in program planning and evaluation. The process and outcome evaluation strategies are designed to document not only the nature and extent of program implementation and reductions in problem behaviors among children and adolescents, but include approaches for measuring neighborhood and community-wide changes--changes that include the extent to which individual, family, neighborhood, and community-wide assets are strengthened.

An Asset-Based Teen Pregnancy Prevention Program

Abstract

Introduction

Asset-Based

Assessment

Evaluation

Discussion

References

In 1995, the Centers for Disease Control and Prevention (CDC) launched a community-based teen pregnancy prevention project. Thirteen communities across the country were selected to participate in a two-year planning process, followed by a five year intervention phase. The planning phase included: conducting a community needs and assets assessment, building a structure and process (coalition, task force, etc.) for input from adults and youth living in the selected areas, and developing a local action plan. In each community, a hub organization worked with a variety of local partners to outline a program model that would provide the framework for the development of the neighborhood and community action plans. Potential interventions were identified, based on the needs and assets assessment and a program model design.

The Oklahoma City (OKC) project covers an 18-zip code area in the central city that represents a population of over 225,000 that has a teen birth rate that is over 50 percent higher than the national average. Central OKC is the most ethnically diverse part of the metropolitan area and includes neighborhoods that are among the poorest. The project, called HEART of OKC (Healthy, Empowered And Responsible Teens of OKC), is based at the Oklahoma Institute for Child Advocacy (OICA), a non-profit organization that promotes issues related to the health and well-being of children and youth.

The HEART of OKC project was designed to be a locally-driven planning process that would actively engage different racial and ethnic populations, involve teens and adults in meaningful ways, and be representative of the range of community sectors and networks. The project is population-based as well as neighborhood-based. The agencies selected as partners each had strong community credibility and a respected track record of programs and services in the selected neighborhoods. They also served diverse racial and ethnic populations and geographic areas in the central city. The partner agencies included: Latino Community Development Agency (Riverside neighborhood); Catholic Charities/ Vietnamese Ministries (23rd/ Classen neighborhood); Oak Grove Teen Pregnancy Prevention Project/ College of Public Health (Oak Grove housing project); and United National Indian Tribal Youth (entire catchment). In addition, the Institute for Child Advocacy provided leadership for the Near Northwest neighborhood.

Partner agencies serving different racial and ethnic populations in central OKC selected staff to work as neighborhood coordinators. The coordinators were responsible for working with individuals and organizations in the priority areas to identify neighborhood boundaries, initiate needs assessment activities, engage neighborhood residents, outline assets and resources, facilitate the neighborhood planning process, promote a positive youth development philosophy to prevention, and coordinate the preparation of a neighborhood youth development plan designed to reduce teen pregnancy.

In addition to neighborhood activities, the project linked with existing planning initiatives working on youth and family issues in central OKC, conducted personal interviews and discussions with hundreds of community leaders and agency directors, presented trainings on the positive youth development approach to prevention, sponsored joint projects, secured initial funding for some activities in the priority neighborhoods, identified potential interventions and funding sources, and outlined elements for a community-wide teen pregnancy prevention action plan.

A HEART of OKC program model was developed to provide a framework for the development of the neighborhood and community-wide action plans. The program model was based on a series of eight prevention strategies that were identified by Philliber & Namerow.⁸ The strategies featured in the project's program model included: life skills, positive relationships with adults, educational achievement, job opportunities, primary pregnancy prevention, community involvement, positive peer influences, and health promotion.

During the planning phase, profiles were developed for each selected neighborhood that included basic demographic data, along with information from the needs assessment

The strategies featured in the project's program model included: life skills, positive relationships with adults, educational achievement, job opportunities, primary pregnancy prevention, community involvement, positive peer influences, and health promotion.

process such as windshield tours, interviews, and youth focus groups. The information in the profiles was then compared with the prevention strategies outlined in the program model to determine activities to include in the action plan and potential interventions to be considered during the five-year implementation phase.

The goals for the five year implementation phase include:

1. Decrease teen birth rates and related risk behaviors to enable central OKC youth to increase their chances for good health, school completion, and economic self-sufficiency as adults.
2. Increase the proportion of youth in central OKC who report having assets that are related to the avoidance of teen pregnancy.
3. Create the social and physical environments in priority neighborhoods that will increase the asset base for youth and promote positive youth development.
4. Build the capacity of central OKC to plan, implement and sustain a comprehensive, coordinated and integrated asset-based approach to youth development and teen pregnancy prevention.

The building assets approach has begun changing the way the community views youth, from a perspective where young people are viewed as problems to be fixed, to one where they are viewed as potential to be nurtured. This has proven to be a popular and compelling message at both the neighborhood and community-wide levels, as well as a message that crosses racial and cultural boundaries.

Community Assessment Methodology

Abstract

Introduction

Asset-Based

Assessment

Evaluation

Discussion

References

The community needs and assets assessment for HEART of OKC focused on selected neighborhoods and populations. At present, the assessments have been completed for a Latino neighborhood (Riverside), a Vietnamese neighborhood (23rd Street and Classen), a typical OKC neighborhood (Near Northwest), an African American/Latino neighborhood (Oak Grove), and the American Indian population. The neighborhoods and populations were selected because of high birth rates, high concentrations of priority populations and because there was a community-based organization with ties to either the neighborhood or the population.

A variety of qualitative and quantitative methods were used in the assessments, including windshield tours, key informant interviews, youth focus groups, and mapping of census and birth data. Each of these methods will be briefly described.

Windshield Tours

A minimum of two windshield tours were conducted for each neighborhood. These were done by project staff and by graduate students from the local college of public health. The tours involved driving through an area, often street by street, and taking detailed notes on what was observed. The notes formed the basis of a neighborhood description. Observations were made on resources such as parks, schools, businesses, restaurants, health care facilities, libraries, etc. The observations also included general condition of the neighborhood such as vacant houses, empty lots, bars on windows, house and yard upkeep, lighting and graffiti. Signs indicating a sense of community and neighborliness were also observed, including chairs on porches, people outside and interacting with one another, and children's toys left outside. These observations aided in developing neighborhood profiles which included descriptions of the living conditions (crime, lighting, empty lots/houses, etc.) and resources (churches, parks, schools, businesses).

Key Informant Interviews

A second qualitative method, key informant interviews, provided another major source of information. These were conducted with approximately 20 people per neighborhood/population, for a total of 100 interviews. The key informants were people who lived in a particular neighborhood or worked with an agency that served the area or population. The interviews covered the following topics: history of the neighborhood, past problem-solving efforts, main problems in the neighborhood, main problems for youth in the neighborhood, barriers or problems that might be encountered in working with the neighborhood, positive aspects of the neighborhood, natural leaders, where teenagers hang out, resources and activities for adolescents, services lacking in the area, and names of other people to interview or involve in the project. Information from the interviews was used to identify assets and needs for each of the neighborhoods. In addition, the process of interviewing gave the coordinators access to new sectors of the neighborhoods and helped to establish relationships.

Focus Groups

Focus groups with youth were a third qualitative method used in the needs and assets assessment. Across all neighborhoods, 19 focus groups were conducted and analyzed. One hundred and fifty-nine youth participated, including 22 white, 22 African American, 46 Vietnamese, 27 American Indian, and 42 Hispanic youth. The focus groups were stratified by race, age and gender. For each neighborhood or population, there was a focus group of young adolescent women (ages 13-15), young adolescent men (ages 13-15), older adolescent women (ages 16-18), and older adolescent men (16-18). The purpose of the focus groups was to gather information from neighborhood youth to identify needs and guide the design of interventions that promote youth development and prevent teen pregnancy. Topics covered in the focus groups included general perceptions of the neighborhood, typical youth activities, where teens hang out, organized activities, schools and social groups, teen pregnancy, jobs, relationships with adults, and aspirations. The information was used to understand youth experiences of their families, neighborhoods and schools, as well as to identify additional assets and needs relevant to the planning process.

Data Mapping

Quantitative methods were also employed in the community assessment, including the collection and analysis of secondary data such as crime statistics, birth records, economic data, voting records, drop out rates and listing of various resources such as churches and neighborhood associations. This data was compiled for each neighborhood or population in a series of tables, graphs and maps. Much of the census data was mapped using ArcView, which is geographic information system software. Birth data were also mapped. This enabled the neighborhood task forces to see where teen births were occurring as well as the distribution of associated risk factors across the neighborhoods.

Evaluation Methodology

Abstract

Introduction

Asset-Based

Assessment

Evaluation

Discussion

References

In addition to the community assessment methods used for program planning during the first two years of the project, the current five year implementation phase includes both process and outcome measures for assessing the extent to which the project goals are accomplished. The outcome evaluation procedures detailed below include measures of individual, family and neighborhood assets, as well as measures of sexual activity, contraceptive use and changes in inter-organizational relationships with and across community sectors. Both qualitative and quantitative evaluation techniques will be used. Copies of the measures are available upon request from the authors.

Cross-Site Process Indicators

CDC formed several evaluation working groups to develop indicators and accompanying measures of 1) community organization, management, mobilization and support, 2) needs and assets assessment, 3) descriptions of interventions, and 4) financial sustainability. Each of the indicators and how they will be measured is described in a guidebook developed for use by the thirteen demonstration projects.¹⁶ Briefly, assessment of community organization, the needs and assets assessment, interventions, and financial sustainability will be done through a questionnaire to be completed by the project director or a knowledgeable staff member. Community organization will also be measured through a survey of coalition members. The indicators will be measured annually.

Neighborhood/Community Mobilization and Planning

Case studies of each of the neighborhood planning groups were conducted during the planning phase of the project. Data sources for these case studies included interviews with the neighborhood coordinators, observation of neighborhood task force meetings, analysis of meeting agendas and minutes, and logs documenting member recruitment and resource mobilization. These data, along with the neighborhood profiles and action plans, will form the basis for case studies which will describe the evolution of the mobilization and planning process within each neighborhood.

Implementation

A series of process logs have been developed for documentation of key activities and events leading up to implementation of the major HEART of OKC activities. For each major activity in the action plan, staff complete process logs that cover four areas: community actions and planning activities, partnerships, resources and contextual factors. Analysis of these logs will be used to assess progress in meeting objectives and implementing activities according to the established timeline and to identify various approaches for effecting community change.

Program and Participant Fidelity

The fidelity of program delivery refers to the extent to which a program, program component or program element is delivered as intended.¹⁷ Measures of program fidelity include, but are not limited to: assessment of the extent to which hypothesized mediating processes in behavior change actually occur, staff commitment and expertise, adherence to treatment protocols, verification of staff activities, target population attendance, participation and coverage, program delivery format, and the sequencing and scheduling of program elements. Measurement of fidelity may include multiple points of view, such as those of participants, staff, administrators and/or funding agencies, and may cover multiple levels, including the overall program, program components, and program elements.

In HEART of OKC, fidelity will be measured at the overall program level by assessing the extent to which project timelines are met, coordination is facilitated, and activities and services are available and accessible to adolescents and families in the priority neighborhoods. Both the implementation process logs and the neighborhood resident surveys will be used to assess fidelity. Each major program component will also be field tested and then monitored to determine selected aspects of fidelity, including coverage of the target population and participation in program elements.

Program Institutionalization

Program institutionalization refers to the extent to which a program is embedded in the host organization.¹⁷ One measure of program institutionalization assesses the extent to which the program has survived various passages, become routinized, and fulfills a niche in subsystems of the organization. In order to asses program institutionalization in the HEART of OKC project, a level of institutionalization scale will be completed annually by the project director and also included in the annual survey of coalition members.

Financial Sustainability

In addition to the financial sustainability indicators assessed as part of the cross-site process measures, the HEART of OKC evaluation includes additional measures of sustainability. While the cross-site indicators focus on progress in securing external and long term sources of funding, the HEART of OKC evaluation also focuses on infrastructure sustainability. This refers to the sustainability of coalitions and planning groups and may depend on the embeddedness of these structures in the community. Indicators of embeddedness may include, 1) commitment of the lead or host agency to sustaining the coordinating activities of developing a free standing not-for-profit organization with other resources or sources of support, 2) perceived need and support in the community and among community organizations for sustaining the coordinating agency, and 3) policy and/or regulatory reforms that support the need for community coordination such as the recent requirements under welfare reform, or California Proposition 99. Commitment of the lead agency will be measured through an institutionalization questionnaire completed by the lead agency staff and coalition members. Organizational support in the community will be measured through an organizational network survey of varied agencies in OKC that are involved in youth and neighborhood development efforts. Community support and perceived need for addressing the problem of adolescent pregnancy and youth development will be assessed in a community leader survey conducted in the second and fifth intervention year. The assessment of policy and/or regulatory reforms will be assessed through process logs completed by the project staff.

Awareness and Concern about Youth Development and Adolescent Pregnancy

An important issue addressed by many community-based programs is raising the level of awareness and concern about a social problem in the community, and increasing the commitment to addressing the problem among community leaders and in the general population. One way of measuring awareness and concern is through population surveys where respondents are asked to rank or rate community problems and the importance of addressing them. Another approach is to survey community leaders under the assumption that increases in awareness and concern among key leaders will eventually be diffused throughout the community.

HEART of OKC is using two strategies to measure changes in awareness and concern about youth development and teen pregnancy prevention. First, a survey of community leaders was conducted in the planning phase and will be repeated in the second and fifth years of the intervention phase. Awareness and concern about these issues will also be included in a population-based survey of neighborhood residents conducted in the first and fourth intervention year.

Interorganizational Relationships

One of the goals of HEART of OKC is to facilitate cooperation and coordination among the varied agencies in OKC that are involved in youth development efforts. In order to assess changes in interorganizational collaboration within and across community sectors such as recreation, religion, communication, criminal justice, health, business, and education a survey of key organizations in each community sector will be conducted in Oklahoma City and a comparison community (Tulsa). For both OKC and Tulsa, 5-8 organizations will be identified per community sector. Each organization will be asked to rate the nature and extent of their relationship with other agencies listed on the survey instrument. Data collection will be initially through the mail, following a telephone call to secure cooperation and to identify the appropriate respondent in each organization to complete the survey. The survey will be conducted in the first and fourth intervention year.

Youth, Family and Neighborhood Assets

Youth, family and neighborhood assets will be assessed through in-person interviews with a random sample of 750 youth and parents living in the target neighborhoods and 750 youth and parents in matched neighborhoods in a comparison community. These interviews will take place in the first and fourth years of the intervention phase. Assets will be measured in the areas of life skills, positive relationships with adults, job opportunities, community involvement and positive peer influences. To the extent possible, each asset will be assessed from the perspective of the youth, the family, the neighborhood and the school. For example, how does the youth perceive his or her relationships with adults? How does the parent perceive his/her relationship with the youth? How does the parent perceive the youth's relationships with other adults? How do both the youth and parent perceive opportunities for other relationships with adults in the neighborhood? In addition to the asset-based portion of the interview, adolescents will be asked a series of questions to assess exposure to the interventions and level of risk behavior, including sexual activity.

Birth Rates

Adolescent birth rates will be calculated for each of the target neighborhoods and for the central city as a whole at three points in time. Due to small numbers of births at the neighborhood-level, data will be pooled in three year intervals. Thus, comparisons will be made with the following years combined: 1994-1996, 1997-1999, and 2000-2002. The primary difficulties with computing and comparing birth rates, particularly at the neighborhood level, are small sample sizes, relatively low rates and outdated denominator data. If possible, year 2000 census data will be used to estimate rates for the years 1997- 1999, and 2000-2002.

Discussion

Abstract

Introduction

Asset-Based

Assessment

Evaluation

Discussion

References

It is clear from previous evaluations of adolescent pregnancy prevention programs that much work is needed to develop and test programs that are likely to be effective with various population groups, particularly on a neighborhood or community-wide basis. While individual factors such as knowledge, skills, and attitudes influence sexual activity and contraceptive use (the two underlying processes in adolescent pregnancy), some research suggests that a variety of contextual factors may also be important including family, neighborhood, school and community characteristics.

The CDC-funded HEART of OKC adolescent pregnancy prevention project is built around a community assets model. The purpose of the project is to implement and evaluate activities and programs designed to strengthen individual assets in adolescents, as well as assets at the family, neighborhood, school, and community levels that are related to early sexual activity and poor use of contraceptives. Table 1 presents the specific assets at four levels of analysis that are targeted through the HEART of OKC project. These include: (1) strengthened relationships between adolescents and adults; (2) life skills; (3) community involvement; (4) employment; (5) relationships with peers; and (6) adolescent pregnancy prevention programs. As suggested in Table 1, the extent to which adolescents experience these assets is assumed to be affected by the availability of these assets through the family, neighborhood, school, and broader community. For example, positive relationships with caring adults will be affected by family characteristics and adult behaviors, as well as characteristics of teachers and other administrators, neighborhood cohesiveness, and community-wide programs that bring adolescents into contact with caring adults, such as mentoring, sports, and the arts.

Table 1. Assets by Levels of Analysis

As with many Federally-funded programs based on a community-development model, direct service dollars are extremely limited. Therefore, in order to build assets in youth, the HEART of OKC is working closely with neighborhood organizations, public schools, health and human service provider agencies, volunteer organizations, city/county governments, a city-wide coalition of concerned citizens and business leaders, and city-wide voluntary agencies to mobilize local resources and target services and programs to specific neighborhoods and schools in Oklahoma City. In addition, project staff are working to strengthen the city-wide coalition with which the project is affiliated and to develop stronger inter-organizational relationships among key city agencies and organizations that will enhance their ability to coordinate activities at the neighborhood and school level.

The evaluation of community-wide projects, such as the HEART of OKC, require assessing changes at multiple levels--including the individual, the family, neighborhoods, organizations, and broader community in order to document the extent to which changes observed in outcomes may be attributable to the project. At the individual, family and neighborhood levels, the project is designed to assess the extent to which the assets listed in Table 1 are strengthened through the project. For example, the survey discussed in the preceding section on youth, family, and neighborhood assets will collect information on the six assets in Table 1 at two time periods-- in years 1 and 4--rom one adolescent and one adult per household in 750 sampled households in the target neighborhoods and 750 sampled households in comparison neighborhoods in Tulsa, Oklahoma. In addition, the survey instrument will collect information on the exposure of adolescents to program components.

At the organizational and community levels, a network survey of key health and human service organizations in Oklahoma City and the comparison community of Tulsa will be conducted during years one and four to document changes in inter-organizational linkages, services provided to youth, and the extent to which the organizations have adopted an asset model for youth services. A separate survey of key community leaders will be conducted to identify changes in the communities' awareness and concern about adolescent issues and the extent to which their organizations are involved in developing assets for youth.

In order to identify the role of the HEART of OKC project in producing community changes identified through the organizational network survey and survey of key community leaders, process logs are being completed by project staff. These process logs are designed to link staff activities to key community events. For example, project staff are working with a key community organization and a community-wide coalition to strengthen their attention to youth development efforts in selected neighborhoods, communities, and schools. Current plans include convening of a community roundtable of essential community voluntary agencies to discuss targeting specific neighborhoods for youth development programs and efforts. Process logs maintained by project staff will document the project's role in convening the community roundtable. In addition, a critical events methodology will be used to collect semi-structured interview information from important actors in key community events, such as the community roundtable, to document their perceptions of critical actors and incidents leading to the community activities or changes.

Information on individual project components or activities is being collected through intervention--descriptions prepared prior to each intervention including descriptions of the target population, numbers to be served, and setting--along with follow-up information to document the extent and fidelity of the intervention.

Finally, essential information on the sustainability of the project will be obtained through three data collection methods. These include a coalition member survey, an organizational network survey, and a community leader survey. The first two of these will assess the extent to which member organizations and organizations from a variety of community sectors have incorporated the asset-building approach into their missions and

The use of quantitative and qualitative methods will provide complementary information that will be useful in understanding why the interventions either work or do not work.

programs. The latter survey will assess community leader awareness of, commitment to and involvement in an asset-building approach to youth development and teen pregnancy prevention.

The major limitation to this overall evaluation design is the inability to link individual participation in selected intervention components to individual-level behavior change. For example, conclusions about the effectiveness of specific interventions in building assets in youth, which then contribute to changes in risk behavior, will not be possible. Rather, conclusions will be limited to statements about levels of exposure to a variety of interventions and whether involvement in these activities builds assets in youth and reduces risk behavior.

A major strength of this evaluation is the use of multiple methods. The use of quantitative and qualitative methods will provide complementary information that will be useful in understanding why the interventions either work or do not work.¹⁸ Quantitative survey data--the youth, family and neighborhood asset survey, the community leader survey, and the organizational network survey--will be useful in assessing the HEART of OKC's effectiveness in building assets and reducing risk behaviors in the target neighborhoods. These surveys will also provide information on HEART of OKC's effectiveness in building capacity for long-term sustainability of an asset-based approach to youth development and teen pregnancy prevention in OKC. Qualitative methods, such as the process logs and neighborhood case studies, will allow for in-depth understanding of how community changes occurred.

References

Abstract

Introduction

Asset-Based

Assessment

Evaluation

Discussion

References

1. Guttmacher Institute. (1994). Sex and America's Teenagers. New York: The Alan Guttmacher Institute.

2. National Campaign to Prevent Teen Pregnancy. (1997). Campaign Update, Fall newsletter of the National Campaign to Prevent Teen Pregnancy.

3. Brown S & Eisenberg L. (Eds.) (1995). The Best Intentions: Unintended Pregnancy and the Well-Being of Children and Families. Institute of Medicine Report. Washington D.C.: National Academy Press.

4. Centers for Disease Control and Prevention. (1997). State-specific birth rates for teenagers United States, 1990-1996. MMWR, 46(36):837-842.

5. Morris L, Warren C, Aral S. (1993). Measuring adolescent sexual behaviors and related health outcomes. Public Health Reports, 108:31-36 (supplement 1).

6. Moore KA., Brent C. Miler DG and Morrison DR. (1995). Adolescent Sex, Contraception, and Childbearing: A Review of Recent Research. Washington, DC: Child Trends, Inc.

7. Kirby D. (1997). No Easy Answers: Research Findings on Programs to Reduce Teen Pregnancy. Washington, DC: National Campaign to Prevent Teen Pregnancy.

8. Philliber S & Namerow P. (1995). Trying to Maximize the Odds: Using What We Know to Prevent Teen Pregnancy. New York: Philliber Research Associates.

9. Moore KA. Sugland BW. January, (1996). Next Steps and Best Bets: Approaches to Preventing Adolescent Childbearing. Washington, DC: Child Trends, Inc.

10. Benson, P. (1997). All Kids Are Our Kids. Jossey-Bass Publishers, San Francisco, CA.

11. Resnick M, Bearman P, Blum R, Bauman K et al. (1997). Protecting adolescents from harm: Findings from the national longitudinal study on adolescent health. JAMA, 278(10):823-832.

12. Coultron C. (1995). Using community-level indicators of children's well-being in comprehensive community initiatives. In: J. Connell, A. Kubisch, L. Schorr, C. Weiss (Eds.). New Approaches to Evaluating Community Initiatives. Washington DC: Aspen Institute, 1995.

13. Sampson R, Raudenbush S, Earls F. (1997). Neighborhoods and violent crime: A multilevel study of collective efficacy. Science, 277:918-924.

14. Roberts E. (1997). Neighborhood social environments and the distribution of low birthweight in Chicago. American Journal of Public Health, 87(4):597-603.

15. O'Campo P, Xue X, Wang M, Caughy M. Neighborhood risk factors for low birthweight in Baltimore: A multilevel analysis. American Journal of Public Health, 87(7):1113-1118.

16. Hollerbach P. (1997). Cross-Site Indicators for Adolescent Pregnancy Prevention. Washington D.C.: Academy for Educational Development.

17. Goodman R, McLeroy K, Steckler A, Hoyle R. (1993). Development of level of institutionalization (LoIn) scales for health promotion programs. Health Education Quarterly, 20(2):161-178.

18. Steckler A, McLeroy K, Goodman R, Bird S, McCormick L. (1992). Toward integrating qualitative and quantitative methods: An Introduction. Health Education Quarterly, 19(1):1-8.

    ---

    [Reprint (PDF) Version of this article]

    

    ---

    
    Copyright © 1998 by IEJHE.