Detailed Information on the
Broadcasting to Africa Assessment

Designing and implementing additional performance measures that capture the quality and credibility of the radio and television programs.

Implementing a new long-term outcome performance measure that assesses the credibility of the radio and television programs throughout Africa.

Ensuring that funding for broadcasting to Africa is explicitly linked to the accomplishment of relevant performance goals.

Revising the current agency strategic plan to build on a market-based approach to broadcasting.

Measure: Audience Reach in Africa. (millions of listeners and viewers)

Explanation:A key indicator of whether or not BBG broadcasts are reaching their intended audience is the number of people who choose to listen to or view BBG programs. Audience reach, also called weekly audience, is a measure of the regular listening/viewing audience. Regular audience is defined as all adults in a given population listening/viewing at least once a week, as determined by an audience survey that has an adequately designed sample. The number is a culmination of several annual common measures used across all African language services, as well as annual performance goals for each individual service. Annual measures of audience size for each language service factor into the overall audience reach throughout Africa. The number of people choosing to tune in to BBG broadcasts is also influenced by the program quality and the audience's awareness of the program.

Measure: Audience Rating of Broadcast Credibility in Africa. (%)

Explanation:An important outcome measure of the external effect of BBG programs is whether or not the audience finds the broadcasts to be reliable and credible, and thereby regularly tunes in and hears the news and information BBG broadcasts. This measure is an industry research standard used to determine the audience's view of the effectiveness of media. This indicator is determined by the survey question about "trustworthiness of news and information" of those sampled respondents who listened at least once a week to each station. The answers are registered on a five-point scale -- very trustworthy, somewhat trustworthy, neither trustworthy nor untrustworthy, somewhat untrustworthy, or very untrustworthy. The credibility index is the percent of those answering the question in the survey (excluding those who did not respond or did not know) that endorsed very or somewhat trustworthy. As the percentage moves towards 100%, increases are more difficult to attain, thus targets are significantly ambitious when the measure is maintained or shows a marginal increase.

Measure: Audience reach in Central Africa. (%)

Explanation:A key indicator of whether or not BBG broadcasts are reaching their intended audience is the number of people who choose to listen to or view BBG programs. Audience reach, also called weekly audience, is an indicator based upon measurement of the regular listening/viewing audience. Regular audience is defined as all adults in a given population listening/viewing at least once a week, as determined by an audience survey that has an adequately designed sample. Annual measures of the number of people tuning in to BBG broadcasts in their particular country or region are tracked by each individual language service, and directly support the long-term measure of growing the size of the audience throughout Africa. There are a total of 8 language services in Africa reaching 48 countries. Of these, two examples of language service measures of audience reach are included: Central Africa and Horn of Africa. The data shown here is specific to Central Africa.

Measure: Audience reach in the Horn of Africa. (%)

Explanation:A key indicator of whether or not BBG broadcasts are reaching their intended audience is the number of people who choose to listen to or view BBG programs. Audience reach, also called weekly audience, is an indicator based upon measurement of the regular listening/viewing audience. Regular audience is defined as all adults in a given population listening/viewing at least once a week, as determined by an audience survey that has an adequately designed sample. Annual measures of the number of people tuning in to BBG broadcasts in their particular country or region are tracked by each individual language service, and directly support the long-term measure of growing the size of the audience throughout Africa. There are a total of 8 language services in Africa reaching 48 countries. Of these, two examples of language service measures of audience reach are included: Central Africa and Horn of Africa. The data shown here is specific to the Horn of Africa.

Measure: Program quality for broadcasting to Nigeria (Hausa). Program quality is measured and scored on a range of 1 to 4; 4 is the highest score, 3.4 or higher is "good or better". Note: Within the range, small variations are common, but maintaining the rating or progressing towards the next rating is significantly ambitious, especially as the rating nears the perfect score of 4.

Explanation:The program quality score is an outcome measure based in part on how the target population perceives the quality of the specific BBG language services. This measure is an industry standard used to determine the target audience's assessment of the quality of media. A monitoring panel of regular listeners from the target population rates the program's accuracy, reliability, authoritativeness, objectivity, comprehensiveness, and other variables reflecting distinct statutory, policy, and mission mandates for the different stations. Another component of the program quality score is the evaluation of IBB delivery systems (engineering and transmission), marketing and program placement efforts, and other broadcasting support functions. Program content and presentation are evaluated by both internal and external reviewers to determine the overall Program Quality Score. In total, the Program Quality Score is a comprehensive, weighted score, which ranges from 1-4, with 1.0-1.3 = poor; 1.4-1.6 = poor to fair; 1.7-2.3 = fair; 2.4-2.6 = fair to good; 2.7-3.3 = good; 3.4-3.6 = good to excellent; 3.7-4.0 = excellent. Annual measures of the perceived quality of BBG programming are tracked by each language service, and directly support the long-term measures of growing the size of the audience and increasing the overall perceived reliability of the programming. There are a total of 8 language services in Africa reaching 48 countries. Of these, two examples of language service measures of program quality are included: Nigeria (Hausa) and Horn of Africa. The data shown here is specific to Hausa.

Measure: Program quality for broadcasting to the Horn of Africa. Program quality is measured and scored on a range of 1 to 4; 4 is the highest score, 3.4 or higher is "good or better". Note: Within the range, small variations are common, but maintaining the rating or progressing towards the next rating is significantly ambitious, especially as the rating nears the perfect score of 4.

Explanation:The program quality score is an outcome measure based in part on how the target population perceives the quality of the specific BBG language services. This measure is an industry standard used to determine the target audience's assessment of the quality of media. A monitoring panel of regular listeners from the target population rates the program's accuracy, reliability, authoritativeness, objectivity, comprehensiveness, and other variables reflecting distinct statutory, policy, and mission mandates for the different stations. Another component of the program quality score is the evaluation of IBB delivery systems (engineering and transmission), marketing and program placement efforts, and other broadcasting support functions. Program content and presentation are evaluated by both internal and external reviewers to determine the overall Program Quality Score. In total, the Program Quality Score is a comprehensive, weighted score, which ranges from 1-4, with 1.0-1.3 = poor; 1.4-1.6 = poor to fair; 1.7-2.3 = fair; 2.4-2.6 = fair to good; 2.7-3.3 = good; 3.4-3.6 = good to excellent; 3.7-4.0 = excellent. Annual measures of the perceived quality of BBG programming are tracked by each language service, and directly support the long-term measures of growing the size of the audience and increasing the overall perceived reliability of the programming. There are a total of 8 language services in Africa reaching 48 countries. Of these, two examples of language service measures of program quality are included: Nigeria (Hausa) and Horn of Africa. The data shown here is specific to the Horn of Africa.

Measure: Audience Awareness for Hausa, measured for Nigeria. (%) Audience Awareness is the percent of the sampled population, prompted or unprompted, that acknowledges BBG station names in an audience survey that has an adequately designed sample. Note: This measure can fluctuate from year to year based on whether programs are transmitted on preferred media, are placed with popular affiliates, or have sufficient advertising. If annual declines are noted, future targets are adjusted.

Explanation:The statistic is expressed as a percent of the sampled population that knows or recognizes BBG stations, which can then translate to the percent of the entire population of the country or region of which the sample was representative. This measure of the extent to which a target population recognizes BBG is influenced by efforts made by each language service such as ensuring that programs are transmitted on preferred media, are placed with popular affiliates, and have sufficient advertising. Annual measures of the percent of a target audience that is aware of BBG programming are tracked by each individual language service, and directly support the long-term measures of growing the size of the audience throughout Africa and increasing the overall perceived reliability of BBG programming. There are a total of 8 language services in Africa reaching 48 countries. Of these, two examples of language service measures of audience awareness are included: Nigeria (Hausa) and Zimbabwe. The data shown here is specific to Hausa.

Measure: Audience awareness in Zimbabwe. (%) Awareness is the percent of the sampled population, prompted or unprompted, that acknowledges BBG station names in an audience survey that has an adequately designed sample. Note: This measure can fluctuate from year to year based on whether programs are transmitted on preferred media, are placed with popular affiliates, or have sufficient advertising. If annual declines are noted, future targets are adjusted.

Explanation:The statistic is expressed as a percent of the sampled population that knows or recognizes BBG stations, which can then translate to the percent of the entire population of the country or region of which the sample was representative. This measure of the extent to which a target population recognizes BBG is influenced by efforts made by each language service such as ensuring that programs are transmitted on preferred media, are placed with popular affiliates, and have sufficient advertising. Annual measures of the percent of a target audience that is aware of BBG programming are tracked by each individual language service, and directly support the long-term measures of growing the size of the audience throughout Africa and increasing the overall perceived reliability of BBG programming. There are a total of 8 language services in Africa reaching 48 countries. Of these, two examples of language service measures of audience awareness are included: Nigeria (Hausa) and Zimbabwe. The data shown here is specific to Zimbabwe.

Measure: Cost per audience member for broadcasting in Swahili. ($)

Explanation:This efficiency measure tracks the cost per audience member of each language service. Cost per audience member is the total cost of programming and delivery divided by the unduplicated audience for all media, and is calculated for each language service. The ratio (measure) must be evaluated in context of the changes in either the cost or audience size. For example, a short-term capital cost increase to improve transmission may not increase audience size until after a lengthy installation. In some cases, audience size may be kept artificially low by restrictions imposed by the host government or increased competition, not necessarily poor programming or inefficient management. Annual efficiency measures of the cost per audience member are tracked by each individual language service. Costs per audience member may fluctuate annually and also differ compared to other language services based on a combination of factors in each location, including: preferred media in the region, hours of programming, competition from other broadcasters, jamming, and government restrictions. There are a total of 8 language services in Africa reaching 48 countries. Of these, two examples of the efficiency measures of cost per audience member are included here: Swahili and Zimbabwe. The data shown here is specific to Swahili.

Measure: Cost per audience member for broadcasting in Zimbabwe. ($) Note: Costs significantly increased in FY 2007 for additional hours of programming and to counter jamming-which also limits audience size.

Explanation:This efficiency measure tracks the cost per audience member of each language service. Cost per audience member is the total cost of programming and delivery divided by the unduplicated audience for all media, and is calculated for each language service. The ratio (measure) must be evaluated in context of the changes in either the cost or audience size. For example, a short-term capital cost increase to improve transmission may not increase audience size until after a lengthy installation. In some cases, audience size may be kept artificially low by intensive jamming by the host government, not necessarily poor programming. Annual efficiency measures of the cost per audience member are tracked by each individual language service. Costs per audience member may fluctuate annually and also differ compared to other language services based on a combination of factors in each location, including: preferred media in the region, hours of programming, competition from other broadcasters, jamming, and government restrictions. There are a total of 8 language services in Africa reaching 48 countries. Of these, two examples of the efficiency measures of cost per audience member are included: Swahili and Zimbabwe. The data shown is specific to the Zimbabwe language service.

Is the program purpose clear?

Explanation: The program purpose is to promote freedom and democracy and to enhance understanding by broadcasting accurate and objective news and information about the United States and the world to audiences overseas -- in this case, audiences in sub-Saharan Africa by the Voice of America (VOA). This BBG mission embraces the longstanding VOA component of U.S. international broadcasting to 1) serve as a consistently reliable and authoritative source of news, 2) represent America's culture, and 3) present the policies of the United States.

Evidence: BBG Strategic Plan (2008-2013); U.S. International Broadcasting Act of 1994; Foreign Affairs Reform and Restructuring Act of 1998; Voice of America (VOA) Charter (Public Law 94-350)

Does the program address a specific and existing problem, interest, or need?

Explanation: The program addresses the need in Africa for accurate news and information about the United States and events in Africa to audiences in their languages. The dearth of free, unbiased media, a large Muslim population, and the existence of strong anti-Americanism create a need for timely and accurate reporting of U.S. policies and actions as well as events in the region and the world at large. To address this need, in 2007 the BBG has 9 language services that broadcast in 12 languages, plus TV to Africa, in sub-Saharan Africa. This does not include Arabic broadcasting that reaches the region by the Middle East Broadcasting Networks.

Evidence: The 2006 Freedom House Press rating is "Not Free" for 21 African countries, "Partly Free" for 18 African countries and only 7 are rated "Free"; BBG FY 2008 Budget Justification to Congress

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Broadcasting Board of Governors (BBG) is the only entity responsible for all U.S. government and government-sponsored, non-military international broadcasting. As such, its broadcasts are not duplicative of other Federal, state, or local contributions to the problems being addressed in Africa. Although other media outlets broadcast to this region, only U.S. international broadcasting (Voice of America) has immediate and guaranteed access to accurate information about U.S. interests and policies and can be relied upon to provide a distinctly American perspective on events. The mission of BBG broadcasting ("to promote freedom and democracy and to enhance understanding by broadcasting accurate and objective news and information about the United States and the world to audiences overseas") distinguishes it from commercial media organizations. Only VOA provides approved editorials on U.S. policy and the BBG is the sole international broadcasting entity that serves as a critical arm of U.S. public diplomacy missions overseas.

Evidence: The U.S. International Broadcasting Act of 1994 and the Foreign Affairs Reform and Restructuring Act of 1998 state that U.S. international broadcasting should not duplicate the activities of private United States broadcasters or the activities of government-supported broadcasting entities of other democratic nations.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no conclusive evidence that another approach would be more efficient or effective at achieving the program's goals. In the July 2003 GAO report "New Strategic Approach Focuses on Reaching Large Audiences but Lacks Measurable Program Objectives" (GAO-03-772) the Broadcasting Board of Governors (BBG) identified the internal challenge of consolidating and rationalizing the overall enterprise. "The diversity of the BBG - organizations with different missions, different frameworks, and different constituencies - makes it hard to bring all the separate parts together in a more effective whole." In the same report the BBG also identified a solution to this challenge, a "single system" approach to broadcasting. In follow-up testimony to Congress (GAO-04-711T) the GAO found that "The Broadcasting Board of Governors has responded to a disparate organizational structure by developing a new strategic approach to broadcasting" and "To streamline its operations, the Board has used its annual language service review process to address such issues as how resources should be allocated among language services on the basis of their priority and impact, what degree of overlap should exist among services, and whether services should be eliminated because they have fulfilled their broadcast mission." For example, VOA TV and the former WORLDNET merged in 2004 and created a focused, multimedia organization that will reach larger audiences with enhanced programming in the most effective media. The BBG has developed measurable program objectives that are integrated with the budget by linking performance with resources and setting performance targets for requested enhancements. In addition, the BBG has annual and long-term performance measures that track the success of each individual language service in supporting the agency's overarching goal of promoting freedom and democracy and enhancing understanding about the United States and the world to audiences overseas. By providing objective and accurate news and information, the BBG plays an important part in the broad support of democracy and the open exchange of news and ideas in countries and regions of the world where, because of geographic, developmental, or political reasons, there is a dearth of free and open information. U.S. international broadcasting by the BBG provides an alternative voice in regions where free speech is stifled, and it empowers and educates oppressed populations about freedom, democracy, and the world.

Evidence: GAO Report: "New Strategic Approach Focuses on Reaching Large Audiences but Lacks Measurable Program Objectives" (GAO-03-772); GAO Testimony: "U.S. International Broadcasting: Challenges Facing the Broadcasting Board of Governors" (GAO-04-711T); FY 2006, 2007 and FY 2008 Congressional Budget Requests; Africa Language Service and Program Reviews.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The BBG promotes freedom and democracy and enhances understanding by broadcasting accurate and objective news and information about the United States and the world to audiences overseas. To support this mission, the BBG must ensure that audiences are actually tuning in, are receiving the broadcasts in their preferred language and media, and are confident that the message is credible and reliable. The 2008-2013 BBG Strategic Plan includes implementation strategies that direct efforts and resources to priority areas of U.S. interests, including "Build on our reach and impact within the Muslim world " and "Help audiences in authoritarian countries understand the principles and practices of democratic, free, and just societies". To assure that the programs are designed effectively and that resources are directly supporting the mission, the BBG uses the annual Program Review process to conduct audience and market research that determines whether or not the target audiences are being reached and recommends actions to increase the effectiveness of the program when the research indicates a need to do so. This information also supports the BBG Language Service Review process in determining the strategic priorities and resource allocations across all BBG audiences. The BBG targeting efforts focus on: 1) Choice of delivery medium: As part of the research conducted for Program Review, InterMedia (an independent research organization under contract to the BBG) conducts audience surveys on the type of media available to and preferred by target audiences. Based on the results of the survey and input from the IBB Office of Engineering about technical feasibility and the Office of Marketing and Program Placement, the BBG selects the media mix that will be most effective at reaching the target audience. 2) Choice of language: BBG programming is broadcast in the local languages of the target areas. 3) Program Content: As part of the research conducted for Program Review, control panels and focus groups are used to determine the information needs of target populations and obtain feedback on several program attributes, such as the relevance and objectivity of content. Internal content analysis and production analysis by the Office of Performance Review also inform program content decisions.

Evidence: BBG Strategic Plan 2008-2013; In May, 2006, a new, research-based Africa Strategy was approved by the BBG that links the BBG strategic priorities to target audiences by country and is described in the Broadcasting to Africa Performance Plan; InterMedia Research presentations during Africa Language Services Program Reviews; State Department Background Notes for countries in Africa; U.S. International Broadcasting Act of 1994; Foreign Affairs Reform and Restructuring Act of 1998; Voice of America Charter; Program Review Scoring Procedures; Language Service Review process

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The BBG's mission of objective journalism will build audiences in areas of U.S. strategic interests that trust U.S international broadcasting news and information and enhance understanding. Measuring the program's direct impact to promote or sustain freedom and democracy in Africa is virtually impossible due to the numerous factors (political, geographic, historical, educational, economic, ethnographic, diplomatic, humanitarian, etc), separate from the influence of BBG broadcasting efforts that may affect the situation in a given country or region. Therefore, having a population choose to tune in to BBG broadcasts, or the program's "audience reach", is the most important and quantifiable measure of the program's success. In addition, whether or not the audience finds the broadcasts to be reliable and credible, and thereby regularly tunes in and hears the information and ideas BBG broadcasts, is another important measure of the external affect of BBG programs. The BBG currently has two long-term performance measures for this program and continues to explore additional or improved measures that address the quality and relevance of broadcasting. Audience reach, also called weekly audience, is an indicator based upon measurement of the regular listening/viewing audience. The Broadcasting to Africa FY 2009 performance plan ties this measure to annual language service activities, the BBG overall strategic goal, and the annual performance objectives. Regular audience is defined as all adults in a given population choosing to listen/view at least once a week, as determined by an audience survey that has an adequately designed sample. The long-term audience reach outcome measure is a culmination of a number of annual common measures used across all African language services (see question 2.3), as well as annual performance goals specific to each individual service. As detailed in the measures section, the overall BBG Africa program has been achieving its long-term goals of expanding audience reach through individual targeted programs in 13 languages broadcast to 48 countries in Africa. In addition to the long-term outcome measure of whether or not the target population decides to tune in to BBG broadcasts, BBG has developed a long-term outcome measure of whether or not the target population considers the broadcasts to be credible/reliable. Having a growing audience tune in to BBG programs and finding them to be credible/reliable sources of news and information means that BBG is achieving its mission and broadcasting its message, with the overarching result being the promotion of freedom and democracy and enhanced understanding of the U.S. and the world. The credibility score is determined by the survey question about "trustworthiness of news and information" of those sampled respondents who listened at least once a week. The answers are registered on a five-point scale -- very trustworthy, somewhat trustworthy, neither trustworthy nor untrustworthy, somewhat untrustworthy, or very untrustworthy. The credibility measure is the percent of those answering the question in the survey (excluding those who did not respond or did not know) that endorsed very or somewhat trustworthy.

Evidence: BBG Strategic Plan (2008-2013); Broadcasting to Africa FY 2009 Performance Plan; VOA Africa Division individual language service FY 2009 performance plans

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: BBG's audience size in Africa was approximately 31.5 million in 2003 and grew by nearly 10 million by 2007 (based on current estimates for 2007). Attracting significant new audiences to VOA's programming of objective news and information during the last several years was primarily due to an unprecedented growth in media choices in Africa, such as FM radio, television and the internet, that is expected to continue, especially for the internet and other new technology. The explosive growth in media availability was combined with audiences searching for information on emerging Muslim terrorism, genocide, health crises plus the lack of local press coverage to propel the audience reach to all-time highs. The opportunities to reach additional new audiences in Africa are now more limited, especially in areas of continued political and press restrictions or in areas of increasing competition in diverse media. Broadcasting to Africa will focus on deepening and sustaining its audience gains- for example, with interactive programming and improved websites- while targeting high priority areas, such as Nigeria, Somalia and Zimbabwe. A similar surge in audiences was experienced in Ukraine during the Orange Revolution, when affiliations with local FM stations were once again allowed and people were searching for news and information, but has since remained steady- in 2004 the audience reach was 7.6% and more than doubled by 2005 to 16.3% and then increased slightly in 2006 to 18%. Therefore, another 5% increase, or 2.1 million more weekly audience members in Africa, from 2007 to 2012, over the recent significant gains is ambitious. The other BBG long term measure is audience credibility, which is the percent of those answering the question in the survey related to the "trustworthiness of news and information" (excluding those who did not respond or did not know) that endorsed the programming as very or somewhat trustworthy. Broadcasting to Africa has already achieved the highest scores for credibility among all VOA broadcasting, from 93% in 2004 (when the measure was first introduced) to 90% in 2005 to 91% in 2006 and estimated to reach 92% in 2007. The BBG has set ambitious goals to sustain scores above 90% with significant incremental increases to 95% over the next five years. The related annual goals and action steps will track the progress toward achieving the long-term goals. For example, an improved program quality score may be a contributing factor to attracting and retaining a regular audience with fair and objective news and information. On a scale of 0 to 4, a program quality rating score of 3.4 or higher is considered "good or better". In 2003, the Horn of Africa language service received a 3.0 program quality score and has improved every year since then to reach a score of 3.5 in 2006. All language services in Africa achieved a score of 3.4 or higher in 2006 and have annual goals to maintain or exceed that score.

Evidence: The long-term goals for broadcasting to Africa is to increase the audience size by 5% and sustain the credibility score above 90% to reach 95% over 5 years from the 2007 baseline. BBG Strategic Plan (2008-2013); Broadcasting to Africa FY 2009 Performance Plan; VOA Africa Division individual language service FY 2009 performance plans

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: In addition to the long-term measures described in 2.1 and 2.2, there are common annual performance measures for each of the 8 language services in Africa: 1) program quality (data for the Nigeria and Horn of Africa language services included in the measures section); 2) percent of audience awareness (data for Nigeria and Zimbabwe language services included in the measures section); and 3) cost per audience member (data for Swahili and Zimbabwe language services included in the measures section). In addition, audience reach is tracked annually for each language service, and this data feeds directly into the long-term measure of audience reach in Africa as a whole. The common measure of cost per audience member is an efficiency measure that tracks the cost per audience member of each language service and is discussed further in 4.3. (1) The program quality score is an outcome measure based in part on how the target population perceives the quality of the specific BBG language services. The target population that rates the program's accuracy, reliability, authoritativeness, objectivity, comprehensiveness and other variables reflecting distinct statutory and policy objectives, provides the content evaluation and mission mandates for the different stations. Another component of the program quality score is the evaluation of IBB delivery systems (engineering and transmission), marketing and program placement efforts, and other broadcasting support functions. Programming content and presentation is also evaluated by both internal and external reviewers. In total, the Program Quality Score is a comprehensive, weighted score, which ranges from 1-4, with 4 being the highest, that indicates whether the program is effective in meeting the over-arching goals and mission of the BBG. (2) Audience awareness is an outcome measure that refers to the total of either prompted or unprompted acknowledgement of BBG station names in an audience survey that has an adequately designed sample. The statistic is expressed as a percent of the sampled population that knows or recognizes BBG stations, which can then translate to the percent of the entire population of the country or region of which the sample was representative. This outcome measure of the extent to which a target population recognizes BBG is influenced by internal output efforts taken by each language service, in coordination with the Office of Marketing and Program Placement, such as ensuring that programs are transmitted on preferred media, are placed with popular affiliates, and have sufficient advertising. To support the goals reflected in these annual performance measures, each individual language service or area has developed output-based performance objectives with annual milestones or measures tailored to the nature of the individual language services' broadcasting and target population. Improved content and availability of the programs will contribute to achieving the common annual goals as measured by program quality and audience awareness. The individual language service improvements to program quality and audience awareness directly impact the long-term outcome goals of increasing the overall audience in Africa that chooses to tune in to BBG programs and improving the audience's perception of the broadcasts as credible/reliable.

Evidence: BBG Strategic Plan (2008-2013); Broadcasting to Africa FY 2009 Performance Plan; VOA Africa division individual language service FY 2009 performance plans

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The annual targets of the measures common to all Africa language services are sufficiently ambitious given the large population of Africa, the political and press freedom restrictions in many countries, explosive growth in competitive broadcasters in diverse media and the closed nature of many of its societies. For example, all language services have set targets to maintain or increase their program quality score to "good or better'. Each language service has also developed tailored annual performance objectives that directly impact each service's progress toward achieving the targets set forth in the annual measures. For example, broadcasting to Swahili has a performance objective in their 2009 Annual Performance Plan to ensure dynamic programming to captivate and build a regular audience and to reach youth in target areas through cooperative efforts with other U.S. partners, such as USAID. If successful, this performance objective could directly impact each of the broader common goals reflected in the annual measures. It could lead to an increase in the population that is aware of Swahili programming, and could impact the audience members' perception of the quality of Swahili programming, thereby increasing the overall audience that chooses to tune into Swahili programming. As in the example, the targets are set based on the expected effects of the annual performance objectives combined with a trend analysis of the measure.

Evidence: The baseline for all of the annual measures is the FY 2003 level. Annual targets are set through FY 2009. See the Measures tab of this report for examples of this program's annual goals; Broadcasting to Africa FY 2009 Performance Plan (see tables of measures and targets); Language Service Review; Program Review (Source: Performance Review Scoring Procedures)

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The VOA Africa Division contributed to establishing the BBG Strategic Plan (2008-2013) and the development of the Broadcasting to Africa FY 2009 Performance Plan. In addition, the Broadcasting to Africa program participates with all other BBG entities, including support elements, in the Program and Language Service Reviews. Each African language service also develops an individual annual plan that includes annual performance goals that support the overall BBG long-term goals of the program. Senior managers within the program commit to and work toward the goals through the BBG performance based appraisal system, which is specifically tied to program objectives and goals. Each year, managers review and include the most current objectives and goals in their annual appraisals. BBG Contract Officers monitor compliance with standard performance clauses that impact and support the goals of the program, including schedule, cost and technical requirements, as they manage contracts for the program. Another example is contract stringers who submit materials that are thoroughly reviewed by an editor for quality and adherence to VOA journalistic standards, and they receive payment only if the report is aired.

Evidence: Language Service Review; Program Review; BBG Strategic Plan (2008-2013); Broadcasting to Africa FY 2009 Performance Plan; VOA Africa Division individual language service FY 2009 performance plans

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The BBG Broadcasting to Africa program and each individual language service within the program undergo annual comprehensive evaluations that meet the scope and quality required. The International Broadcasting Bureau's Office of Performance Review, an independent program evaluation office within BBG, conducts the annual Program Reviews and the BBG conducts the annual Language Service Review. The Office of Performance Review operates independently from all BBG language services. Annual Program Reviews include a rigorous assessment of each aspect of the program, including content, IBB delivery systems (engineering and transmission), marketing and program placement efforts, Internet development, and production content and presentation. Of each language service and program, including the overall impact of the program , with regularly scheduled follow-up meetings to discuss areas of improvement. Within ten days of the review, the VOA Director or program director, the Division Director, the Service Chief, and appropriate Performance Review personnel reconvene to consider the results of the review, and to determine the issues that need to be addressed in an Action Plan. The Action Plan includes items for all IBB and VOA elements, not just the broadcast service; support elements are also tasked with action items. Three months after the Program review, the group that assembled the Action Plan convenes again to check the progress on recommendations generated at the Action Plan meeting. In addition to the comprehensive annual performance review, BBG's annual Language Service Review looks at BBG's overall broadcasting portfolio to assess two basic issues: (1) where should BBG broadcast and (2) how well is BBG broadcasting. Through Language Service Review the importance of reaching each of the target audiences is evaluated and the highest priority audiences among all of those targeted are identified. Resource allocations or reallocations to reach specific target audiences are made based on the results of this evaluation. In the case of broadcasting to Africa, the language services are included in the evaluation of BBG priorities throughout the world. Within each African language service, performance measures are reviewed and effectiveness assessed, in concert with all other BBG broadcasting to Africa. In addition to the comprehensive evaluations described above, BBG also regularly receives more targeted independent reviews focused on the impact of the language services, as well as broader agency-wide evaluations focused on financial and other management issues. BBG contracts with InterMedia, an independent firm that conducts annual surveys for each language service and entity, evaluating a variety of factors: market profile, media access and use, and international broadcasting impact and trends. An independent annual audit by an external auditing firm has been conducted for the BBG since 2004, including the management of funds and evaluation of BBG performance, as reported in the Performance and Accountability Report (PAR). The BBG has received clean audit opinions for 2004, 2005 and 2006. Government review: Between September 2000 and February 2004 the GAO conducted three evaluations of the BBG. While none of the reports focused specifically on broadcasting to Africa, the findings, recommendations and resulting actions by the BBG impacted all broadcast services, including those targeting Africa. Follow-up testimony was provided in 2004.

Evidence: Language Service Review; Program Review (Source: Performance Review Scoring Procedures and Analysis); InterMedia Research Presentations and research documents for VOA language services; PAR; GAO Report: "Strategic Planning and Performance Management System Could be Improved" (GAO/NSIAD-00-222); GAO Report: "New Strategic Approach Focuses on Reaching Large Audiences but Lacks Measurable Program Objectives" (GAO-03-772); GAO Report: "Enhanced Measure of Local Media Conditions Would Facilitate Decisions to Terminate Language Services" (GAO-04-374); GAO Testimony: "Challenges Facing the Broadcasting Board of Governors" (GAO-04-711T)

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: BBG has excelled at clarifying its mission, establishing implementation strategies and performance objectives that support the mission, as well as establishing performance indicators. Actions taken in developing the plans for 2006, 2007 and 2008 have enhanced the linkages between these strategic tools and increasingly show how they are supported and implemented by BBG's budget each year. In its Budget requests the BBG explicitly linked all costs to six annual performance objectives that support BBG ' s mission, and will do so again in its FY 2009 budget request. The performance objectives include direct and indirect costs as appropriate. For example, the performance objective to " Target African broadcasting to areas prone to terror incidents, genocide or failed states" include all programming expenses and expenses for program delivery leases, power, and parts. In addition, each objective will link annual performance goals to the requested resources with performance measures that are monitored against the targets set for FY 2009. In FY 2009, the BBG will show how the performance objectives for the various broadcasting entities and support elements support the overall BBG strategic goal and annual performance goals. In FY 2009, the BBG will again include the agency's annual performance plan and performance measures in its budget requests.

Evidence: FY 2006, 2007 and FY 2008 Congressional Budget Requests; Language Service Review; Program Review; PAR

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The BBG developed a new strategic plan for 2008-2013. The plan reflects the experiences gained over the past several years and is a forward looking roadmap to achieve the goals of U.S. public diplomacy in reaching significant audiences in support of U.S. strategic interests. The Board increased involvement in the planning process (based on lessons learned regarding implementation of the prior plan) by enlisting all BBG entities in defining internal and external challenges, identifying organizational strengths and weaknesses, and drafting updates to strategies contained in the earlier plan. The new plan also considered other sources, such as global market and audience research findings, feedback from the Department of State, U.S. national security strategy documents and think tank studies, in identifying challenges and deriving the implementation strategies to address them. BBG also developed a performance plan for Broadcasting to Africa for FY 2009. The performance plan is guided by the VOA Africa Strategy (approved by the Board in May 2006), which is based on extensive research and BBG strategic priorities, and is linked to the BBG mission. Each language service also developed their performance plan based on the guidance of the Africa Strategy and with the performance goals specifically linked to the BBG implementation strategies. The annual performance goals link to the long term goal of increasing the African audience by specific objectives, such as increasing the number of affiliates, providing more interactive programming, continuing cooperative efforts with other government agencies or enhancing web sites.

Evidence: BBG Strategic Plan (2008-2013); Africa Strategy that links the BBG strategic priorities to target audiences by country, as described in the Broadcasting to Africa Performance Plan; VOA Africa Division individual language service FY 2009 performance plans; research and results from Language Service Review and Program Review processes

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The BBG analyzes the performance data gathered through its annual Language Service Reviews and Program Reviews, including polling data gathered from InterMedia (from statistically valid surveys and focus groups) and uses the results to adjust resources and set strategic priorities. VOA, Radio and TV Marti, and each of the BBG grantees (RFA, RFE/RL, MBN) undergo annual Language Service and Program Reviews - with the VOA Africa services specifically relevant to this PART. Program Reviews also include content and production analysis that assist managers in improving their specific programming in such areas as music bridges, lively pace, interactive call-in shows, or younger voices to attract a larger audience. Annual Program Plans include long-term and annual goals for each Language Service, as well as relevant measures to track these goals. For the Broadcasting to Africa Program, the baseline is 2003 actuals. The BBG also conducts follow-up meetings to ensure that programs have successfully made adjustments and changes that were recommended during the Language Service Reviews and Program Reviews. Recommendations may include management or performance issues and improvements. The Board uses this information to adjust resources and set strategic priorities.

Evidence: Language Service Review; Program Review

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Senior managers are under a performance based appraisal system that requires critical elements and ratings that are tied to the objectives of the BBG with measurable goals and targets. Through the ongoing appraisal reviews and final annual rating, managers are held accountable for the performance results. Senior Executive Service (SES) employee evaluations are also tied to the performance results reported in the PAR. The annual Language Service Review holds managers and partners directly accountable for results with the same common measures of audience reach, audience awareness, program quality and cost per audience member considered among other factors. The entities develop an annual financial plan that is reviewed in conjunction with the CFO's mid-year, 3rd quarter and year-end reviews of budget. The reviews hold managers accountable for their annual resource level against expected expenditures, schedules, including enhancements or major changes, and accomplishments. Capital projects are also monitored monthly for schedule and budget status. In addition, entities submit a monthly report to the Board on the status of resources, performance and key program initiatives. Contractors are accountable to the same program standards and the Contracting Officer monitors the quality of the deliverables. Many contractors work on a "pay-for-product" basis, with payment dependent upon the timely receipt of quality products. For example, many "stringers" receive payment for only the quality news stories they deliver, after review by a VOA editor for adherence to VOA standards.

Evidence: Performance based Employee Standards/Appraisals; SES performance based appraisal process; Language Service Review; CFO's mid-year, 3rd quarter and year-end Reviews; monthly BBG status reports; Capital Projects process; Program Review.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: All requisitions must include verification that funds are available, and the funds are obligated when a contract or agreement is signed or a purchase order is done. The mid-year, 3rd quarter and year-end review provides an opportunity for managers to verify that unliquidated obligations are valid and that once an obligation has been fully expended, remaining balances are deobligated. Financial transactions are coded to both an organization and to a function or project. Given that BBG has a high percentage of fixed costs (salaries, transmission and infrastructure expenses, rent, etc.), funds must be obligated in a timely manner in order to continue operations. The BBG requires an annual financial plan and monthly status reports from the IBB and the grantees. The financial plans must detail anticipated costs for specific line items by each month of the fiscal year. BBG conducts a mid-year, 3rd quarter, and year-end review of each entity's financial plans and budgets to monitor funding and expenditures. BBG also requires a monthly status report of expenditures, disbursements and obligations, as well of the submission of the SF-169 Financial Status Report, and SF-172 Report of Federal Cash Transactions from the grantees. The reviews and the allotment control procedures enable the Agency to track obligations to ensure that funds are used for the intended purpose by the appropriate Agency office.

Evidence: SF-169 Financial Status Report; SF-172 Report of Federal Cash Transactions; Monthly status reports on the IBB, RFA and RFE/RL on expenditures, disbursements, and obligations; Quarterly review reports

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The annual Language Service Review process includes reviewing each program's progress on an efficiency measure: cost per audience member. The Program Review process also assesses the efficiency and productivity of each language service in support of this efficiency measure. Cost per audience member is the total cost of programming and delivery divided by the unduplicated audience for all media, and is calculated for each language service. The ratio (measure) must be evaluated in context of the changes in either the cost or audience size. For example, a short-term capital cost increase to improve transmission may not increase audience size until after a lengthy installation. In some cases, audience size may be kept artificially low by intensive jamming by the host government, not necessarily poor programming. In addition to the efficiency measure described above and included in the PART measures section, BBG tracks efficiencies in a number of different ways. The CFO's reviews of budget and performance results hold managers accountable for their annual resource level against expected accomplishments. Enhancements to programs are tracked separately with specific performance targets based on the increased resources. The program also achieves efficiencies by using incentives in its contracts, for example using competitive sourcing to assure cost effectiveness. Contracts may include "negative incentives" in accordance with the Federal Acquisitions Regulations to discourage changes from the time or performance indicated in the scope of contract.

Evidence: Broadcasting to Africa FY 2009 Performance Plan; VOA Africa Division individual language service FY 2009 performance plans; Language Service Review process; Program Review process; Federal Acquisitions Regulations 52.211-11, 52.211-12, 11.503

Does the program collaborate and coordinate effectively with related programs?

Explanation: In terms of broadcasting, BBG is solely responsible for all government and government sponsored non-military, U.S. international broadcasting. It is the only entity with the express objective of promoting freedom and democracy through the broadcasting of objective news and information about the U.S. and the world to audiences in Africa. Although there are no related programs that share similar broadcasting goals and objectives, the BBG does collaborate with other entities, including the Department of State and USAID, for funding to provide programming on issues of mutual concern in Africa, such as HIV prevention, polio, malaria and other health issues. For example, the Department of State is funding a 30-minute daily VOA radio program to reach millions of Somali speakers in Somalia, Djibouti and the greater Horn of Africa. A new VOA program, Evening Edition launched in Feb. 2006, is targeting Muslim youth with accurate representation of American values and policies to the region. Another successful coordinated effort is Studio 7 broadcasts to Zimbabwe, a partnership with USAID that promotes democracy and good governance by providing accurate, credible and effective news and information to the country's citizens.

Evidence: U.S. International Broadcasting Act of 1994; Foreign Affairs Reform and Restructuring Act of 1998; U.S. International Broadcasting Act of 1994; Voice of America Charter; USAID and State Dept. funded programming; Broadcasting to Africa Performance Plan 2009, Language Service Performance Plans 2009

Does the program use strong financial management practices?

Explanation: The BBG is cross-serviced by the State Department to provide financial services using the Broadcasting Financial Management System (BFMS) system. Internal controls are in place to provide for the separation of duties, minimize erroneous payments, ensure recording of receipts, and the accurate reporting to external entities. The BBG Administrative Officers obligate financial transactions accurately and reconcile their allotments on a monthly basis. In the CFO Office, the Financial Operations, records and makes disbursements, reconciles the Fund Balances with Treasury, records and makes deposits, and prepares the monthly, quarterly and annual statement reports for the agency. FY 2004 was the first year the BBG prepared the Performance and Accountability Report (PAR) and conducted a financial statement audit. The PAR was submitted on time and the BBG received an unqualified opinion on its Balance Sheet. Again in 2005 and 2006, the BBG submitted an expanded PAR and received an unqualified opinion.

Evidence: BFMS System; PAR: An independent audit conducted in October 2004, October 2005 and October 2006 by the OIG offered an unqualified opinion of the BBG financial statements; The State Department has also received clean audit opinions

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Annually, the Office of Performance Review conducts an independent analysis of program content and presentation, which includes the management of the program. The Action Plans resulting from the Program Reviews lay out specific and meaningful steps to correct deficiencies when they are identified. The Office of Performance Review regularly follows up with managers to track progress. The senior managers' performance- based appraisal system requires critical elements and ratings based on the mission, and are tied to the objectives of the BBG. Specific goals with measurable targets are also set for each element within the appraisal. SES performance evaluations are linked to the BBG's results as reported in the PAR. Between September 2000 and February 2004 the GAO conducted three evaluations of the BBG. While none of the reports focused specifically on BBG Broadcasting to Africa, the findings, recommendations and resulting actions by the BBG impacted the entire agency, including this program. Follow-up testimony provided in 2004 shows that the Board has taken meaningful steps to address its management deficiencies. No other current evaluations regarding BBG management have been completed.

Evidence: Program Reviews; BBG Performance Based Appraisal System; GAO Report: "Strategic Planning and Performance Management System Could be Improved" (GAO/NSIAD-00-222); GAO Report: "New Strategic Approach Focuses on Reaching Large Audiences but Lacks Measurable Program Objectives" (GAO-03-772); GAO Testimony: "Challenges Facing the Broadcasting Board of Governors" (GAO-04-711T)

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The number of people choosing to listen to or view BBG programming in Africa, or the weekly audience rates, in recent years show a trend toward increasing audience in high priority areas, maintaining audience rates in many countries and showing only slight declines in a few countries. Total unduplicated audience for all media in Africa shows an overall upward trend - rising from 31.5 million in 2003 to 37.8 million in 2004, then to 38.5 million in 2005 and again rising to 42 million in 2006. Significant weekly audience percentage increases were recorded in the Central Africa Service (52.0% in 2004 to 77.2% in 2005); in the Horn of Africa (5.6% in 2005 to 18.4% in 2006); and in the Zimbabwe Studio 7 Service (3.2% in 2005 to 7.3% in 2006). Since 2003, the audience reach for VOA in Africa has increased over 10 million and is on track to increase another 5% in the next five years. At the same time audience reach expanded significantly, BBG maintained a credibility score of over 90% - demonstrating progress in achieving its long term outcome goal of broadcasting fair and objective news and information, with the overarching result being the promotion of freedom and democracy and enhanced understanding of the U.S. and the world. The credibility score is determined by the survey question about "trustworthiness of news and information" of those sampled respondents who listened at least once a week. The answers are registered on a five-point scale -- very trustworthy, somewhat trustworthy, neither trustworthy nor untrustworthy, somewhat untrustworthy, or very untrustworthy. The credibility measure is the percent of those answering the question in the survey (excluding those who did not respond or did not know) that endorsed very or somewhat trustworthy. Despite the influx in new audience members who need to build trust in BBG programming, increasing anti-Americanism, and polarizing crisis situations in many areas of Africa, holding the scores at such high levels with only slight changes from 93% in 2004 (when the measure was first introduced) to 90% in 2005 to 91% in 2006 and estimated to reach 92% in 2007 further demonstrates the BBG commitment to objective journalism that provides audiences with reliable news and information for making informed decisions. Over the next five years, Africa broadcasting will sustain a credibility level above 90% and incrementally reach a significant goal of 95%.

Evidence: U.S. International Broadcasting to Africa FY 2009 Performance Plan

Does the program (including program partners) achieve its annual performance goals?

Explanation: In addition to the long-term weekly audience and credibility measures, there are three annual performance measures for each language service in Africa: 1) percent of audience awareness, 2) program quality, and 3) cost per audience member (addressed in question 4.3). Broadcasting to Africa has made progress toward achieving its performance goals using these measures. Data available from FY 2003 to FY 2006 for these measures show increases in key regions, maintaining rates in many countries and slight reductions in few countries. For example, audience awareness increased in several broadcast areas and remained steadily high in others, such as Central Africa at 98%. By 2006, all language services reached a range of "good" or better scores (at least 3.4 out of 4.0) for program quality and will continue to maintain or improve their scores.

Evidence: U.S. International Broadcasting to Africa FY 2007 Performance Plan

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The BBG has a common measure for all language services, cost per audience member, which tracks the cost effectiveness of broadcasting. This is the ratio of the total cost of programming and delivery divided by the unduplicated audience for all media, and is calculated for each language service. The cost per audience member in sub-Saharan Africa continues to be one of the lowest at VOA. The program's efficiencies and increased weekly audience clearly demonstrate improved cost effectiveness. For example, as detailed in the measures section, in Zimbabwe, cost per audience member went from $4.90 in 2005 to $0.70 in 2006 while the percentage of weekly audience more than doubled from 3.2% to 7.3% in the same period. Similarly, as detailed in the measures section, cost per audience member of the Swahili language service dropped from $0.27 in 2003 to $0.11 in 2006. Further examples not included in the measures section are: the Horn of Africa cost per audience member reduced from $0.68 to $0.64 and the percentage of weekly audience more than tripled from 5.6% to 18.4%; and the Hausa Service cost per audience reduced to $0.06 from $0.11 while the percentage of weekly audience for the peak range increased from 14.6% in 2005 to 26.5% in 2006. Other efficiencies developed by individual Africa language services, but not tracked in the measures section, include the Kano and Abuja Reporting Centers in Nigeria, supported financially by USAID and staffed with contractors signed up by VOA's Hausa Service, which provide field support, production facilities, and satellite feed capacity to other Africa Division services, primarily English. The Reporting Centers also provide stringer reports on major stories that are translated into English and distributed for adaptation and broadcast by other African language services, at no increased cost.

Evidence: Broadcasting to Africa FY 2009 Performance Plan; VOA Africa Division individual language service FY 2009 performance plans; Program Review

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: BBG is solely responsible for all U.S. government and government sponsored, non-military international broadcasting and is an important part of U.S. public diplomacy efforts overseas. No comparable programs exist. BBG promotes freedom and democracy overseas and the VOA charter specifically calls for VOA to represent America and present the policies of the United States. Although there are other international broadcasters in Africa, such as BBC, CNN, Deutsche Welle and Radio France Internationale (RFI), they do not have immediate and guaranteed access to accurate information about U.S. interests and policies and do not share in VOA's mission to provide information about the U.S. Only VOA provides approved editorials on U.S. policy. Other international broadcasters also do not share in BBG's mission of promoting freedom and democracy and enhancing understanding by broadcasting accurate and objective news and information about the U.S. and the world to audiences overseas. BBG is the only international broadcasting entity that serves as a critical arm of U.S. public diplomacy missions overseas.

Evidence: U.S. International Broadcasting Act of 1994; Foreign Affairs Reform and Restructuring Act of 1998; Voice of America Charter

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The International Broadcasting Bureau's independent Office of Performance Review conducts focused comprehensive evaluations of programs, as requested, that include independent, external (non-BBG) expert input to ensure effective compliance with the VOA Charter and success at meeting goals specific to the program, as well as evaluations conducted by experienced BBG broadcasters, not from the Africa Division. These Program Reviews assess each aspect of the program, including content, IBB delivery systems (engineering and transmission), marketing and program placement efforts, Internet development, and production content and presentation. The annual Program Reviews have shown improvements in the Program Quality scores, a weighted score that includes factors measuring all aspects of program quality, to indicate that the program is effective in meeting the over-arching goals and mission of the BBG. In addition to the comprehensive, independent Program Reviews described above, the BBG annual Language Service Review evaluates a wide range of performance measures against strategic priorities and indicates that the Africa Division is an effective program and is achieving results, especially in areas with large Muslim populations or lacking access to accurate news and information. Audience research conducted by InterMedia, an independent research organization, indicates that broadcasting services in sub-Saharan Africa reach target audiences with appropriate and effective material.

Evidence: InterMedia research including surveys, monitoring panels, and listener panels; Program Review schedules, results, action plans and independent evaluations; Language Service Review analysis and results