Detailed Information on the
Programming Support for US International Broadcasting Assessment

The long-term goal for Programming Support is to effectively provide support to BBG's strategic programming initiatives by providing integrated multi-media content and sufficient, state-of-the- art technical infrastructure. The expansion to new media requires state-of the art technical infrastructure and the integration of content to be used by more than one media while still customizing content that draws audiences via their preferred media. Meeting this outcome goal will be tracked by the growth in BBG audience size, as measured by the worldwide, unduplicated number of weekly audience members. The long-term goal is to increase the worldwide reach to 180 million by 2013 from 140 million in 2005.

The Programming Support office at BBG is tracking the overall credibility of each broadcasting entity with five separate annual measures focused on the five distinct broadcasting entities: Voice of America, Radio Free Europe/Radio Liberty, Radio Free Asia, Radio Sawa, and Alhurra Television. Annual surveys focused on each entity will be conducted amongst target populations. These surveys will ask participants about the trustworthiness of the news and information presented by the broadcasting entity in question. The Programming Support office's success with writing and editing news stories, which are the audience members' basis of judgment, will have a direct impact on the perceived credibility of each broadcasting entity.

Revising the current agency strategic plan to build on a market-based approach to broadcasting.

Measure: Overall audience size for U.S. international broadcasting (measured in millions of listeners/viewers) Note: Measure reflects program's effectiveness in supporting strategic programming initiatives by providing integrated multi-media content and sufficient, state-of-the- art technical infrastructure, which contributes to the Agency's success in reaching the target audiences, as measured by the overall audience size for BBG broadcasting.

Explanation:A key indicator of whether or not BBG broadcasts are reaching their intended audience is the number of people who choose to listen to or view BBG programs. Audience reach, also called weekly audience, is a measure of the regular listening/viewing audience. Regular audience is defined as all adults in a given population listening/viewing at least once a week, as determined by an audience survey that has an adequately designed sample. The effectiveness of Programming Support functions such as centralized information collection and technical and operational support directly impact the quality of BBG programming, and thereby support growth in the overall audience size.

Measure: The contribution of Programming Support to the trustworthiness of news and information can be measured with the credibility score (by entity), as a percentage of the total viewing/listening audience who rate the broadcasts as "credible/trustworthy" or "very credible/trustworthy." The measure is provided for the Voice of America (VOA). Note: As the percentage moves towards 100%, increases are more difficult to attain, thus targets are significantly ambitious when the measure is maintained or shows a marginal increase.

Explanation:The functions of the Programming Support program are to collect information and news from a variety of sources, write news and feature stories and disseminate to broadcasters as well as provide centralized technical and operational programming support, such as technical infrastructure, production services, centralized music, video editing, audio feeds plus archiving and research capabilities. An important outcome measure of the external effect of BBG programs is whether or not the audience finds the broadcasts to be reliable and credible, and thereby regularly tunes in and hears the news and information BBG broadcasts. This measure is an industry research standard used to determine the audience's view of the effectiveness of media. This indicator is determined by the survey question about "trustworthiness of news and information" of those sampled respondents who listened at least once a week to each station. The answers are registered on a five-point scale -- very trustworthy, somewhat trustworthy, neither trustworthy nor untrustworthy, somewhat untrustworthy, or very untrustworthy. The credibility index is the percent of those answering the question in the survey (excluding those who did not respond or did not know) that endorsed very or somewhat trustworthy.

Measure: The contribution of Programming Support to the trustworthiness of news and information can be measured with the credibility score (by entity), as a percentage of the total viewing/listening audience who rate the broadcasts as "credible/trustworthy" or "very credible/trustworthy." The measure is provided for Radio Free Europe/Radio Liberty (RFE/RL). Note: As the percentage moves towards 100%, increases are more difficult to attain, thus targets are significantly ambitious when the measure is maintained or shows a marginal increase.

Explanation:The functions of the Programming Support program are to collect information and news from a variety of sources, write news and feature stories and disseminate to broadcasters as well as provide centralized technical and operational programming support, such as technical infrastructure, production services, centralized music, video editing, audio feeds plus archiving and research capabilities. An important outcome measure of the external effect of BBG programs is whether or not the audience finds the broadcasts to be reliable and credible, and thereby regularly tunes in and hears the news and information BBG broadcasts. This measure is an industry research standard used to determine the audience's view of the effectiveness of media. This indicator is determined by the survey question about "trustworthiness of news and information" of those sampled respondents who listened at least once a week to each station. The answers are registered on a five-point scale -- very trustworthy, somewhat trustworthy, neither trustworthy nor untrustworthy, somewhat untrustworthy, or very untrustworthy. The credibility index is the percent of those answering the question in the survey (excluding those who did not respond or did not know) that endorsed very or somewhat trustworthy.

Measure: The contribution of Programming Support to the trustworthiness of news and information can be measured with the credibility score (by entity), as a percentage of the total viewing/listening audience who rate the broadcasts as "credible/trustworthy" or "very credible/trustworthy." The measure is provided for Radio Free Asia (RFA). Note: As the percentage moves towards 100%, increases are more difficult to attain, thus targets are significantly ambitious when the measure is maintained or shows a marginal increase.

Explanation:The functions of the Programming Support program are to collect information and news from a variety of sources, write news and feature stories and disseminate to broadcasters as well as provide centralized technical and operational programming support, such as technical infrastructure, production services, centralized music, video editing, audio feeds plus archiving and research capabilities. An important outcome measure of the external effect of BBG programs is whether or not the audience finds the broadcasts to be reliable and credible, and thereby regularly tunes in and hears the news and information BBG broadcasts. This measure is an industry research standard used to determine the audience's view of the effectiveness of media. This indicator is determined by the survey question about "trustworthiness of news and information" of those sampled respondents who listened at least once a week to each station. The answers are registered on a five-point scale -- very trustworthy, somewhat trustworthy, neither trustworthy nor untrustworthy, somewhat untrustworthy, or very untrustworthy. The credibility index is the percent of those answering the question in the survey (excluding those who did not respond or did not know) that endorsed very or somewhat trustworthy.

Measure: The contribution of Programming Support to the trustworthiness of news and information can be measured with the credibility score (by entity), as a percentage of the total viewing/listening audience who rate the broadcasts as "credible/trustworthy" or "very credible/trustworthy." The measure is provided for Radio Sawa. Note: As the percentage moves towards 100%, increases are more difficult to attain, thus targets are significantly ambitious when the measure is maintained or shows a marginal increase.

Explanation:The functions of the Programming Support program are to collect information and news from a variety of sources, write news and feature stories and disseminate to broadcasters as well as provide centralized technical and operational programming support, such as technical infrastructure, production services, centralized music, video editing, audio feeds plus archiving and research capabilities. An important outcome measure of the external effect of BBG programs is whether or not the audience finds the broadcasts to be reliable and credible, and thereby regularly tunes in and hears the news and information BBG broadcasts. This measure is an industry research standard used to determine the audience's view of the effectiveness of media. This indicator is determined by the survey question about "trustworthiness of news and information" of those sampled respondents who listened at least once a week to each station. The answers are registered on a five-point scale -- very trustworthy, somewhat trustworthy, neither trustworthy nor untrustworthy, somewhat untrustworthy, or very untrustworthy. The credibility index is the percent of those answering the question in the survey (excluding those who did not respond or did not know) that endorsed very or somewhat trustworthy.

Measure: The contribution of Programming Support to the trustworthiness of news and information can be measured with the credibility score (by entity), as a percentage of the total viewing/listening audience who rate the broadcasts as "credible/trustworthy" or "very credible/trustworthy." The measure is provided for Alhurra Television. Note: As the percentage moves toward 100%, increases are more difficult to attain, thus targets are significantly ambitious when the measure is maintained or shows a marginal increase. Note: In 2008, there was a slight decline from 2007, but Alhurra's credibility remains at a high level for the entire Arabic speaking region.

Explanation:The functions of the Programming Support program are to collect information and news from a variety of sources, write news and feature stories and disseminate to broadcasters as well as provide centralized technical and operational programming support, such as technical infrastructure, production services, centralized music, video editing, audio feeds plus archiving and research capabilities. An important outcome measure of the external effect of BBG programs is whether or not the audience finds the broadcasts to be reliable and credible, and thereby regularly tunes in and hears the news and information BBG broadcasts. This measure is an industry research standard used to determine the audience's view of the effectiveness of media. This indicator is determined by the survey question about "trustworthiness of news and information" of those sampled respondents who listened at least once a week to each station. The answers are registered on a five-point scale -- very trustworthy, somewhat trustworthy, neither trustworthy nor untrustworthy, somewhat untrustworthy, or very untrustworthy. The credibility index is the percent of those answering the question in the survey (excluding those who did not respond or did not know) that endorsed very or somewhat trustworthy.

Measure: Program Cost per Audience Member (total program cost divided by total number of listeners/viewers) Note: The transition to multi-media broadcasting and content integration required initial investments starting in 2005, but the result will be to reach larger audiences via the preferred media and increasing efficiency over the long term.

Explanation:This efficiency measure is the total annual cost for Programming Support, including central news, news bureaus and the operational support essential to broadcasting production, divided by the unduplicated BBG audience for all media. The ratio (measure) must be evaluated in context of the changes in either the total cost or audience size. For example, a short-term cost increase to facilitate multi-media broadcasting that audiences prefer may not increase audience size for several years during the transition period. In some BBG strategic areas, audience size may be kept artificially low by restrictions imposed by the host government or increased competition, not necessarily inefficient management.

Is the program purpose clear?

Explanation: The purpose of this program is to provide programming support to the Broadcasting Board of Governors (BBG) broadcasting entities in their efforts to promote and sustain freedom and democracy by broadcasting accurate and objective news and information about the United States and the world to audiences overseas. The are two functions of this program: 1) to collect information and news from a variety of sources, write news and feature stories and disseminate to broadcasters, and 2) to provide centralized technical and operational programming support, such as technical infrastructure, production services, centralized music, video editing, audio feeds plus archiving and research capabilities. These two functions are carried out in the newsrooms, production, technical infrastructure and programming support offices of the Voice of America (VOA), Office of Cuba Broadcasting (OCB), Radio Free Asia (RFA), and Radio Free Europe/Radio Liberty (RFE/RL) and Middle East Broadcasting Networks (MBN). Programming support serves as a hub to provide radio, television and web content and infrastructure services on a 24/7 basis to audiences nearly any place in the world.

Evidence: ?? BBG Strategic Plan ?? U.S. International Broadcasting Act of 1994 ?? Foreign Affairs Reform and Restructuring Act of 1998

Does the program address a specific and existing problem, interest, or need?

Explanation: In order to carry out its mission of broadcasting accurate and objective news and information about the United States and the world to audiences overseas the BBG broadcasting entities must have up to the minute, reliable and verified news content provided by the central news organizations and strategically located bureaus. The production and broadcasting infrastructure provides the studios, editing, directing and other operations support to enhance the appeal of BBG programming worldwide.

Evidence: ?? BBG Strategic Plan (2002-2007), updated BBG Strategic Plan (2008-2013) ?? U.S. International Broadcasting Act of 1994; ?? Foreign Affairs Reform and Restructuring Act of 1998

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Within the agency, the Voice of America (VOA), Office of Cuba Broadcasting (OCB), Radio Free Asia (RFA), and Radio Free Europe/Radio Liberty (RFE/RL) and Middle East Broadcasting Networks (MBN) each have their own news and information collection functions and their own technical support groups. Duplication exists in that each organization has their own staff performing both functions, but there are several reasons why duplication is acceptable and areas where duplication has been minimized. Regarding news and information collection, duplication has been minimized through a central contract for outside news from the Associated Press and other news providers. Regarding field reporting, while each organization has their own stringers, correspondents, and overseas bureaus (in different quantities), in certain locations these resources are shared. There is no evidence however that all field reporting functions at all the grantee organizations have been or are continually and systematically reviewed for potential duplication and redundancies, and the PART recommends that the BBG perform this analysis and implement any restructurings or reductions that would result in cost savings, while not compromising performance significantly. With regard to the technical infrastructure and programming support work, since these services are provided during the actual broadcasting production, and this production occurs around the world, there is no way to consolidate functions across the broadcasting organizations and duplication is inevitable among broadcasting organizations. There is no comparable program for either function within the federal, state, or local government. With regard to duplication of private efforts, the news and information gathering function and the technical infrastructure and production support do duplicate private efforts. The majority of the work performed under these functions is commercial in nature. The BBG has sought to minimize the duplication of the news and information gathering function with the private sector by making use of private contractors to provide news content, including both news organizations and individual stringers. The BBG has retained responsibility for the editing and news writing, which has been deemed "inherently governmental" and unsuitable for private sector performance. With regard to the technical infrastructure and production support, this program employs outside contractors and the function is generically duplicated in the private sector, but there is no evidence that another private sector organization could perform this entire function "off the shelf." This program is responsible for producing broadcasting content that reaches a wide range of overseas audiences in several languages and focuses on a particular target audience at the same time. However, the BBG does not have a methodology for comparing the cost or performance of this function with the private sector. This PART recommends that the BBG research and develop a measure that would benchmark this function to the private sector and/or consider performing a streamlined or standard competition for this function or sub-functions within this function in order to determine the optimum level of efficiency and whether their costs are competitive with the private sector.

Evidence: ?? The U.S. International Broadcasting Act of 1994 and the Foreign Affairs Reform and Restructuring Act of 1998 state that U.S. international broadcasting should not duplicate the activities of private United States broadcasters or the activities of government-supported broadcasting entities of other democratic nations. ?? U.S. International Broadcasting Act of 1994; ?? Foreign Affairs Reform and Restructuring Act of 1998 ?? State Dept. MOU to acquire material ?? VOA Charter, RFA Charter, RFE/RL Charter

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence that a different approach or mechanism would be more effective or efficient at meeting the programs' goals. The Voice of America, Office of Cuba Broadcasting, Radio Free Asia, Radio Free Europe/Radio Liberty and MBN have centralized their respective newsrooms and provide oversight of the news bureaus and correspondents to coordinate coverage and editorial control. While it is possible that further centralization across organizations could improve the efficiency of news gathering, and the BBG is encouraged to look at potential efficiencies (as explained in question 1.3) there is no evidence that the current program design is limiting the program's effectiveness or efficiency. The production, broadcasting infrastructure and programming support are configured to maximize resources for 24/7 operations to cover all time zones around the world.

Evidence: ?? U.S. International Broadcasting Act of 1994; ?? Foreign Affairs Reform and Restructuring Act of 1998

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Programming Support reaches two groups of intended beneficiaries. The first is the assortment of internal broadcasting organizations; the second is composed of worldwide listeners. BBG beneficiaries: Through the centralized newsrooms and operations, broadcasters can be assured of reliable and verified news and information. Overseas news bureaus, correspondents and stringers provide up to the minute local and regional coverage, in some cases up to 24 hours per day. Centralized libraries and other reference resources are maintained for use by all broadcasters and individual language services can request special reports or feature articles. Broadcasters depend on the centralized newsrooms to provide the highest quality news coverage of major world events, which directly relates to their ability to attract audiences (explained below). This program thereby enables other organizations within the agency to meet their goals. The newsrooms are available 24/7 and are staffed with journalists, analysts, and editors knowledgeable in many geographical regions of the world. Production and programming support, including professional directing, editing, graphic services and remote shoot capability are available to broadcasters, in some cases up to 24 hours per day. In many areas of the world, the level of production provided to BBG's broadcasting distinguishes it from other media available. Features such as editing, on-scene shooting, and popular music often provide significant audience appeal, which enables broadcasters to meet their goals of growing audience size. Listeners: Programming support, including the newsrooms, works closely with the language services to provide the content and format that the audience deems credible and reliable while producing programming that attracts the targeted audience. They participate in each of the language service's annual Program Review to receive research information and to help inform decisions about programming content to carryout the broadcasting mission. They receive feedback about audience likes and dislikes, which impacts future planning.

Evidence: ?? U.S. International Broadcasting Act of 1994 ?? Foreign Affairs Reform and Restructuring Act of 1998 ?? Capital and Information Technology Planning Process ?? Program Review ?? Language Service Review ?? Action plans/correspondence from Program Reviews demonstrating Programming Support responsiveness to issues raised by BBG language services.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: In accordance with the mission, vision, and strategic goals of the agency, BBG Programming Support supports BBG broadcasting through centralized news and information functions and technical infrastructure. The long-term goal for Programming Support is to effectively provide support to BBG's strategic programming initiatives by providing integrated multi-media content and sufficient, state-of-the- art technical infrastructure. The expansion to new media requires state-of the art technical infrastructure and integration of content to be used by more than one media while still customizing content that draws audiences via their preferred media. Meeting this outcome goal will be tracked by the growth in BBG audience size, as measured by the worldwide, unduplicated number of weekly audience members. The long-term goal is to increase the worldwide reach to 160 million by 2010 from 140 million in 2005.

Evidence: ?? BBG Strategic Plan ?? BBG Programming Support FY 2008 Performance Plan

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The long-term target for increasing the audience reach is sufficiently ambitious since they entail successful growth in audience size compared with actual resources available to the program. Compared to previous time periods, the target set for 2010 is sufficiently ambitious.

Evidence: In FY04 and FY05, the audience reach was 130 million and 140 million. Estimates for FY06 (150,000 million) and FY07 (156,000) show slower but contined positive growth. The surge in audience size in 2004 and 2005 is attributed to budget initiatives that enabled new programming in new markets, which may not continue at the same pace due to constrained resources and/or more competitive markets. The percent increase for each year is as follows: FY05 (8%); FY06 (7%); FY07 (4%); FY10, compared to FY05 (14%). Thus the estimate for meeting the 2010 target is sufficiently ambitious for the BBG.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: There are three components of measuring annual progress in achieving the long-term goals of the Programming Support Program: 1) all of the broadcasting entities share the long-term goal to build audience size in their target regions. In this way, when the entire agency undergoes the annual planning exercise with the goal of meeting targets related to building audience size, this program's progress toward achieving its long terms goals is reinforced. 2) The Program has a work-plan with detailed and measurable annual action steps over several years. The action steps are organized by performance objective and also have annual performance goals and targets, both numerical (discussed below) and anecdotal. 3) Each performance objective also has key measures for each function (seven total, including the efficiency measure, described in question 3.4) that support the long-term goal. These objectives and measures will be monitored each year as they contribute to both the long-term goal of audience reach and the overall efficiency of the program.

Evidence: Programming Support will track six key performance measures (not including the efficiency measure). First, the program will track the overall credibility of each broadcasting entity, for a total of five annual measures. This is determined through annual surveys of target populations, by asking about the trustworthiness of news and information. This measure is relevant to the program since this program is responsible for writing and editing news stories, which are the audience members' basis of judgement. Second, the program will track the number of annual program reviews that are attended by a representative of this program. Since this program contributes to the program quality, credibility, and overall audience size for each language program, their participation in the annual review will support the broadcasters' ability to reach target audiences. This program tracks progress toward the efficiency goal of program cost per audience member. As explained in question 3.4, this measure tracks growth in program cost relative to the growth in audience size. As this program aims to be more successful in growing the overall audience size, the program also aims to increase its efficiency by growing audience size at a higher rate than program cost. ?? BBG Strategic Plan ?? BBG Programming Support FY 2008 Performance Plan

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The annual targets for BBG's annual measures are sufficiently ambitious given the transition to multi-media broadcasting and content integration. The expansion to new media requiring state-of the art technical infrastructure and integration of content to be used by more than one media while still customizing content that draws audiences via their preferred media directly addresses BBG's strategic programming initiatives.

Evidence: - Overall Credibility - The baseline for this measure is by entity: for VOA, the baseline is 89%, RFE/RL is 85%, RFA is 88%, MBN (Radio Sawa) is 69% and MBN (Alhurra) is 73%. Each measure shows increasing levels of credibility each year, which is sufficiently ambitious. - Percentage of Program Reviews Attended in a Year - The baseline for this measure is 100% in FY05, and the program plans to meet this goal in FY06. - Program Cost per Audience Member -- (See question 4.3).

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The VOA Operations Support and Central News Directorates and the corresponding offices in the BBG's grantee organizations contributed to establishing the Strategic Plan and in the development of the annual performance plan, which include the annual and long-term goals of the program. Senior managers within the BBG Programming Support Services Programs commit to and work towards the goals through the BBG performance based appraisal system, which is specifically tied to program objectives and goals. Contracts for the program are managed by contract officers who monitor compliance with performance standard clauses that impact the goals of the program, such as performance, schedule, cost, and technical requirements. The Programming Support participation in all language service Program Reviews and the annual Language Service Reviews also enforce commitment to the annual and long-term goals of the agency, thereby supporting the broadcasting mission of the BBG.

Evidence: ?? BBG Strategic Plan ?? Language Service Review ?? Program Review ?? Performance Based Employee Standards/Appraisals

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Programming Support (both central news and technical support functions) is included in the annual Language Service Reviews and Program Reviews conducted by the IBB's Office of Program Review, and RFA's and RFE/RL's program review offices. VOA Central News has also undergone a complete Program Review in 2005. The offices operate independently from all BBG language services. Annual reviews include an initial assessment of how this program is meeting its objectives with regularly scheduled follow-up meetings to check the progress on recommendations generated by the review. The reviews assess content and production for quality, effectiveness and relevance. Independent focus groups facilitated by the research contractor provide the data to derive the credibility score as well as providing information on credibility and audience likes/dislikes, which is used to support improvements and evaluate effectiveness.

Evidence: ?? Language Service Review ?? Program Review

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: BBG has excelled at clarifying its mission, establishing implementation strategies and performance objectives that support the mission, as well as establishing performance indicators. Actions taken in 2006 and 2007 have enhanced the linkages between these strategic tools and increasingly shown how they are supported and implemented by BBG's budget each year. In its Budget requests the BBG explicitly linked all costs to six annual performance objectives that support BBG ' s mission, and will do so again in its FY 2008 budget request. In addition, each objective will link annual performance goals to the requested resources with performance measures will be monitored against the targets set for FY 2008. In FY 2008, the BBG will show how the performance objectives for the various broadcasting entities (including VOA and OCB) and support elements support the overall BBG strategic goal and annual performance goals. In FY 2008, the will again include the agency's annual performance plan and performance measures in budget requests. Budgeting and output for central news are related, and the costs and levels of output are fairly consistent from year to year. The budget for the news wire contracts is based on [some level of output]. While the program's goal is to increase news content in individual broadcasts, the program does not anticipate a large increase in budgetary resources to meet this goal. The budget and output for technical support are also related, but the costs and levels of output fluctuate according to the level of output required. For example, if a broadcast program is planned to expand the number of hours it broadcasts, the budget for technical support would increase too.

Evidence: ?? FY 2006 Congressional Justification ?? Language Service Review ?? Program Review ?? FY 2006, FY 2007 and FY 2008 Budget Requests to OMB ?? Capital and Information Technology Planning Process

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The BBG has a performance plan that reflects the functions of the BBG Programming Support program. The performance plan supports the BBG Strategic Plan, and it ties individual program goals to the strategic goals of the agency. The BBG Strategic Plan is in the process of being revised, and will be complete by Fall 2006. The RFE/RL Newsroom was reorganized in November 2005 with three regional desks (Europe, Asia and the World) to enhance cooperation between the Newsroom and the language services. RFE/RL has also begun to move more of its news writers out to the field to collect, write, and even produce live news stories from the field. OCB and MBN have also reorganized their newsroom to improve effectiveness and synergy among the different media broadcasts, such as moving Radio Sawa to the shared newsroom with Alhurra. When RFA was created, it chose not to create a central newsroom, in favor of country desks compiling their own news.

Evidence: ?? BBG Strategic Plan ?? BBG Programming Support FY 2008 Performance Plan ?? FY 2006, FY 2007 and FY 2008 Budget Requests to OMB ?? Capital and Information Technology Planning Process

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The BBG analyzes the performance data related to VOA's central news function through the annual Language Service Review and Program Reviews. Annual Program Reviews include an initial assessment of this program's content and production, with regularly scheduled follow-up meetings to check the progress on recommendations generated by the review. The Board uses this information to adjust resources and set strategic priorities. The Program Review data is also used in the development of the annual performance plans for programming support and to check whether existing performance targets are being met. The annual plans include action steps that the management of BBG Programming Support has identified as necessary for the accomplishment of the annual and long-term goals. The performance of the central news functions at other organizations is assessed through annual performance data such as "trustworthiness of news and information" as perceived by audience members and "usage of news reports" by the broadcasting programs themselves. Management also uses this performance information to inform their evaluations of and performance plans for individual employees. It is not clear that the technical support functions collect regular performance information. As stated in question 1.3, the BBG should develop a measure that would gauge the specific performance of this function. However, this suggested improvement is relatively minor to the program and does not warrant a "no" answer for this question.

Evidence: ?? Language Service Review ?? Program Review

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Senior managers, including the Senior Executive Service (SES) and other senior staff (GS 13-15), are under the BBG performance based appraisal system that requires annual "critical elements" and ratings based on the mission and performance objectives of the BBG. Specific goals with measurable targets are set for each element within the appraisal. The performance based appraisal system also includes individual accountability for cost and schedule results as appropriate. Through the ongoing appraisal reviews and final annual rating, all mangers and senior staff are held accountable for performance results. The annual Language Service Review holds managers and partners directly accountable for results. The impact of marketing and advertising services in achieving individual language service goals and objectives are reviewed and evaluated throughout the process. In addition to the individual employee appraisal system described above, the CFO's mid-year reviews of budget and performance results hold managers accountable for their annual resource level against expected expenditures, schedules and accomplishments.

Evidence: ?? Project Status Review ?? BBG Performance- based Employee Appraisals ?? Language Service Review

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: All requisitions must include verification that funds are available, and the funds are obligated when a contract or agreement is signed or a purchase order is done. The mid-year, 3rd quarter and year-end review provides an opportunity for managers to verify that unliquidated obligations are valid and that once an obligation has been fully expended, remaining balances are deobligated. Financial transactions are coded to both an organization and to a function or project. Given that BBG has a high percentage of fixed costs (salaries, transmission and infrastructure expenses, rent, etc.), funds must be obligated in a timely manner in order to continue operations. The BBG requires an annual financial plan from the IBB and the grantees, including Radio Free Asia and Radio Free Europe/Radio Liberty. The financial plans must detail anticipated costs for specific line items by each month of the fiscal year. BBG conducts a mid-year, 3rd quarter, and year-end review of each entity's financial plans and budgets to monitor funding and expenditures. BBG also requires a monthly status report of expenditures, disbursements and obligations, as well of the submission of the SF-169 Financial Status Report, and SF-172 Report of Federal Cash Transactions from the grantees. The reviews and the allotment control procedures enable the Agency to track obligations to ensure that funds are used for the intended purpose by the appropriate Agency office.

Evidence: ?? SF-169 Financial Status Report ?? SF-172 Report of Federal Cash Transactions ?? Monthly status reports on the IBB, RFA and RFE/RL on expenditures, disbursements, and obligations ?? Quarterly review reports

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: In accordance with the mission, vision, and strategic goals of the agency, BBG Programming Support supports BBG broadcasting through centralized news and information functions and technical infrastructure. The BBG uses the measure of BBG Programming Support Program Cost per Audience Member to evaluate the efficiency of the program. The efficiency indicator (program cost per audience member) is used to track improvements in efficiency over time. The program is expected to maintain or improve the achievement of all performance indicators at a relatively constant resource level. In fact the program has initiated increased capacity, both in content and infrastructure, transition to multi-media broadcasting and content integration which although required enhanced investment, is expected result in reaching significantly larger audiences via the preferred media in many targeted areas of the world and in the process increase net efficiency and assist the BBG in meeting its long-term goals. The program also achieves efficiencies by using incentives in its news contracts, for example using competitive sourcing to assure cost effectiveness. These projects are managed by the BBG Programming Support program to assure compliance with the cost, schedule and performance dictated in the contract. Contracts include "negative incentives" in accordance with the Federal Acquisitions Regulations to discourage changes from the time or performance indicated in the scope of contract. As explained in question 1.3, it is not clear that the overseas news gathering function has processes in place to measure and achieve efficiencies across the grantee organizations. Also, as explained in question 1.3, the technical support function is advised to develop a measure that would monitor its own performance or efficiency. However, these suggested improvements are relatively minor to the program and do not warrant a "no" answer for this question.

Evidence: ?? Language Service Review process ?? Program Review process ?? Federal Acquisitions Regulations 52.211-11, 52.211-12, 11.503

Does the program collaborate and coordinate effectively with related programs?

Explanation: The central news functions of all entities must maintain and support good working relationships with the language to provide timely, accurate and relevant news and information. VOA and RFE/RL have shifted resources to regional desks in order to improve coordination with local news gathering and language service needs. Where possible, the entities share resources, including studios and news bureau facilities. For example, in Hong Kong, RFA and VOA share studios and RFE/RL and VOA share resources in Kabul. The transmission of programming for all BBG entities is coordinated and controlled through one network control center. During the annual Language Service Review process, the collaboration and coordination in a language or region are examined and management or resource allocation actions are recommended.

Evidence: ?? U.S. International Broadcasting Act of 1994 ?? Foreign Affairs Reform and Restructuring Act of 1998 ?? Language Service Review process

Does the program use strong financial management practices?

Explanation: The BBG is cross-serviced by the State Department to provide financial services using the Broadcasting Financial Management System (BFMS) system. Internal controls are in place to provide for the separation of duties, minimize erroneous payments, ensure recording of receipts, and the accurate reporting to external entities. The BBG Administrative Officers obligate financial transactions accurately and reconcile their allotments on a monthly basis. In the CFO Office, the Financial Operations, records and makes disbursements, reconciles the Fund Balances with Treasury, records and makes deposits, and prepares the monthly, quarterly and annual statement reports for the agency. FY2004 was the first year the BBG prepared the Performance and Accountability Report (PAR) and conducted a financial statement audit. The PAR was submitted on time and the BBG received an unqualified opinion on its Balance Sheet. Again in 2005, the BBG submitted an expanded PAR and received an unqualified opinion. (Will be updated for 2006)

Evidence: ?? BFMS System ?? An independent audit conducted in October 2004 and October 2005 offered an unqualified opinion of the BBG financial statements

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Office of Program Review conducts an independent annual analysis of content and production for each language service, in addition to radio content, as well as a review of VOA's Central News separately. The Central News and operational support functions have recognized the importance of participating in the language service reviews and improved management of work schedules and coordination meetings to serve the need of the broadcasters. Between September 2000 and February 2004 the GAO conducted three evaluations of the BBG. While none of the reports focused specifically on BBG Programming Support services, the findings, recommendations and resulting actions by the BBG impacted the entire agency, including this program. Follow-up testimony provided in 2004 shows that the Board has taken meaningful steps to address its management deficiencies. No other current evaluations regarding the BBG have been completed.

Evidence: ?? Program Reviews ?? BBG Performance Based Appraisal System ?? GAO Report: "Strategic Planning and Performance Management System Could be Improved" (GAO/NSIAD-00-222) ?? GAO Report: "New Strategic Approach Focuses on Reaching Large Audiences but Lacks Measurable Program Objectives" (GAO-03-772) ?? GAO Testimony: "Challenges Facing the Broadcasting Board of Governors" (GAO-04-711T)

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: As explained in question 2.2, in FY04 and FY05, the audience reach was 130 million and 140 million. Estimates for FY06 (150 million) and the targets for FY07 (156 million) and FY08 (TBD) demonstrate adequate progress in achieving the long-term goal of 160 million by 2010.

Evidence: ?? BBG Programming Support FY 2008 Performance Plan

Does the program (including program partners) achieve its annual performance goals?

Explanation: Five out of the seven annual measures are new to this program and do not have data or show results beyond the baseline 2005 data. There is no reason to believe that the program will not meet its 2006 goals, but these data are necessary to show that the program meets its annual goals and to warrant a "yes" for this question. Two measures do have data from before 2005, which provides clear evidence that the program is meeting these goals. For this reason, a Small Extent is warranted.

Evidence: Annual measures (as shown in the Audience Development FY 2008 Performance Plan) ?? Overall Credibility - The baseline for this measure is by entity and the program plans to meet the goals for FY06. For VOA, the baseline is 89%, RFE/RL is 85%, RFA is 88%, MBN (Radio Sawa) is 69% and MBN (Alhurra) is 73% ?? Percentage of Program Reviews Attended in a Year - The baseline for this measure is 100% in FY05, and the program achieved 100% from years 2002-2004. The program plans to meet its goal of 100% in FY06 as well. ?? Program Cost per Audience Member -- (See question 4.3).

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Programming Support key performance measures include an efficiency indicator, which measures cost per audience member. There is sufficient demonstrated evidence Programming Support will show improved efficiencies or cost effectiveness in achieving efficiency goals each year. There is also clear evidence that significantly larger audience reach via the preferred media in many targeted areas of the world is estimated to start bringing net benefits to efficiencies progressively from the $0.78 in FY05 to $0.73 in FY06 and $0.69 in FY07.

Evidence: ?? BBG Programming Support FY 2008 Performance Plan

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: There are no government programs that share similar goals and objectives with which the performance of this program can be compared. With regard to the private sector, the news writing function employs private stringers and central news contracts where necessary, which adheres to market research industry standards and indicates that this function compares favorably with the private sector. The technical infrastructure and production support is generically duplicated in the private sector (as explained in question 1.3) since the BBG has no basis to compare performance of this function to the private sector (as explained in question 1.3), this warrants a "large extent" answer for this question. This PART recommends that the BBG research and develop a measure that would benchmark this function to the private sector and/or consider performing a streamlined or standard competition for this function or sub-functions within this function in order to determine the optimum level of efficiency and whether their costs are competitive with the private sector. A section of this program (the overnight Asian news writing function) was "competed" with the private sector in 2005 and the comparison found that it would be more efficient and effective to transition the function to performance by the private sector. The transition is pending resolution of some issues as of fall 2006.

Evidence: ?? U.S. International Broadcasting Act of 1994 ?? Foreign Affairs Reform and Restructuring Act of 1998

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Annual Program Reviews and Language Service Reviews indicate that Programming Support is effective in supporting the broadcasting entities and is achieving results. There has been no applicable OIG and GAO reports.

Evidence: ?? Program Review schedules, results and action plans ?? Language Service Review analysis and results ?? InterMedia research including surveys, monitoring panels, and listener panels