[This on-line version does not contain any figures, charts, graphs, or tables. To receive a published copy of this report, contact DISC, 1611 North Kent Street, Suite 200, Arlington, VA 22209-2111. Phone 703-351-4006; fax 703-351-4039; internet docorder@disc.mhs.compuserve.com] USAID Program and Operations Assessment Report No. 13 Strengthening the Public-Private Partnership: An Assessment of USAID's Management of PVO and NGO Activities By Patricia L. Jordan, CDIE April 1996 (PN-ABS-548) PREFACE In September 1993, the U.S. Agency for International Development's Center for Development Information and Evaluation (CDIE) initiated an assessment of the Agency's management of its relationship with U.S. private voluntary organizations (PVOs) and indigenous nongovernmental organizations (NGOs). During the subsequent eight months, CDIE teams interviewed staff of PVOs, NGOs, USAID, and other donor organizations in eight countries and the United States. The study benefited greatly from the expertise and goodwill of people and organizations too numerous to thank individually--in particular, the many PVO, NGO, and USAID staff who generously arranged or participated in interviews and provided suggestions and insights to enrich the study. The efforts of four people--Carl E. Wisler, Ronald D. Levin, Janice K. Stallard, and Jonathan M. Conly--were invaluable to this assessment. Mr. Wisler of Development Alternatives, Inc. (DAI), the study methodologist, was instrumental in the design of the assessment and development of data collection and analysis procedures. He also assisted with data collection, including participating on the Bangladesh field team. Mr. Levin of DAI served on the study's core assessment team from its inception through completion of data collection. He was team leader for the Mali and Senegal visits, team member for the Bangladesh field visit, and a participant in U.S. interviews. Ms. Stallard, of the Academy for Educational Development, was a member of the core assessment team. In addition to preparing draft report materials, she conducted fieldwork in Bangladesh, Swaziland, and South Africa. Mr. Conly, formerly chief of CDIE's Program and Operations Assessment (POA) division, was team leader for the Bangladesh and Poland visits and provided useful guidance and insights throughout the development of the assessment. The contributions of Leslie Fox of DAI also are greatly appreciated. In addition to preparing draft report materials, Mr. Fox was a member of the Mali and Senegal field teams. John R. Eriksson, former CDIE director, and Michael M. Calavan, CDIE/POA division chief, were especially helpful in their review of drafts of the report. The field team members' time, effort, and suggestions to improve the quality of the study also are greatly appreciated. In addition to those mentioned above, team members included Edward Glaeser (Bangladesh, Mexico, and Guatemala); Christina Schoux (South Africa and Swaziland); Stark Biddle (Poland and South Africa); Rachel Patterson (Mali, Senegal, and the United States); Ken Koehn (Mali and Senegal); Joseph Lombardo (Mexico and the United States); Mari Clark and Carol Finnegan (Poland); Scott Smith, Rachel Peterson, and Cathryn Thorup (Guatemala); and Mary Pat Selvaggio (South Africa). The study also benefited from the services of several research assistants, notably Sara Hurwitch, Dawn Emling, and John McDermott. The efforts of Cheryl Gwynn and Rachel Peterson in analyzing interview responses were particularly appreciated. Excellent background literature reviews were prepared by Virginia Lambert, Tami Fries, Cheryl Gwynn, and Margo Kelly. Finally, the assistance of Laurie Denton, Ross Bankson, Chuck Wilkinson, and Pamela McDade Frankel in preparing this report for publication is greatly appreciated. The author retains full responsibility for any errors in this report and hopes any such errors will not detract from the valuable contributions of the people mentioned here. SUMMARY USAID and its predecessors have worked for almost 5 decades with U.S. private voluntary organizations (PVOs) in providing humanitarian and development assistance. The Agency first turned to PVOs to handle short-term disaster relief and food distribution. Over the past 2 decades, USAID has worked with nongovernmental organizations (NGOs) indigenous to the countries where the Agency operates. Gradually, the scope of PVO and NGO work has broadened to programs aimed at the root causes of poverty and vulnerability to disasters. In recent years, donors, academics, development professionals, and members of Congress have recommended that PVOs and NGOs play a larger role in development and advocacy efforts. They reason that use of grass-roots citizens' groups is the best approach to accomplishing many development objectives. In most countries with USAID Missions, PVOs and NGOs are hard at work delivering disaster relief, handling food distribution, or implementing development activities. This assessment examines the working relationship between USAID and the PVO/NGO communities and suggests ways to improve that relationship. The study looks at how USAID does business--that is, the processes--as opposed to examining the development impact of USAID-funded PVO and NGO activities. U.S.- and field-based interviews, research, analysis, and report preparation took place over a 1-year period, from September 1993 to September 1994. Structured interviews were conducted with 259 PVO, NGO, and USAID staff in eight countries and the United States. Topics covered included registration, negotiation, funding instruments, general implementation, financial management concerns, and location of the USAID manager. The study also gathered information on several other facets of the USAID PVO/NGO relationship: sustainability; other donors' work with NGOs; use of financial intermediaries, or "umbrellas"; and PVO/NGO collaboration in establishing USAID development strategies. Information on these topics was collected through literature review as well as additional field interviews. Principal Findings Although PVO, NGO, and USAID respondents were able to accurately describe the differences between the Agency's funding instruments (grants, contracts, and cooperative agreements), a lack of consistency pervades USAID's use of each instrument. There seem to be few practical differences in the management of grants and cooperative agreements, as many USAID project officers impose onerous control measures for each instrument. In fact, some officers state they manage all activities the same way, regardless of the funding instrument. In addition, a contradiction is perceived between the presumed "hands off" management of grants and the Agency's accountability requirements. USAID project officers' management style and relationship with the implementer are seen as more important to project success than officers' location. However, the respective roles of USAID/Washington and Mission staff in the management of USAID/W funded activities are not clear. Many interviewees do not understand the purposes of USAID registration or at what point organizations must register. NGOs appear to have a more difficult time than PVOs with registering, particularly in meeting accounting and financial management requirements. Despite the difficulties, however, becoming registered seldom is a major stumbling block to obtaining Agency funding, since funds are available for nonregistered organizations. Negotiation is not seen as an especially difficult process. Most implementers say that time spent up front in proposal preparation and project design facilitates smooth, timely negotiations. Moreover, negotiations go more smoothly when the Agency uses a team approach or when implementers have a single USAID contact person, as opposed to separate negotiation sessions with several USAID staff. According to interviewees, key factors in implementation success are well-developed activity designs, work plans and budgets, and collaborative, competent USAID and implementer staff. Implementers say they would prefer dealing with just one USAID contact person during project implementation, preferably the project officer. Agency regulations are seen as too complex for small organizations to understand. Moreover, many NGOs receive little or no administrative orientation before or during project implementation. The USAID "system" for implementation approvals is not clear to implementers; it appears to depend on the preferences of individual USAID project officers. USAID reporting requirements and time spent on them also are seen as excessive. (The Agency recently made changes in key implementation approvals and reporting requirements for grants and cooperative agreements. They should ease the administrative burden.) It was not a specific interview topic, but many PVOs and NGOs said they would like USAID to stress the importance of--and provide funding for--regular project evaluations. Many PVOs and NGOs said the Agency's financial reporting requirements are excessive and time consuming; at the same time, however, many NGOs report their financial management systems have improved as a result of meeting Agency requirements. In addition, NGOs generally see audits as a positive management tool. Owing to recent USAID changes, NGOs that receive less than $100,000 a year per agreement in direct USAID funding (up to a total of $250,000 a year) no longer are required to follow OMB Circular A-133 audit requirements. Neither PVOs nor NGOs are consulted extensively in the development of USAID's global and country strategies, and both would like more opportunities for dialog with the Agency during strategy development. (Recently, though, the Agency's climate for strategic collaboration with PVOs and NGOs has improved dramatically.) Cost sharing involves dividing project expenses between USAID and the implementing PVO or NGO. PVOs and NGOs strongly support the concept of cost sharing, but would like more flexibility in its application. In July 1994 the Agency eliminated the 25 percent cost-sharing requirement for registered PVOs. Instead it encouraged the "largest reasonable and possible financial participation" of recipients. PVOs and NGOs raised concerns about sustainability in project design, including having USAID assess community demand for an activity's benefits and the capabilities of the proposed implementer. In addition, they see beneficiary participation in project design and implementation as an important ingredient of success. Umbrella activities involve a USAID award to a lead organization, which then makes subgrants to a number of PVOs and NGOs. Capacity building of subgrantees is an increasingly important part of umbrella activities. NGO subgrantees benefit from the opportunities for networking and institutional strengthening (with assistance from the lead organization). USAID staff say some management savings are realized with umbrella activities, because only one grant is managed, not several. With the notable exception of the Canadian International Development Agency, donors generally don't emphasize donor-NGO partnerships or NGO capacity building. Many donors provide indirect funding to indigenous NGOs through a donor country NGO and have cost-sharing requirements. Another funding method, the "consortium" approach of some donors, is a useful method for multiple donor funding of indigenous NGOs. Compared with other donors, USAID's strengths are its in-country presence and its ability to directly fund indigenous NGO activities. Management Recommendations Three sets of management recommendations flow from the evaluation. One regards USAID's inconsistent management of grants and cooperative agreements. The second relates to policy changes. The third involves other areas in which USAID staff can take immediate management actions. This last group is by far the largest, as the study found that most of the needed changes do not require revisions to the Agency's policies and procedures. Inconsistent Management Over several months of interviews, it became clear that a pervasive problem affects USAID's management of PVO/NGO activities. It is the inconsistency of USAID staff's management of grants and cooperative agreements. Staff appear knowledgeable about technical differences between funding instruments, but in practice these differences often are ignored. Preferences of individual officers, rather than requirements of specific funding instruments, seem to determine how activities are managed. Overall, the Agency needs to provide clearer guidance on the use of each funding instrument and necessary controls associated with each--for example, in cooperative agreements, drawing the line between substantive programmatic involvement and administrative micromanagement. Several specific ways suggest themselves for developing USAID staff capacity to manage PVO and NGO activities more consistently and equitably: 1. Reinforce collaborative management. The Agency's senior managers should make dedicated efforts to communicate to staff the importance of collaborating with USAID's development partners. 2. Structure an incentive system. As part of USAID's reengineering efforts, senior managers should develop an incentive system that recognizes and rewards hands-on, results-oriented project officers for excellence in working in partnership with PVOs and NGOs. 3. Provide additional training. Additional training of USAID staff, particularly project officers, is needed to ensure reasonable consistency worldwide in applying USAID regulations and procedures to the various funding instruments. 4. Certify project officers. Agency managers should consider developing a certification program for project officers to ensure that project staff have the knowledge and skills needed for managing PVO/NGO activities. Recommended Policy Changes Senior managers should review current policy regarding indigenous NGOs in these areas: 1. Simplify NGO registration. Current legislation requires USAID to register indigenous NGOs that meet the criteria for classification as PVOs. Given the Agency's emphasis on increased direct funding of NGO activities, current registration requirements for NGOs should be reviewed and simplified. USAID also should consider the feasibility of direct Mission registration of NGOs. 2. Clarify requirements of the Office of Management and Budget (OMB) for NGOs. As a matter of policy, the Agency has determined that OMB circulars pertaining to the management of grants and cooperative agreements apply to NGOs as well as PVOs. Keeping in mind the importance of sound management principles, and recognizing the need for NGO institutional strengthening, the Agency should review the current OMB requirements to determine which ones are essential for application to indigenous NGOs. It should release NGOs from inappropriate or unproductive management requirements. 3. Consider using more donor consortia. To provide more funding options for indigenous NGOs and to streamline their reporting requirements, USAID should investigate the possibility of joining or establishing country-level donor consortia to provide joint funding for indigenous NGO activities. Immediate Management Actions In addition to addressing the problem of USAID officers' inconsistency in PVO/NGO grant management, many changes can be made by USAID staff to improve management of PVO/NGO activities. These suggestions can be carried out without revising existing Agency policy or procedures. 1. Senior manager actions. Senior managers--at field Missions or in USAID/Washington--can help strengthen USAID's partnership with the PVO and NGO communities in several ways. Among them: -- Systematically tap PVO and NGO expertise through regular consultation in the development of USAID's global and country development strategies -- When capacity building is identified as an objective, consider providing longer term funding -- Establish a single point of contact. Senior managers should encourage a team approach within the Mission for project design, negotiation, and implementation. To improve communication, senior managers can establish one point of contact for the PVO or NGO. Preferably that contact person is the project officer 2. Project officer actions. Project officers can take immediate steps to improve the management of PVO/NGO activities: -- Encourage the development of PVO/NGO networks to help strengthen indigenous NGOs -- Use more support grants to fund organizations' broad portfolios of activities -- Strengthen NGO financial management skills by using pre-award reviews and small initial awards for new or inexperienced NGOs -- Better educate PVOs and NGOs about USAID's minimum accountability requirements and the differences in funding instruments -- Strengthen PVOs and NGOs by including capacity building activities as program outcomes or intermediate results -- Regularly include funds for evaluation in activity budgets GLOSSARY CDIE Center for Development Information and Evaluation (USAID) NGO Nongovernmental organization OMB Office of Management and Budget POA Program and Operations Assessment division PVO Private voluntary organization USAID/W USAID Washington 1. SEEKING A LARGER ROLE FOR PVOS AND NGOS USAID has worked for many years with U.S. private voluntary organizations in providing humanitarian and development assistance. The Agency first turned to PVOs to handle short-term disaster relief and food distribution. Gradually, the scope of PVOs' work was broadened to programs aimed at addressing the root causes of poverty and vulnerability to disasters. More recently, PVOs also have directed their attention to developing the management and technical capabilities of their indigenous partners. USAID has developed umbrella activities to allow the transfer of funds and technical assistance through PVOs to indigenous nongovernmental organizations. In most countries with USAID Missions, PVOs and NGOs are at work delivering disaster relief, handling food distribution, or implementing development activities. This assessment examines the working relationship between USAID and the PVO/NGO communities and suggests ways for USAID policymakers and project managers to improve that relationship. Thus, the study looks at how USAID does business--that is, the processes--as opposed to looking at the development impact of USAID-funded PVO and NGO activities. Information on USAID management of PVO/NGO activities was collected through structured interviews with 259 PVO, NGO, and USAID staff in eight countries and the United States. Interview topics included location of the USAID project officer (whether in USAID/W or a field Mission), choice of funding instrument, USAID requirements for PVO registration, negotiation of grants and cooperative agreements, general implementation concerns, approvals and reporting requirements, and financial management. Interview information was sorted by topic with the help of a computer and then analyzed to determine key themes or trends. Interview results are summarized in chapters 2 and 3. The study also gathered information on several other concerns related to the USAID-PVO/NGO relationship. Among them: sustainability, strategic collaboration, other donors' work with NGOs, and use of financial intermediaries (umbrellas). Information on these topics was collected through literature review and additional field interviews. These topics are summarized in chapter 4, with fuller discussions found in appendix D. Appendices A and B contain information on the USAID PVO/NGO policy framework and a summary of recent and proposed Agency reforms, respectively. Appendix C discusses the study methodology, and appendix E presents the perceptions of USAID senior managers on related study topics. Background In recent years, a number of donors, academics, development professionals, and members of Congress have recommended that PVOs and NGOs play a larger role in development efforts and in advocacy roles. They reason that use of grass-roots citizens' groups is the best approach to accomplishing many development objectives. The dynamics of the USAID-PVO partnership are influenced by changing world conditions and revised development strategies. The ascendance of democracy, the shift toward market-based economies, the new centrality of environmental concerns, and political shifts in Europe and the former Soviet Union have altered the context in which USAID and PVOs function. Groups such as the Advisory Committee on Voluntary Foreign Aid and the USAID-PVO Task Force have suggested significant changes in the nature of the aid partnership with the PVO and NGO communities. As of October 1, 1994, there were 419 U.S. PVOs registered with USAID, not all of which were receiving U.S. Government support. During fiscal year 1993, the government provided more than $1.6 billion in grants, contracts, government-owned excess property, ocean freight subsidies, and P.L. 480-donated food in support of PVO-administered programs. More than 400 NGOs also are registered with USAID. In fiscal year 1993 they and other NGOs received more than $200 million in U.S. Government support from bilateral funds. Critical Management Issues Until the mid-1980s, the professional literature on PVO management generally concentrated on the internal management of PVOs--fund raising, strategic planning, and the like--rather than on the mechanics of managing field-based PVO activities. Change came in the late 1980s as development professionals and scholars began exploring the link between good PVO management and sustainable development. However, a literature review conducted for this assessment found little material specifically representing the USAID manager's view or dealing with management issues concerning PVO or NGO implementation of USAID-funded activities. Three recent reports document difficulties in the USAID-PVO/NGO relationship. They include the report of a joint USAID-PVO task force, a survey conducted for a USAID-funded regional environmental project, and a General Accounting Office report on USAID's management of grants and cooperative agreements. The issues discussed below in large measure also reflect concerns expressed during this assessment. The USAID-PVO task force (USAID and the Private Voluntary Community 1993) met in August and September 1993 and issued its final report in December 1993. The task force comprised six working groups that dealt with three sets of related issues: 1) program focus, balance, and direction; 2) administrative systems and procedures; and 3) innovative funding mechanisms. Among the task force's observations are several closely related to the scope of this assessment: -- USAID should capitalize on the growing role, importance, and ability of NGOs -- Institutional capacity building for PVOs and NGOs is an essential component of a results-oriented development strategy -- USAID's current registration, procurement, grant negotiation, and oversight system impedes the USAID-PVO partnership and reduces the effectiveness of USAID resources and PVO performance A December 1993 report (MSI 1993) on a survey of Central American environmental PVOs and NGOs contained several recommendations on matters of policy, communications, administrative issues, and financial management. It found that USAID should -- Adopt more flexible administrative procedures for working with NGOs -- Promote more collaboration among NGOs -- Take greater advantage of local knowledge and expertise when designing new USAID activities -- Strive for improved coordination within USAID A 1993 study (GAO 1993) by the General Accounting Office investigated USAID's policies and procedures for administering grants and cooperative agreements with PVOs, educational institutions, and other not-for-profit organizations. It noted that -- USAID did not ensure that all grantees were qualified, inasmuch as contracts officers did not routinely document their determinations that prospective recipients had adequate financial and management capabilities -- USAID had limited ability to monitor grant recipients, because project officers often did not use measurable benchmarks and target dates -- USAID often was not able to make needed field trips for monitoring, owing to budgetary or time constraints, yet Washington-based project officers rarely delegated monitoring responsibilities to field Missions. Definitions of 'PVO' and 'NGO' According to a 1988 cable, to be recognized as a private voluntary organization, an organization must meet the six conditions listed below. They relate to the organization's privateness and voluntary nature. NGO is a broader category than PVO. The NGO category includes PVOs, but also other organizations such as educational or research institutions and profit-making firms. An NGO that is not a PVO remains eligible for assistance without registration, but it cannot take part in activities reserved by policy or law for registered PVOs. These include USAID's payment for ocean freight shipping, procuring goods or services from federal agencies, participating as cooperating sponsors under section 202(A) of PL 480, or participating in the PVO matching grant, child survival, and Mission PVO programs. However, for ease of reference, "PVO" is used in this assessment to mean a U.S.-based not-for-profit organization engaged in overseas development work, and "NGO" is defined as an indigenous not-for-profit organization. Use of the terms "PVO" and "NGO" in this study does not imply presence or absence of USAID registration as a private voluntary organization. 2. PREAWARD CONSIDERATIONS This chapter and the next examine several facets of USAID's management relationship with PVOs and NGOs. Discussion and recommendations are based on 259 interviews with USAID staff and representatives of PVOs and NGOs in developing countries and the United States. Interviews concentrated on management areas in which significant simplification and streamlining might be possible. The remainder of this chapter explores five elements of the USAID management equation that must be dealt with before USAID actually awards funds to a PVO or NGO. They are 1) choice of funding instrument, 2) location of the USAID manager, 3) USAID's registration requirements, 4) preaward reviews, and 5) negotiation of the funding agreement. Funding Instruments Choice of Instrument Differences between the three funding instruments and their uses can be characterized as follows.1 Contracts represent the normal legal relationship for USAID's acquisition of goods or services. A contract is a legally binding agreement under which an organization agrees to provide USAID with specified goods or services required for USAID programs in return for payment. With a contract, USAID has greater rights to remedies for breach of contract than are appropriate under a grant or cooperative agreement. Grants and cooperative agreements, by contrast, are assistance instruments under which USAID's principal purpose is to support or stimulate the recipient's own program or project (consistent with objectives of the Foreign Assistance Act). Both grants and cooperative agreements are conditional gifts to an individual or an organization, such as a PVO. Under a grant, the grantee has considerable freedom to pursue its own stated program, without substantial involvement by USAID during the performance of the activity. The Agency expects the grantee to make its best efforts in achieving the purpose of the grant. Unlike a grant, a cooperative agreement entails a significant degree of direct participation by USAID in performance of the recipient's program. USAID has several types of agreements2 that can be used to fund activities of registered PVOs or other organizations, such as NGOs. Two forms of "specific support grant"--matching grants and operational program grants (both of which also may be awarded as cooperative agreements)--are reserved for registered PVOs. Two other agreements--"other" specific support grants and cooperative agreements--can be used to fund activities of other not-for-profit organizations, such as NGOs. Since interviewees generally had limited experience working with contracts, comments in this section pertain to perceived differences between grants and cooperative agreements. Overall, respondents gave appropriate, accurate descriptions of each funding instrument. However, PVOs and NGOs report they do not understand how USAID determines which funding instrument to use, since there is a lack of consistency throughout the Agency. Some NGOs remarked that choice of instrument seems to depend on the personal preference of the USAID contracts officer. USAID staff gave a variety of reasons for choosing one instrument over another: level of trust and respect for the recipient PVO/NGO, capabilities of the PVO/NGO recipient, nature of the activity, convenience (for example, to avoid the procurement requirements of contracts), preferences of the recipient (for example, some will accept only grants), and a Mission habit of using one instrument over another. Overall, most respondents believe that implementation success, regardless of the funding instrument, depends primarily on the capabilities of the implementer. Some USAID staff add other factors related to effectiveness: level of trust between the Agency and the PVO/NGO, management style of the project officer, and the Agency's level of involvement in the activity. In fact, many USAID staff argue that the more control they have over a project, the greater the likelihood of success. Some project officers also claim that regardless of the type of funding agreement, they manage all their activities the same way. For practical purposes, they see no differences between instruments. There were some reports of innovative uses of funding instruments to develop the capacities of organizations new to working with USAID. For example, some respondents report that the Agency has awarded small contracts to PVOs/NGOs for development of USAID-approved financial management systems, particularly in cases where the organization fails to meet preaward standards. Contracts Contracts are not often used with PVOs and NGOs. When they are used, they often are small purchase orders for specific products. PVOs say that contracts are the hardest funding mechanism to work with, because of the technical nature of the agreement and the relative inflexibility of a contract to meet changes in the scope or implementation of an activity. But many USAID project officers say they prefer contracts, because they provide the highest level of control over deliverables, reporting, and accounting. Grants Appropriately, all respondents see grants as the most flexible funding instrument. They observe that USAID has minimal management and oversight requirements under grants, compared with the other funding instruments. However, PVOs and NGOs note that the Agency too often is overly involved in the implementation of grant activities and in budget management after a grant is awarded. Most USAID staff interviewed note that grants offer less control than cooperative agreements and contracts regarding program issues, management, and finances. As a result, they say, grants are more difficult to monitor. They see a contradiction between the hands-off management of grants and the Agency's program and financial accountability requirements. Staff also believe that in auditing grant activities, the office of the USAID Inspector General needs to better understand that grants, by their nature, allow PVOs to receive and spend funds with considerable discretion. However, an Inspector General representative says that since the advent of the Recipient-Contracted Audit Program, his office performs few financial audits of PVO/NGO activities. Instead, the Inspector General relies on audits conducted by private accounting firms. According to the representative, the Inspector General does not expect USAID project officers to be financial managers; rather, they should be concerned with the overall accountability of the funding recipient. In that regard, he thinks USAID should make wider use of preaward reviews. They are, he believes, one of the most effective management tools available to the Agency. Cooperative Agreements USAID staff believe cooperative agreements are useful for building partnerships between the Agency and implementers, and for longer term development activities. Some PVOs, however, question this view. They claim that with cooperative agreements, USAID attempts to control and intrude on organizations and activities, indicating a lack of trust toward the implementer. PVOs also note that the negotiation of cooperative agreements is time consuming. The Agency, they say, tries to insert micromanagement aspects into the agreement, while PVOs try to negotiate out these same elements. One NGO respondent provided an example of USAID's inconsistency in managing its funding agreements. This NGO had two concurrent Mission-funded cooperative agreements, both in the same program area. According to the NGO, the major difference in the agreements was the difference in personality of the two USAID project officers. With one agreement, the project officer was cooperative and supportive, and the NGO felt as if it were carrying out its own program with the Agency's help. With the other agreement, because of micromanagement by the project officer, the NGO felt distrusted, as if it were not implementing its own program. Some USAID staff believe cooperative agreements should be used with more experienced organizations. Others contradict this, saying the agreements' primary use is to allow the Agency to recognize and correct implementer weaknesses. These views skirt the central issue, though: the use of cooperative agreements should depend on USAID's desired level of programmatic--not administrative--involvement. Other mechanisms, such as preaward reviews or the use of graduated (initially small) awards, should be used when there are concerns about administrative capabilities of the recipient. Location of the USAID Manager Overall, respondents agree that substantial differences exist between USAID/W and Mission management of PVO/NGO activities.3 But some interviewees--PVOs, NGOs, and USAID staff--say the location of the USAID project officer is not a major factor in successful implementation of PVO/NGO activities. They feel the personality or management style of the USAID project officer and the relationship between the project officer and implementer are more important to implementation success than the project officer's location. PVO and NGO comments regarding location of USAID management fall into two camps. A larger group wants a high level of USAID involvement in its projects (a hands-on approach), whereas a smaller group wants more autonomy and minimal USAID involvement (hands off). Respondents associated Mission-funded projects with the more hands-on approach and identified USAID/W-managed activities with the hands-off approach. PVO, NGO, and USAID respondents all pointed to confusion regarding the respective roles of USAID/W and Mission staff in managing USAID/W-funded activities. Many PVOs are not clear about the division of responsibility between Missions and USAID/W when both are involved in program management. Also, Missions often appear not to be knowledgeable about USAID/W-funded activities being implemented in their countries. Some Mission staff argue that managing projects from USAID/W when a field Mission is present is inefficient and creates unnecessary difficulties and duplication during grant approval and project monitoring. PVOs agree there is time-consuming duplication in communications, clearances, and approvals when both USAID/W and Missions have roles in a USAID/W project. In some instances, required periodic reports travel from the PVO field adviser to the PVO's U.S. headquarters to a USAID/W officer and then back to the Mission project officer who monitors the activity in-country. Clearances for some USAID/W-funded activities also are considered a painful process, as they travel the same convoluted route as reports. USAID/W Management U.S.-based PVO staff tend to favor USAID/W-funded activities, owing to their own U.S. location, which makes it easier for them to deal with USAID/W than with overseas Missions. These PVOs see USAID/W-managed activities as more efficient, since they can receive funding for activities in more than one country, thus reducing or eliminating the need to obtain funding from individual Missions. Most PVO staff who favor USAID/W-funded activities also stress the benefits of hands-off management by USAID/W. These PVOs say that USAID/W provides clear instructions for implementation as well as advice on, for example, the use of matching funds to simplify procurement. Mission Management Compared with USAID/W-funded activities, respondents agree that Mission-funded activities offer more direct interaction between implementers and USAID on project management issues--a more intense, individualized approach. PVO field staff say that substantive input and timely communications increase with Mission-funded activities. The full-time presence of Mission staff appears to make networking and coordination easier. Many PVO, NGO, and USAID respondents see the field location of the USAID project officer as important to project success, because the officer is more likely to know local NGOs and their capabilities. As one NGO staff member put it, "The closer to the action it is, the more effective USAID management is." Some PVO staff members say, however, that implementation changes are difficult to make at the Mission level. Reasons given are largely personnel related. Among them: frequent changes in U.S. direct-hire project officers, working with USAID personal services contractors or foreign service nationals who lack authority or experience, and Mission use of regional contracts officers or legal advisers who are physically distant and overworked. Registration Since 1988, annual appropriations legislation has required USAID to register all U.S. and foreign PVOs that will receive direct funding from a Development Assistance account. However, registration is not a requirement for organizations not defined as PVOs,4 those receiving contracts or subgrants, or those receiving funding for disaster assistance activities. Registration serves three purposes: 1) it identifies PVOs (U.S. or indigenous) that engage in or intend to engage in voluntary foreign aid operations, 2) it determines whether PVOs meet certain general operating guidelines and accountability standards, and 3) it certifies eligibility of PVOs to apply for USAID funding (USAID n.d.). Registration Requirements To become a registered PVO, an organization must meet eight conditions proving the entity's status and demonstrating its financial integrity. The eight registration conditions for U.S. PVOs are shown in the box below. Requirements for organizations in developing countries are essentially the same but may be modified slightly to reflect local laws and customs. Many USAID staff, PVOs, and in particular, NGOs say they do not understand the rationale for and process of registration. Registration is not seen as a major stumbling block to obtaining funds, though, since only a portion of USAID funding is reserved for registered PVOs. The most frequent registration problem is meeting accounting and financial management requirements. For that reason, many interviewees say that registration is significantly more difficult for NGOs. Still, many NGOs say they recognize the appropriateness of USAID requirements that they meet strict financial and legal requirements to receive funding. Some NGOs say their organizations are better managed as a result of changes implemented while preparing for USAID registration. Do registration requirements favor "mature" organizations? Many PVO and USAID staff think so. They suggest that USAID tailor registration requirements to allow smaller, less experienced NGOs to participate more easily. In fact, in January 1995, USAID's Office of Private and Voluntary Cooperation revised the PVO registration process, reducing the number of required documents from 18 to 6. The Agency has other alternatives, as well, such as use of subgrants (which are not restricted to registered PVOs) for encouraging participation of smaller, less experienced NGOs. The Registration Process Registration requirements are essentially the same for U.S. PVOs and indigenous NGOs, but the process for becoming registered differs. The Office of Private and Voluntary Cooperation directly registers all U.S. PVOs: required documentation is submitted directly to that office, which reviews the materials and, if acceptable, issues a certificate. Indigenous NGOs submit required documentation to the local USAID Mission, which may provisionally register them. Once an NGO receives provisional registration, it is eligible to receive USAID funding reserved for registered PVOs. The Mission prepares a report detailing how the NGO met USAID registration requirements, including any special circumstances due to country-specific cultural or legal practices. The Mission then forwards this report to the Office of Private and Voluntary Cooperation. After review and approval of the Mission registration report, the office issues a permanent registration certificate to the NGO. Most PVOs consider registration fairly easy, presumably because they generally are well established and have sound accounting and management systems in place. For many NGOs, though, registration is slow, complex, and costly, and they frequently become overwhelmed by the requirements. Mission staff, too, find the registration process difficult. Many say they take it upon themselves to see NGOs through the process. One Mission, for example, appointed a USAID officer specifically to oversee the registration process. The officer set up, in effect, a one-stop operation for NGOs. Preaward Reviews In addition to PVO registration requirements, USAID must carry out preaward reviews to ensure that recipients have technical and financial management capabilities to prudently manage USAID funded activities. Although preaward reviews were not an explicit interview topic, several interviewees made spontaneous comments on it. The basic requirements are outlined here to present a fuller picture of USAID's preaward activities and to complement the discussion on registration. Before receiving a grant or cooperative agreement, a potential recipient must convince USAID that it has, or has the ability to obtain, the overall management competence to plan and carry out the proposed program. More specifically, the USAID contracts officer must determine whether the recipient is able to meet the Agency's accounting and reporting standards.5 This determination can be through a formal or informal survey, depending on the circumstances. For example, organizations that never have received USAID funding, or that have not had U.S. government funds for the past five years, usually are required to have a formal preaward review. Some USAID respondents indicate they regularly follow the Agency's preaward survey requirements, particularly when awards are being made to nonregistered NGOs. However, the study did not attempt to assess the extent to which USAID staff follow prescribed procedures. Consistent use of preaward reviews, when applicable, would help USAID determine the management capabilities of potential funding recipients and could lessen the Agency's need for extensive implementation reviews and approvals. The Negotiation Process The great majority of respondents do not find the award negotiation process difficult. In fact, many respondents say that if negotiation is handled well, it can positively affect project success. The theme of "early collaboration" between USAID and PVOs/NGOs was heard frequently throughout the interviews. Implementers say that time spent in proposal development and project design, especially in close collaboration with USAID, makes subsequent negotiation easier. Many PVOs and NGOs note there is no consistent structure for the negotiation process; negotiation arrangements vary by Mission and by project. In some cases, only project officers are involved in negotiations. In others, both a project officer and a contracts officer negotiate with the PVO or NGO, often meeting with them separately (the so-called serial approach to negotiation). At other times, several USAID staff members (project officer, contracts officer, and controller) are involved in negotiations, often meeting as a team with the PVO or NGO. All respondents agree that when negotiations are well coordinated--especially when USAID uses a team approach--the negotiation process works well. Many field-based PVOs and NGOs--particularly NGOs--find it helpful to have a single contact person in the Mission to coordinate the negotiation process. When there is no single contact person, PVOs and NGOs have more complaints about the negotiation process. For example, one PVO reported it underwent serial negotiations with three different USAID staff members. By the time the process was completed, the scope of the proposed activity had changed so dramatically that the PVO chose to withdraw from the project. Most respondents report that USAID project officers carry the burden for negotiations and for ensuring coordination among Agency staff. PVOs and NGOs frequently commented on the importance of the personality and knowledge of USAID staff for successful negotiations. They noted that when the USAID project officer plays a strong role, negotiations go more smoothly. Conversely, some PVOs and NGOs say that when negotiations are not coordinated within the Agency, too much power ends up in the hands of the contracts officer, who frequently does not clearly understand the proposed program. Some PVOs report that yearly incremental funding obligations for grants, which require agreement amendments, greatly increase their administrative workloads. To justify receiving the planned incremental funding, PVOs say they often have to submit and negotiate yearly miniproposals (presumably referring to annual work plans), and they suggest that USAID consider ways to streamline the incremental obligation process. However, USAID's Procurement Policy office points out there are no formal U.S. Government or USAID requirements tying work plan approvals to incremental funding obligations; it suggests the USAID contracts officer's preparation of an incremental funding amendment for a grant should be a routine matter. USAID staff made several suggestions to streamline the negotiation process. In particular they emphasized the importance of preaward reviews. They suggested providing small amounts of funds to potential implementers, as needed, to help them upgrade their financial systems before awarding a larger agreement. USAID staff also recommended developing a simplified set of requirements for smaller grants (for example, up to $250,000) and streamlining required documentation for obtaining grants. In fact, the Procurement Policy office says that many of these documents are required annually, not on a per grant basis. It adds that the Agency is exploring establishment of a central repository for this information. Conclusions Major problems in funding instruments spring from USAID's inconsistency in choice of instrument and inappropriate (and inconsistent) use of associated monitoring and reporting mechanisms. As a result, successful implementation appears to depend more on the personality or management style of the USAID project officer than on USAID policies and procedures. The Agency can take a number of steps to deal with these problems. It can, for example, provide clearer guidance on use of each funding instrument and the necessary controls associated with each. For cooperative agreements, for example, the Agency should draw a clear line between substantive programmatic involvement and administrative micromanagement. Another possible step is to consider establishing a certification program for project officers in management of grants and cooperative agreements. Regarding the location of USAID project officers, the major concern of respondents is that in the management of USAID/W activities, the respective roles of USAID/W and Mission staff are not clearly distinguished. To remedy this, the roles of all parties need to be defined at the start of the activity. The Agency also should consider establishing guidelines for Mission monitoring of USAID/W-funded activities. Registration is a legal requirement for U.S. and indigenous organizations that meet the criteria for classification as PVOs. The key issue here is that NGOs have a difficult time meeting registration requirements, particularly those for accounting and financial management. Although the Agency revised registration requirements for U.S. PVOs in January 1995, these should be reviewed to determine if they can be further simplified to encourage and ease NGO registration--for example, by using preaward surveys to fulfill all or some of the registration requirements. USAID also should consider direct Mission registration of NGOs as a way to further streamline the process. Finally, the Agency should continue to use other funding sources, such as subgrants, to channel USAID funds to NGOs that do not meet registration requirements. Because this assessment did not examine preaward reviews, the extent to which they are used throughout the Agency cannot be determined. But interviewees who spontaneously discussed this topic indicate preaward reviews are a valuable tool in assessing organizations' capability to manage USAID funds. Agencywide use of preaward surveys could lessen the need for many administrative implementation approvals. Negotiation of funding agreements is not seen as an especially difficult process. Negotiation seems to work best, though, when the Agency adopts a team approach or when one staff member is designated as the contact person. 3. IMPLEMENTATION CONCERNS The preceding chapter examined management issues before funding is awarded; this chapter looks at project implementation. General implementation concerns include factors in implementation success, management principles, USAID staff roles, time management, USAID regulations, and procurement issues. Subsequent sections deal with implementation approvals, reporting requirements, financial management, and auditing. General Implementation Concerns Implementation Success PVOs, NGOs, and USAID staff consistently say implementation success requires careful development of activity design, work plan, and budget before the award of project funding. Many NGOs also believe that monitoring inputs and outputs and assessing project results can positively affect implementation. Many respondents suggest paying more attention to project development, to avoid implementation problems. In that way, benchmarks and reporting requirements (for example) are clearly laid out before the funding agreement is signed. NGOs, in particular, say they would like USAID to help them develop and monitor project benchmarks. Almost all respondents feel strongly that project success is due in large part to collaborative and competent project staff--on the part of both the implementer and USAID--rather than the use of any particular implementation system or mechanism. Many PVOs and NGOs add that having a USAID project officer in-country aids implementation. Some PVOs and NGOs cite the positive effect of regular project implementation meetings. PVOs and NGOs also advocate Mission-sponsored coordinating or networking meetings for donors and implementers working in the same sector. Many PVOs and NGOs contend the overall quality of implementation would improve if USAID would allow them to more freely direct and manage their own activities. Respondents frequently used words like "respect," "autonomy," and "trust" to reflect this view. But USAID project officers believe that strict Agency accountability requirements often preclude more autonomy for implementers. Management Principles According to USAID guidance for grants and cooperative agreements, there are three aspects of prudent management of grants and cooperative agreements: 1) self-restraint in imposition of program controls, 2) prudent financial management of public funds, and 3) minimal requirements for approvals, reports, and restrictions.6 Self-restraint in imposition of program controls means that if a great deal of operational control is required, USAID should use a contract instead of a grant or cooperative agreement. Limiting requirements means that USAID officers should avoid imposing requirements for program approvals, reports, or restrictions that go beyond applicable circulars of the Office of Management and Budget (OMB) or the needs of prudent fiscal management. Prudent financial management involves a number of principles related to stewardship of public funds. To exercise prudent management, USAID officers should provide adequate information on the Agency's assistance procedures to prospective recipients and ensure that prospective recipients have technical, managerial, and fiscal competence. Agency officers should make sure all understandings between the recipient and USAID are put in writing. USAID also should: 1) review at least an annual substantive report on the recipient's operations and management; 2) ensure adequate financial controls, such as recordkeeping, financial reporting, audits, limitations on expenditures, and provisions for return of funds; 3) ensure evaluation of program effectiveness; and 4) have provisions for termination of the award. These principles are consistent with OMB Circular A-110, which outlines requirements for management of grants and cooperative agreements. The same provisions also are incorporated into USAID's mandatory and optional standard provisions, which form part of each grant or cooperative agreement. USAID Staff Roles Oversight of a grant or cooperative agreement is shared between the project officer and the contracts officer.7 Although each has distinct responsibilities, close cooperation is essential. The contracts officer is responsible for three areas: 1) interpreting provisions of the agreement, 2) resolving questions in connection with agreement financing (such as final determination of overhead rates), and 3) receiving and analyzing financial and audit reports. USAID custom is for recipients to communicate directly with the contracts officer on these issues. The project officer is responsible for working with the recipient on all program or technical matters related to the agreement. This includes making site visits, administratively approving vouchers, maintaining routine liaison with the recipient, ensuring that the recipient submits required reports, and referring the recipient's questions on interpretation of the agreement to the contracts officer. But many PVOs and NGOs say the system is seldom this clear-cut. Most find they continually have to deal with a number of different USAID staff members (project officer, contracts officer, controller) on various project issues, and they find this confusing. They would prefer to have one contact person, preferably the project officer. Although this runs counter to the Agency's customary practice, a one-stop system for recipients would work if USAID staff are clear regarding their respective roles. Some interview respondents find that USAID staff are overwhelmed by the Agency's internal systems and requirements. This is seen as a disincentive for staff involvement in certain activities, particularly in making site visits. Some USAID staff (and a surprising number of PVOs and NGOs) say USAID staff manage too many projects to be effective with any single one. When forced to choose among possible monitoring activities, the USAID system generally forces staff to concentrate on meeting a project's accountability requirements. Although all categories of USAID staff--U.S. direct hires, personal services contractors, and foreign service nationals--were interviewed, the extent to which this concern applies to all staff is not clear. In general, PVOs and NGOs hold that USAID staff tell them what to do, rather than engaging in a dialogue to help them learn how to do something. This observation was countered, though, by some NGO staff, who report that USAID staff have been helpful in implementation of their projects. Many PVOs and NGOs note that during the course of an activity, they deal with a series of USAID project officers because of staff turnover. This adversely affects project implementation. For one thing, each new project officer must get up to speed on the project; for another, each new officer brings a unique management style to the activity. The role of foreign service national staff, who ostensibly could provide needed continuity, was not explicitly addressed by interviewees. Time Management Almost every respondent reports some time-management problems. PVOs, NGOs, and USAID staff all say too much time is spent on financial management, reports, procurement, and project design/proposal preparation. For PVOs procurement issues are the principal source of excessive implementation time; NGOs report that financial management is the most time consuming. USAID staff say that meeting internal USAID reporting requirements is their biggest time-management concern. An overwhelming number of respondents--from USAID as well as PVOs and NGOs--say they spend insufficient time monitoring outputs and assessing impact. Almost all USAID staff say they would rather spend time assessing a project's results, but they don't find the time. Respondents generally agree that USAID should be more involved in the programmatic substance of project activities, rather than in administrative micromanagement. USAID Regulations In general, USAID staff and PVOs say USAID regulations are too complex for most small organizations to understand and follow. Some PVOs report that regulations on compensation, in particular, create difficulties in hiring and keeping competent, educated local employees. In addition, some NGOs report that USAID staff spend little or no time explaining how the USAID system works. However, in general, NGOs believe that meeting USAID's administrative and accountability requirements leads to institutional strengthening. Many NGOs say they are given little or no orientation to USAID regulations before the start of the activity and that they are unprepared for the challenge of meeting USAID's administrative requirements. Many USAID staff and PVOs support this assertion. They say that, in general, NGOs are technically capable but lack experience dealing with USAID management requirements. Moreover, some respondents complain that the Agency's administrative and regulatory materials are available only in English, making the problem worse. Procurement Issues Most comments on procurement matters were anecdotal accounts involving vehicles, computers, and other equipment that is expensive and cannot be serviced in-country. Other comments concerned USAID regulations themselves, particularly the Buy American requirement. Most implementers say it is extremely costly and time consuming, even though they understand why it exists. Most respondents, including USAID staff, believe the level of competition required for small procurements is not efficient, since too much time is spent monitoring small sums of money. Some implementers are unhappy about costs and delays caused by bidding procedures they are forced to follow for minor items. Respondents note that obtaining exemptions for relatively small procurements sometimes requires more justification than for larger ones. To avoid procurement problems, some PVOs purchase goods and services with non-USAID (or cost-sharing) portions of their funding. In January 1994, the Agency established a procurement-reform task force. One task force initiative is streamlining USAID's procurement procedures. The task force conducted a vendor town meeting in March 1994 to gather information from contractors, PVOs, and others on procurement problems. Several reforms have been carried out recently; they are discussed in the following section. Approvals and Reports Approvals Although many PVOs and NGOs say they are concerned about the need to obtain USAID approvals more than once for the same implementation actions, most agree that the Agency's approval system is workable, once they know the system. However, "knowing the system" may be an elusive goal for most implementers. The majority of PVOs say criteria for required approvals are never defined, and there is great variability from activity to activity in the types of required approvals and their timing. Many PVOs and NGOs believe their relationship with the USAID project officer, or the project officer's management style, is the greatest determinant of the level of implementation approvals: the better the relationship, the more flexible the project officer and the fewer the number of required approvals. Many PVOs complain that obtaining implementation approvals is time consuming, cumbersome, and leads to unnecessary project delays. They note they often must obtain follow-up approvals for actions approved during negotiations. Among items frequently requiring such double approval are procurements, personnel compensation, selection of trainees, overall project work plans, and plans for specific project activities. Difficulty in obtaining approvals arises, for example, when Mission approval is required for individual subgrants made by a PVO acting as an umbrella grantee. According to the Procurement Policy office, however, USAID approval of individual subgrants is not required under OMB Circular A-110, so long as the description and budget for the grant or cooperative agreement include provisions for subgrants. USAID staff and implementers expressed confusion as to whether and when advance approval is required for international travel--even though the requirement for advance travel approval was simplified in 1992 and reiterated to USAID staff in 1994. According to the revised policy, recipients must notify the USAID project officer in advance of international travel plans, but they need not obtain formal approval before beginning travel. For USAID/W-funded activities, the project officer is responsible for checking with the Mission(s) and notifying the implementer if travel is not approved. Disapproval is expected to occur only on an exceptional basis for example, for security reasons. Some PVOs suggest that USAID provide clarification on which approvals are mandatory (either required by law or part of USAID regulations) and which are subject to flexibility on the part of the project or contracts officer. These PVOs also say clarification is needed on which USAID officer is authorized to issue approvals. In addition, several USAID staff members suggest the Agency review and, if appropriate, approve the implementer's overall system (for example, the procurement or travel or personnel system) rather than requiring approvals for individual actions under the system. In August 1994 USAID announced several reforms in administering grants and cooperative agreements (see box, next page). They deal primarily with approval and reporting requirements. One change is new procedures for approving recipients' financial systems and procurement, personnel, and travel policies so that additional approvals during implementation will no longer be required. Reports Requirements for progress reports are specified in OMB Circular A-110 and form part of USAID Handbook 13, chapter 1. Recipients are required to submit a progress report for each grant or cooperative agreement. The report presents a brief comparison of actual accomplishments with the goals for the period, reasons why goals were not met, and other pertinent information. According to the circular, progress reports are to be submitted for the same period and with the same frequency as required financial reports. Generally, this means that progress reports are required on a frequent basis. However, the revised OMB Circular A-110 eliminates the requirement for submitting these reports on the same schedule. That makes it possible for grantees to submit reports at more appropriate intervals, perhaps annually. The Agency incorporated the circular into its own directives in February 1995. Several PVOs say they would like less confusing, more consistent reporting formats and reporting systems that are compatible with those required for other purposes, such as for the host government. Some PVOs and NGOs suggest that USAID develop consistent reporting formats for each type of report. Overall, USAID staff say they find required progress reports useful. Among other things, they help staff monitor activity progress more easily and economically than through site visits. Many PVO and NGO respondents also find the reports useful for internal project management. Although most Agency project officers dispute this, a large majority of PVOs and NGOs report they receive little or no feedback from USAID on their reports, especially in regard to implementation concerns. Project officers who don't provide feedback say it is because they lack time to read the reports. This was verified by one PVO staff member, who spotted a pile of obviously unread reports from his organization stacked in a corner of his USAID counterpart's office. (But one USAID officer noted that no feedback was provided because "the reports are so good.") Some NGOs state that particular USAID Missions have streamlined reporting requirements. One NGO notes it can submit one five-page report covering all its USAID-funded activities. This has simplified the reporting process and made it more useful while allowing NGO staff to concentrate more on program activities. Another NGO is allowed to use its own reporting system rather than a format specified by the Agency. Still another submits the same progress and financial reports to USAID that it submits to its board of directors. Project Evaluations Although not an interview topic, many interviewees volunteered opinions about project evaluations. PVOs and NGOs generally seem to like having an evaluation requirement as part of their grant or cooperative agreement. NGOs, in particular, say that without USAID's help, they do not always have the resources to conduct evaluations, but that evaluation findings help them document project achievements and can assist with fund-raising. PVOs agree that they learn a great deal from evaluations, and they apply this learning to their organizations and to subsequent projects. Some PVOs suggest that midterm evaluations be undertaken for more activities. They see this as useful for modifying or correcting the course of the activity. In addition, some PVOs suggest conducting more end-of-project impact evaluations to establish a track record of PVO accomplishments and improve the design of future activities. Financial Management Financing Arrangements USAID uses several financing schemes, depending on the financial management capability of the recipient.8 The most restrictive and least convenient schemes are reserved for those that do not meet USAID's financial management standards. The methods of financing PVO and NGO activities, in order of preference, are letters of credit, advance by U.S. Treasury check, and reimbursement by Treasury check.9 Letters of credit can be used only by U.S. organizations that meet a number of requirements, including total yearly USAID advances of at least $50,000 and a continuing relationship with USAID for at least one year. U.S. PVOs that do not meet the requirements for letters of credit, as well as NGOs, are considered for the "periodic advance" payment scheme. Recipients must meet the financial management requirements outlined in Handbook 13, chapter 1, and eligible recipients are required to submit requests for advances at least monthly. NGOs and PVOs that do not meet the requirements for either a letter of credit or periodic advances must follow cost reimbursement procedures. Financial Reports Each payment method has its own financial reporting requirements. For letters of credit, two quarterly financial reports are required, the Financial Status Report (SF-269) and the Federal Cash Transactions Report (SF-272). U.S. recipients of periodic advances must submit the same two quarterly financial reports. However, non-U.S. recipients of advances instead are required to submit the Public Voucher for Purchases and Services Other than Personal (SF-1034 and SF-1034A) on a quarterly basis. In addition, the voucher must be accompanied by a Federal Cash Advance Status Report (format provided in the Standard Provisions). Under the cost reimbursement scheme, U.S. recipients are required to submit the Request for Advance or Reimbursement (SF-270) on a monthly basis. Indigenous NGOs are required to submit a voucher (SF-1034 and SF-1034A) on a monthly or a quarterly basis. About a third of PVO and NGO respondents say that monthly financial reports are required for their activities. They criticize this as inconvenient and inefficient. From the other side, Mission controllers in two countries argue that monthly financial reports are useful for keeping track of project activities. They say monthly reports usually are required when the implementer lacks strong financial management skills. Not surprisingly, most PVOs and NGOs say USAID's financial reporting requirements are excessive. In addition, many NGOs argue that reporting requirements are disproportionate to the level of funding and the nature of project tasks, since small grants have the same financial reporting requirements as large ones. Numerous respondents suggest streamlining financial reporting requirements or developing different financial accountability requirements for small grants. Financial Management Respondents generally agree that too much time is spent on financial management concerns. Most PVOs say they are not particularly impressed with the benefits of USAID financial management, primarily because they already have well-established financial systems. However, most USAID officers and many NGOs say that strong financial management is needed for most NGOs, particularly smaller, less experienced organizations. Most USAID staff feel USAID's attention to financial management has led to increased NGO accountability and to the overall institutional development of NGOs. Some NGOs say they have put together solid, effective financial systems to meet USAID's requirements. These systems, they believe, will serve them well in the future. There were numerous suggestions for improving financial management requirements. Contracts officers recommend establishing a central repository of financial management resources (like a library), so that NGOs don't have to reinvent the wheel every time they apply for USAID funds. They suggest that NGOs use these resources as their own preaward tool. Audits USAID's audit requirements are contained in OMB Circular A-133. It establishes uniform standards for single or organizationwide audits (as opposed to audits on a grant-by-grant basis). It also requires organizational audits for all U.S. and non-U.S. recipients of grants or cooperative agreements that receive $25,000 or more annually in federal funds. The same requirement applies to subrecipients. PVO and USAID staff had mostly negative comments about audits, but most NGO comments about audits were positive. Respondents generally find the quality of audits satisfactory, however, and many implementers say that audits help improve their financial management and strengthen their organization's internal controls. In many cases, project officers do not know the results of audits. They are notified by the controller's office only if problems are found. USAID staff who read audit reports say audits help their management of PVO/NGO activities. They note that audits provide legitimacy, help the organization develop its institutional capabilities, and help the project officer determine what needs to be done, since problems are identified and recommendations can be made and implemented immediately. The major complaint of interviewees is the expense of conducting audits for organizations that receive small amounts of funding. Several respondents provided examples of audits costing as much as or more than the grant itself. Owing to concerns about audit costs for small awards, USAID in May 1994 relaxed its requirements for A-133 audits of non-U.S. NGOs. As a result, NGOs that receive direct USAID awards now must comply with A-133 audit requirements only if they receive $100,000 or more in USAID funds under one agreement annually (or more than an annual total of $250,000). The threshold for U.S. PVOs and for NGO subgrantees remains at $25,000; however, according to the Procurement Policy office, the subgrant audit provision may be changed soon. Conclusions Interviewees identify several factors that appear to be related to implementation success: well-developed activity designs, work plans and budgets, and collaborative, competent USAID and implementer staff. USAID and PVO/NGO professionals would do well to keep these factors in mind as they develop new activities. More specifically, PVOs and NGOs are concerned about the number of USAID staff they must regularly deal with. They would prefer to have one contact person, preferably the project officer. USAID Missions should investigate the possibility of establishing a one-stop shop for each activity. For this to succeed, however, roles of Mission staff must be clearly defined. Implementers also are concerned that USAID regulations are too complex for small organizations to digest quickly. In addition, many NGOs say they receive little or no administrative orientation from the Agency before the start of their activity. USAID's procurement reform task force is working on reforming many of the Agency's procurement requirements. In the meantime, regular orientations by USAID staff would help organizations new to USAID better understand and respond to the administrative requirements of their agreements. Many implementers say they do not understand USAID's approvals "system." For example, it's unclear which approvals are mandatory and who can grant approvals. Implementers argue that too much seems to depend on project officers' personal preferences. USAID reporting requirements and the time spent on them also are seen as excessive. In this regard, the Agency recently made several changes in key implementation approvals and reporting requirements. That should eliminate the need for too many repetitive implementation approvals. Another way to simplify PVO/NGO reporting requirements is for USAID to develop standard reporting formats for each report type. This would eliminate agreement-specific requirements and allow implementers to complete the same report for all USAID-funded activities. Many PVOs and NGOs volunteered comments about the importance of project evaluations. They believe USAID should provide more funding for midterm and end-of-project evaluations as an integral part of grants and cooperative agreements. That would provide a basis for making midproject corrections, measuring project results, and providing guidance for future programs. Project officers should include sufficient funds for project evaluation in activity budgets and make sure resultant evaluation reports are widely disseminated in the PVO/NGO community, the donor community, and within USAID (as by providing CDIE copies of evaluations that can be made available electronically). Although many PVOs and NGOs say USAID's financial reporting requirements are excessive and time consuming, these comments presumably come from the one third of respondents who say their organizations are required to submit monthly financial reports. (Monthly reports are a requirement of the "reimbursement" method of financing, which is reserved for implementers that do not meet financial management criteria for obtaining fund advances.) Frequent reports of financial management problems indicate USAID staff should work more intensively with PVOs and NGOs to improve their financial management skills--for example, by providing small awards for development of acceptable financial management systems or by offering periodic training sessions. NGOs generally see audits as a positive management tool, even though implementers in general complain about the frequency and expense of audits, especially for small awards. However, USAID recently has relaxed its audit requirements for NGOs receiving direct USAID funding. 4. OTHER CONCERNS This chapter examines several additional facets of the USAID-PVO/NGO relationship: sustainability; other donors' work with NGOs; use of financial intermediaries, or "umbrellas"; and PVO/NGO collaboration in establishing USAID development strategies. Information on these topics was collected through literature review as well as additional field interviews. Strategic Collaboration The August 1993 USAID-PVO task force (USAID and the Private Voluntary Community 1993) discussed how to improve the strategic collaboration process--in particular, how PVOs and NGOs can play stronger roles in shaping and defining the Agency's development priorities. One recommendation was that USAID develop and issue a framework for strategic dialogue among PVOs, NGOs, and USAID, including principles for a formal consultative process. In November 1993 Administrator J. Brian Atwood outlined the Agency's new principles on participatory development (Atwood 1993). Among the principles is practicing a "respectful partnership" with indigenous and U.S. organizations in defining USAID's global objectives and country strategies. During data collection for the study, interviewers gauged the quality of the partnership by asking PVOs, NGOs, and USAID staff about the extent of PVO and NGO involvement in the formation of the Agency's development strategies in the U.S. and developing countries. Evaluators also asked about the organizations' satisfaction with the current level of input. Conducted from December 1993 through May 1994, interviews came shortly after the participatory development principles were issued. Overall, it is apparent that USAID still tends to form its development strategies without extensive consultation with PVOs or NGOs. But there were numerous observations that the climate for strategic collaboration has changed dramatically in recent months. A much more positive dialogue between USAID and PVOs/NGOs has developed. USAID senior managers who were interviewed support these observations. In general, senior managers feel that PVOs have had a reasonable level of influence in the formation of USAID's strategic plans, although probably not so much as the PVOs would have liked. NGOs, by contrast, have had far less influence. A majority of the senior managers feel the Agency needs to increase its efforts in this area. Although a formal consultative process has not been widely adopted throughout the Agency, there were promising examples of Missions holding formal meetings with PVO and NGO counterparts to obtain input on country strategies. In addition, many PVOs and NGOs feel they have an informal influence on country development strategies through contacts with Mission staff. Many USAID staff cited specific instances in which PVOs or NGOs provided them with significant information and guidance that resulted in changes or improvements to Mission development strategies. Many interviewees note that when positive, collaborative dialogue occurs, it is due to the initiative of USAID staff. They note too that a wide range of consultation practices exists, depending on the preferences of the USAID project officer. Overall, most PVOs and NGOs say they would welcome USAID efforts to initiate more dialogues with PVO and NGO counterparts while developing country strategies. PVOs About two thirds of the PVOs interviewed say they have some degree of input in the formation of USAID's development strategies, either in the United States or in a developing country. Barely half, however, say their input level is satisfactory. Many PVOs praised the efforts of InterAction and the Advisory Committee on Voluntary Foreign Aid to represent them in the USAID-PVO dialogue. In general, PVOs are more concerned about increasing strategic collaboration than NGOs are. About two thirds of USAID staff say PVOs have some input in USAID strategy development. USAID staff are a little more likely than PVOs to say the level of PVO involvement is satisfactory. However, some USAID staff feel PVOs already have too much influence on USAID policy formulation, especially given new policy guidance that encourages direct work with NGOs. These staff believe PVOs tend to promote their own agendas. NGOs There is some discrepancy between perceptions of NGOs and USAID on NGO input into USAID's country development strategies. While more than half of the NGOs interviewed say they have no input into USAID's country development strategies, about the same proportion of USAID staff say that NGOs do have some influence. In any case, neither NGOs nor USAID are satisfied with NGOs' current level of consultation; more than half of NGOs and USAID staff say the level of input from NGOs is not satisfactory. In general, NGOs don't believe USAID appreciates what they can bring to a strategy dialogue. USAID staff dispute that assertion, saying they appreciate and willingly consider the views of smaller, specialized NGOs that know their country settings well. Several NGOs note that their own communities should be better organized; they would have a greater chance of making a difference if they could present themselves as a united front. Several NGOs and PVOs suggest that USAID serve as a host or mentor for country networks of PVOs and NGOs. Sustainability When designing this assessment, USAID staff gave special attention to the issue of sustainability. This section summarizes findings from interviews with NGO, PVO, and USAID staff; a field survey of more than 20 completed projects; and an extensive literature review. Topics covered here are cost sharing, project design considerations, and implementation concerns, including project management and the role of monitoring and evaluation. Appendix D1 contains a fuller discussion of these issues. In field interviews, PVOs and NGOs distinguished three types of sustainability: benefit, organizational, and financial.10 Defining "sustainability" as the continuation of benefits is the most comprehensive view; it also is the definition most often used by PVO and NGO respondents and is considered the most prevalent in USAID programs. However, both PVO and USAID staff recognize that achieving benefit sustainability takes a great deal of time, effort, and resources. Many PVOs and NGOs say that achieving benefit sustainability takes from 10 to 20 years. PVOs and NGOs argue that the USAID programming cycle runs counter to this need, since project funding averages around three to five years. Cost Sharing The cost-sharing concept calls for PVOs and NGOs to make financial or in-kind contributions to the overall costs of a project. For some time, USAID's Handbook 3 has included a 25 percent cost-sharing requirement for awards to U.S. and indigenous organizations registered as PVOs. PVO, NGO, and USAID interviewees overwhelmingly agree that USAID should require cost-sharing for PVO/NGO activities. Respondents who favor cost-sharing in general, however, oppose it under certain circumstances, such as unique or difficult country settings or for young or small organizations. In general, respondents believe USAID should have flexible cost-sharing guidelines. Some suggested that the Agency review its cost-sharing requirements on a case-by-case basis. Interviewees recommend that cost-sharing requirements be established at the Mission level, with enough flexibility to take into account a PVO's or NGO's capacity and the activity's goals and objectives. In some cases, USAID may want to provide full funding (without cost sharing) to an organization because it is implementing a USAID initiative or because it simply is unable to contribute to costs. In July 1994, in response to recommendations of a joint PVO-USAID task force, USAID eliminated the 25 percent cost-sharing requirement for registered PVOs. Instead, all PVOs and NGOs will follow USAID Policy Determination 16, which encourages the "largest reasonable and possible financial participation" of recipients. In collaboration with PVOs and NGOs, USAID staff now have the flexibility to develop situation- or organization-specific cost-sharing formulas for PVO/NGO activities. Project Design Considerations Asked if they believe sustainability is an appropriate goal for USAID-funded PVO/NGO activities, interviewees overwhelmingly said "yes." PVOs and NGOs feel strongly about achieving greater sustainability, and they made a variety of suggestions for improvements. Among them: involving the target community, ensuring that beneficiaries are decision-makers, emphasizing empowerment of women, using community volunteers for project implementation, coordinating with appropriate government programs, developing management information systems, and ensuring adequate training. Project planners need to address sustainability early in the design of new activities. They should include such issues as -- Level of community demand for the project's benefits -- The project's probable contributions to sustainable development of the local area -- Host government attitudes toward NGO activities -- Project benefits that must be sustained by the local community Field case studies indicate that beneficiary participation in the design and implementation of projects is an important ingredient of success. Beneficiaries, after all, have the most to gain or lose from an activity. Their participation increases the probability that interventions will be relevant to local needs. It also provides a sense of local ownership. PVO/NGO implementers also need to work in partnership with community groups, private associations, local government, and other donors. Respondents stressed that the implementation effectiveness of a PVO or NGO is critical to sustainability. USAID project planners must consider a number of factors when selecting an implementer. Some topics suggested for the screening process are the organization's leadership, experience in the sector, current funding sources, ability to recover its program costs, number of years in the community, and whether the organization has a strong local constituency. In essence, program planners need to look for organizations (and leaders) with the following qualities: -- They have an entrepreneurial spirit -- They are motivated by the desire to create significant grass-roots change -- They are pragmatic and have problem-solving skills -- They are strategic planners able to develop management structures appropriate to carrying out their ideas USAID senior managers were asked if there are differences between PVOs and NGOs in achieving sustainability. The consensus is that neither PVOs nor NGOs are very successful in ensuring benefit sustainability. However, significantly more senior managers feel that NGOs are more successful in achieving sustainability and have far greater potential to do so. Project designers need to maintain realistic expectations of the time necessary to achieve results. Since benefit sustainability is a long-term process, activities often need to continue beyond the three- to five-year funding span normal for USAID awards. Also, prospects for self-funded project activities--and the phasing out of donor (and PVO/NGO) funding--should be considered during project design. Possible alternate funding sources are income-generating schemes, cost-recovery approaches (such as fees for service), use of endowments, and local fund-raising. Management Considerations The Agency's management of PVO and NGO activities can affect their sustainability. Management considerations include the Agency's rules and regulations, its short-term funding approach, attention to monitoring and evaluation, and measuring project impact. These topics are discussed more fully in appendix D1. Several case studies and interviews suggest that USAID's procedures and regulations can have a negative, indirect effect on sustainability--for example, if grantees are not permitted to adapt their projects to changing conditions. From the time of project conception, all partners need to agree on the activity's objectives. Objectives closely linked to the immediate needs of local communities have a greater likelihood of being sustained. Benchmarks must be established during project design to facilitate progress monitoring, and indicators that assess sustainable impact are particularly important for monitoring and evaluation. Umbrella Mechanisms During the design of this assessment, numerous questions were raised about USAID's increasing use of umbrella mechanisms for PVO and NGO activities,particularly as it relates to management efficiency. This section summarizes the major findings from the study's literature review and field interviews. A fuller discussion is found in appendix D2. An umbrella is a funding mechanism designed to deliver relatively small amounts of USAID funds to a number of organizations, such as PVOs and NGOs, through one financial award to a lead organization. Most commonly, a PVO receives a block grant award from USAID and subsequently makes smaller subgrants to NGOs or PVOs for project implementation. In addition to providing funds through subgrants, the lead organization often is responsible for providing administrative and technical assistance to subgrantees. Reported advantages to using umbrella mechanisms are numerous. They include reducing the Agency's management and technical assistance demands, since USAID deals with only one grantee; USAID funding of a wider range of local organizations, including those not registered with USAID; providing NGOs the opportunity to work in partnership with a more experienced lead PVO or NGO; and relieving subgrantees of some of the administrative requirements of direct grants. One possible disadvantage of using an umbrella mechanism is that USAID may not receive "credit" for funding subgrant activities. From the NGO perspective, umbrella grants may hurt their ability to develop long-term relations with donors, since the lead organization can compete for funding and visibility. Umbrella Models There are several models for managing umbrella grants. The lead organization may be a contractor, a PVO or an NGO, or an association of PVOs or NGOs. The most common arrangement is use of a PVO, although some larger or more experienced NGOs also serve as lead organizations. USAID/Bangladesh has developed a range of options for working with NGOs, including use of umbrella activities. Although the Mission has direct funding relationships with only 12 PVOs/NGOs, through these agencies USAID resources reach hundreds of indigenous NGOs. For example, the Mission's Family Planning and Health Services project provides direct funding to five organizations (two PVOs, two indigenous NGOs, and an international NGO). They in turn provide funding to 106 indigenous NGOs operating at more than 300 project sites. A number of factors should be considered in choosing a lead organization. They include characteristics of the local NGO community, the presence of PVOs and the nature of their local relationships, and the compatibility of USAID's objectives with those of the lead organization. In general, NGOs are likely to be more familiar with local needs, whereas PVOs tend to be better managed but more expensive. Project Management Concerns A majority of USAID staff believe that umbrella arrangements improve activity management and that using an umbrella works well in controlling costs. The cost efficiency of umbrella activities is a complex issue, though, and has not been adequately addressed in the literature. USAID senior managers generally agree that most Missions prefer funding NGOs directly rather than through an umbrella arrangement, which is usually managed by a U.S. PVO. But the reality of Mission management capacity (or lack of it) to work directly with NGOs often limits this preference and leads in most cases to use of a PVO intermediary. One problem cited by Mission staff is their limited ability to monitor technical quality of umbrella subgrant activities, which they say is necessary when the lead organization does not have this capability. Several implementers argue, however, that Mission staff need to let go of their overinvolvement, since monitoring subgrants is the responsibility of the lead organization. It seems reasonable to expect USAID to select lead organizations with adequate monitoring capabilities. Lack of coordination and communication problems also were cited by PVO, NGO, and USAID interviewees. Respondents note that poorly defined roles and lack of communication between the lead organization and subgrantees may lead to unsuccessful programs. In addition, many NGOs feel isolated from lead PVOs that manage subgrants from the United States. They say they are not kept informed about grant-related issues. PVO and NGO respondents recommend developing clear lines of authority and clearly defining the roles of USAID, the lead organization, and subgrantees. In addition, requiring PVO lead organizations to have local offices or representation may improve communications with subgrantees. Many USAID respondents expressed concern about financial accountability for subgrantees. They believe subgrantees' finances are not always being monitored properly. Project Impact and Capacity Building According to interview respondents, the most critical factor leading to successful umbrella projects is the competence of the lead organization. Respondents also say USAID needs to devote more time, attention, and resources to selecting the lead organization. Essential characteristics include technical assistance skills, ability to choose appropriate subgrantees, adequate management capacity to monitor and oversee subgrants, and ability to work in partnership with the local NGO community. Capacity building is seen as increasingly important in umbrella activities. It takes many forms: networking, fund-raising training, guidance during strategic planning, guidance on USAID's systems and procedures, and provision of technical, management, and membership-development assistance. An effective partnership between the lead organization and subgrantees is seen as critical to capacity building. The partnership should be a learning relationship for each organization. The lead organization should become increasingly confident in its mentoring role. For its part, the NGO subgrantees should gain new technical competencies, adopt new management practices, improve monitoring and evaluation systems, and master new methods of project design and implementation. Two Approaches Compared During field interviews, one team visited two Missions, each with an umbrella project. The projects have similar purposes: to increase the quality and quantity of PVO/NGO activities in priority sectors and to develop indigenous NGO capacity in those sectors. Team members compared the approaches used in these two umbrella activities, examining management arrangements and assessing success in developing indigenous NGO capacity. They discovered that although the Missions' approaches differ, both projects require substantial Mission management time. Neither project appears to emphasize NGO capacity building to the extent envisioned when each project was designed. One project uses a U.S. foundation as lead organization under a contract arrangement. The role of the lead organization is to provide training, technical assistance, and grants management for subgrantees; the lead organization does not directly implement project activities. In theory the use of one lead implementer should reduce the Mission management burden. This has not been the case, though, since Mission staff review and approve all subawards made under the contract. The team did not ascertain the exact breakdown of subawards between U.S. PVOs and indigenous NGOs, but it believes the PVOs have received a disproportionately large amount of project funding. There are many reasons for this, but the overall effect has been to deemphasize the project purpose of NGO capacity building. The second project works with eight U.S. PVOs as lead organizations. These groups have a dual role. They directly implement project activities, and they grant funds to 12 NGO partners. Although the lead organizations handle many project management details, the Mission also has significant responsibilities in managing eight direct grants. Because the lead organizations are directly responsible for implementing their own activities, this appears to be their priority. Only a small percentage of their grants are passed on to their NGO partners for project implementation; thus, they devote only limited time to capacity building for the indigenous groups. Other Donors' Work With NGOs During assessment planning, evaluation designers felt it would be useful to obtain information on the work of other multilateral and bilateral donors, to supplement findings on USAID's work.11 Data were collected primarily through field interviews in eight countries with other donors, U.S. and indigenous NGO representatives, and USAID officials. In addition, evaluators conducted a literature review of the management practices of European, Canadian, and multilateral donors. A fuller discussion of this topic appears in appendix D3. Donor Programming Approaches Although donor program objectives vary, there is considerable similarity among their NGO programs and substantial overlap in the reasons donors choose to work with NGOs. These include beliefs that NGOs -- Apply low-cost solutions and maintain streamlined, cost-effective services -- Have innovative and flexible approaches to improving local conditions -- Have a good sense of the needs of local communities -- Employ a high degree of participation in project implementation -- Successfully work to empower indigenous NGOs -- Work well in poverty reduction -- Reach less accessible communities and individuals--the poorest of the poor Donors vary greatly in the extent to which they invite NGO involvement in strategic planning. Although most pay some attention to it, the effect of NGO involvement on donor programming strategies seems to be limited. In field interviews, NGOs in general say they appreciate the overtures made by USAID and other donors to include them in their strategic planning processes. However, they report limited participation in actual strategic planning sessions. A few larger NGOs actively participate in country-level strategic planning with donors and are encouraged by their level of participation and impact. Indigenous and U.S. NGOs would like to see donors take a proactive approach to understanding their organizations' goals, objectives, and needs, and then to fund these, rather than predesigned projects that fit a donor's overall development strategy. They argue that the donor-implementer partnership should incorporate NGO involvement in every aspect of the development process: strategic planning, needs assessment, project design, implementation, and evaluation. By developing a broader partnership, U.S. and indigenous NGOs say, USAID and other donors could achieve better, more sustainable development results. Funding Arrangements Few donors set aside large amounts of money for directly funding indigenous NGO activities. Instead, they generally channel funding through NGOs based in their home country. These NGOs in turn team with indigenous NGOs. The donor NGOs work with indigenous NGOs in a variety of capacities, such as awarding subgrants, providing technical assistance, or jointly implementing development projects. In interviews, U.S. and indigenous NGOs criticized the short-term funding arrangements of many donors. U.S. groups pointed to this strategy as an impediment to their own long-term strategic plans; many argue that longer-term funding is essential for successful, sustainable projects. U.S. groups also expressed concern about the practice of USAID and other donors of funding discrete activities rather than entire NGO programs. Several funding mechanisms appear standard, including matching grants, block grants, and umbrella mechanisms. Many donors require NGOs to contribute to projects as a way of demonstrating commitment or developing financial independence. The typical cofinancing mechanism among other donors is a 50-50 matching grant. Consortium funding is a promising approach used by the Swedish International Development Authority, the Ford Foundation, and a few others. Donors contribute to a single fund, so that NGOs receive and administer funding in a simplified manner. Standard formats for financial reports, progress reports, and project monitoring are used, and the recipient sends the same reports to all donors. Project Design and Implementation U.S. and indigenous NGOs agree that donors need to give greater attention to local input before or during project design, saying that NGOs should be consulted about the needs of local communities. Although several donors have tried to include NGOs in the design stage, these efforts often have little effect because of practical limitations--inflexibility of donor funding mechanisms, limited staff resources available for collaboration with NGOs, and continued resistance of some governments and NGOs. Donors interviewed for the study frequently praised USAID's rules and regulations. They commented that Agency procedures hold NGOs accountable for project implementation and for meeting goals, requirements that many donor programs inadequately address. In many respects, donors view USAID's procedures as good standards to follow. U.S. and indigenous NGOs had mixed reactions, however, when comparing USAID's procedures and regulations with those of other donors. They overwhelmingly agree USAID's procedures and regulations are stricter; they perceive both positive and negative consequences of that strictness. Many U.S. and local NGOs observed that USAID gives more attention than other donors to monitoring and evaluation activities. Nonetheless, they repeatedly asked for more support to develop their capacity to undertake data gathering, analysis, and evaluation activities. Implications for USAID In comparison with other donors, USAID's U.S. and indigenous NGO programming has a number of widely recognized strengths, including USAID's in-country presence and its direct relationship with implementers. USAID's in-country presence facilitates more frequent interaction with NGOs and greater speed and flexibility in decision-making. Many U.S. and local NGOs believe other donors should adopt USAID's strategy of directly funding indigenous NGOs, as it increases opportunities for strategic partnerships in development. However, three weaknesses in USAID's programming were sometimes mentioned: 1) the Agency's lack of attention to capacity building and sustainability, 2) its short-term approach to project funding, and 3) excessive attention to detail and process. Conclusions Although PVO and NGO participation in setting USAID's country and global strategies is far from ideal, the climate for strategic collaboration recently has improved. PVOs, NGOs, and USAID staff all report more opportunities for dialogue with Missions and USAID/W. To maximize opportunities for PVO/NGO consultation, however, USAID/W and Missions should develop explicit, formal consultation strategies. Interviewees overwhelmingly support the concept of cost-sharing for PVO/NGO activities but feel the 25 percent cost-sharing requirement for registered PVOs should be relaxed. In point of fact, the requirement was eliminated in July 1994; the Agency now calls for the "largest reasonable and possible financial participation" of recipients. Study results confirm that sustainability should be considered early in project design. To the extent possible, projects should have flexible designs that allow revisions based on changing environments. In addition, extensive beneficiary participation in design and implementation, the involvement of local leaders and community groups, and an understanding of the capabilities of the proposed implementer are essential to benefit sustainability. The study confirms that umbrella activities can be a useful way to reach more NGOs than traditional projects afford. Well-designed umbrella projects also can minimize the USAID management burden. During project design it is important to consider the need for NGO capacity building and to carefully consider the characteristics and qualifications of the project's lead organization. When appropriate, USAID project officers should make NGO capacity building an explicit objective of umbrella activities. They also should ensure adequate levels of human and financial resources to address capacity-building concerns. In addition, to minimize the USAID management burden, the lead organization should have well-developed skills for selecting and monitoring subgrantees. USAID can assist with the development of NGO capacity in several ways. When appropriate, USAID projects can emphasize NGO capacity building as a project objective, including adequate human and financial resources in activity budgets to ensure progress. Project officers also should consider longer-term funding, when appropriate, to address capacity building and sustainability concerns, as well as participation in in-country funding consortia with other donors. The Agency can build on its ability to directly fund indigenous NGO activities by adopting a conscious strategy to do so in USAID priority sectors. 5. MANAGEMENT RECOMMENDATIONS Because the purpose of this assessment is to improve USAID's management of PVO/NGO activities, the following recommendations are aimed primarily at policymakers and project managers. There are obvious implications for PVO and NGO implementers, however, and suggestions for them are outlined at the end of this chapter. Cross-cutting Themes These general themes emerged from the study: PVOs have an experience edge over NGOs. Conventional wisdom says that, in general, PVOs are more experienced than NGOs in doing development work and managing USAID funds. This study confirms that. Although a great deal of diversity exists in the PVO and NGO communities, overall PVOs and NGOs do seem to have different levels of organizational development, implementation capabilities and implementation preferences. Reflecting their perceived shortcomings, NGOs are interested in having USAID staff assist with building their financial management capabilities and their project implementation skills. PVOs--which generally have a wider repertoire of skills--are more capable of (and prefer) implementing their activities with minimal USAID assistance. The USAID "system" is confusing. PVOs, NGOs, and USAID staff all have difficulty figuring out how the Agency's "system" works--if indeed there is such a system. Confusion prevails over what actual legal or Agency requirements are, as opposed to individual Mission, contracts officer, or project officer preferences. "It depends on the project officer." As a result of the above, interviewees frequently commented that successful implementation depends more on the personality or management style of the USAID project officer than on application of specific regulations or procedures. On the one hand, can-do project officers display trust and a collaborative attitude, and they try to cut red tape. On the other hand, some project officers believe successful implementation is directly related to the amount of control they have over the project activity. They insert frequent or unnecessary control measures, such as additional approvals and reports, into the implementation process. PVOs and NGOs empathize with USAID staff. PVOs and NGOs showed a surprising amount of empathy and understanding for the lot of the USAID project officer. They believe that USAID project officers would like to be more involved in the substance of their activities, but that they generally have to manage too many projects and are overwhelmed by the Agency's administrative demands to be effective managers. "Collaboration, not strangulation." PVOs and NGOs stressed they would like to collaborate with their USAID colleagues on programmatic matters. NGOs, in particular, are interested in technical assistance and assistance in building local networks of PVOs and NGOs. But implementers, particularly PVOs, said that USAID staff often interpret "collaboration" and "substantial involvement" to mean "micromanagement" and "more administrative control." Implementers said that Agency staff get lost in myriad administrative details and requirements. As a result, they lack time to provide the substantive programmatic support PVOs and NGOs desire. Inconsistent Management Over several months of interviews, it became clear that a pervasive problem affects the Agency's management of PVO/NGO activities: inconsistency of USAID staff in managing grants and cooperative agreements. Although staff appear knowledgeable about the technical differences between funding instruments, in practice these differences often are ignored. Preferences of individual project officers, rather than requirements of specific funding instruments, seem to determine how individual activities are managed.12 The following recommendations pertain to development of USAID staff capacity to manage PVO and NGO activities more consistently and equitably. Reinforce collaborative management. Senior managers should make a determined effort to communicate to staff management principles that reinforce the importance of collaboration with USAID's development partners. This could be accomplished by updating directives or by broadcasting Agencywide notices, for example. The Bureau for Policy and Program Coordination should take the lead in these efforts. Get incentive system right. As part of USAID's reinvention efforts, senior managers should develop an incentive system to recognize and reward hands-on, results-oriented project officers for excellence in working in partnership with PVOs and NGOs. The Office of Human Resources in the Bureau for Management should take the lead in developing such an incentive program. Provide additional training. Additional staff training, particularly of project officers, is needed to ensure there is reasonable consistency worldwide in applying USAID regulations and procedures. Training efforts could include collaborative, results-oriented project management and could aim at improving staff's "people" skills. The Office of Human Resources should take the lead in developing or revising Agency training courses. Certify project officers. The Agency needs to develop a certification program for project officers (similar to that for contracts officers) to ensure that project staff have the needed skills and knowledge for managing PVO/NGO activities. The Office of Human Resources should take the lead in this. Recommended Policy Changes Most of the study's recommendations, outlined in later sections, point to actions that require no changes to Agency policies and procedures. The following recommendations, however, do require policy review: Simplify NGO registration. Current legislation requires the Agency to register indigenous NGOs that meet the criteria for classification as PVOs. Given the Agency's emphasis on increased direct funding of indigenous NGO activities, registration requirements for NGOs should be reviewed and simplified, with only essential requirements retained. The Office of Private and Voluntary Cooperation in the Bureau for Humanitarian Response should take the lead in this effort. It would be assisted by the Bureau for Policy and Program Coordination and the Office of the General Counsel. Consider Mission registration of NGOs. The Agency also should consider the feasibility of direct Mission registration of NGOs, with Missions providing reports on registration activity to USAID/W for central data management. The Office of Private and Voluntary Cooperation should take the lead, working with the Bureau for Policy and Program Coordination and the Office of the General Counsel. Simplify preaward reviews. The Agency's preaward review procedures need to be reviewed to determine whether modified preaward reviews could fulfill all or some of the local NGO registration requirements. The Bureau for Policy and Program Coordination should take the lead in these efforts, assisted by the Office of Private and Voluntary Cooperation, the Office of Procurement, and the Office of the General Counsel. Clarify OMB requirements for NGOs. The Agency has determined that OMB circulars pertaining to management of grants and cooperative agreements (in particular, OMB Circulars A-110 and A-133) apply to both indigenous NGOs and U.S. PVOs. Many respondents indicated that smaller, newer NGOs can organize effective programs but have difficulty meeting OMB management requirements. Keeping in mind the importance of sound management principles and recognizing the need for NGO institutional strengthening, the Agency should review OMB requirements for the management of grants and cooperative agreements. The review would determine which requirements are essential for application to indigenous NGOs. The Office of Procurement should take the lead in this task. Explore wider use of donor consortia. To provide more funding options for indigenous NGOs and to streamline their reporting requirements, USAID should investigate the possibility of joining or establishing country-level donor consortia to provide joint funding for indigenous NGO activities. Using consortium funding would allow NGOs to greatly simplify their reporting and recordkeeping responsibilities. They could use one accounting system to track all donor funds and could prepare one set of financial and progress reports for submission to all donors. The Bureau for Policy and Program Coordination should take the lead role in the exploration of donor consortia. Immediate Management Actions In addition to addressing the problem of USAID officers' inconsistency in PVO/NGO grant management, many other changes can be made to improve management of PVO/NGO activities. The recommendations below are grouped according to proposed implementer. Suggestions in this section can be carried out by Mission managers, contracts officers, project officers, or other Agency staff without revisions to existing policy or procedures. Senior Management Actions Empowerment and capacity building. Senior managers--at field Missions or in USAID/W--can help strengthen the Agency's partnership with the PVO and NGO communities in a number of ways. 1. Formalize regular consultations. PVOs are well known for their international development expertise. As suggested in the 1993 Principles of Participation, senior managers should systematically tap the expertise of PVOs by including their managers in discussions on the Agency's global and country development strategies. USAID senior managers also should encourage NGO participation in country strategy development by formalizing a process of local consultation. 2. Support use of mentors and networking. Senior managers can encourage NGOs to learn from more seasoned organizations by supporting PVOs and NGOs as informal mentors for newer or smaller organizations. Encouraging development of implementer networks among PVOs and NGOs also will help the learning process. 3. Build capacity. Senior managers should encourage project officers to consider capacity building as an explicit project objective. They should include appropriate human and financial resources for this purpose in activity budgets. New funding approaches. Senior managers can take at least two steps in developing new approaches to funding PVO/NGO activities: 1. Go the consortium route. To provide more stable, sustainable options to NGOs and streamline their reporting requirements, senior managers should investigate the possibility of joining or establishing donor consortia to provide joint funding for NGO activities. 2. Provide longer term funding. When capacity building is identified as an activity objective, senior managers should consider allowing longer-term funding (5 years or more) to the funding recipient. At the same time they should ensure that appropriate, measurable institution-strengthening goals are included in the activity design. Strengthening design and implementation. Senior managers can encourage stronger activity designs and smoother implementation in three ways: 1. Design benchmarks. During project design, senior managers can encourage project officers to establish clear development-oriented project benchmarks so that the activity is geared toward measuring results. 2. Use a team approach. Senior managers can encourage a team approach to project design and negotiation. 3. Designate a contact person. For the implementation of each activity, senior managers can ensure that one USAID point of contact is established for the PVO or NGO. That contact person is preferably the project officer. Office of Procurement Actions Strengthening design and implementation. The Office of Procurement can facilitate improvements in the Agency's design and implementation processes in several ways: 1. Set clear standards. The Office of Procurement should communicate to USAID staff the differences in operational controls among funding instruments. It should monitor the appropriate uses of each instrument. 2. Specify required approvals. The office also should clarify for project officers and contracts officers the mandatory implementation approvals. 3. Provide translations. To smooth the Agency's work with indigenous NGOs, the office should make available to contracts officers Spanish- and French-language versions of the standard provisions for grants, cooperative agreements, and other relevant project implementation guidance. The office also should consider making small amounts of funds available to Missions for translation of key regulations into other local languages. 4. Use standardized formats. Standardizing reporting formats for progress and financial reports for grants and cooperative agreements also would strengthen project implementation. 5. Encourage prompt incremental funding. To help ensure regular, timely fund flows to PVOs and NGOs, the office should encourage contracts officers and project officers to separate the work plan approval and incremental obligation processes for grants and cooperative agreements (as appropriate) and to expeditiously obligate incremental funds. Project Officer Actions Project officers can make immediate improvements in the Agency's work with PVOs and NGOs during design and implementation of individual activities as well as in other actions aimed at institutional strengthening or networking. Because Agency policies or procedures do not need to be changed, these improvements can be implemented immediately. Empowerment and capacity building. PVOs generally are experienced in implementing development activities and in managing USAID funds, but NGOs often have a difficult time meeting financial and administrative requirements. USAID can take definite steps to overcome these problems. In addition, many NGOs have asked for USAID's help in sponsoring or encouraging PVO/NGO networks that increase opportunities for collaboration, information exchange, and professional development. Project officers can help increase NGO capacity and support networking in the following ways: 1. Promote mentoring, networking. Project officers can foster NGO development by encouraging seasoned PVOs and NGOs to serve as informal mentors for newer, smaller organizations. They also can encourage development of networks of PVOs/NGOs working in the same country or the same sector. 2. Increase funding opportunities. Project officers can help the Agency broaden its range of partners by providing more funding opportunities for smaller, inexperienced NGOs or PVOs. Or they can set up programs exclusively for organizations that have never received USAID funding. For example, small grants can be given to help less experienced organizations develop needed skills (such as in financial management) for working with larger awards. 3. Seek new funding approaches. The Agency's traditional approach to PVO/NGO activities has been to fund discrete, short-term project activities. Although this approach provides flexibility in budget management for USAID managers, it doesn't encourage sustainable activities or sustainable implementer organizations. In addition to making small initial awards to new or inexperienced organizations, USAID project and contracts officers can consider using more support grants that is, providing funding to an organization's broad portfolio of activities instead of funding individual activities. 4. Bolster NGO financial management. Project officers can help strengthen NGO financial management skills by a) using preaward reviews to assess existing management capacity, b) starting with small initial awards, and c) arranging for experienced organizations (PVOs, NGOs, or contracted accounting firms) to provide financial management services to smaller organizations. 5. Provide better education. By providing regular orientation sessions, project officers and contracts officers can more effectively educate PVO and NGO implementers about the Agency's minimum accountability requirements and differences in funding instruments. These sessions should be held with NGO funding recipients both before signing funding agreements and during project implementation. A U.S. or indigenous accounting firm might be contracted to provide these services. 6. Expand use of umbrella arrangements. Project officers should consider more frequent strategic use of umbrella mechanisms, especially when institutional strengthening of NGOs is desired. The lead organization should have demonstrated ability to administer and monitor subgrants and provide technical management assistance to subgrantees. Communications problems with subgrantees and USAID will be minimized when the lead organization has an in-country office and when roles and expectations for all parties (including USAID) are spelled out in advance. 7. Intensify capacity building. Project officers can help strengthen NGOs and PVOs by including capacity building as an explicit project objective, when appropriate, and by including adequate human and financial resources for capacity building in activity budgets. Strengthening activity designs. PVOs and NGOs often complain that USAID spends too much time on project development and review. But the study found that attention to design is one of the Agency's strengths, particularly in comparison with other donors. Careful attention to reviewing and critiquing activity designs appears to increase the likelihood of successful activities. Among the ways USAID project officers can strengthen the design of PVO/NGO activities are these: 1. Think sustainability. The study's literature review suggests that attending to sustainability concerns during project design and eliciting participation of the beneficiaries increase the likelihood of success. 2. Develop ways to measure results. During activity design, USAID project officers should attend to establishing development-oriented project benchmarks. That will enable project officers (and implementers) to concentrate on activity results rather than on process. 3. Provide for evaluation. Project officers should regularly include funds for evaluation in activity budgets. Many PVOs and NGOs commented on the importance of regular evaluation as a management tool. Midterm evaluations allow for needed project modifications. Final evaluations, by contrast, a) measure project impact, b) enhance institutional wisdom, and c) help implementers build a track record of accomplishments. Having a wide range of PVO/NGO evaluations broadens the knowledge base for PVO, NGO, and USAID project planners. In this way, lessons learned from earlier projects can be incorporated into new activities. 4. Clarify respective roles. To encourage smooth project implementation, project officers should ensure that the roles of all parties, including USAID staff, are well defined before the funding agreement is signed. Implementation. USAID managers can work to improve project implementation (thus increasing the likelihood of successful activities) by 1. Following "prudent management." USAID project managers should follow the principles of prudent management, which emphasize placing the fewest necessary controls on implementers. Also, when project requirements are clearly defined and agreed to before the start of an activity, there is less need for micromanagement. The recipient can concentrate on carrying out the project. 2. Calling regular meetings. Many interviewees recommend regular meetings between the USAID project officer and implementers to improve communications and quickly resolve implementation problems. 3. Favoring prompt incremental funding. PVOs and NGOs often report delays in receiving yearly incremental funds. They complain they often have to submit miniproposals before receiving the next year's funds. According to the Procurement Policy Office, however, work plan review is not an OMB requirement for incremental funding. Given the "substantial involvement" aspect of cooperative agreements, USAID project officers should determine when it is absolutely necessary to tie incremental funding to work plan approval. They should avoid this approach in other cases. PVO/NGO Actions Although recommendations in this chapter are aimed primarily at USAID managers, PVOs and NGOs can take steps to strengthen their collaboration with USAID: 1. Use USAID information sources. PVOs and NGOs should become more knowledgeable about the Agency's rules and regulations, particularly recent and proposed changes. Information sources include Mission contracts officers, Office of Procurement staff, and Agency notices posted to the Internet. Groups of PVOs and NGOs, in the United States or in a host country, also can request regular briefings from Agency procurement officers to explain current procedures or planned changes. 2. Network. Other PVOs and NGOs can be good sources of information regarding the successes and problems of their own USAID funded activities. Discussing common problems is an excellent way to discover innovative solutions or to pinpoint systemic weaknesses in USAID policies or operating procedures. 3. Clarify funding agreements. Before signing a funding agreement with USAID, PVOs and NGOs should make sure they understand all the provisions. They should examine the procurement procedures, required approvals, financial management requirements, progress and financial reporting requirements, and the roles and responsibilities of each party. This is particularly important for organizations that have never before received USAID funding. Before signing--not after--is the time to clarify any ambiguities. APPENDIX A: USAID POLICY FRAMEWORK A number of sources of U.S. policy and guidance govern USAID's work with PVOs. The most fundamental is Section 123 of the Foreign Assistance Act, which authorizes a program to assist the development assistance activities of U.S. PVOs and cooperatives. USAID's assistance policy is based on the Federal Grant and Cooperative Agreement Act of 1977 and the Office of Management and Budget's (OMB's) final implementation guidance on that act. USAID's most recent PVO policy statement (USAID 1995) was issued in April 1995 replacing the September 1982 policy. Although the policy primarily is aimed at USAID's relationship with U.S. organizations, USAID's relationship with indigenous NGOs is addressed in several areas, and a separate policy on USAID's relationship with indigenous organizations is planned. The policy includes statements of policy principles regarding: consultation in establishing country development priorities; participation in developing and implementing activities; program integration and managing for results; U.S. PVO independence; support for relationships of U.S. PVOs and indigenous NGOs; capacity building; USAID-U.S. PVO cost-sharing; and administrative simplification. USAID issued new "principles on participatory development" in October 1993. These principles include practicing a "respectful partnership" with indigenous and U.S. organizations that collaborate with USAID in providing development and humanitarian assistance, and supporting the initiatives of indigenous communities and organizations in defining USAID's global objectives and country strategies. OMB Circular A-110, Grants and Agreements with Institutions of Higher Education, Hospitals, and Other Nonprofit Organizations, provides the basic standards for management of USAID-funded PVO activities. While OMB Circular A-110 is applicable only to U.S. organizations, as a matter of USAID policy it is applied to non U.S. organizations to the extent practicable. OMB Circulars A-122 (cost principles for nonprofit organizations) and A-133 (audit requirements) also apply to USAID-funded PVO and NGO activities. In addition, USAID's implementation guidelines for PVO and NGO activities are found in several Agency handbooks, primarily Handbook 13 (Grants), Handbook 1B (Procurement Policy) and Handbook 3 (Project Assistance). USAID Policy Determination 16, dated October 1987, outlines financing arrangements with independent organizations, including PVOs and NGOs. Handbook 3, Supplement A, Project Officers' Guidebook for the Management of Direct USAID Contracts, Grants and Cooperative Agreements, is a useful summary of information contained in the various Agency handbooks. APPENDIX B: SUMMARY OF RECENT AND PROPOSED REFORMS Recent Changes Customer service standards: PREVIOUS: No guidance provided. NEW: In August 1994, USAID developed and published its initial customer service standards covering three areas: quality (improving processes and simplifying business practices); timeliness (improving turnaround time for USAID processes); and greater access and more transparency to Agency activities and information. Specific reform targets were included for each area. (Source: Phase I Customer Service Plan: Establishing Quality Service Standards for Working with USAID's Development Partners: Private Voluntary Organizations, Universities, and Private Businesses, August 1994) Registration: PREVIOUS: To become registered, new applicants were required to submit 18 documents. Annual registration renewal required 6 documents. NEW: USAID's Office of Private and Voluntary Cooperation revised the U.S. PVO registration requirements in January 1995; the number of required documents has been reduced from 18 to 6 for new applicants, and from 6 to 3 for annual registration renewals. (Source: BHR/PVC materials dated 1/24/95) Cost sharing: PREVIOUS: USAID required PVOs to make a 25 percent contribution in order to receive an operational program grant or specific support grant. NEW: All PVOs and NGOs will follow USAID Policy Determination 16 (as revised), which encourages the "largest reasonable and possible financial participation" of recipients. (Source: Cable 1994 State 184498 dated 7/11/94) Systems approvals: PREVIOUS: There was no provision for approval of a recipient's management system. NEW: Procedures have been developed for approving recipients' financial systems and procurement, personnel and travel policies. (Source: USAID General Notice dated 8/8/94) Salary approvals: PREVIOUS: The policy for contracts (requiring special approval for salaries above the ES-6 [previously FS-1] level) often was applied to assistance instruments. NEW: Unlike contracts, however, there is no requirement under grants and cooperative agreements for approval of salaries above the FS-1 level. (Source: USAID General Notice dated 8/8/94) Individual consultants: PREVIOUS: Common practice was to require approval of individual consultants under a grant or cooperative agreement. NEW: No approvals are required for hiring individual consultants. (Source: USAID General Notice dated 8/8/94) Key personnel approvals: PREVIOUS: Assistance instruments often contained requirements to approve unlimited numbers of key personnel. NEW: No more than five persons, or 5 percent of the total number of employees, may be designated as key personnel, over which USAID is permitted to exercise approval. In addition, there is no requirement to submit multiple candidates for key positions. (Source: USAID General Notice dated 8/8/94) International travel: PREVIOUS: Recipients were required to secure approval from USAID before beginning travel. NEW: As of August 1992, the standard provision under assistance instruments requires the recipient only to notify the USAID project officer of international travel plans. (Source: USAID General Notice dated 8/8/94) Trip reports: PREVIOUS: Individual grants/cooperative agreements often required separate trip reports. NEW: Trip reports are to be included in regular technical (progress) reports, and not submitted separately. (Source: USAID General Notice dated 8/8/94) Audit: PREVIOUS: USAID required all NGO recipients or subrecipients of grants/cooperative agreements to comply with OMB Circular A-133 audit requirements if they received $25,000 or more annually in USAID funds. NEW: USAID and the Office of the Inspector General have increased the audit threshold for NGOs. Now, NGOs that receive less than $100,000 per year per agreement in direct USAID funding (up to a total of $250,000 per year) no longer are required to follow OMB Circular A-133 audit requirements. (Source: USAID General Notice dated 5/6/94) USAID codification of OMB Circular A-110 (administrative requirements for grants and cooperative agreements): These USAID-specific revisions were published in the Federal Register on 1/19/95 and were effective on 2/21/95. (Source: M/OP/POL) -- Additional requirements: PREVIOUS: No specific guidance. NEW: Prohibits imposing requirements additional to those outlined in A-110. -- Exceptions: PREVIOUS: No specific guidance re: less restrictive requirements. NEW: USAID may apply less restrictive requirements when making small awards (currently defined as less than $25,000). -- Funding instruments: PREVIOUS: Not covered. NEW: Provides guidance on the appropriate use of grants and cooperative agreements as funding instruments. -- Annual certifications: PREVIOUS: Not covered. NEW: USAID encouraged to accept annual certifications and representations on a recipient basis rather than an award-by-award basis. To avoid duplicate submission of these routine documents by prospective recipients, the Agency is exploring setting up a central repository for this information. -- Procurement: PREVIOUS: No specific requirement for written procurement procedures. NEW: Recipient must establish written procurement procedures that cover three minimum standards: that unnecessary items not be purchased; that a lease/purchase analysis be conducted; and that all solicitations meet certain minimum standards. -- Progress reports: PREVIOUS: Progress and financial reports were required with the same frequency. NEW: Eliminates the requirement for submitting progress and financial reports on the same schedule, allowing for progress reports to be submitted less frequently. -- Report copies: PREVIOUS: Not covered. NEW: Recipients will not be required to submit more than the original and two copies of progress reports. Proposed Changes "Reengineering": CURRENT: Procedures outlined in USAID handbooks and other official Agency guidance. PROPOSED: In November 1994, USAID completed its Operations Business Area Analysis report, which contains recommendations for restructuring the way USAID programs its development assistance. The Agency began implementing its "reengineered" operating systems on October 1, 1995. Key aspects of the new operating system include: participatory planning with USAID's partners and customers; focusing on the achievement of activity objectives; empowering customers and partners (that is, investing them with authority to make and implement decisions), as well as holding them accountable for those decisions; and the use of teams to achieve agreed-to objectives or results. (Source: Operations Business Area Analysis Core Report, November 1994) Principles for award of assistance instruments: CURRENT: Guidance as provided in USAID handbooks. PROPOSED: At its January 1995 meeting, the Advisory Committee on Voluntary Foreign Aid approved a set of principles for the award of assistance instruments (grants and cooperative agreements) to PVOs and NGOs. These principles, which were developed in collaboration with USAID's Office of Procurement, were forwarded to the USAID Administrator for review and action. The principles include: achieving a higher degree of standardization of policy and procedure in assistance to PVOs/NGOs; selection of grant or cooperative agreement; clarification of "substantial involvement"; and focusing on activity results rather than inputs or outputs. (Source: ACVFA January 1995 Quarterly Report) Proposed revisions to OMB Circular A-133 (audits): As of February 1996, these proposed changes had not yet been published in final. (Source: M/OP notice on Internet, dated 11/13/94) -- Application: CURRENT: USAID policy is that A-133 requirements apply to foreign recipients. PROPOSED: USAID will determine the appropriate dollar level of review for foreign recipients of awards. -- Audit threshold: CURRENT: $25,000 per year. PROPOSED: $250,000 in awards per year. -- Frequency of audit: CURRENT: Usually annually, but not less than every two years. PROPOSED: Annually. -- Timing of audits: CURRENT: Thirteen months after the end of the fiscal year. PROPOSED: Nine months after the end of the fiscal year. -- Major programs: CURRENT: Opinion on compliance with laws and regulations based on review of all major programs. PROPOSED: Opinion on compliance based on review of 50 percent of federal expenditures. -- Audit findings: CURRENT: Not covered. PROPOSED: Requires schedule of status of findings and corrective actions. -- Subrecipients: CURRENT: Not covered. PROPOSED: Increased prime guidance to its subrecipients; management decision on subrecipient audit findings. -- Quality managed: CURRENT: Not covered. PROPOSED: Two years of clean audit reports would result in less burdensome audits--i.e., testing of 25 percent of federal expenditures or A-133 audit every 2 years and GAGAS (generally accepted government auditing standard) audit every other year. APPENDIX C: METHODOLOGY This appendix describes the methodology used for data collection and analysis for the study's principal evaluation question and for supplementary studies on senior managers' views, strategic collaboration, sustainability, "umbrella" mechanisms, and other donors' work with NGOs. The original study design included an additional evaluation question on the effectiveness of PVO- and NGO-implemented activities. Data for that part of the study were collected through the review of 151 completed PVO/NGO activities and were analyzed under several statistical procedures. However, because of the technical complexity of the data analysis, to present a clear, understandable discussion of the findings, the results of that portion of the study are being published separately. Key Evaluation Question The purpose of this assessment is to critically examine the working relationship between USAID and the PVO/NGO communities. Thus, the study looks at how USAID does business--that is, the processes--as opposed to looking at the development impact of USAID-funded PVO and NGO activities. To examine the working relationship, 259 interviews were conducted with PVO, NGO, and USAID officials in the United States and eight countries where USAID works. The interviews dealt with a set of predetermined topics, and the findings were synthesized across locations. Data Collection Data collection instruments: To answer the key evaluation question, eight topics on management of USAID-funded PVO/NGO activities were developed during assessment design: location of the USAID project officer ("management locus"); choice of funding instrument; PVO registration; the negotiation process; general implementation concerns; approvals; reporting requirements; and financial management. These topics became the basis for the categories of questions used in the structured interviews. Four separate data collection instruments were developed, for different categories of interviewee, as shown in the below table. Interview form 1, the most frequently used instrument, is included in this appendix. (Four additional data collection instruments, developed for other study purposes, are discussed in the next section.) The data collection instruments were field-tested and refined during the first two country visits to Bangladesh and Poland. Subsequently, identical forms were used for data collection in the remaining six field visits and for U.S. interviewing. Country selection: To ensure a broad range of views would be captured in study interviews, plans called for field visits to at least six countries. Criteria for selecting countries for field interviews included: -- Wide geographic representation -- At least one country where USAID does not work directly with the host government--that is, where PVOs and NGOs are the principal implementers of USAID assistance activities -- At least one advanced developing country--that is, with a minimal USAID presence in-country -- At least one country where USAID has taken an innovative approach to using PVOs and NGOs for development activities >From these criteria, eight countries were selected for fieldwork: Bangladesh, Guatemala, Mali, Mexico, Poland, Senegal, Swaziland and South Africa, as well as the United States. Members of the Bangladesh field team made up the core team for subsequent field visits; at least one member of the Bangladesh team served on each of the remaining field teams, usually as team leader. As mentioned earlier, the Bangladesh fieldwork, in October 1993, and the Poland visit, in December 1993, served as opportunities for testing and refining data collection instruments. The remaining six field visits, which took place in January and February 1994, and the U.S. interviews, in May 1994, utilized identical data collection instruments. Interview procedures: The general purpose of the interviews was to obtain a range of views about how USAID manages PVO- and NGO-implemented activities, and to obtain suggestions for changes that might improve project performance. Field teams of two to four people spent two weeks in each country, interviewing PVO and NGO staff, USAID staff, and representatives of other donor agencies. Interviewees were selected using a "target of opportunity" approach, that is, team members attempted to interview as many PVO, NGO, and USAID staff as possible during the two-week field visit. USAID/Washington interviewees were selected on the basis of the extent of their work with USAID/W-funded PVO and NGO activities. A range of U.S. based PVO representatives was selected for U.S. interviews; criteria included geographic location (within and outside the Washington, D.C. area) as well as USAID-funded program size (those with small activities as well as those who receive extensive USAID funding). Two-person teams conducted most interviews; one team member served primarily as the questioner, with the other as the primary note-taker. The note-taker was responsible for word processing of interview notes at the end of the day. Composition of the interview teams was rotated on a daily basis, as were roles as interviewer and note-taker. Interviewers were provided the preprinted interview forms and instructed to ask the questions as worded and in the prescribed sequence, unless the circumstances of particular interviews dictated otherwise. For example, communication problems might require a restatement of a question using different words. Responses to some questions were closed-ended; others were open-ended. On open-ended questions, prompts were offered when the interviewee's response was more limited than expected. When interviewees were non English speaking, interviewers provided the necessary translation. Spanish-language versions of the pertinent interview forms were prepared in advance of the field work; however, due to time constraints, no advance French-language translations were provided to team members. PVO/NGO interviews were sometimes conducted with groups of respondents, usually a senior official plus one or more assistants, who provided more detailed answers to particular questions. Individual responses in group interviews were combined to yield a single interview protocol. Data Analysis Note-takers were responsible for entering the qualitative responses to study questions on WordPerfect computer files, and for maintaining hard copies of interview forms, which contained responses to closed-ended (yes/no or scaled) questions. To ensure confidentiality, an identification number was assigned to each interviewee, and subsequent data analysis used only the identification number. Because the interviews resulted in more than 400 pages of interview notes, a computer program, "Analysis of Qualitative Data" (AQUAD), was employed to help sort responses. From the predetermined interview categories, a scheme was developed for coding both the qualitative material and the closed-ended interview responses. Two research assistants reviewed the material from each interview and entered the appropriate codes into each interview file. AQUAD then was employed to sort the responses by category as well as to provide frequency counts of the closed-ended responses. This sorted information was analyzed to determine major concerns and themes. A table showing the coding scheme and some representative pages displaying AQUAD data are included in this appendix. Within each interview record, distinct comments on each topic were coded individually. For example, when a respondent made several observations on the subject of registration, each comment was coded separately. This procedure allowed more in-depth analysis of responses in each topic area. However, since a respondent may have made either no comments or several comments on an individual topic, frequency counts of the total number of responses in an interview category were not a useful way to determine response trends. Similarly, it was not possible to accurately determine the percent of respondents in each response category. Instead, the analysts looked at the pattern of responses in each interview category, and used less precise terms such as "nearly all," "most," "some," and the like to reflect response trends. Supplementary Study Areas In addition to the key evaluation question, information was collected on several other topics: the views of USAID senior managers; strategic collaboration; sustainability; use of "umbrella" mechanisms; and other donors' work with NGOs. As mentioned earlier, several other data collection instruments were developed for the study, primarily to gather information on issues not included in this report. But these data collection instruments also included questions related to the supplementary study areas. Information on the additional data collection instruments is shown below. Interviewers also conducted 17 unstructured interviews, for example, with PVOs/NGOs that do not receive USAID funding, or background interviews with U.S. Embassy officials. A summary of the data collection and analysis procedures for the supplementary study areas is shown in the table below. Additional information on each topic is provided in the following sections. Senior managers' views: As part of the larger study looking at the issue of USAID's working relationship with PVOs and NGOs, the views of senior USAID managers were gathered to complement those of USAID officers who worked directly with PVOs and NGOs. A total of 19 interviews were conducted with senior officers based in USAID/Washington and with Mission Directors and Deputy Directors based in USAID Missions overseas. Similar interview forms were used with both Washington and field-based senior managers. These were far less detailed than those used with PVO, NGO and USAID project managers, and were designed to elicit more broad-based perspectives and insights on a range of PVO/NGO and USAID programmatic and policy concerns. As with the study's principal investigation, two-person teams conducted the interviews, and the results were entered onto computers. Because of the small number of respondents, data were analyzed manually on a topic-by-topic basis. The findings are summarized in Appendix E and are cited at appropriate points in the main study report. Strategic collaboration: The topic of "strategic collaboration" explored the extent to which PVOs and NGOs were consulted during development of USAID's global and country assistance strategies. Responses on this topic were received from 291 interviewees: 146 PVO/NGO representatives (68 PVOs and 78 NGOs); 124 USAID staff (94 project officers, 11 contracts officers and 19 senior managers); and 21 other donor representatives. These interview responses were sorted through use of the AQUAD program and then analyzed. Sustainability: Information on sustainability was collected from a literature review of approximately 51 documents as well as responses from 308 interviewees. Of these respondents, 170 were PVO or NGO representatives; 117 were USAID staff (102 project officers, 11 contracts officers and 8 financial management officers); and 21 were representatives of other donors. Interview information was sorted using the AQUAD program, and then was analyzed and synthesized with the findings of the literature review. "Umbrella" mechanisms: Information on umbrella mechanisms, or financial intermediaries, was collected from a literature review of approximately 22 documents as well as responses from 280 interviewees. The interview respondents included 146 PVO or NGO representatives; 113 USAID staff; and 21 other donor representatives. Interview information was sorted using the AQUAD program, and then was analyzed and synthesized with the findings of the literature review. Other donors: Other donor practices with PVO/NGO activities were assessed through a review of 16 documents, interviews with 21 other donor representatives, and responses from 270 interviews. The 270 interview respondents included 146 PVO/NGO representatives and 124 USAID staff. Information from the 270 interviews was sorted using the AQUAD program. All interview data then were analyzed and synthesized with the literature review findings. APPENDIX D1: RESEARCH SUMMARY: SUSTAINABILITY Introduction PVOs and NGOs have a special place in USAID's development strategy, largely because of the Agency's belief in NGOs' and PVOs' capacity to implement sustainable development activities. This perception is founded on a number of assumptions. PVOs and NGOs -- Adapt simple, often innovative, labor-intensive technology to local conditions -- Build long-term relationships with local communities -- Have wide information networks, local counterparts, and other institutional contacts -- Are less dependent on government inputs for their programs, compared with other organizations -- Historically, have made deliberate attempts to foster and draw on local capacities for self-help (VanSant 1987) Although these assumptions may be true in certain circumstances, they are not always valid. Characteristics of NGOs and PVOs that may decrease the likelihood of project sustainability include -- Limited ability to leverage formal systems or access decision makers -- Inadequate attention to market forces -- Staffing problems, such as high staff turnover and lack of professional (technical) qualifications -- Inconsistencies in program implementation -- Dependency on external financial support (VanSant 1986) This appendix presents findings from a number of sources: interviews with several hundred NGO, PVO, and USAID staff; a field survey of more than 20 completed projects; and an extensive literature review.13 Topics discussed in this appendix include definitions of sustainability, cost sharing, project design considerations, and implementation considerations, including monitoring and evaluation. Definitions of 'Sustainability' "Sustainability" is best defined in the broadest possible way. Definitions limited to financial, social, or institutional sustainability are insufficient. Issues concerning impact and potential trade-offs need to be addressed and incorporated into a workable, flexible definition. Addressing local conditions is important to defining sustainability. Not all project components may be sustainable; they may change according to circumstances. Sustainability of one project component may have unintended repercussions or hinder other ongoing community initiatives. A useful definition of "sustainability" will be broad and well suited to individual environments. Interviewees generally expressed a strong interest in sustainability.14 Most PVOs and NGOs believe sustainability to be a critical part of their work. Beyond this enthusiasm for the concept, respondents noted that sustainability needs to be better defined by both donors and implementers. The literature review supports this, illustrating that USAID does not apply a uniform definition of sustainability to its activities. PVOs and NGOs distinguished three types of sustainability: benefit, institutional, and financial. Benefit Sustainability Benefit sustainability involves the indefinite continuation of benefits that result from or flow from an activity, rather than continuation of specific project outputs. It is the demand for particular outcomes that gives them value and provides a test of whether they continue. According to VanSant, benefit sustainability is the continuation of valued benefit flows or outcomes with or without the programs or organizations that stimulated those benefits in the first place. The nature of these benefits may change, their source may shift, or responsibility for their costs may be assumed by a new mix of benefactors. What is important is that the benefits or outcomes be valued by the intended beneficiaries and that they continue (VanSant 1987). Defining "sustainability" as the continuation of benefits is the most comprehensive view. This also is the definition most often used by PVO and NGO respondents and is considered the most prevalent in USAID programs. But both PVO and USAID staff recognize that achieving benefit sustainability takes a great deal of time, effort, and resources. Many PVOs and NGOs expressed concern about the amount of time and resources required to achieve benefit sustainability. Most respondents say that achieving benefit sustainability takes from 10 to 20 years, depending on the level of institutional development. PVOs and NGOs believe USAID programming generally runs counter to this need, since project funding averages 3 to 5 years. Discussions of benefit sustainability with PVOs, NGOs, and USAID staff included deliberations about differences in implementation methods. NGOs and PVOs often said their development approach deals more with sustaining behavioral changes (as a result of project activities), whereas USAID seems to pay more attention to project sustainability. PVOs and NGOs see their long-term local commitment as critical to achieving benefit sustainability. A 1988 ACVFA report supports this. It states, "PVOs have a comparative advantage in achieving sustainable benefits where sensitivity to local conditions, cooperation of local institutions, and lean operating costs are particularly critical to project success" (ACVFA 1988). Several of the assessment's field case studies provide examples of benefit sustainability, with mixed results. One USAID-funded South African NGO has labored to develop programs that deal with community needs, such as adult literacy, income generation, and children's issues. Although the NGO's goal is developing sustainable projects, staff question whether the programs will achieve sustainable benefits. They are taking steps to realize greater success in this. Mexico provides another example of an activity emphasizing benefit sustainability. A USAID-funded microenterprise initiative there has attempted to build the confidence and skills needed to bring sustainable benefits to local communities. According to the NGO implementer, an important aspect of the project has been to prepare people to participate in development projects. Beneficiary participation is seen as having a strong influence on the success and sustainability of the activity. Institutional Sustainability Key components of a sustainable organization include the following: -- A clear organizational mission -- Strong leadership, including housing necessary technical and management resources -- Ability to plan strategically and an aptitude for recovering costs -- Support from the local community and capacity to mobilize beneficiary participation -- Ability to supply a continuous stream of benefits (Kean 1987) Recent studies indicate the importance of building sustainable organizations to achieve sustainable development benefits. The 1988 ACVFA report is a well-referenced account that links organizational capacity to sustainable benefits (ACVFA 1988). PVO and NGO interviewees voiced their desire for greater attention to organizational capacity building. Many respondents say the Agency's projects and programs provide insufficient resources for capacity building. Several suggested that USAID emphasize this area by helping NGOs develop long-range strategic plans, financial management capabilities, and human resources. A recent USAID-funded study noted that The best way for USAID to ensure institutional sustainability among its grantees is to enable them to attain management skills necessary to continue securing . . . funding from a variety of donors as well as to continue to implement effective development programs [MSI 1992]. PVOs and NGOs offered several reasons why, in addition to providing activity funding, donors should support institutional strengthening. First, NGOs serve as a voice for the public and provide a system of checks and balances for government actions.15 Second, supporting and strengthening NGOs provides them with the capacity to serve marginal and underserved groups. PVOs and NGOs suggest that USAID provide funding for longer periods--for 10 to 15 years instead of 3 to 5 years. This would allow them time and resources to strengthen their organizations while devoting appropriate attention to project implementation. Interviewees also recommend that USAID encourage greater networking and sharing of knowledge and resources among the PVO/NGO community. PVOs and NGOs say USAID's programs could foster better collaboration and learning from both sides. Local networking is seen as critical to long-term organizational sustainability as well as the sustainability of the entire NGO community. Financial Sustainability There are two types of financial sustainability. One pertains to NGOs and PVOs as organizations. The other concerns project activities. Research suggests the importance of gaining control over key resources, including financial and human resources (Fries 1994). Sustainability thus goes beyond organizational development to developing income-producing activities and control over funds generated. Financial sustainability of an organization refers to a group's ability to recover costs, allowing it to continue to provide services. It can be considered a component of organizational sustainability; the two often are confused. Many USAID evaluations state or imply that self-reliance--the ability of an organization to develop a mixed portfolio of financial support--is a valid measure of financial sustainability. This mix of financial resources might include fees for service, local or national government funding, donor grant funds, charitable contributions, and endowment proceeds (Fries 1994). Project financial sustainability is achieved when an activity is financially supported by those it serves. This can include in-kind support, financial or technical assistance, or both. As beneficiaries support ownership of the project, including a role in its design and cost, the activity can take on more importance to the community, bolstering its potential for sustainability. Informants gave examples of project financial sustainability when beneficiaries shouldered part of project costs. Ideally, beneficiaries would be involved in project design, support project activities, be willing to accept a portion of the project's expenses, or have the capacity to maintain the project once initial funding ends. PVOs and NGOs encourage the Agency to think more creatively about activity funding. Support for endowment and trust funds, income-generating activities, fund-raising techniques (including obtaining funding from U.S. private sources, other donors, and the host country government), and developing innovative cost-recovery techniques all are seen as avenues that can contribute to long-term growth and financial sustainability. Replication Many project implementers consider replication an element of a sustainable program. A host of new problems can arise, though, when a project that is designed and implemented by local beneficiaries is expanded into new areas. Some evidence links expansion or replication of benefits with risks to benefit sustainability. The risks are that expansion or duplication of a program requires different skills from those of the original project concept and may generate significantly different kinds of costs. At least in some instances, expansion of benefits to new persons and new areas may not be sustainable. The trade-offs between these concepts should be explored before a project is designed and implemented, since replication may not always be an attainable goal (Borton 1992). Cost Sharing Cost sharing is a mechanism that requires PVOs and NGOs to make financial or in-kind contributions to the overall costs of a project. The Agency's Handbook 3 (Project Design) long has included a 25 percent cost-sharing requirement for U.S. and indigenous organizations registered with USAID as PVOs. This requirement is in addition to the guidance in USAID Policy Determination 16, of October 1987, which simply encourages the financial participation of recipients. USAID began requiring cost sharing for PVO/NGO activities for a number of reasons: -- Cost sharing is seen as a means of leveraging USAID development assistance with other sources of money -- Cost sharing is considered a mechanism to prevent NGO and PVO dependence, both financially and programmatically -- Cost sharing encourages PVOs and NGOs to gain a sense of ownership of programs by financing a portion of the costs -- Cost sharing makes it less likely that USAID will be able to force its own agenda on organizations that are accountable to other sources or constituencies (USAID 1982) NGO and PVO interviewees provided a number of reasons why cost sharing is useful. Cost sharing -- Fosters ownership and commitment -- Promotes greater financial involvement of beneficiaries -- Encourages PVO and NGO financial independence -- Allows PVOs and NGOs to simplify procurement by using their own funds for certain purchases -- Prevents reliance on one donor for funding -- Requires strategic planning for cost recovery once project funding ends Asked if the Agency should require cost sharing for PVO/NGO activities, PVO, NGO, and USAID interviewees overwhelmingly said "yes." Nearly all USAID staff support cost-sharing requirements, closely followed by PVOs. About three fourths of the NGOs interviewed support USAID cost-sharing requirements. Most NGO and PVO respondents indicate that USAID's cost-sharing requirements are understandable and are a way for them to show their commitment to a given activity. But many expressed their concern about the inflexibility of cost-sharing rules. They felt cost sharing should be approached with greater flexibility, depending on circumstances, and should take into account an organization's financial limitations. Respondents who favor cost sharing in general oppose it under certain circumstances, such as when activities are not intended to be financially self-sufficient. Examples are education, policy research, legal assistance to the poor, drug prevention and education, nutrition and family planning programs, women's empowerment activities, and human-rights initiatives. On the negative side, some USAID and PVO respondents say monitoring PVO and NGO contributions is time consuming. There appear to be no set guidelines on establishing dollar values for in-kind contributions, making them difficult to calculate. Some Agency staff say cost-sharing arrangements have entailed additional monitoring, management, and paperwork requirements. Several NGO staff feel the standard 25 percent contribution is inappropriate for many organizations. They consider this percentage onerous, especially for young or smaller organizations. NGOs feel this financial burden could prevent them from seeking initial funding from USAID and could limit the types of organizations that work with the Agency.16 USAID staff also expressed the belief that NGOs should not be required to cost-share until they have established a certain level of institutional viability. The literature review supports NGO, PVO, and USAID perceptions. Although the objective of cost sharing may be to reduce dependence on Agency funding, it does not always succeed. Strict cost-sharing formulas may not be as useful as other arrangements that more accurately reflect an organization's financial capacity. In addition, some groups need assistance to identify other mechanisms to achieve financial sustainability. In general, respondents believe the Agency should have flexible cost-sharing guidelines. Some NGOs suggest that USAID review its cost-sharing requirements on a case-by-case basis. Interviewees recommend that cost-sharing requirements be established in Missions and that they show flexibility in relation to an NGO's capacity and the activity's goals and objectives. In some cases the Agency may want to provide funding (without cost sharing) to an organization that is implementing a USAID initiative, or if it cannot contribute to costs. There also were suggestions that USAID require limited cost sharing for smaller organizations. These cost-sharing arrangements could range from 5 to 10 percent and increase each year. Resources also could be devoted to training organization staff in fund-raising and providing information and training on other income-generating activities. In July 1994, in response to recommendations made by a joint PVO-USAID task force (part of the Agency's Advisory Committee on Voluntary Foreign Aid), the Agency eliminated the 25 percent cost-sharing requirement for registered PVOs that was mandated in USAID Handbook 3. Instead, all PVOs and NGOs will follow Policy Determination 16. It encourages the "largest reasonable and possible financial participation" of recipients but does not prescribe a formula. In collaboration with PVOs and NGOs, USAID staff now have flexibility to develop situation- or organization-specific cost-sharing plans for PVO/NGO activities. Project Design Considerations Defining and incorporating sustainability concerns into projects is difficult. But methods do exist for incorporating sustainability elements into project designs. They are discussed in this section. Asked if they believe sustainability to be an appropriate goal for USAID-funded PVO/NGO activities, respondents overwhelmingly said "yes." PVOs and NGOs feel strongly about achieving greater sustainability. They made a variety of suggestions for improvements. Among them: -- Build from the most basic grass-roots level -- Involve the target community at all levels -- Ensure that project decision-makers are the project beneficiaries -- Help groups make their own decisions -- Emphasize empowerment of women -- Use volunteers from the community to implement projects -- Coordinate with available government programs -- Develop implementers' management information system capabilities -- Ensure that adequate training is incorporated into every project, for project beneficiaries as well as project managers Other themes regarding the design of sustainable activities emerge from the literature review, field case studies, and key informant interviews. Flexibility Project planners need to consider sustainability early in the design of new activities.17 The appropriate definition of "sustainability" must be determined for each new project, taking into account its particular context and developing indicators against which sustainability can be measured. Issues that must be considered early in the project design include the following: -- Is there community level demand for the project and its benefits? -- How will the project contribute to sustainable development of a community (or region or country)? -- What are the attitudes of the central and local governments toward NGO activity in the sector? -- How will the project be sustained by the local community? (Russell et al. 1994) Ability to test and revise the project design--and its underlying assumptions--is critical to sustainability. Initial errors can be corrected if the implementer and its partners are sufficiently informed and flexible in making adjustments. When the implementer encourages feedback and project modification, sustainability is enhanced (USAID 1989). Broad Participation Some sources indicate there is a link between beneficiary involvement and sustainability. According to a 1988 ACVFA report: Of all the factors that affect sustainability, local organizational capacity to identify and solve problems with the involvement of beneficiaries is the most important (ACVFA 1988). The field case studies also indicate that beneficiary participation in design and implementation of projects is a main ingredient of success, since it is the beneficiaries who have the most to gain or lose from an activity. Their participation creates an opportunity for interventions that are relevant to local needs and that provide a sense of ownership for beneficiaries. PVO and NGO implementers also need to work in partnership with community groups, private associations, local government, and other donors, among others.18 Project designers should try to incorporate into the planning process powerful local interests and local decision-making bodies (e.g., village chiefs, the military) (Russell et al. 1994). Successful NGOs often learn to collaborate with government officials as well as with targeted beneficiaries and local groups. These NGOs keep the government fully informed of ongoing and planned activities and give officials public credit for project success. At the same time they maintain credibility with their constituents. Project planners should be cautious, though, about proposed NGO activities that replace or bypass government services. These efforts can foster a difficult relationship between implementer and government. Or they could result in government neglect of a service area.19 Sustainable projects need the support of local institutions to run and maintain activities. Sometimes local institutions must be established, but often existing organizations can be adapted or strengthened.20 Project planners must assess the capacity and the interest of existing local institutions to continue activity benefits (VanSant 1987). Implementer Capability Capacity of an organization to implement a project is critical to sustainability. While there is no formula for selection, time must be set aside to learn about and understand proposed implementers. For example, many foundations have an interview process, including multiple site visits, for prospective implementers. Identification of an organization's strengths and weaknesses during this period is important. Since no implementer is likely to have all the technical and administrative skills that designers find important, it is essential to understand and address implementer limitations, and to build on the implementer's strengths.21 Morss stipulates that institutional capacity is a critical element of project sustainability, saying that few project ideas are so compelling that they will perpetuate benefits without institutions equipped to carry them forward (Morss 1982). Other studies support this assertion, such as a 1987 study that found a high correlation between organizational capacity and overall sustainability.22 Project planners must consider a number of factors when selecting an implementer.23 Screening questions include the following: -- What is the organization's leadership like? Have leaders been involved in setting up other NGOs? If so, were they successful? -- Does the organization have a sectoral specialization? -- What are the organization's current funding sources? -- Does the organization recover its program costs? -- How long has the organization worked in the region? country? community? Has the organization a strong constituency? In essence, program planners need to look for NGOs (and leaders) with an entrepreneurial spirit, that is, they are pragmatic problem solvers. They are strategic planners able to develop the organization to carry out their ideas. They are, moreover, motivated by the desire to create significant grass-roots change. inancial Concerns Activities often take longer to attain goals and objectives than project planners anticipate. Designers need to maintain realistic expectations of the time frame necessary to achieve project results. For several reasons, donors may need to consider longer term funding arrangements. First, benefit sustainability is a long-term process. If sustainability is to be achieved, activities may need to continue for more than 3 to 5 years (the usual funding span). Second, longer term funding arrangements allow NGOs and PVOs to look to longer range financial and strategic planning for continuation of activity benefits. Prospects for self-funded project activities--the phasing-out of donor resources--should be considered during project design. Issues to consider include possible income-generating schemes, cost-recovery approaches (e.g., fees for service), use of endowments, and fund-raising techniques (Fries 1994). Another issue raised by development practitioners is whether activities can be sustainable if they compete with government services.24 The issue of private sector provision of public services has several sides. NGOs may be able to provide services more efficiently and effectively than the public sector, but some point out that equal access to services may be less likely when impoverished users must pay private providers. Others contend that public service delivery is more easily controlled by elite and ethnic groups, making equal access less likely (Russell et al. 1994). Management Considerations USAID's management of PVO and NGO activities can affect their sustainability. Some management considerations are the Agency's procedures and regulations, its short-term funding approach, attention to monitoring and evaluation, and measuring project impact. These are discussed below. USAID Practices Several case studies and interviewees suggest that the Agency's procedures and regulations can hinder sustainability. That, in the long run, has negative implications for NGOs' capacity to achieve benefit sustainability. One example is the inflexibility of grants: once a grant is signed, NGOs say, they have difficulty adapting their projects to changing conditions even though adaptation is critical to sustainability. Other unsustainable practices mentioned by interviewees include English-language reporting requirements, infrequent USAID site visits, and minimal implementation support. Funding Approach USAID's project-oriented approach to development usually means short-term perspectives and programming. According to several interviewees, the Agency's short-term funding practices hinder sustainability. Projects often are designed with a 3- to 5-year time frame; after this period, implementers are encouraged to seek other funding sources. This short-term approach often is unrealistic for long-term project results, which can require many years or even decades of continuous support. As a result, NGOs and PVOs continually search for new funds and are limited in their ability to undertake long-term financial and management planning. Monitoring and Evaluation Evaluations and reports consistently indicate the importance of devoting resources to monitoring and evaluation, but managers often give this area insufficient attention. From the time of project conception, short-term objectives and long-term goals must be clear and realistic. Responsibility for defining goals and objectives is central to sustainability, since all partners need to agree, explicitly and in advance, on the project's scope and what it hopes to accomplish. Case studies, interviews, and the literature review all suggest that the demand for an activity drives its success, and that goals and objectives that are closely linked to the immediate needs of local communities have a greater likelihood of sustainability. Benchmarks must be established during project design to allow for progress monitoring. In addition, indicators that look at sustainable impact also must be incorporated into a monitoring and evaluation system. Developing benchmarks and indicators allows implementers to track progress while ensuring activity sustainability. Both USAID and implementers can do more in the areas of monitoring and evaluation. The Agency can provide NGOs and PVOs with additional training and resources for establishing benchmarks, developing realistic impact indicators, setting up management information systems, and conducting evaluations. NGOs and PVOs can increase the amount of resources devoted to their own monitoring and evaluation activities and can work to improve their skills in these areas. Measuring Impact Several interviewees raised the issue of trade-offs between sustainability and project impact. They say that some development objectives, such as behavioral changes (e.g., for health projects) or acquiring new skills (e.g., through training programs) are difficult to measure and to sustain. In these examples, respondents consider project impact to be more important than sustainability.25 An ACVFA report notes that although sustainability is vital, it is not the sole indicator of success; "[while] project benefits may be sustained over time, the magnitude of a project's impact may be insignificant. Sustainability is not synonymous with impact" (ACVFA 1988). Other studies indicate that donors often overlook NGO project impact while centering attention on project implementation activities and operational performance. In addition, failure to establish clear and measurable indicators at the project level makes impact evaluation difficult (VanSant 1986). The Agency currently is undertaking a major initiative on design and measurement of performance-based activities. USAID staff are developing indicators to effectively measure results. But NGOs and PVOs often have difficulty demonstrating broad-based results. They may devote limited resources to developing impact indicators and monitoring results. In addition, small-scale initiatives may work effectively in one village but have limited application in others. Conclusions and Recommendations USAID has a long history of employing sustainable development concepts in its projects and programs. Its NGO and PVO activities are considered central to these endeavors. In the assessment, three types of sustainability have been defined: benefit, organizational, and financial (pertaining to both the organization and project activities). Benefit sustainability is the continuation of valued benefit flows after USAID funding ends, with or without the programs or organizations that stimulated those benefits in the first place. Benefit sustainability is the approach generally used by PVOs, NGOs, and USAID. Cost sharing was overwhelmingly supported by interviewees, but there was widespread agreement that the strict 25 percent cost-sharing requirement for registered PVOs should be relaxed and that PVO/NGO shares of project costs should be determined more flexibly. In fact, the requirement was eliminated in July 1994 (after completion of study interviews). The Agency's guidance now is for the "largest reasonable and possible financial participation" of recipients. Sustainability should be considered early in project design. To the extent possible, projects should have flexible designs that allow revisions based on changing environments. In addition, extensive beneficiary participation and local involvement in project design and implementation, and the capabilities of the proposed implementer, are essential to the sustainability of project benefits. PVOs and NGOs indicated that USAID's short-term funding approach--financing activities for 3- to 5-year periods--may have a negative effect on sustainability. PVO/NGO experience indicates that long-term project impact often requires 10 years or more of continuous intervention. The literature review and interview respondents indicate the Agency places insufficient attention on sustainability aspects of project monitoring and evaluation. Clear, measurable benchmarks and impact indicators should be established at the beginning of activities. There can be trade-offs, however, between sustainability and impact, as projects that have significant impact may not be sustainable. Appendix D1: References Advisory Committee on Voluntary Foreign Aid. 1988. The Effectiveness of Private Voluntary Organizations. Washington: USAID. Borton, Nan. 1992. Strategies for Sustainable Development: Essential Elements in Planning and Implementation for Benefit Sustainability. Washington: Automation Research Systems. Cotter, Jim. 1988. USAID-Financed PVO Co-Financing Projects in the Philippines and Indonesia: Reshaping the Arms of PVO Collaboration. Washington: USAID. Finsterbusch, Maussolff, and Van Wilkin. 1992. Factors Contributing to the Effectiveness and Sustainability of Development Projects. Washington: International Development Management Center. Fries, Tami. 1994. Sustainability: A Literature Review. Washington: Development Alternatives, Inc. General Accounting Office. 1982. Changes Needed to Forge an Effective Relationship Between A.I.D. and Voluntary Agencies. Washington: GAO. Harkins, Allen, and Azucena Quan. 1990. Evaluation of Belize Enterprise for Sustained Technology. Washington: USAID. Institutional Reform and the Informal Sector Project. 1991. Project Implementation Document: FSDP. Washington: USAID. John Snow, Inc. 1991. Promoting NGO Sustainability: Lessons From Enterprise. Washington. Kean, John, et al. 1987. Analysis of Institutional Sustainability: Issues for USAID 1985-86 Project Evaluation Reports. Devres, Inc. Washington: USAID. Management Systems International. 1992. A Review of the Quality and Coverage of A.I.D. Evaluations, FY 1989 and FY 1991. Washington: MSI. Morss, Elliot, et al. 1982. Toward Self-Reliant Development: A Guide for Evaluating the Sustainability of Project Benefits. Washington: Development Alternatives, Inc. Russell, Diane, et al. 1994. Sustaining the Vision: Lessons for USAID's Move Toward Sustainability and Sustainable Development. Center for Development Information and Evaluation. Washington: USAID. Tendler, Judith. 1982. Turning Private Voluntary Organizations into Development Agencies: Questions for Evaluation. CDIE Discussion Paper. Washington: USAID. USAID. 1982. A.I.D. Partnership in International Development with Private and Voluntary Organizations. Policy Paper. Washington: USAID. USAID. 1989a. Sustainability of Development Programs: A Compendium of Donor Experience. CDIE Discussion Paper. Washington: USAID. USAID. 1989b. Project Assistance Completion Report: Zio River Development Project, Togo. Washington: USAID. USAID. 1993. PVO Co-Financing II Project: PACR. Asia Bureau. Washington: USAID. VanSant, Jerry. 1987. Benefit Sustainability. Washington: Development Alternatives, Inc. VanSant, Jerry. 1986. The Role of Non-Governmental Organizations as Intermediaries in Decentralized Rural Development. Chapel Hill: Development Alternatives, Inc. APPENDIX D2: RESEARCH SUMMARY: UMBRELLA MECHANISMS Introduction During the design of this assessment, a number of questions were raised about USAID's increasing use of "umbrella" funding mechanisms for PVO and NGO activities, particularly for management efficiency. To obtain more information on this topic, a literature review was prepared by Development Associates, Inc. To supplement the literature review, evaluators collected information on the effectiveness of umbrellas through more than 250 field interviews. This appendix summarizes the major findings from the literature review and field interviews. An umbrella is a funding mechanism designed to deliver relatively small amounts of Agency funds to a number of PVOs and NGOs through one financial award to a lead recipient. Most commonly, a U.S. PVO receives a block grant award from USAID and subsequently makes smaller subgrants to PVOs and NGOs for project implementation. There are several umbrella models, described below. The umbrella mechanism originally was developed for expediency. USAID first used umbrellas as an administrative mechanism to channel funds to more PVOs and NGOs than was possible through direct grants. Also, in politically sensitive situations, USAID was able to provide funds indirectly--through an intermediary--to the intended recipient. However, as time progressed, it became evident that subgrantees often needed assistance managing their grants. Consequently, capacity building has become a major component of umbrella activities. In addition to providing funds through subgrants, the lead organization often is responsible for providing administrative and technical assistance to subgrantees. This may include providing training, assisting with project implementation, helping to develop or improve standardized financial and administrative practices, or assisting with the development of monitoring and evaluation systems. Umbrella mechanisms have been used to implement activities in a number of sectors, including agriculture, education, training, income generation, rural development, health and family planning, and environmental and natural resources management. Advantages and Disadvantages There is a range of assumptions regarding the advantages of umbrellas. The most widely asserted is that umbrella mechanisms reduce USAID's management demands, since the Agency deals with only one grantee instead of several. Other potential advantages, provided by NGO, PVO, and USAID interview respondents, include the following: -- Umbrellas permit the Agency to fund a wider range of small rural or grass-roots community groups -- Umbrella activities provide NGOs the opportunity to work as partners with a lead PVO or NGO, to obtain implementation guidance and capacity building assistance -- Umbrellas minimize the need for USAID to directly provide technical assistance, training, and supervision to grantees One possible disadvantage of using an umbrella mechanism is that USAID may not receive "credit" for funding subgrant activities. Although this may be desirable in politically sensitive situations, in other cases the Agency may want its presence known. Project planners need to recognize and address the limitations of the umbrella approach on the Agency's in-country visibility. >From the subgrantee perspective, there also are pros and cons to receiving subgrants under umbrella activities. On the one hand, subgrants can be a way to avoid many of the administrative requirements of direct grants. On the other hand, NGOs say that umbrella grants can hurt their ability to develop long-term relations with donors such as USAID, since the lead implementer can compete for funding and visibility. Some PVOs and NGOs say that when USAID uses the lead implementer simply to administer subgrants--instead of also assisting with project implementation and institutional strengthening--PVOs and NGOs are being used "like contractors." Respondents say the strengths of PVOs and NGOs are their project implementation skills, innovative development approaches, and knowledge of local issues. These advantages are lost when the organizations are employed merely as financial intermediaries. Selection of Lead Organization Several models exist for management of umbrella grants. The lead organization may be a PVO or an NGO, an association of PVOs or NGOs, or a contractor. The most common arrangement seems to be use of a U.S. PVO, although some larger or more experienced NGOs also serve as lead implementer. Significant factors in choosing a lead implementer include the following: -- Size and organizational capacity of the local NGO community -- Extent of U.S. PVO presence and PVO relationships with local organizations -- Whether USAID's activity objectives are compatible with those of an existing organization, and whether the organization can carry on the project's objectives without hurting its own -- Whether USAID's project objectives as carried out by the lead organization complement services already being performed by other PVOs and NGOs -- Whether the local political situation will enable the lead organization to function effectively to carry out the activity objectives According to an evaluation of Latin American umbrella activities, if USAID's principal purpose is to provide funding for NGO activities (as opposed to institutional development), then using a PVO or NGO--rather than a membership association--as lead implementer/financial intermediary is simpler and more efficient. But if the purpose of the project is to strengthen an organization to coordinate with and represent NGOs, then using a membership association works better. If USAID's purpose is to do both, then either type of organization can be used, depending on local circumstances (Checchi 1989). Although membership associations are best employed when the objective is to strengthen the NGO sector, several issues arise regarding use of membership associations. These issues include 1) attempts by individual member PVOs or NGOs to control the association and its policies, 2) members' lack of understanding about the appropriate roles and functions of the board of directors, and 3) staff relations with the board and its members. In addition, one evaluation found that the financial sustainability of the membership organization was questionable, owing to lack of dues payments and absence of other income-generating activities (Buzzard 1991). A number of factors must be considered in choosing a PVO or NGO as lead implementer; each strategy has its own strengths and weaknesses. In general, NGOs may be more familiar with local needs, whereas PVOs may be better managed but more expensive. In some countries there may be no local NGOs that can effectively award and administer subgrants. In other countries, a strong NGO sector skilled in grant administration and management may exist. NGOs usually will demonstrate a good understanding of the local economic, political, and social situation. Although NGOs may have a better sense of local realities, they may not have the capacity to effectively implement complex projects. NGOs also may bring with them political biases and perspectives about other NGOs. Such attitudes may hurt their capacity to act as neutral decision-makers. PVOs, by contrast, may provide a stronger management infrastructure26 and may come with a good understanding of USAID regulations and the complexities of umbrella activities. PVOs may bring needed skills to local development projects but may lack sensitivity to local perceptions and practices. U.S. PVOs can be considerably more expensive than NGOs. In addition, PVOs usually have expatriate staff that rotate every 2 or 3 years. This can create continuity problems and questions about whether PVOs can comprehend local development needs. Project Management Concerns Most problems noted by interviewees involve project implementation and financial management. Respondents' major themes, as well as pertinent findings from the literature review, are provided in this section. Management Efficiency During field interviews, all categories of respondents stressed the overall management efficiency (including cost efficiency) of umbrella projects. A majority of USAID staff state that umbrella arrangements improve activity management and that using an umbrella mechanism works well in controlling costs.27 But the cost efficiency of umbrella activities is a complex issue that has not been adequately addressed in the literature. An assessment of costs must include an accurate comparison of USAID's usual costs for direct grant administration with costs under an umbrella arrangement. Cost estimates need to include 1) staff time and resources to manage multiple NGO/PVO activities, 2) opportunity costs of the Agency's direct management of multiple PVO grants, and 3) costs and benefits of reaching more NGOs. There are two conflicting views regarding the cost efficiency of umbrella activities. The first considers umbrellas to be more cost efficient than direct USAID management for both USAID and the implementers, for a number of reasons: -- Less duplication and reduced competition for funding among NGOs -- Greater reliance on local resources and expertise -- Streamlined reporting and monitoring requirements -- USAID's increased capacity to work with multiple NGOs through one grant mechanism28 -- Less USAID staff time and resources needed to manage and monitor multiple small grants -- The Agency's limited role in providing technical assistance and training The second view holds that umbrellas are a more costly alternative than direct grants. That, critics say, is because 1) USAID pays overhead costs for both the lead implementer and subgrantees, 2) activity start-up times are longer, 3) subgrant disbursements are slower, and 4) subgrantees need capacity building and technical assistance. A 1989 evaluation of several USAID-funded Latin America membership associations found that providing funds through these associations did not always result in decreased Mission involvement, at least in the short term. According to the evaluation, Missions should recognize that working with such organizations will involve them in problems affecting the operations and development of these organizations; and that resolving [those problems] will require staff time, including that of Mission management. This will be particularly the case if the Mission also seeks to have the organization assume a leadership role among the NGOs and achieve institutional sustainability. Consequently, the saving on staff time is most likely to occur over the long run rather than in the short run [Checchi 1989]. Given the absence of much empirical evidence regarding management (and cost) efficiency of umbrella activities, several unanswered questions remain: -- Does the award of umbrella grants permit USAID to fund more organizations and activities while using fewer human and financial resources? -- Do umbrella activities minimize USAID's need to provide technical assistance and training to PVOs and NGOs? -- Do umbrella activities reduce USAID staff time for administrative and oversight responsibilities? Communications and Coordination USAID's experience implementing umbrella activities illustrates problems with communications and poor coordination. A major finding of a 1991 evaluation of a USAID/Senegal umbrella activity was that the project designers failed to take into account the need for coordination and communication among the various actors. The design did not reflect the complexities of the project, its multiple partners, or its levels of intervention. The evaluation recommended that future activities include resources for PVO and NGO communications and coordination in order to provide an opportunity for the organizations to learn from each other and share experiences and expertise (Ba et al. 1991). In another case, the office of the USAID representative to Afghanistan had grants with a PVO and a foundation to fund a women's education program through subgrants to local NGOs. A 1992 program evaluation found a lack of coordination and communication among the various donors and the implementing organizations. It recommended that all delivery organizations work together more closely to share resources and expertise (El-Sanabary 1992). Lack of coordination and communication problems also were frequently cited by PVO, NGO, and USAID interviewees. Respondents say undefined roles and lack of communication between the lead implementer and subgrantees contribute to unsuccessful programs. In addition, many NGOs feel distant from lead PVOs that manage subgrants from the United States. The NGOs say they are not kept informed about issues concerning the grant and, as a result, they are unable to plan their own activities well.29 PVOs also admit a need for better communication and efficiency in financial and administrative matters. The use of umbrella grants also can make it more difficult to communicate USAID's goals and objectives to subgrantees (Kelly 1994). Informants conclude that umbrella activities can be more difficult to manage when poor communications exist between USAID, the lead implementer, and subgrantees. PVO and NGO respondents recommend that lines of authority and roles and responsibilities of USAID, the lead implementer, and the subgrantees be clearly developed before the signing of grant agreements. Project Monitoring USAID staff have limited time and resources to monitor and assess direct grants and even less capacity to monitor subgrants. One problem cited by Mission staff is their limited ability to monitor the technical aspects of umbrella subgrant activities. Agency staff say it is necessary to monitor subgrants because they believe the lead implementer may not always have this capability. Several NGO respondents expressed the view, however, that Mission staff need to "let go" of their overinvolvement, and trust the PVO/NGO community to do its job. PVOs and NGOs feel that monitoring subgrant activities is their responsibility, and unless they are permitted to do so, the umbrella process will not work properly. Financial Management Many USAID respondents expressed their concern about financial accountability matters when the Agency does not have a direct relationship with subgrantees. Although there is a sense that the Agency's administrative oversight is reduced through the use of subgrants, some USAID staff are concerned that subgrantees' finances are not being monitored properly. The range of interviewee comments on financial procedures reflects the extent of PVO/NGO knowledge about USAID's rules and regulations. Some PVOs and larger NGOs appear to understand the various financial and auditing requirements, but there are many others that do not. Also a great deal of confusion exists among key informants about the financial requirements of umbrella agreements and subgrants. Some PVOs and NGOs feel that umbrellas make it easier to deal with USAID's requirements. Others criticize the process as being the same for both direct and subgrantee recipients. Some respondents say there are no requirements for lead implementers to exercise strong financial management oversight over subgrantees. They add that only lead implementers are bound by USAID rules regarding procurement, travel and per diem, personnel, and the like. However, all procurement rules and regulations, as outlined in USAID's standard provisions, apply to subgrant recipients. The Agency has developed two sets of standard provisions, based on OMB Circular A-110, which are attached to each grant or subgrant. One set of standard provisions applies to U.S. organizations; the other, to non-U.S. organizations. No distinction is made between principal and subgrantee; rather, the distinction is between U.S. and non-U.S. When a U.S. PVO makes a subgrant to another U.S. PVO, both organizations follow the same procedures. But when a U.S. PVO makes a subgrant to a local NGO, different, more relaxed standard procedures apply. Interviewees had many complaints about USAID's financial rules and auditing requirements, including the expense of audits, the time and resources devoted to establishing "appropriate" financial systems, and not having capable local expertise to conduct audits. Recommendations include changes to OMB requirements to more accurately reflect realities faced by local NGOs. NGOs and PVOs also believe that USAID should consider different standards of financial accountability for local NGOs than for other not-for-profit organizations and contractors. The audit requirements recently were relaxed for NGOs that receive direct USAID grants. In March 1994, the OMB Circular A-133 audit threshold was raised; A-133 now applies to NGO direct grantees if they receive more than $100,000 annually in Agency funds. The threshold for subgrantees, however, remains at $25,000. In other words, it is simpler--as regards auditing requirements--for NGOs to receive a grant directly from USAID. According to USAID's Procurement Policy office, this subgrant audit provision may be changed in the future. Project Impact As discussed above, using an umbrella mechanism to implement a project involves trade-offs. It may, on the one hand, take longer to set up the project and make subgrants under an umbrella. On the other hand, a wider range of organizations may be reached and strengthened through umbrella grants. Questions remain, however, regarding the extent to which project-level impact is achieved through the use of umbrella grants. For the most part, evaluators have found that umbrella activities do achieve their objectives. For example, the 1992 evaluation of the Bolivia Regional Development project found that the NGO component was helping USAID achieve its strategic objectives of increases in crop yields and incomes (Kraljevic 1993). A 1989 evaluation of umbrella groups in Latin America found that subprojects were meeting their objectives and having a beneficial effect on recipients (Checchi 1989). According to field interview respondents, the most critical factor leading to successful projects is the competency of the lead implementer. Respondents feel that USAID project designers need to give more time, attention, and resources to the selection process, including obtaining a thorough understanding of an organization's capabilities and experience in implementing similar programs. "Competency" also includes concerns such as the ability to choose appropriate subgrantees, adequate management capacity to oversee subproject activities, the skills to provide necessary technical assistance to subgrantees, and the ability to work in partnership with the local NGO community. Capacity Building Capacity building is taking on an increasingly important role in umbrella activities. According to a 1992 USAID study, the ability of a lead implementer to strengthen subgrantee NGOs is appealing.30 The PVO and NGO community welcome this finding and have rallied around the task of strengthening their community and building new networks of collaboration and information sharing. Many of the umbrella activities reviewed for this assessment included efforts by the lead implementer to build the capacity of subgrantees. This assistance takes many forms, including technical assistance, training in fund-raising, membership development, strategic planning, management assistance, networking and information sharing, and guidance on USAID's systems and procedures. A collaborative partnership between the lead implementer and subgrantees is critical to the capacity-building process. This partnership can be a learning relationship where NGO subgrantees master methods of project design and implementation, gain new technical competencies, adopt new management and administrative practices, and develop and use monitoring and evaluation systems. A midterm evaluation of a Bolivian NGO umbrella activity found that NGOs look to the lead implementer as a source of guidance and assistance--not just for funds. They perceive the lead organization as their "broker" with regional development agencies and donors (Kraljevic 1993). A 1992 study of a Colombian membership association supports these assertions, concluding that a membership association can be an excellent vehicle for institutional strengthening, program expansion, and channeling of financial resources (Alfonso et al. 1992). Several USAID staff interviewed for the study question whether the need for capacity building overshadows the issue of achieving project impact. A 1989 evaluation of Guatemalan umbrella activities concluded that the use of umbrella grants appears to have limited potential for contributing to the institutional sustainability of PVOs assisting microenterprises (Canellas et al. 1989). However, a 1992 evaluation of African PVO/NGO projects notes that most of the umbrella activities evaluated "included training and technical assistance for the NGOs--although the importance of this activity to the achievement of the projects' objectives is not always appreciated" (Otto and Drabek 1992). The literature review illustrates the difficulty of assessing the success of umbrella activities in NGO capacity building; the evidence to date is inconclusive. However, NGO and PVO respondents believe USAID should devote greater resources to capacity building to enable lead implementers to strengthen smaller NGOs and build sustainable, community-based organizations.31 NGO and PVO interviewees say the effectiveness of umbrella arrangements would be greatly enhanced if lead implementers are allowed to invest more resources in strengthening indigenous NGOs. Conclusions and Recommendations Key informants made a variety of suggestions regarding the use of umbrella mechanisms. Recommendations include greater attention to communications and information sharing, the unique role of U.S. PVOs in capacity building, and subgrantee selection criteria. NGOs and PVOs stress the important role of capacity building in umbrella activities; in general, the positive contributions of capacity-building efforts appear to outweigh weaknesses. NGOs in particular recognize the opportunity umbrella mechanisms offer to work in partnership with PVOs. Many NGOs feel this collaboration can provide them with the financial, technical and administrative skills necessary to strengthen their organizations. The NGO/PVO community also recommends that USAID devote more resources to NGO capacity building as well as toward improving communication among grantees. Increased NGO networking and information sharing is seen as critical to NGO organizational development. Although PVOs like the flexibility of umbrella grants in selecting subgrant recipients, many state that clearer selection criteria should be established. PVO recommendations include USAID's development of country-specific guidelines for the selection of subgrantees. These guidelines should include organizational criteria as well as guidelines for project identification and funding. Given the appropriate conditions and attention to design and implementation, umbrella grants can be a useful way for USAID to reach greater numbers of NGOs throughout the developing world. However, umbrella grants do not always provide the Agency with the intended management savings, such as reduced administrative and oversight responsibilities. A 1989 evaluation of Latin America and Caribbean umbrella activities concluded that USAID Missions can conserve staff time by utilizing intermediary organizations [U.S. PVOs and local NGOs] to prepare, monitor, and evaluate project activities by NGOs. However, it is unlikely that, without losing its own coherence, one organization will be able to handle the necessary range of required NGO activities; and [it is likely] that a USAID Mission will be unable to utilize it for all the types of activities it wants to support. Thus, it will not be able to avoid all direct relationships with NGOs unless it has a very highly focused, limited program [Checchi 1989]. Umbrella mechanisms offer one approach to the Agency's provision of assistance to NGOs and PVOs. The attention that USAID and the NGO/PVO community devote to working in partnership are key to successful design and implementation of umbrella activities. Through the joint planning and decision making of all parties--including subgrantees--umbrella mechanisms provide USAID with an opportunity to support local sustainable development initiatives. Appendix D2: References Alfonso, Arelis G¢mez; Nan Borton; and Carlos Castello. 1992. Apex Study of the Asociaci¢n de Grupos Solid rios de Colombia. Washington: USAID. Ba, Thierno; Robert H. Brandsetter; and George Butler. 1991. Community and Enterprise Development Project/PVO Component. USAID/Senegal. Washington. Buzzard, Shirley. 1991. The Cooperative Agreement Between AID/FVA/PVC and Private Agencies Collaborating Together (PACT): An Evaluation Report. Washington: Heartlands International Consultants. Canellas, Andrew; Michael Hirsh; and Cressida McKean. 1989. Microenterprise Stock-taking: Guatemala. Washington: USAID. Checchi and Company. 1989. Evaluation of Experience of USAID Missions with PVO Umbrella Groups in Costa Rica, Guatemala, Honduras, Haiti, and Jamaica. Washington: USAID. El-Sanabary, Nagat. 1992. Evaluation of Programs for Afghan Women Funded by the O/AID/Rep and the Asia Foundation. Washington: USAID. Kelly, Margo. 1994. Umbrella Activities Literature Review. Working paper. Washington: Development Alternatives, Inc. Kraljevic, Kvo J. 1993. Midterm Evaluation of the Nongovernmental Organizations Component of the Cochabamba Regional Development Project. Washington: USAID. Otto, Jonathan, and Anne Drabek. 1992. Designs for Collaboration: A Study of PVO/NGO Umbrella Projects in Africa. Washington: USAID. USAID. 1990. Project Paper: PVO/NGO Support Project. USAID/Senegal. Washington. USAID. 1990. Project Paper: South Pacific Regional Family Planning. Bureau for Asia, Near East, and Europe. Washington. APPENDIX D3: RESEARCH SUMMARY: OTHER DONOR'S WORK WITH NGOS During assessment planning, evaluation designers decided it would be useful to obtain information on the work of other multilateral and bilateral donors, to supplement findings on USAID's work with U.S. and indigenous NGOs.32 Data were collected primarily through field interviews in eight countries with donors, U.S. and indigenous NGO representatives, and USAID officials. In addition, a literature review of the management practices of European, Canadian, and multilateral donors was conducted. The overall size of an organization (i.e., number of staff and budget) appears to have an effect on reported problems. Smaller NGOs seem to have more difficulty interacting with donors, understanding their bureaucracies, and working in partnership arrangements. U.S. and larger indigenous NGOs seem better prepared to handle donors' implementation guidelines, reporting requirements, and auditing procedures and, perhaps, to have more productive relationships with donors. Topics discussed in this appendix are donor programming approaches, partnership approach, funding arrangements, project design, project implementation, capacity building, and implications for USAID. Donor Programming Approaches International development assistance to NGOs varies from donor to donor. Each donor has perceptions about NGOs, which engender different programmatic objectives. Sometimes NGOs are viewed as a contracting vehicle to execute a donor's development strategy. More often, NGOs are considered an appropriate entity to implement grass-roots poverty alleviation programs. In other cases, an NGO's entire portfolio may be supported through institutional-strengthening grants. Although country program objectives vary, there is considerable similarity among donor NGO programs and substantial overlap in the reasons donors choose to work with NGOs.33 These include the beliefs that NGOs -- Apply low-cost solutions and maintain streamlined, cost-effective services -- Have innovative and flexible approaches to improving local conditions -- Have a good sense of the needs of local communities -- Employ a high degree of participation in project implementation -- Successfully work to empower local NGOs -- Work well in poverty reduction -- Reach less accessible communities and individuals--that is, the poorest of the poor Donors' views vary greatly regarding the importance of NGO involvement in strategic planning. Most pay some attention to it, but the impact of NGO involvement in donor programming strategies seems to be limited. The Canadian International Development Agency makes a concerted effort to include Canadian NGOs in its strategic-planning process; this also is done, to some extent, with local NGOs. The European Commission in South Africa makes a significant effort to include local NGOs in the formation of its development strategy. The Swedish International Development Authority also includes NGOs in the formation of Swedish development assistance policies (Gwynn 1994). In field interviews, NGOs in general say they appreciate the overtures being made by USAID and other donors to include them in their strategic-planning processes. Nevertheless, many NGOs are concerned that their voices are not being heard. Several NGOs cite USAID's decision to close a number of Missions as one instance in which NGO perspectives were not considered. NGOs report feeling removed from donor strategic-planning processes. Several NGOs acknowledge some overtures by donors but say they are halfhearted and often come too late in the decision-making process. Many NGOs also note their own lack of time or ability to be involved in strategic programming. A few NGOs--most notably, larger ones--actively participate in country-level strategic planning and are encouraged about their level of participation and impact. Partnership Approach Many implementer comments address other donors' preoccupation with "process" concerns (as opposed to a "results" orientation). In their view, other donors pay relatively limited attention to the substance of NGO activities. Many implementers say they would like donors to limit their attention to details even more, while not adopting an entirely hands-off approach. Striking an appropriate balance between detail orientation and a hands-off approach is viewed as a "partnership" approach to development. In addition, some U.S. and indigenous NGOs believe that donors, including USAID, tend to use them as "contractors." Such an association does not advance the principle of two equal sides in the development relationship. NGOs say they would like to see donors take a more proactive approach to understanding organizations' goals, objectives, and needs--and funding these instead of predesigned projects that fit into a donor's overall development strategy. To work effectively, U.S. and indigenous NGOs say the donor-implementer partnership should incorporate NGO involvement in every aspect of the development process: strategic planning, needs assessment, and project design, implementation, and evaluation. By developing a more holistic approach, NGOs say, USAID and other donors can achieve better, more sustainable development results. Funding Arrangements Most donors do not program large amounts of money for directly funding indigenous NGOs.34 Instead, donors generally channel funding through NGOs based in their home country; these NGOs in turn team with local NGOs. The donor NGOs work with indigenous NGOs in a variety of capacities, such as awarding subgrants to NGOs, providing technical assistance, or jointly implementing a development project (SECOMA 1992). In interviews, U.S. and indigenous NGOs criticized the short-term funding arrangements of many donors. Numerous U.S. NGOs point to this strategy as an impediment to their own long-term strategic plans. Many interviewees link longer term funding to successful and sustainable project implementation. U.S. NGOs also expressed concern about USAID and other donors' practices of funding discrete activities rather than entire NGO programs. As a result, many organizations are left with the impression that donors are seeking a vehicle (or "contractor") to implement an activity rather than selecting organizations that have a commitment to a particular community or sector. U.S. and indigenous NGOs in general believe that greater donor resources should be devoted to capacity building of indigenous NGOs. Several funding mechanisms are widely used by all donors. These include matching grants, block grants, and umbrella mechanisms. The main NGO cofinancing mechanism of other donors is a 50-50 matching grant (Gwynn 1994). Many donors require NGOs to contribute to projects as a way of demonstrating commitment. Cost sharing is considered a way to ensure that U.S. or local organizations have "legitimacy" as independent entities with community-based support. Cost sharing also is viewed by donors as a way to encourage the financial independence of NGOs. Many indigenous NGOs claim to enjoy indirect relationships with donors. They say they do not want to deal with the various rules, regulations, audits, and reports demanded by donors. Through a partner or subgrant relationship, many local groups believe they are buffered from these issues. But a significant number of indigenous NGOs advocate direct relationships with donors. They feel this is the best way to develop true partnerships with donors and is essential to understanding donor organizations and developing long-term relationships. Consortium funding is a unique approach used by a number of donors, including Swedish International Development Authority, the Ford Foundation, the German development agency Deutsche Gesellschaft fur Technische Zusammenarbeit, the Norwegian Agency for International Development, the Danish International Development Agency, and the European Community. Consortium funding was developed as a mechanism to enable NGOs to receive and administer funding from multiple sources in a simplified manner. For example, standard formats for financial reports, progress reports, and project monitoring are developed; the recipient sends the same forms to all donors. In Bangladesh, some NGOs have begun to accept consortium funding. Such groups include Grameen Bank, Bangladesh Rural Advancement Committee, and Proshika. Their use of consortium funding has reduced administrative costs associated with reporting and monitoring, completing separate evaluations, and conducting multiple audits (Stallard 1993). Although U.S. NGOs, indigenous NGOs, and donors all like the concept of consortium funding, many pointed out the potential pitfalls of this approach, including limited flexibility and adaptability to changing local conditions. In addition, setting up "standard" procedures acceptable to all donors can be a time-consuming process. U.S. NGO, indigenous NGO, and donor interview responses indicate a need for more coordination among donors on types of programs funded, funding mechanisms, and reporting requirements. Among problems cited by implementers are excessive reporting requirements, poor management coordination, overlapping and competing development agendas, and overlaps or duplication in auditing and other financial requirements. To advance the effectiveness of NGO programming, implementers recommend that donors develop a common set of procedures, quality guidelines, and strategic plans. Project Design The study's literature review revealed that involvement of funding recipients in project design is critical to project success. However, donors have not been highly successful in this area (Gwynn 1994). Donors have made efforts to include NGOs in the design stage, but these efforts often do not work well. Among the reasons: inflexibility of many funding mechanisms, limited staff resources available for collaboration with NGOs, and continued resistance of some host country governments and NGOs to closer collaboration. U.S. and indigenous NGOs agree that donors need to give greater attention to local input before or during project design. They say NGOs should be consulted about the needs of local communities and should participate in strategic planning. Respondents say that this will result in more successful and sustainable development activities. U.S. NGOs, indigenous NGOs, and donors all say that USAID has a much better project design process than other donors. Although it is seen as detailed and labor-intensive, interviewees concede that USAID's attention to project design results in more carefully developed activities. Project Implementation Donors frequently praised USAID's rules and regulations, saying that these procedures hold the NGOs accountable for project implementation and for meeting goals, something many donor programs inadequately address. In many respects, donors view USAID's procedures as good standards to follow. U.S. and indigenous NGOs had mixed reactions on questions comparing USAID's rules and regulations with those of other donors. Although they overwhelmingly agree that USAID's rules and regulations are more difficult than those of other donors, they made both positive and negative comments about the consequences of USAID's regulations. Overall, U.S. NGOs seem to have less difficulty with USAID's rules. Although the NGO community experiences more problems following Agency procedures, NGOs often commented on the benefits of these rules. Advantages include gaining credibility by working with a large donor organization, learning how to handle large sums of money, and acquiring skills in proposal design and preparation. In general, NGOs agree they learn a great deal while working with USAID that enables them to subsequently approach other donors for funding. Negative remarks from U.S. and indigenous NGOs centered on the implied lack of trust resulting from the Agency's strict application of its rules and regulations. Several respondents say that NGOs' decision-making responsibility seldom is commensurate with their project implementation accountability. NGOs say they are confused about USAID's and other donors' administrative processes, such as decision-making procedures and funding sources. Recipients of USAID funding claim to be overwhelmed by its multiple funding sources (central, regional, and Mission) and its layers of decision-makers (civil service, foreign service, foreign service nationals, personal services contractors, political appointees, and others). U.S. and indigenous NGOs admonished all donors, including USAID, for not having a better balance between accountability and flexibility. Implementers criticized donors for simply giving out funds and then disappearing. At the same time, USAID is seen as too process oriented and paying too much attention to rules. U.S. and indigenous NGOs would like to see the donor community develop a true partnership, with both parties treated as equals in decision-making responsibility, management, and accountability. Many U.S. and indigenous NGOs say USAID gives more attention than other donors to monitoring and evaluation activities. The literature review supports this, finding that donors have not been good at systematically evaluating NGO initiatives or helping them develop their internal capacity for conducting benchmark studies or project evaluations. In addition, evaluations appear to have examined more closely administrative and implementation problems than impact or sustainability (Gwynn 1994). For their part, donors often say that it is NGOs that limit monitoring and evaluation efforts, owing to their lack of understanding of the benefits of these activities. Many NGOs say they have a limited capacity to evaluate their own activities because of the high cost of evaluations and the limited value donors place on them. U.S. and indigenous NGOs repeatedly asked for more support to develop their capacity to undertake data gathering, analysis, and evaluation. Several indigenous NGOs recommend that donors increase their emphasis on monitoring and evaluation during project design. This could be accomplished by devoting more time and attention to developing the necessary skills within the NGO community and by providing resources to educate and raise the awareness of NGOs regarding the benefits of establishing benchmarks, gathering data, and monitoring results. Capacity Building Donors need to pay greater attention to the institutional strengthening of NGOs. Reasons for this lack of attention are, first, the insufficient funds that donors have thus far devoted to capacity building and, second, donors' inability to develop a systematic approach to institutional development (Gwynn 1994). Although many Canadian and European donors use a "partnership" approach to funding (i.e., linking international and local NGOs), the lead organizations have not performed well in developing their counterparts' capacity (Cameron and Cockey 1993). NGOs interviewed for this assessment believe donors should give greater attention and resources to institutional strengthening. They feel that donors' emphasis on funding projects for short periods runs counter to the institutional needs of their organizations. Most NGOs say that longer term funding of activities, coupled with resources to support capacity building, are essential. Implications for USAID Each donor has its strengths and weaknesses in project administration. Highlights of USAID characteristics, identified through informant interviews and the literature review, are given in this section. USAID's NGO programming has a number of strengths, including USAID's in-country presence and its direct relationship with U.S. and indigenous NGOs. In comparison with donors using centralized decision-making, USAID's in-country presence gives the Agency the ability to work in a more interactive, partnership manner. It also provides greater speed and flexibility in decision-making. And maintaining locally based staff provides the Agency with substantive information on local conditions and changing political, social, and economic environments. In contrast to some donors, USAID often funds NGOs directly, with no U.S. NGO partner. When there is sufficient NGO capacity, U.S. and indigenous NGOs agree that this probably is the most beneficial arrangement. Although working directly with USAID can entail more paperwork and financial regulations, it also permits greater dialogue with Agency officials. Many organizations feel the Agency's hands-on attitude allows the NGO community to learn about Agency operations while simultaneously receiving continuous input and assistance with implementation problems. Many indigenous and U.S. NGOs believe other donors should adopt similar strategies that increase the likelihood of partnership approaches to development. Informants mentioned three weaknesses regarding USAID's NGO programming; these are supported by the literature review. Weaknesses are the Agency's lack of attention to capacity building and sustainability issues, its short-term approach to project funding, and its emphasis on detail and process. U.S. and indigenous NGOs unanimously agree that the Agency's limited funding for capacity building is a deficiency. Rhetoric about institutional strengthening and sustainability abounds, but concrete ideas often are left out of project designs and implementation procedures. Respondents note that among donors, the Canadian International Development Agency is known for its emphasis on project and organizational sustainability.35 As for short-term funding, U.S. and indigenous NGOs in several countries cited instances in which Missions funded an activity for 2 or 3 years and then expected the organization to find outside funding. In several cases, Mission staff perceived the activity as successful (and sustainable) because the organization obtained funding from other donors. Many NGOs question this logic and applaud the efforts of some donors, such as the Swedish International Development Authority,36 to fund entire NGO portfolios for longer periods. Although several positive outcomes were noted regarding the Agency's rules and regulations, U.S. NGOs, indigenous NGOs and donors generally feel these requirements are a stumbling block to effective implementation. Some respondents believe the Agency's attention to rules and regulations may limit its ability to be innovative. In addition, key informants say that USAID's strict attention to details, such as accounting and reporting procedures, limits the Agency's ability to concentrate on project impact. NGOs report that USAID's rules sometimes hinder implementation and that they also can impede flexibility and trust--two qualities greatly appreciated by the NGO community. The Agency's preoccupation with holding its staff accountable for details limits its capacity to work in a more hands-on, partnership style. More direct involvement with NGOs requires greater time, resources, and energy, which may be precluded by mandated reporting requirements. Conclusions The most important issue emerging from the analysis of other donor practices is that USAID and other donors need to change their philosophies regarding NGOs away from process concerns to those of substance and impact. Although practices differ from setting to setting, in the long run a process-oriented approach may be more likely to have a negative influence on effective project implementation. Other themes that emerged from the review of other donor practices include the necessity for longer-term funding of NGO portfolios and for capacity building and development of more substantive relationships with the NGO community. Some donors have successfully addressed some of these issues. The Swedish aid agency, for example, has been able to establish longer term relationships aimed at institutional as well as project-level sustainability issues. By applying these lessons, USAID should be able to develop long-lasting partnerships with the NGO community and obtain better results for its development activities. Appendix D3: References Cameron, Catherine, and Jane Cockey. 1993. The Evaluation of NGO Activities, Organizations, Methodology and Results. Paris: Overseas Development Administration. Gwynn, Cheryl. 1994. Literature Review of Other Donor Funding of NGOs. Development Alternatives, Inc. Washington: USAID. SECOMA. 1992. Evaluation of CIDA's Non-Governmental Organization Program-Final Report. Vol. I. Ontario: Canadian International Development Agency. Stallard, Janice. 1993. Donor NGO Funding and Collaboration: Summary of Findings from Bangladesh. Center for Development Information and Evaluation. Washington: USAID. APPENDIX E: PERSPECTIVES OF USAID SENIOR MANAGERS Evaluators solicited the views of senior USAID managers to complement those received from project officers who work directly with PVOs and NGOs. Nineteen interviews were conducted with senior officers based in Washington and Mission directors and their deputies based in the field.37 Although the sample was relatively small, as a percentage of total Agency policymakers, it was substantial. The experience and knowledge of these officers varied significantly and yet, cumulatively, covered a wide range of issues and a diversity of perspectives regarding the Agency in general and PVOs/NGOs in particular. Those interviewed included political appointees with little more than 2 months on the job, to "old field hands" with well over 20 years of international experience and service with the Agency. In most cases, interviewees have had experience both in the field and in Washington. Interviews of senior managers were far less detailed than those of PVO, NGO, and USAID project managers, and were designed to elicit broad-based perspectives and insights on a range of PVO/NGO and USAID programmatic and policy concerns. This appendix provides findings and conclusions from the interviews. Management Capabilities Regarding the respective management capabilities of PVOs and NGOs, interviewees considered both PVOs and NGOs capable of attracting and retaining qualified staff, but for different reasons. Although PVOs generally were viewed as having stronger internal management capacity and being better able to comply with USAID regulations and procedures, respondents also believed that over time NGOs could achieve a level of institutional capacity adequate to meet USAID requirements. In general, senior managers saw no distinction between PVOs and NGOs regarding potential effectiveness. The following responses elaborate these views: NGOs have thin management capacity and often depend on a charismatic leader; PVOs are starting to attract more well-rounded staff, but high turnover is decreasing the number of "old hands" who work for them, with possible negative effects. Internal management warrants higher and more focused USAID attention--more so for NGOs than PVOs, but there is a wide diversity of capacity in both communities. . . . USAID regulations are a horror, and we must be willing to help both PVOs and NGOs work within it in a more partnership approach that achieves both sets of institutional objectives. PVO skills have improved over the years, both in the United States and in the field, and in part due to USAID-financed institutional strengthening. PVOs' ability to attract and retain qualified staff is due in large measure to USAID funding. Local NGOs have weak management capacity and operate in an autocratic manner, which discourages highly qualified staff from staying. . . . NGOs that received capacity strengthening through PVOs tended to be stronger than those that received it directly from USAID. PVO staff are motivated with career choices available to them; if individuals choose to join a PVO it is because they want to. . . . People who join NGOs are normally more technically competent than [those who join] PVOs, reflecting the lack of employment opportunities in developing countries. There is tremendous variability in PVO management ability. . . . NGOs are weak and often burdened by domestic political agendas. They have problems trying to compete with the private sector in low-income countries over the same pool of experienced individuals. PVO/NGO Sustainability The consensus among senior managers was that neither PVOs nor NGOs are successful in ensuring sustainability of their organizations or programs. More respondents felt that NGOs were more successful in achieving sustainability and had far greater potential to do so: Sustainability is a good idea, but we need flexibility. . . . With NGOs it is almost impossible to get a 25 percent matching contribution and it is difficult to calculate in-kind support. Cost recovery is always a good idea but must be achieved over time for sustainability to occur. PVOs are not interested in developing capability for achieving sustainable results. . . . They get mixed messages from USAID on the importance of sustainability. NGOs are better at achieving sustainability. They can't leave. PVOs have questionable capability to achieve sustainability. Sustainability is a function of many factors over which PVOs and NGOs have no control, including host government support, a stable economy, environmental changes, population growth, peace, etc. The issue is a waste of time. The U.S. has not been able to make it happen at home or in USAID projects generally, so why the preoccupation with making it happen in other countries? PVOs' success at sustainability is a mixed bag. It depends on what their role in the country is. Do they want to leave anything behind, and if so, what? Sustainability is harder for NGOs working in poor countries where they cannot build a membership base or raise funds easily. USAID's Objectives in Funding PVOs and NGOs One line of questioning was aimed at understanding senior manager views on 1) the purposes of USAID funding for PVOs and NGOs by central and regional Bureaus and field Missions; 2) the worthiness and actual attainment of these objectives; 3) the relative priority of funding among these three levels, including allocation of budgetary resources; and 4) the capabilities of each level to manage and support PVO/NGO activities. Discussion of centrally funded programs dwelled primarily on the Matching Grant Program (MGP) of the Bureau for Humanitarian Response's Office of Private and Voluntary Cooperation (BHR/PVC). Also discussed was the PVO/NGO program of the Bureau's Office of Food for Peace (BHR/FFP). Responses about regional Bureau and field Mission objectives tended to address more their overall rationale for supporting PVOs and NGOs than to address specific programs. These questions were asked only of Washington-based managers. Central (Bureau for Humanitarian Response) Program Support According to BHR/PVC, the principal objective of the Matching Grant Program is to build the capacity of fairly well established PVOs to branch out into new program areas. PVOs with a good track record, that can provide a 50 percent match, are eligible for support. The following are senior managers' descriptions of MGP objectives: To provide a means for PVOs to leverage additional funding to that gained in their own fund-raising; or, put differently, to leverage additional resources to those provided by USAID in centrally funded activities. To permit the Agency to meet congressionally mandated earmark requirements. To strengthen the Bureau's FFP implementing partners, some of whom work in difficult circumstances. To build a PVO constituency for USAID and American foreign assistance. To avoid the negative views held by field Missions of PVO programs, primarily in food aid and humanitarian assistance. To provide easier cross-country, cross-regional sources of funding; or, to provide a central service for multicountry, multiregional activities. To fund expansion of ongoing PVO programs (let them do more of the same). The feeling was that these objectives were worthwhile given "real world" conditions, including political realities. But the opinion was more negative when these objectives were discussed in "truly" developmental terms, or in light of whether they were actually being accomplished or if they meet Mission objectives and needs. Roughly half the respondents argued that if the Agency is hit with budget cuts, central Bureau grants to PVOs should be among the first to go or, at least, should not be protected from such cuts. A slight majority felt funds for PVOs could be better utilized and managed by field Missions. A minority felt that central bureau PVO programs could be shut down altogether, with funds transferred to the field. The outlook of BHR managers was, not surprisingly, different. They felt that moving these programs to the Missions would be to place them in the hands of "unsympathetic" field staff who would sacrifice important Agency objectives for the more narrow interests of the Missions (e.g., through "projectizing" food aid). The following comments reflect the views, pro and con: If the PVO is sufficiently mature and competent to qualify for this [MGP] program, it should be able to undertake its own further capacity growth without USAID assistance. MGP has been very successful in helping PVOs gain expertise and develop program models in such areas as microenterprise and child survival, which will have far-reaching spinoffs in the developing world, and at a relatively small overall budget ($45.0 million in 1993). MGP is not worth it because you are not going to leverage very much additional funds for the amounts involved. I don't think the purposes are being accomplished because PVOs are still beating up on us; they are not happy. Regional Bureau Program Support All the regional Bureaus provide funding to PVOs. The perceived objectives of these programs are at least as diverse as those of the centrally funded MGP. Senior managers identified these program objectives: -- To fund PVO participation in regional sectoral programs (e.g., the environment, microenterprise) -- To get PVOs involved in new sectors (e.g., democracy and governance, natural resource management) of strategic importance to the regions; or to promote new ways of doing business, such as with private companies or regional networks (e.g., women's political network) -- To permit achievement of a regional Bureau mandate such as the Development Fund for Africa -- To stimulate indigenous NGO growth and, in a larger context, civil society through PVOs -- To respond to PVO lobbying efforts--that is, providing funding in response to political clout -- To maintain a USAID presence in countries where a Mission has closed out; or as a means to provide assistance in countries with no USAID Mission A majority of respondents distinguished between the BHR and regional Bureau programs in that the latter provide support to PVOs within a strategic framework developed with and for their field Missions, while the former responds directly to PVO plans and needs. Thus, while the regional Bureaus also fund PVO capacity building, it is for the express purpose of encouraging them to tackle new sectoral activities consistent with new program directions the Bureaus have set for themselves. It is clearly understood that once their portfolios have become better defined and programmatically sound, regional Bureau funded PVO activities will be phased out and left for the Missions to fund or not as needed. In a similar vein, regional Bureaus have funded PVO programs because Missions did not have the staff to adequately design and manage such programs (the example given was the large number of new country programs begun in Europe and former Soviet Union). But the principle remained the same: once Missions were capable of assuming management responsibility, the regional Bureau would phase out of this role. In general, respondents felt Bureau objectives were worthwhile and, overall, were being achieved. A significant number of respondents expressed views similar to those about BHR programs that is, the need to maintain Bureau responsibility for funding of PVO activities and not turn them over to the Missions until, in the words of one respondent, "the negative attitudes of Mission directors toward PVOs have changed." In this regard, some directors apparently see PVOs as annoyances, simply creating "patches of green" amidst general conditions of poverty and thus not contributing directly to development. As for which locus merits priority funding for PVO activities, opinion split about evenly between regional Bureaus and field Missions. When faced with the possibility of budget cuts for the Bureaus, though, most respondents felt PVO programs should hold no special priority within the larger program portfolio. Mission Program Support Senior managers listed a variety of objectives that reflect their perceptions of why field Missions fund PVOs and NGOs. At least one senior manager noted that a major reason for funding PVOs involved domestic politics, in some cases when Missions believe that programs funded are not consistent with their country strategic plans. There was, however, virtual unanimity that the only valid approach for supporting PVOs and NGOs is within the framework of a Mission's strategic objectives. The following represent senior managers' perceptions of Mission objectives for funding PVO/NGO activities: -- To use PVOs/NGOs as the most efficient way to carry out projects and achieve country objectives -- To complement host country governments in providing public services -- To provide continuity, through PVOs, to Mission-funded NGO programs -- To carry out whatever the Mission's development objectives are at either project or program levels (i.e., to serve as implementing agents) -- To support politically sensitive activities (e.g., human rights, family planning), which may be inappropriate through direct bilateral programs -- To promote (through NGOs) a force for change in both sectoral activities and in the larger society (strengthening civil society) -- To respond to the political pressure of PVO lobbying or congressional earmarks The respondents felt all these objectives, except the last one, were worthwhile, although their achievement was by no means uniform. For instance, while most senior managers agreed PVOs bring motivation and energy to the work of development, they feel the qualities are largely being negated because of USAID's management regulations and accountability requirements, which keep them, as one manager put it, "behind their desks pushing papers rather than in the field promoting change." As discussed above, a significant number of senior managers feel that, while in principle the Missions may be the priority locus of Agency support of PVOs/NGOs, in practice their attitudes are not always positive toward these groups (PVOs in particular). Some doubted that overall achievement of Agency objectives and mandates can be ensured through field Mission activities, so there remains a justification for central and regionally funded PVO/NGO programs. Umbrella Mechanisms Respondents generally agreed that most Missions prefer funding NGOs directly rather than through an umbrella intermediary, normally a PVO. A number of senior managers argued a direct relationship contributes to achievement of important Mission objectives. However, they noted that the reality of limited Mission capacity to work directly with NGOs often limits this preference and leads, in most cases, to using a PVO intermediary. One senior manager pointed out that umbrella mechanisms have little to do with increasing or decreasing the chances of achieving a Mission's developmental objectives; they are used for administrative ease for management (not program) purposes. The following responses provide a cross-section of senior manager views: While considerable funding can be wasted through umbrella projects, it is a good compromise in an imperfect world. Umbrellas are more costly and time consuming, and the intermediary organizations are less likely to develop a partnership arrangement with USAID; they have their own agenda, which gets in the way. Every opportunity to work directly with NGOs should be taken if they are strong enough; however, we must question USAID's capacity to work directly with local groups, given staff resources, time, management, etc. Umbrellas are better because they lighten a Mission's management role and burden, and circumvent attitude problems at the Mission level. In exchange for management efficiency, Missions lose their hands-on relationship with local NGO communities because the umbrella becomes a buffer; moreover, PVOs can be patronizing. USAID's Evolving Relationship with PVOs and NGOs As discussed in various ways in the preceding sections, PVOs and USAID have engaged in dialog for the better part of two decades over the appropriate role of PVOs in formulating and implementing American foreign assistance policy. The next two sections deal with senior managers' comments on questions concerning 1) the validity of funding "autonomous" PVO/NGO programs with public funds and 2) the appropriate degree and actual influence of PVO/NGO collaboration in developing strategic plans at all levels of the Agency. PVO/NGO Program Autonomy or Alignment Should USAID funding to PVOs/NGOs be undertaken to support their own programs and thus ensure their autonomy, or should funding be in alignment with USAID-defined country program objectives? This was the principal issue raised of senior managers in this series of questions. Interviewees generally supported the view that PVOs and NGOs deserve support only if they can assist in fulfilling USAID's objectives. But a significant minority believed funding should sometimes support PVO/NGO activities independent of USAID objectives. The following responses provide rationales for these views: If USAID is going to be held accountable for the impact of PVO/NGO activities, their programs must be closely aligned with our own. Perhaps if the nature of the USAID PVO/NGO relationship were different--a truly competitive grants program, for instance--then we could lean toward funding independent activities. PVOs have the perception that they deserve funding, if for no other reason than they are PVOs. Even if this were valid, a programmatic rationale also must be present. I'm not sure whether USAID has found that rationale. We should fund both independent and "instrumentalist" programs of PVOs/NGOs. At the macro or programmatic level, funding should support USAID's purposes; at the micro or project level, funding can be made to PVO/NGO initiatives. It is important to fund PVO/NGO-initiated activities as they are likely to have greater commitment and thus become sustainable. PVOs/NGOs should work in close alignment with USAID objectives. Missions are responsible to the Agency, Congress, and ultimately the American people for identifying problems and judging how to address them. . . . We will make partnerships with those who share our views. PVOs/NGOs should work in close strategy alignment with USAID. The Agency should provide broader, long-term strategy direction since it understands the economics of development. PVOs and NGOs understand only micro, local-scale development. Projects should be closely aligned with USAID objectives and strategy, but when a PVO/NGO project is well chosen, then give them the go-ahead to implement it without close supervision. There is no correlation between heavy USAID involvement and a successful PVO/NGO activity. Strategic Collaboration In general, senior managers felt PVOs have had a reasonable level of influence in developing USAID's strategic plans, although probably not so much as PVOs would have liked. NGOs, by contrast, have had far less influence, and a majority of respondents felt this is an area in which the Agency needs to increase its efforts. The following responses illuminate these findings: PVOs have had some influence on strategy formulation in that they have caused the Agency to pay more attention to agriculture, natural resources, and equity issues; however, it is probably not as much as they [PVOs] would like. There needs to be more input in terms of substantive dialog, but not in the review of USAID documents. . . . Let USAID be the bureaucrats. Both PVOs and NGOs have had an influence on our strategies, but we could do more at the Mission level. In particular, we could do a better job of explaining why we do or do not fund their activities. PVOs have had an influence on the formulation of USAID's strategies. It has been satisfactory but not well rounded in that the process has been dominated by a small number of larger, more active organizations. NGOs need to be more intensively consulted. More recently PVO/NGO collaboration and influence have increased. . . . PVOs have had a historic influence on the Hill with such issues as microenterprise and child survival. USAID must determine its own general areas of involvement in the beginning, but then it is important to increase PVO/NGO involvement. Once broad sectors are identified, PVOs/NGOs should be brought in to help define the specifics, given their knowledge of country conditions. Ours is a politically driven program. While PVO/NGO input and collaboration are desirable in most countries, that is not the case here where bilateral issues (such as trade and immigration) define the nature of our programs and objectives. Implicating PVOs and NGOs in strategy development is not useful. They often want things that are not sustainable and don't increase per capita income, but rather meet immediate needs. Perspective is too short-term to be able to focus on long-term development. [They have] only marginal influence in USAID's development strategies because they don't have the wherewithal, the human capital. . . . Would like to see much more involvement of NGOs because they serve as proxies for the grass roots. PVO influence is too strong, and they exercise undue influence. Too much of our time is spent in trying to satisfy the felt "greeds" of PVOs. Conclusions This section analyzes and interprets interviewees' statements. The purpose is to raise issues and stimulate thinking in areas where much discussion has taken place but no definitive actions have been taken. Management capability and sustainability: Senior managers saw no overwhelming differences between PVOs and NGOs regarding their capacity to undertake successful development projects or to ensure project or organizational sustainability. The far longer experience of PVOs in international development and their greater access to financial resources need to be balanced against NGOs' local presence and commitment to grass roots development. Agency policymakers should reflect on where budget allocations can be best invested for long-term effect. USAID objectives: Interviewees raised a number of critical issues on USAID's objectives in funding PVOs and NGOs. The most important issue is finding the proper locus of decision-making within the Agency. Should it take place at the central Bureau level or in the field Missions? This age-old debate, not limited to PVO/NGO policy considerations, concerns the proper role and functions of central Bureaus relative to field Missions. Policy concerns that flow from this issue, and corresponding decisions that must be made, include the following: The role of PVOs in USAID implementation: What is the appropriate role of PVOs in program and project implementation as the Agency downsizes and Missions lose staff or are closed out altogether? While this issue pertains primarily to field programs, there are a number of central and regional Bureau programs that utilize or support PVOs to accomplish Agency objectives. In an era of dwindling resources for official development assistance, this question raises a number of policy concerns: 1) Should USAID support PVOs only insofar as they contribute to achievement of USAID objectives (at any level), or should the Agency support activities that are initiated by PVOs and which may or may not contribute to USAID's objectives? 2) What does "partnership" between the Agency and PVOs mean if implementation of USAID strategies is the only or the primary role of PVOs in USAID programs? 3) What is the relative advantage of using PVOs in an implementation capacity versus using contractors or universities? PVOs versus NGOs: As we enter the last half of the fourth development decade, a major question is the proper role of PVOs (or international NGOs in general) in the national development efforts of our partner countries. More specifically, what is the role of PVOs relative to their NGO counterparts? How does USAID's capacity to work with and manage support to NGOs affect its development objectives in host countries, particularly the objective of promoting sustainability? Should a fund for NGO capacity building be created along the same lines and for the same objective as the Matching Grants Program is for PVOs? Is it possible or necessary to develop criteria for supporting one set of actors over another, particularly in the absence of a declared policy? These two sets of policy concerns as well as the larger issue of the proper locus of decision-making and implementation responsibility for PVO and NGO support must be dealt with if USAID programming is to lead to achievement of overall Agency objectives. Absent such decisions, relationships within USAID and between it and the PVO/NGO communities will continue to be marked by tension and less-than-optimal use of our foreign assistance resources. USAID's evolving relationship: On the issue of PVO/NGO participation in USAID strategy development and program implementation, senior managers generally oppose greater involvement. And yet, these same managers admit PVOs and NGOs have played an indispensable role in implementing U.S. foreign assistance. One suspects there has been far greater PVO/NGO influence in policy and strategy formulation, albeit indirectly and informally, than is indicated by the specific responses cited above. Senior managers say the USAID PVO/NGO relationship has an inherent element of conflict, and this is not necessarily bad. In a democracy, the making and execution of public policy depends on expression of a variety of views and interests and a recognition that nongovernmental actors have a legitimate role in both formulation and implementation. Much of USAID's relationship with PVOs and NGOs has been driven by the need for accountability, for it is after all a public agency entrusted with public funds. If PVOs and NGOs are going to enter the public domain and participate with USAID in the delivery of American foreign assistance, they must accept and share the responsibility that goes with it. BIBLIOGRAPHY Advisory Committee on Voluntary Foreign Aid. 1988. The Effectiveness of Private Voluntary Organizations. Washington: USAID. Alfonso, Arelis G¢mez; Nan Borton; and Carlos Castello. 1992. Apex Study of the Asociaci¢n de Grupos Solidarios de Colombia. Washington: USAID. Atwood, J. Brian. 1993. Statement of Principles on Participatory Development. Washington: USAID. Ba, Thierno; Robert H. Brandsetter; and George Butler. 1991. Community and Enterprise Development Project/PVO Component. Washington: USAID. Borton, Nan. 1992. Strategies for Sustainable Development: Essential Elements in Planning and Implementation for Benefit Sustainability. Washington: Automation Research Systems. Buzzard, Shirley. 1991. The Cooperative Agreement Between AID/FVA/PVC and Private Agencies Collaborating Together (PACT): An Evaluation Report. Washington: Heartlands International Consultants. Cameron, Catherine, and Jane Cockey. 1993. The Evaluation of NGO Activities, Organizations, Methodology and Results. Paris: Overseas Development Administration. Canellas, Andrew; Michael Hirsh; and Cressida McKean. 1989. Microenterprise Stock-taking: Guatemala. Washington: USAID. Checchi and Company. 1989. Evaluation of Experience of USAID Missions with PVO Umbrella Groups in Costa Rica, Guatemala, Honduras, Haiti, and Jamaica. Washington: USAID. Cotter, Jim. 1988. USAID-Financed PVO Co-Financing Projects in the Philippines and Indonesia: Reshaping the Arms of PVO Collaboration. Washington: USAID. El-Sanabary, Nagat. 1992. Evaluation of Programs for Afghan Women Funded by the O/AID/Rep and the Asia Foundation. Washington: USAID. Finsterbusch, Maussolff, and Van Wilkin. 1992. Factors Contributing to the Effectiveness and Sustainability of Development Projects. Washington: International Development Management Center. Fries, Tami. 1994. Sustainability: A Literature Review. Washington: Development Alternatives, Inc. General Accounting Office. 1982. Changes Needed to Forge an Effective Relationship Between A.I.D. and Voluntary Agencies. Washington: GAO. General Accounting Office. 1993. Foreign Assistance: Improvements Needed in A.I.D.'s Oversight of Grants and Cooperative Agreements. Washington: GAO. Gwynn, Cheryl. 1994. Literature Review of Other Donor Funding of NGOs. Development Alternatives, Inc. Washington: USAID. Harkins, Allen, and Azucena Quan. 1990. Evaluation of Belize Enterprise for Sustained Technology. Washington: USAID. Institutional Reform and the Informal Sector. 1991. Project Implementation Document: FSDP. IRIS. Washington: USAID. John Snow, Inc. 1991. Promoting NGO Sustainability: Lessons from Enterprise. Washington: John Snow, Inc. Kean, John, et al. 1987. Analysis of Institutional Sustainability: Issues for USAID 1985-86 Project Evaluation Reports. Devres, Inc. Washington: USAID. Kelly, Margo. 1994. Umbrella Activities Literature Review. Working paper. Development Alternatives, Inc. Washington: DAI. Kraljevic, Kvo J. 1993. Midterm Evaluation of the Nongovernmental Organizations Component of the Cochabamba Regional Development Project. Washington: USAID. Management Systems International. 1992. A Review of the Quality and Coverage of A.I.D. Evaluations, FY 1989 and FY 1991. Washington: MSI. Management Systems International. 1993. RENARM, USAID, and Central American NGOs: Voices from the Field; a survey of perceptions and opinions. Washington: MSI. Morss, Elliot, et al. 1982. Toward Self-Reliant Development: A Guide for Evaluating the Sustainability of Project Benefits. Washington: Development Alternatives, Inc. Otto, Jonathan, and Anne Drabek. 1992. Designs for Collaboration: A Study of PVO/NGO Umbrella Projects in Africa. Washington: USAID. Russell, Diane, et al. 1994. Sustaining the Vision: Lessons for USAID's Move Toward Sustainability and Sustainable Development. Washington: USAID. SECOMA. 1992. Evaluation of CIDA's Nongovernmental Organization Program-Final Report. Volume I. Ontario: Canadian International Development Agency. Stallard, Janice. 1993. Donor NGO Funding and Collaboration: Summary of Findings from Bangladesh. Washington: USAID. Tendler, Judith. 1982. Turning Private Voluntary Organizations Into Development Agencies: Questions for Evaluation. Discussion paper. Washington: USAID. USAID. 1982. A.I.D. Partnership in International Development With Private and Voluntary Organizations. Policy paper. Washington: USAID. USAID. 1989. Project Assistance Completion Report: Zio River Development Project, Togo. Washington: USAID. USAID. 1989. Sustainability of Development Programs: A Compendium of Donor Experience. Discussion paper. Washington: USAID. USAID. 1990. Project Paper: PVO/NGO Support Project. USAID/Senegal. Washington: USAID. USAID. 1990. Project Paper: South Pacific Regional Family Planning. Washington: USAID. USAID. 1993. PVO Cofinancing II Project: PACR. Washington: USAID. USAID. Undated. The A.I.D-PVO Partnership. Washington: USAID. USAID and the Private Voluntary Community. 1993. Policies for a More Effective Partnership: A Report to the Administrator of the Agency for International Development and the Advisory Committee on Voluntary Foreign Aid from a Joint Task Force established by A.I.D. and InterAction. Washington: USAID. VanSant, Jerry. 1986. The Role of Nongovernmental Organizations as Intermediaries in Decentralized Rural Development. Chapel Hill: Development Alternatives, Inc. VanSant, Jerry. 1987. Benefit Sustainability. Washington: Development Alternatives, Inc. NOTES 1This summary is based on material found in USAID Handbook 13, chapter 1, section B.2.a., and in Handbook 3S, supplement A, chapter 1, section 3. 2See USAID Handbook 1B, chapter 25, section 25E, for the selection criteria for choice of implementation instrument. 3This section looks only at differences in USAID/W and Mission management of PVO/NGO activities, since interviewees provided little information on management of regionally funded activities. 4See the general definition of "PVO" in chapter 1. USAID refers to registered organizations as either "U.S. PVOs" or "local PVOs." 5See USAID Handbook 13, chapter 4, for a fuller discussion of preaward reviews. 6Source: USAID Handbook 13, chapter 1, section b.2.b. 7This summary of the roles of contracts officers and project officers is taken from Handbook 3S, supplement A, chapter 3. A useful table illustrating the respective roles of the contracts and project officers can be found in appendix B of USAID Handbook 3S. 8Requirements for establishment and operation of financial management systems, based on OMB Circular A-110, are extensively documented in USAID Handbook 13. In addition, payment provisions are outlined in Treasury Circulars 1075 and 1083 and summarized in Handbook 13. Audit requirements are found in OMB Circular A-133. 9These methods are explained in more detail in OMB Circular A-110 and the Standard Provisions for U.S. and non-U.S. NGOs, contained in Handbook 13. 10Definitions of "sustainability" are discussed more fully in appendix D1. Briefly, "organizational (or institutional) sustainability" refers to the continued existence of the PVO or NGO. "Financial sustainability" refers to the organization's ability to recover costs, allowing it to continue to provide services. Project financial sustainability is achieved when an activity is financially supported by those it serves, including in-kind support or financial or technical assistance. 11To conform to international conventions, in this section the term "NGO" refers to a not-for-profit organization engaged in international development work. It can be based either in the donor's home country or in a developing country. 12Of course, there is a distinction between "inconsistency" and the flexibility needed to deal with varying country circumstances. But respondent comments indicated that, other than individual USAID staff preferences, in many cases there appears to be no underlying reason for management inconsistencies. 13The literature review, completed by Tami Fries of Development Alternatives, Inc., consisted of analysis of 51 reports, evaluations, and studies on USAID's PVO and NGO programs. It is available from CDIE as a working document. 14Sustainability is one aspect of the sustainable development process. For a further discussion on sustainability and sustainable development, see Russell et al. 1994. 15A 1987 evaluation of the Indonesia PVO Cofinancing Project (Betts, A Strategic Assessment of NGO Development in Indonesia) found that local NGOs' capacity to create a positive policy environment is a more important contribution than direct delivery of services. 16This finding is supported by a 1982 General Accounting Office report, which noted that rigid cost-sharing requirements discouraged smaller (and perhaps more innovative) organizations from applying for programs that require a matching component (GAO 1982). 17See International Development Management Center, 1991, The Logical Framework and Benefit Sustainability, for further information on incorporating benefit flows into project designs. 18For example, the success of a USAID-funded activity in Belize suggests it was the solid partnership between the government, the private sector, and PVOs that allowed resources to be directed most effectively and that directly contributed to the project's sustainability (Harkins and Quan 1990). 19This occurred in the Zio River project in Togo, where a PVO took over provision of many agricultural services in a given area. As a result, the government was discouraged from providing assistance to the region and discontinued extension services to this area (and other areas) that had been taken over by PVOs or donor programs (USAID 1989). 20Efforts also can be made to look to informal groups (e.g., neighborhood associations, ethnic groups, and extended families) through which investments can be channeled. 21Case studies from Indonesia and the Philippines show that training exercises to increase NGO skills in financial management, project monitoring, and evaluation are critical to project success (Cotter 1988). 22In this study, 55 percent of the projects with high organizational capacity ratings also had high overall sustainability ratings (Kean 1987). 23The Agency has developed a set of guidelines to be followed when working with local organizations in microenterprise development. These guidelines are outlined in Policy Determination 17, Microenterprise Development Program Guidelines. 24A USAID respondent referred to a U.S. PVO in Swaziland that worked in a sector the government seemed unable to service. After several years of PVO success in distributing commodities, the host government appeared resentful of the organization's funding. Although the PVO continues to operate, it now receives no government support. The PVO may be forced to cease services if outside donor assistance ends. 25Some informants say that USAID places too much emphasis on sustainability, to the detriment of examining project impact. 26One USAID contracts officer believes it is better to use PVOs as umbrellas because they have more developed management capabilities than local NGOs and can better isolate themselves from political pressures or corruption in selecting local NGOs for subgrants. 27One USAID respondent says that a Mission would need six to eight personal services contractors to do what one PVO umbrella grantee does. 28For example, the USAID South Pacific Regional Development Office funded a project through a PVO and the South Pacific Alliance for Family Health to promote population and family planning activities in the region. Using this mechanism, USAID was able to provide assistance to ten cooperating countries and their NGO communities through one funding arrangement (USAID 1990). 29Both local NGOs and Mission staff comment that they feel out of the loop regarding disbursements and expenditures because these activities are often handled by a PVO's home office in the United States. There often seems to be a lack of transparency between the PVO, the local NGO, and USAID. 30A 1992 study of African NGO umbrella grants found that umbrella activities are a flexible mechanism for enlarging NGO operations and improving their institutional capacities (Otto and Drabek 1992). 31In general, PVOs and NGOs appeal for more funding in areas such as operating costs (e.g., personnel, equipment, and other expenses), training, marketing, strategic planning, and technical assistance. 32To conform to international conventions, in this appendix the term "NGO" refers to a not-for-profit organization engaged in international development work and based either in the donor's home country or in a developing country. 33See USAID 1982, for its guiding principles for working with U.S. and indigenous NGOs. 34Several donors say they set aside small sums of money for small-scale NGO initiatives, which seems to be a major source of direct funding for local NGOs. These small grant programs often award noncompetitive, short-term grants, which allow local NGOs a great deal of flexibility and independence. NGOs have mixed feelings about these programs, though; they enjoy the flexibility and creativity the funds allow but are limited by the grants' short-term nature and the donors' lack of participation in activity design, implementation, and evaluation. 35This includes project sustainability that concentrates on community involvement and on organization sustainability, including technical assistance on how to run an organization--such as training, strategic planning, and management and administration. 36SIDA was mentioned as an example of how donors should work with NGOs. SIDA's approach to NGOs is longer term; it often funds an NGO's entire portfolio. In general, SIDA does not use NGOs to implement its own strategic objectives, but to complement SIDA's bilateral government funding. 37USAID/Washington respondents came from the Bureau of Humanitarian Response (BHR) and within it both the Office of Food for Peace (FFP) and the Office of Private and Voluntary Cooperation (PVC); Bureau for Management; Bureau for Africa; Bureau for Europe and the New Independent States; Bureau for Asia and the Near East; and Bureau for Latin America and the Caribbean. Directors and Deputy Directors were interviewed from Missions in Guatemala, Mali, Mexico, Senegal, South Africa, and Swaziland.