Detailed Information on the
Self-help Homeownership Opportunity Program (SHOP) Assessment

Completing an independent evaluation of program effectiveness, and identify areas for improvement.

Developing additional annual and long-term outcome measures to assess community level impact of the SHOP program.

Measure: The number of homeowners who have used sweat equity to earn assistance with SHOP funding reaches 1,500

Explanation:This results for this measure are for the 12-month period ending June 30 preceding the end of the fiscal year since fourth quarter fiscal year results are not available in time for inclusion in the PAR.

Measure: In the SHOP Program, create $225 million in household equity from FY 2006 through FY 2011

Explanation:The measure represents the cumulative total of purchase prices less outstanding mortgage balances at closing for assisted households during a 12 month period. Cumulative total is in parentheses.

Measure: The SHOP Program will maintain a default rate that is lower that reported in the USDA Section 502 Direct Loan FHA Program.

Explanation:This measure represents the number of households that are foreclosed upon as a percentage of all assisted households, net of recoveries. Since SHOP-assisted units and households share largely the same profile as other units and households assisted generally through self-help housing programs, the results presented will reflect the default rate for the overall pool of households assisted by SHOP grantees. Using the Credit Supplement of FY2007, the current Section 502 Direct Loan Program default rate is 5.37 percent; this figure will be adjusted annually. (The SHOP program used the FHA program default rate of 3.58 percent previously).

Measure: Maintain a leveraging ratio of at least 8.5:1 for private and other funds and resources invested in the housing versus SHOP funds

Explanation:This measure describes the ratio of the dollar value of private and other funds and resources that are invested in a SHOP-assisted unit for every SHOP dollar invested. Private and other funds and resource do not include the value of sweat equity contributed by homebuyers. HUD and OMB have agreed to establish a more ambitious target of 8.5:1 for 2006 and forward because SHOP has historically outperformed the previous target.

Measure: The average SHOP investment in a property will increase by no more than 3 percent a year.

Explanation:The average per-unit level of SHOP investment in a project is a function of the overall development cost and the availability of other funds, federal and non-Federal, for the project. If development costs such as land, labor, materials, and fees, rise, or higher subsidy levels are required in order to make units affordable to lower income households, the required SHOP investment will rise accordingly. Given the above factors, HUD estimates that the annual increase in the average SHOP investment per unit will be less than 3 percent.

Is the program purpose clear?

Explanation: The purpose of the Self-Help Homeownership Opportunity Program (SHOP) is to develop new, decent, safe and sanitary, non-luxury dwellings for low-income families and persons who would otherwise be unable to afford to purchase a dwelling. To achieve this purpose, the program requires homeowners to contribute a significant amount of sweat equity toward the construction of the new dwellings. Community participation through which volunteers assist in the construction of the dwellings is a required element of the program as well. The program targets funding to be used for a limited number of eligible expenses??land acquisition, infrastructure improvement, and minimal administrative costs??that set the stage for development and the homeownership opportunities.

Evidence: The program purpose of SHOP is provided in the authorizing legislation: Section 11 of the Housing Opportunity Program Extension Act of 1996, as amended (42 U.S.C. 12805) (the "Extension Act"), paragraph d.1. - "Purpose". The statute may be accessed through the SHOP web-site at: http://www.hud.gov/offices/cpd/affordablehousing/programs/shop/index.cfm

Does the program address a specific and existing problem, interest, or need?

Explanation: The SHOP program addresses two specific barriers to the provision of homeownership opportunities: (1) the inability of families with incomes at or below 80 percent of median income for their area (i.e., poor households) and/or those who are not bankable borrowers for credit or other reasons to become homeowners (HUD generally targets 50 to 65 percent of median area income); and (2) the lack of suitable building sites and the funding to improve those sites and their basic infrastructure to set the stage for the construction of new decent, safe, and sanitary non-luxury dwellings. Both these barriers were identified during the drafting of the legislation as critical to increasing affordable housing and they continue to be relevant needs.

Evidence: Paragraph d.1. of the SHOP statute - Purpose - identifies the intended beneficiaries as: "...families and persons who otherwise would be unable to afford to purchase a dwelling". Data from HUD's specialized SHOP reporting system indicates that SHOP serves this category..." . The SHOP Program serves over 80 percent of families who have incomes at or below 50 percent of the median income for their area, as reported by Habitat for Humanity. By statute (paragraph d.2), the eligible activities under SHOP are limited to land acquisition and infrastructure improvements (costs associated with the administration of the program were later made eligible through appropriations language). The use of funds on other activities that were identified barriers to housing development targeted by the statute, were specifically precluded (i.e., the costs associated with new construction and rehabilitation).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program design is unique in that it combines three features: 1) targets majority of funding to predevelopment activities, 2) uses the self-help housing development model that requires significant amounts of sweat equity from low-income homeowners, and 3) serves both rural and urban areas. Although there are a variety of Federal programs that help low-income homebuyers, there is no comparable program delivery mechanism with the above combined features.

Evidence: Federal programs, such as Community Development Block Grant (CDBG) and HOME Investment Partnerships Program, can fund predevelopment activities but these are block grant programs that have a broad range of activities, including increasing homeownership for low-income homebuyers. CDBG and HOME is sometimes used as complementary funding sources that focus on rehabilitation and construction, which are activities that are ineligible under SHOP statute. The USDA Mutual Self-Help program is similar in its sweat equity approach in housing development, but the USDA program is primarily a technical assistance grants program to non-profits (i.e., recruitment, training), with a small direct loan program component. Further, USDA targets only rural areas while SHOP-assisted units are about 65 percent rural. The SHOP program was specifically designed to assist the largest private nonprofit self-help providers, Habitat for Humanity and the Housing Assistance Council (HAC), to expand their already successful activities, and not to be redundant of existing efforts.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: SHOP benefits from a carefully crafted statute that keeps government involvement to a minimum and directs assistance toward reducing or eliminating the barriers to self-help housing development. It also directs SHOP funds to be awarded to experienced national or regional organizations, and consortia which minimize administrative costs. These groups coordinate networks of local affiliates and are in the best position to know which of these local affiliates are most capable of administering projects successfully or are best situated geographically to administer and meet the objectives of the program. SHOP is also efficient because SHOP funds are recaptured when they are not used within 24 months (or 36 months in the case of affiliates that develop five or more dwellings) and when grantees have not substantially fulfilled their obligations under the grant agreement, including completing the number of units specified. These factors and the limited cost per unit enable the program to leverage significant amounts of funds from private and public sources and thereby keep average unit costs low.

Evidence: Paragraph b.4. of the SHOP statute describes one of the goals of the program as being "the efficient development of affordable housing with minimal governmental intervention, limited governmental regulation, and significant involvement by private entities." The average length of time for a SHOP grantee or affiliate to complete a unit, from start to finish, is one year for rural programs and three to six months for medium to high producing affiliates in urban areas. Also, paragraph i.5 of the SHOP statute provides that HUD must recapture any grant amounts for slow expenditure. The expenditure rate of 89 percent was determined using information contained in the SHOP Expenditure Report which may be accessed through the SHOP web-site at: http://www.hud.gov/offices/cpd/affordablehousing/programs/shop/index.cfm. In addition, SHOP is effective in leveraging Federal dollars with public and private funds to achieve program objectives. The leveraging ratio for the SHOP Program since the program's inception in 1996 has risen over the years to 7.7:1 in 2004 and 9.7:1 in 2005. Paragraphs (f)(2) & (i)(3) of the SHOP statute mandate the leveraging of funds. The actual per unit average of SHOP funds invested in an assisted unit since 1996 is approximately $9,400 versus the $10,000 limit that was in place up until 2004 and the $15,000 limit in place since. To further motivate grantees, points are awarded in the annual SHOP NOFA competition based upon the extent to which applicants leverage these funds.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The SHOP statutory purpose requires, and the annual NOFAs establishes as a condition for award, that the targeted funds are used only for eligible activities in connection with developing new housing and that the program beneficiaries are very low and low-income households (e.g., incomes at or below 80 percent of the area median) who would otherwise be unable to afford to purchase a dwelling. These households also have to be willing to contribute significant amounts of sweat equity (100 hours).

Evidence: The primary evidence that the SHOP program design effectively targets resources that reach intended beneficiaries is twofold: (1) the average amount of equity created for each low-income buyer in their new home (appraisal value less purchase price) through their own sweat equity and as a result of the SHOP program is approximately $28,000 in 2005, as reported by Habitat for Humanity and the Housing Assistance Council (HAC). These two organizations are responsible for creating 94 percent of the 14,932 new homeowners created through SHOP since 1996; and (2) almost 80 percent of homebuyers of SHOP assisted units have incomes under 50 percent of median for their area, clearly addressing the target population consisting of those low-income households that otherwise would not be able to afford to purchase a home. The FY 2006 NOFA can be found at: http://www.hud.gov/offices/cpd/affordablehousing/programs/shop/index.cfm.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: SHOP has established two long-term outcome measures that focus on the purpose of the program: 1) Maintain a low default rate, net of recoveries, compared to other comparable programs; and 2) Create a minimum of $225 million in home equity among low-income homebuyers assisted through SHOP (cumulative total of appraised value less purchase price at closing for assisted households over the six-year period from FY 2006 through FY 2011).

Evidence: The long-term SHOP outcome measures are included in HUD's Strategic Plan for FY 2006 through 2011, and the 2007 Annual Performance Plan. In the Strategic Plan, the default measure uses the FHA-insured default rate as a benchmark. However, this outcome measure is slightly revised in the 2007 Annual Performance Plan to use USDA Section 502 loan program's default rate (5.37%) as a benchmark because the majority of SHOP programs are located in rural areas. Both the HUD Strategic Plan and Annual Performance Plan are available on the HUD website.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Yes, to maintain a loan default rate of less than the USDA Section 502 rate is quite ambitious. The loan qualification criteria for low-income homebuyers of units assisted by the SHOP program, such as credit history, are applied more liberally than is the case with federal insured loan programs, such as the Section 502 program. It is even more so when considering that almost 80 percent of homebuyers of SHOP assisted units have incomes under 50 percent of median for their area. The creation of a minimum of $225 million in home equity for low-income homebuyers assisted through SHOP by 2011 is premised on an average of $25,000 per household, to be achieved largely though the sweat equity contributions of the households themselves. This is a significant amount of wealth created within a relatively short period of time and an ambitious target -- approximately $2 of household equity would be created for each $1 of SHOP funds invested.

Evidence: The targets for the two outcome measures described above in response to question 2.1 (i.e., no more than 5.37 percent default rate, net of recoveries, and no less than $225 million (equity created), respectively) are included in HUD's Strategic Plan for FY 2006 through 2011, and the Annual Performance Plan of 2007.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: SHOP has an annual performance measure that can gauge progress towards its long-term goals. The SHOP program has an annual performance measure contained in HUD's FY 2006 Annual Performance Plan: 1,500 homeowners will have used sweat equity to earn assistance with SHOP funding. In order to achieve one of its long-term goals, at least 9,000 new homeownership opportunities will have to be created through SHOP over the next six years based on an average accumulation of $25,000 in home equity per household. SHOP is working with OMB to develop a viable annual performance measure that will continue the program's low default rate amongst its homeowners. The program already requires that all participants receive pre- and post-purchase housing counseling, which has contributed to its historically low default rate. In the interim, SHOP will review the default rate on an annual basis to track any fluctuations.

Evidence: The SHOP annual performance measure can be found in HUD's Strategic Plan and in the FY 2006 and 2007 Annual Performance Plan. Also, please see Performance Measures.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The SHOP annual measure of homeownership opportunities has baseline data since 2003 and sets ambitious targets based on results from previous years, and other factors that may impact results, such as funding levels, changes in program requirements, and applicant projections of completions.

Evidence: The number of homeownership opportunities remain ambitious from year to year, except for the decreased target level in 2005 due to the lowered appropriations level and the increase in allowable per unit cost from $10,000 to $15,000 because of escalating land prices. Please see measures section.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: HUD and its grantee partners and their respective affiliates share the short and long-term goals of the SHOP program: (1) the production of units and homeownership opportunities for families who could not otherwise afford it; (2) the leveraging of private and other funds and resources; (3) the selection and counseling of beneficiary families to ensure their successful transition to homeownership; and (4) the creation of household wealth through homeownership and equity growth. Beginning in FY 2006, grantees can include additional measure through the SHOP NOFA Logic Model that would also contribute to SHOP's annual and long-term goals.

Evidence: The annual SHOP NOFA supports these goals by setting minimum threshold requirements for participation and awarding points to highlight important program goals, such as leveraging. SHOP's grant agreement requires that these and other goals be included in the written agreements entered into between grantees and their local affiliates. To reinforce this commitment at all levels, HUD regularly reviews quarterly and annual performance reports and, in several instances, has recaptured funds from grantees who have not substantially fulfilled their objectives as specified in their grant agreement during the 36-month performance period. The "Logic Model" requires applicants to identify performance outcomes and to develop an effective and quantifiable evaluation plan for determining whether short and long-term goals are met. The responses may lead to the development of additional measures tailored to local interests and needs. See the SHOP web-page for access to program reports, the Logic Model, and other information identified above: http://www.hud.gov/offices/cpd/affordablehousing/programs/shop/index.cfm

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: There are past evaluations that focused on individual SHOP grantees, but they are either limited in scope or not independent. SHOP is under contract with HUD's Office of Policy and Development and Research to have an independent organization assess the effectiveness of the SHOP Program. When this is completed, the program will have an evaluation that should meet the criteria of an independent evaluation to help it improve.

Evidence: An evaluation of the SHOP Program is now under contract through HUD's Office of Policy Development and Research and is currently underway to assess the effectiveness of the SHOP Program. After a full and open competition, HUD has contracted the evaluation to be conducted by Applied Real Estate Analysis (AREA), Inc. The final report is expected in early 2007. In regard to quality, the SHOP evaluation underway is based on the most appropriate design that is feasible. A strong statistical design that gathered much buyer-level data on outputs and outcomes would be very costly to implement. The study underway, which gathers data primarily from grantees and then from a limited number of affiliates and buyers, will characterize the program broadly and provide illustrative information on what the program is achieving. There are three major methodological pieces of this project. First, the Contractor will conduct interviews with staff from the six FY 2001 and FY 2002 SHOP grantees. Second, the Contractor will conduct loosely-structured reconnaissance discussions with staff from 20 to 25 of the SHOP grantees' affiliates in order to obtain background on the SHOP program and to select a sub-sample of 8 to 10 affiliates as case study sites. The contractor will analyze the data they gather from the FY 2001 and FY 2002 SHOP grantees. As far as scope is concerned, the SHOP evaluation has four goals: (1) Describe the implementation of the SHOP program in terms of grantees, their affiliates, and how they get money to specific localities and properties; (2) Describe affiliates and how the SHOP program is administered at the affiliate level; (3) Describe homebuyers, properties completed and transferred to homebuyers, and neighborhood characteristics and stabilization; and (4) Analyze housing cost and ongoing affordability. The Statement of Work for this independent study is available upon request.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: SHOP uses average per unit costs and leveraged resources data to estimate the total annual homeownership opportunities to use for its budget requests. For example, since the program aims to achieve a specific amount of home equity ($225 million) over six years, it has budgeted amounts to construct 9,000 homes to reach this long-term goal. These estimates also include indirect costs, such as monitoring and program management (7 field and headquarters FTEs). The SHOP budget also proposed $990,000 for technical assistance (TA) in the FY 2007 Budget. This TA is intended to increase the capacity of grantees and their local affiliates to successfully carry out SHOP-assisted homebuyer programs.

Evidence: See the 2007 Congressional Budget Justification for the SHOP Program, Performance Measurement Table and Staff Requirement FTE table on page C-2, available at: http://www.hud.gov/offices/cfo/reports/2007/cjs/part2/cpd/selfhelpprogram.pdf For example, in regard to the annual and long-term production goals, SHOP's FY 2007 budget justification reads as follows: "Expanded demand for this program is evidenced by the fact that the availability of only $27 million in fiscal year 2004 generated $57 million in funding requests...It is projected that no less than the FY 2007 requested amount will be needed to assist at least 2,000 households."

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: In the course of managing the SHOP program over the past ten years, the Office of Affordable Housing Programs (OAHP) identified and addressed several planning deficiencies. Examples of identified deficiencies that have been corrected are: lack of a standard, electronic performance reporting system by developing such a system in-house; absence of a definition for "significant amount of sweat equity" for homebuyers by imposing a required minimum number of sweat equity hours; general increases in land costs across the country that threatened to make the program irrelevant by raising the SHOP maximum per-unit subsidy limit from $10,000 to $15,000; time-consuming and burdensome environmental review requirements by issuing a notice that streamlined and explained the process; and too narrow a definition for eligible sweat equity activities by allowing the sweat equity contributed by a family toward the construction of other self-help housing to count toward the minimum required sweat equity hours

Evidence: Once identified, corrective action was taken through various means, including the issuance of formal written guidance, revisions to requirements in the annual NOFA, modifications to grant agreement language, and even statutory change. OAHP relies on changes to the annual NOFA to implement program modifications, a process which is, by its nature, easier and more expeditious than the regulation amendment process. Some examples are: Notice CPD 98-10 to facilitate environmental review requirements and processes; Notice CPD-01-09 permitted reimbursement to grantees for their acquisition of land prior to environmental review approval; FY 2004 NOFA increased maximum per subsidy unit from $10,000 to $15,000 due to increasing land costs; statute amended in 2005 to allow the sweat equity contributed by a family toward the construction of other self-help housing to count toward the minimum required sweat equity hours for that family. Furthermore, in order to allow grantees and their affiliates to improve their individual programs, HUD published a SHOP 'Model Guide' in 2005. The Guide provides case studies of successful self-help projects.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: SHOP has routine mechanisms for collecting data from grantees and using the information for program management purposes. Since 2003, grantee are required to submit reports electronically at the end of each quarter to ensure a uniform method for tracking such information as fund expenditures, property accomplishments, property characteristics, income ranges, and racial/ethnic data. In addition to the quarterly reports, an annual report identifying each affiliate's SHOP accomplishments is submitted by grantees. A "logic model" has been incorporated into the 2006 annual SHOP competition where applicants are required to adopt and report minimum and some optional output and outcome goals for their SHOP program, should they be funded. Options for long-term measures to be selected by applicants include: average reduction in sales price to homebuyer, average increase in household assets (a long-term measure for SHOP) and the Return on Investment (ROI) comparing the cost of market rate construction versus the actual cost of constructing units using sweat equity and volunteer labor.

Evidence: The information provided through the reports described above are used, among other things, to determine that grantees are "substantially fulfilling" their objectives within specified timeframes - as required by statute, and to complete required reports, such as the Performance and Accountability Report (PAR). It also provides baseline performance data necessary to set meaningful, ambitious performance targets and to test the validity of the data provided. Based upon management review of the quarterly reports, $1,059,395 has been deobligated from SHOP grantees for having failed to substantially fulfill their obligations during the 36-month performance period. In addition, the scores of applications submitted by existing grantees that are seeking additional SHOP funding have been reduced based upon performance information collected through these reports. In some instances, this has resulted in additional funding being denied. Specialized reports on grantee performance as well as the logic model from the FY 2006 NOFA may be accessed though the SHOP web-page at: http://www.hud.gov/offices/cpd/affordablehousing/programs/shop/index.cfm

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Current numeric performance goals are directly incorporated into manager performance plans. SHOP Program Managers have in their current annual job performance assessment the following performance objective under the critical element "Increase Homeownership Opportunities: 1,500 homeowners assisted through the Self-help Homeownership Opportunity (SHOP) program." This job performance objective is directly linked to the FY 2006 Annual Performance Plan indicator for SHOP. In regard to grantees, the annual SHOP NOFA includes an element specific to past performance. Without being able to demonstrate successful past performance on the same or similar self-help housing activities as authorized under SHOP, an applicant has only a remote chance for being funded. In several instances, grantees have failed to reach the minimum score necessary under this "capacity" factor to be considered eligible for funding, including existing grantees seeking additional awards.

Evidence: The HUD personnel performance evaluation system(the HUD Performance Accountability and Communication System (PACS), as it is applied in the Office of Affordable Housing Programs, holds managers accountable for meeting program performance goals, including Annual Performance Plan goals. In regard to grantees, "Past Performance" is a crucial element in the review and scoring of SHOP applications submitted by existing grantees or new applicants seeking funding. The applicant instructions for addressing this factor read as follows: "You must describe the past experience of your organization and key staff in carrying out self-help housing activities (specify the time frame during which these activities occurred) that are the same as, or similar to, the activities you propose for funding, and demonstrate that you have had reasonable success in carrying out and completing those activities. You must include the average number of sweat equity hours provided per homebuyer family, and the average number of volunteer labor hours provided per unit. You may demonstrate reasonable success by showing that your previous activities were carried out as proposed, consistent with the time frame you proposed for completion of all work."

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: SHOP Grant funds are typically obligated (i.e., the grant agreement is jointly executed with grantees) within two months of the award announcements. Funds are then set up in HUD's Line of Credit Control System (LOCCS). (See response to question 3.6 as well.) The 36-month performance period for drawing funds begins when the SHOP funds are set up in LOCCS. Grantees cannot be reimbursed for costs incurred after the 36-month performance period during which the goals of the grant agreement in terms of number of properties completed and number conveyed to homebuyers must be, by statute, "substantially fulfilled."

Evidence: HUD regularly monitors the expenditure of funds and the number of properties underway, completed and conveyed to homebuyers in order to determine compliance with program requirements and adherence to the individual grantee's performance plan as described in their original funding application. This monitoring is accomplished through a specialized in-house performance reporting system that collects information quarterly on grantee activities, including, but not limited to, fund expenditures, housing starts and completions. Funds remaining unspent as of the end of the 36-month performance period are deobligated. $1,059,395 has been deobligated from three SHOP grantees for this reason. Completion and expenditure reports for SHOP grantees may be accessed through the SHOP web-page at: http://www.hud.gov/offices/cpd/affordablehousing/programs/shop/index.cfm

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The awarding of points in the annual SHOP competition announced through a NOFA published in the Federal Register is based largely upon such program elements as the amount of firm commitments of leveraging provided, the amount of equity contributed by homebuyers, and past performance of the grantee. The SHOP NOFA specifically requires applicants to address the cost effectiveness of sweat equity and volunteer labor in reducing the cost of the units to the homebuyer and requires comparison of SHOP-funded units to non-SHOP-funded units in the project area. This selection procedure ensures that the applicants most likely to invest SHOP dollars efficiently and effectively are funded and those who are least likely are not.

Evidence: The SHOP Program uses the following long-term/annual efficiency measure: "Maintain a leveraging ratio of at least 7.5:1 for private and other funds and resources invested in the housing versus SHOP funds". As reported in 1.4 of the PART, SHOP exceeds this ratio. This measure is intended to demonstrate the ratio of program outputs and outcomes to the input of program funds. Private and other funds and resources are typically used on the actual construction or rehabilitation of the new homes. Under the FY 2006 SHOP NOFA, the per-unit average of SHOP assistance cannot exceed $15,000. The awarding of points in the SHOP competition is based upon the amount of leveraging proposed by applicants that, taken together with other scoring elements described above, act as an incentive to make the program more efficient and effective.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The SHOP program was initially and specifically designed to assist the largest private nonprofit self-help providers, Habitat for Humanity and the Housing Assistance Council (HAC), expand their already successful self-help housing activities. As the program has matured, coordination with additional regional and national self-help consortia has occurred. Also, SHOP Program staff meets with or discusses program progress and issues with grantee staff on an on-going basis.

Evidence: In regard to coordination with other Federal housing programs, such as CDBG and HOME, communication among the programs led to having the HUD Affordable Housing Web-site provide both HOME Program and SHOP information for prospective applicants in one place. SHOP and HOME staff also collaborate on the scoring and evaluation of annual competitions. In rural areas, there is collaboration and coordination with USDA Mutual Self-Help Housing and the 502 Direct Loan program. SHOP staff and grantees, and USDA program staff attend the annual HAC conference. At the HAC conference and other similar conferences, attendees learn about issues confronting communities, best practices in the self-help housing field, etc. As noted in the 2004 PART for the USDA Mutual Self-Help Housing Program, "Self-help partners (grantees, contractors, related interest groups and other funding sources, such as HUD and State and local agencies) meet regularly on a regional and national basis to exchange ideas and assure best practices." HUD concurs with that assessment.

Does the program use strong financial management practices?

Explanation: SHOP NOFA requires applicants to have administrative and fiscal management capability to administer the grant. Applicants provide HUD an overview of their system for managing SHOP funds. When funded, SHOP award funds are managed through HUD's Line of Credit Control System/Voice Response System (LOCCS/VRS), the same system used for the financial management of other HUD formula and competitive programs. The use of this system ensures that SHOP payments are made properly and erroneous payments minimized. LOCCS meets all statutory requirements for such systems and provides financial information that is accurate and up-to-date. LOCCS has no identified material internal control weaknesses. Upon award of SHOP funds, HUD reserves funds in LOCCS, and conveys a fully executed grant agreement and current Grantee Financial Instructions to each grantee. Grantees are provided the appropriate banking and user ID and password forms to set up their accounts. Grantees have up to 36 months to spend SHOP funds once there is a fully executed grant agreement and the funds are "verified" in LOCCS. To further ensure proper financial management, SHOP program staff accesses grantees' LOCCS accounts at least monthly to monitor draw activities. On a monthly basis, grantees also submit grant payment vouchers that support their draws. On a quarterly basis, grantees reporting requirements include a breakdown of funds expended by eligible activity. At grant closeout, the grantee submits a financial closeout report that covers total program activity, including the final disposition of SHOP funds in LOCCS. As a result of these reviews, HUD has deobligated $1,059,395 in unspent funds from three grants.

Evidence: See the SHOP Program NOFA published in the Federal Register on March 8, 2006 (FR Vol. 70, No.173) and the FY 2006 NOFA Policy Requirements and General Section to the SuperNOFA for HUD 's Discretionary Programs published on January 20, 2006 (FR Vol. 71, No.13), both available through SHOP's web-site at: http://www.hud.gov/offices/cpd/affordablehousing/programs/shop/index.cfm. The following is available upon request: SHOP Grant Agreement; FY 2005 SHOP Grantee Financial Instructions; SHOP Grantee Reporting System Instructions and Excel Spreadsheets; and SHOP Closeout Instructions, September 27, 2004, including financial report and agreement.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: SHOP Program staff meets with or discusses program progress and issues internally, with other HUD offices, and with grantee staff on an on-going basis. Staff discuss and determine actions to address program needs, management issues, and timelines for completions of deliverables.

Evidence: In the course of managing the SHOP program over the past 10 years, the Office of Affordable Housing Programs (OAHP) identified several management deficiencies as having a negative impact on program operations. An example of an identified management deficiency that has been addressed is in the area of Information Technology: the lack of a standard, electronic performance reporting system for SHOP. In response, a system for SHOP was developed in-house and currently provides timely grantee reports on program progress. In the area of personnel, an effort has been made to reduce the staff workload and the review time related to the annual SHOP competition by automating the receipt of applications through e-grants, streamlining the application review process by simplifying the review criteria and imposing page-limitations on the applications themselves. All of these efforts are intended to improve the efficiency of the application review process and, as a consequence, make grant funds available earlier than would otherwise be the case. Also, in the area of personnel, recruitment action is in process and should be completed by fall of 2006 that will add a staff in OAHP to support SHOP activities and address gaps in monitoring and program oversight.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: The SHOP competition is an open, fair and transparent process with safeguards built into the review procedures to ensure that awards are made based on merit, and is conducted consistent with stringent HUD Reform Act requirements. All SHOP funds are now distributed through competition. (A portion of the initial SHOP appropriation in FY 1996 was earmarked to Habitat for Humanity.) However, SHOP legislation restricts the types of self-help organizations that can apply to national and regional organizations and consortia, which limits outreach to the number of potentially eligible applicants.

Evidence: The SHOP NOFA and the SHOP website and Grants.gov provide contact information for the application process for downloading and electronically submitting an application and technical assistance. All application reviewers are required to receive ethics training preceding the competition each year on HUD Reform Act requirements, as well as training specific to the review and scoring of SHOP applications. Each application is generally reviewed by three persons (not less than two) and presented before a panel comprised of all SHOP reviewers--six to seven staff members??to ensure review consistency across applications. Panel members must reach consensus on final scores of each application. In the interests of transparency, a debriefing of strengths and weaknesses is provided to any applicant??funded or non-funded??that requests it. The SHOP statute limits participation to nonprofit national and regional organizations or consortia that have experience providing self-help housing. Nevertheless, SHOP attempts to make the availability of SHOP funds known through widely publicized e.grants training, satellite training broadcasts, web-site postings. This being the case, the eligible pool of potential applicants is relatively few in number. Currently, there are five active grantees carrying out self-help projects through hundreds of local organizations and affiliates. Four national and regional organizations were awarded SHOP funding in FY 2005. FY 2006 SHOP NOFA can be accessed from the SHOP web-page: http://www.hud.gov/offices/cpd/affordablehousing/programs/shop/index.cfm.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The office with oversight responsibilities for SHOP is the Office of Affordable Housing Programs (OAHP). OAHP has put in place a specialized in-house performance reporting system that collects information quarterly on activities such as fund expenditures, property accomplishments, property characteristics, income ranges, and racial/ethnic data of beneficiaries. Monitoring visits are conducted and technical assistance is provided to all grantees on an on-going basis. HUD field offices provide oversight by conducting environmental reviews themselves or designate responsible entities who conduct these reviews on behalf of grantees and their affiliates. Field staff also oversees grantee compliance with Relocation Assistance and Real Properties Act. The SHOP NOFA requires grantees to provide a plan for monitoring affiliates at least once during the 3-year term for drawing funds. Housing Assistance Council alone has 32 site visits planned for the coming year. Grantees' LOCCS accounts (i.e., HUD's automated fund disbursement system) are reviewed monthly at headquarters to ensure that the period of draws do not exceed the statutorily mandated 36-month period for expenditure of funds. $1,059,395 has been deobligated from three grantees that were unable to expend funds within this period.

Evidence: The staff responsible for managing the SHOP Program in the Office of Affordable Housing Programs (OAHP) reviews this information collected in the in-house performance reporting system on an on-going basis to determine that, among other things, grantees are "substantially fulfilling" their objectives as set forth in their applications within the specified timeframes and that expenditures are spent on eligible activities. The roll-out of the new data-collection system described above late in 2003 resulted in an immediate improvement in the quality of data through the discovery of duplications in reporting of past accomplishments by Habitat for Humanity. In the last several years, on-site monitoring/technical assistance visits by OAHP staff have included the two largest grantees, Habitat and HAC, and several smaller ones at locations in Georgia, Illinois, Wisconsin, Virginia and Maryland.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: SHOP grantee performance information is collected electronically by a specialized reporting system developed by the Office of Affordable Housing Program, quarterly and annually. The information collected from individual grantees is then reformatted for ease of use, aggregated - if appropriate - and posted on the SHOP web-site for public scrutiny. Consequently, the public can obtain full access to information such as how many first-time homebuyers have achieved homeownership through the sweat equity process, annual appropriations, quarterly property activities, and expenditures by grantee. A special report specifically on grantee progress in meeting annual and long-term performance goals set by HUD is also available through the site. Other items of interest to the public can be easily accessed as well: annual announcements of funding availability and awards, the SHOP Model Guide, which provides a representative sample of affiliate case studies, and grantee contact information for those interested in participating in local programs.

Evidence: The SHOP web-site with grantee performance data may be accessed at: http://www.hud.gov/offices/cpd/affordablehousing/programs/shop/index.cfm

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: SHOP has established two long-term outcome measures, and is making continual progress towards them. The first goal is to maintain a low default rate. In the HUD Strategic Plan, HUD used FHA-insured loan programs as its benchmark, but will revise its performance plan to use the USDA Section 502 Single Family loan program because the majority of its projects are in rural areas. In either case, the default rate for self-help housing programs administered by the major SHOP grantees, Habitat for Humanity and Housing Assistance Counciil (HAC) and others was well under both of the actual default rates, net of recoveries, over the past three years. The second long-term measure is to create a minimum of $225 million in home equity among low-income homebuyers assisted through SHOP. The creation of a minimum of $225 million in home equity for low-income homebuyers assisted through SHOP by 2011 is premised on an average of $25,000 per household, to be achieved largely though the sweat equity contributions of the households themselves. Habitat for Humanity, HAC, and other self-help providers have created in excess of the $37.5 million annual target for the past three years and are on track to create $225 million in household equity over the six year period beginning in FY 2006.

Evidence: The targets for the two outcome measures described above are included in HUD's Strategic Plan for FY 2006 through 2011. In consultation with OMB, they will be added to HUD's 2007 APP. The results for the past three years and projections for the next six may be found in the Performance Measurement section (indicators no. 2 and 3).

Does the program (including program partners) achieve its annual performance goals?

Explanation: The established annual target is based upon preceding years' results and adjusted to take into consideration factors that may impact results, such as funding levels, and changes in program requirements. The SHOP annual targets and actual accomplishments are available from 2003 to 2005. SHOP exceeded its targets in both 2003 and 2005. The results for FY 2004 were significantly impacted by the introduction of new reporting procedures by HUD late in FY 2003. As a result of the conversion to the new system, Habitat for Humanity, the largest recipient of SHOP funding and responsible for approximately 62 percent of the completed units discovered that some affiliates had duplication errors in their records. The one-time deletion of these records was largely if not entirely responsible for the shortfall in the number of units reported as completed during the period. The targets for FY 2006 and 2007 have been reduced due to 1,500 due to an increase in the maximum allowable average per-unit assistance level from $10,000 to $15,000 that took effect with FY 2004 funding, and adjusted due to decreases in annual appropriation level. This increase was a response to the dramatic increases in land costs across the country. SHOP will begin collecting baseline data on the annual performance measure regarding assisting homeowners with mortgage delinquencies avoid foreclosures. Nevertheless, based on the program's history of the default rates, the new annual performance measure should be achieved.

Evidence: Please see Performance Measures section.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: SHOP's efficiency measure to "maintain a leveraging ratio of at least 7.5:1 for private and other funds and resources invested in the housing versus SHOP funds." A high leveraging ratio has characterized the SHOP program and continually exceeded the 7.5:1 level. Private and other funds and resources are typically used on the actual construction or rehabilitation of the new homes made possible through the land acquisition and infrastructure improvements paid for with SHOP funds. SHOP Program staff has also taken steps to maximize leveraging to the extent possible.

Evidence: The SHOP program has improved leveraging over recent years to where approximately $7.50 in private and other funds and resources are generated by each SHOP dollar invested in a project. This return on investment does not include the sweat equity and volunteer labor contributions of homebuyers and the local community. Leveraging of other funds in SHOP-assisted projects has been increasing steadily over time. For example, the combined leveraging reported by Habitat and HAC, the two largest recipients of SHOP funding, has increased from an average of $72,219 per unit in 2003, to $102,546 in 2005. While the efficiencies achieved have been commendable, OMB recommends that the efficiency measure goal should be higher than what has been achieved to make them meaningful. Please see performance measures.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: While it is not possible to compare SHOP to Habitat and HAC's programs since it is now actually an extension of these nonprofit efforts, and program goals have been merged, SHOP compares favorably with other housing programs. It is effective at targeting very low and low-income households and promotes homeownership; its eligible activities are limited to land acquisition and infrastructure improvements, with some administrative costs and therefore, complement other funding sources that increase affordable housing units; and it is a flexible, efficient and effective method for allocating and leveraging funds that limit government involvement.

Evidence: SHOP differ in significant and advantageous ways from the other federal program with a similar purpose and goals - the USDA Section 523 Mutual Self-help Housing. SHOP assistance may be in the form of either a loan or a grant and families can be served in both rural and urban areas; and while the household sweat-equity contributions are significant under SHOP, as required by statute and minimum borrower qualifications are applied, they are both more flexible under SHOP and, therefore, increase the pool of potentially eligible low- and very low-income homebuyers. OMB noted in it 2004 PART review of the USDA Section 523 Mutual Self-help Housing Program that "the program's ability to grow is slow", in part because of land acquisition and environmental clearance issues, two areas that the SHOP Program has addressed successfully. The SHOP statute may be accessed through: http://www.hud.gov/offices/cpd/affordablehousing/programs/shop/index.cfm Compared to larger block grant programs such as CDBG and HOME, the community participation and sweat equity features of the program help participants feel more invested in the project. Annie E. Casey Foundation entitled "Creating the Village - How Mutual Self-Help Housing Builds Community" found, among other things, that new homeowner households assisted through these programs had been given "a sense of empowerment that can, and in some cases has, translated into stronger, more stable communities."

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: There are past evaluations that focused on individual SHOP grantees, but they are either limited in scope or not independent. SHOP is under contract with HUD's Office of Policy and Development and Research to have an independent organization assess the effectiveness of the SHOP Program. When this is completed, the program will have an evaluation that should meet the criteria of an independent evaluation to help it improve.

Evidence: The studies available on SHOP are limited, and not independent; they provide some indication on this program's performance. Examples of past evaluations are the 2004 study, funded by the Annie E. Casey Foundation and conducted by HAC to determine the long-term impact of self-help housing efforts, and the 1998 HUD study on Habitat for Humanity entitled Making Homeownership a Reality: Survey of Habitat for Humanity (HFHI), Inc. Homeowners and Affiliates, which was conducted for the Department by Applied Real Estate Analysis (AREA), Inc.