This is the accessible text file for GAO report number GAO-05-342 
entitled 'Rental Housing: HUD Can Improve Its Process for Estimating 
Fair Market Rents' which was released on May 2, 2005.

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Ranking Minority Member, Subcommittee on Housing and 
Transportation, Committee on Banking, Housing, and Urban Affairs, U.S. 
Senate: 

March 2005: 

Rental Housing: 

HUD Can Improve Its Process for Estimating Fair Market Rents: 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-342]: 

GAO Highlights: 

Highlights of GAO-05-342, a report to the Ranking Minority Member, 
Subcommittee on Housing and Transportation, Committee on Banking, 
Housing, and Urban Affairs, U.S. Senate: 

Why GAO Did This Study: 

The Department of Housing and Urban Development (HUD) annually 
estimates fair market rents (FMR) for standard quality rental units 
throughout the United States. Among other uses, FMRs help determine 
subsidies for almost 2 million low-income families in the nation’s 
largest rental assistance program. However, concerns exist that FMRs 
can be inaccurate—often, too low, preventing program participants from 
finding affordable housing. Also, HUD will soon derive FMRs from a new 
source, the American Community Survey (ACS), which processes data 
somewhat differently than HUD’s current data sources, including the 
decennial census. You asked us to review (1) how HUD estimates FMRs, 
(2) how accurate FMRs have been, (3) how ACS data may affect accuracy, 
and (4) other changes HUD can make to improve the estimates. 

What GAO Found: 

According to HUD, the typical process for estimating FMRs includes 
benchmarking, or developing baseline rents for each FMR area (generally 
county-based) using census data or other surveys for the years between 
censuses; adjusting those rents to bring them up to date; and seeking 
public comment before finalizing the numbers. HUD generally uses 
Consumer Price Index and telephone survey data to adjust baseline 
rents—that is, to account for rent changes since data used for baseline 
estimates were collected and to project the estimates into the next 
fiscal year (when they will be in use for subsidy purposes). HUD then 
lists the proposed FMRs in the Federal Register for public comment. 
These comments can lead to changes in FMRs, but only when they include 
new data or lead HUD to conduct a new survey.

About 69 percent of all areas had FMR estimates in use in 2000 that 
were within 10 percent of rents indicated by the 2000 decennial 
census—the most accurate comparison data available for each FMR area. 
This represents an improvement over HUD’s 1990 estimates, as the table 
below shows. Similarly, about 73 percent of 153 areas whose FMRs HUD 
rebenchmarked after 2000 were within 10 percent of rents derived from 
recent surveys. In general, GAO found that areas that are rebenchmarked 
with more recent data tended to have FMRs in the most accurate range 
(within 10 percent). 

Using ACS data could improve the accuracy of FMRs by allowing HUD to 
benchmark more areas more frequently than is possible with current data 
sources, using more recent data—a factor that GAO’s analysis suggests 
is related to accuracy. HUD’s first use of ACS data will be to update 
existing baseline estimates for the fiscal year 2006 FMRs; HUD expects 
to use ACS data to set baseline rents for some fiscal year 2008 FMRs. 

HUD could improve its FMR estimation process by consistently following 
its guidelines relating to the transparency of FMRs and ensuring that 
it can assess the accuracy of ACS-based FMRs. Transparency would be 
improved by fully documenting the estimation process so that FMRs can 
be independently reproduced. Even ACS-based FMRs may not always be 
accurate, and HUD’s policies require mechanisms to correct information 
it disseminates. 

Accuracy of HUD’s Fiscal Years 2000 and 1990 FMR Estimates: 

[See PDF for image]

Sources: GAO analysis of HUD data (2000 figures) and HUD (1990 
figures). 

[End of table]

What GAO Recommends: 

To improve the usefulness of its FMR estimates, GAO recommends that HUD 
fully document its methods for estimating FMRs by following all of its 
data quality guidelines; use, to the extent possible, state-level ACS 
data to update the fiscal year 2006 FMRs; and develop a mechanism to 
assess the accuracy of future FMRs. In response to a draft of this 
report, HUD agreed to better document methods for estimating FMRs and 
said it is exploring options to assess accuracy. 

www.gao.gov/cgi-bin/getrpt?GAO-05-342.

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact David G. Wood at (202) 
512-8678 or woodd@gao.gov. 

[End of section]

Contents: 

Letter: 

Results in Brief: 

Background: 

HUD Estimates FMRs by Defining Housing Markets, Choosing Data Sources, 
Updating Rent Data, and Evaluating Public Input: 

Most FMR Estimates Were Accurate within 10 Percent of the Census or 
Other Rebenchmarking Surveys: 

ACS Could Improve the Accuracy of FMRs by Providing HUD with More 
Recent, Better Data: 

HUD Did Not Follow One of Its Data Quality Guidelines and May Lack Data 
Sources to Assess the Accuracy of Future FMRs: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendixes: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Comments from the Department of Housing and Urban 
Development: 

Appendix III: GAO Contacts and Staff Acknowledgments: 

GAO Contacts: 

Staff Acknowledgments: 

Tables: 

Table 1: Accuracy of HUD's Fiscal Years 2000 and 1990 FMR Estimates 
Compared with Rents from Census: 

Table 2: Accuracy of HUD's FMR Estimates Compared with Rents from RDD 
Surveys (by Reason for Survey, 2001-05): 

Table 3: Accuracy of FMR Estimates in 2000 Compared with Rents from 
Census (Based on Age of Baseline FMR Data): 

Table 4: Accuracy of FMR Estimates in 2000 Compared with Rents from 
Census (Based on Type of Rebenchmarking Survey): 

Table 5: Accuracy of FMR Estimates in 2000 Compared with Rents from 
Census (by Type of Update Factor): 

Figures: 

Figure 1: Example of 40th Percentile of Rent: 

Figure 2: HUD's Typical Process for Estimating FMRs12: 

Figure 3: HUD Regions: 

Figure 4: Accuracy of HUD's Fiscal Year 2000 FMR Estimates: 

Figure 5: Scope of ACS Rebenchmarking as Related to FMR Area Size and 
Housing Choice Voucher Program Data: 

Abbreviations: 

ACS: American Community Survey: 

AHS: American Housing Survey: 

BAH: basic allowance for housing: 

CPI: Consumer Price Index: 

DOD: Department of Defense: 

FMR: fair market rent: 

HOPWA: Housing Opportunities for Persons with AIDS: 

HUD: Department of Housing and Urban Development: 

LIHTC: Low Income Housing Tax Credit: 

NAS: National Academy of Sciences: 

OMB: Office of Management and Budget: 

PHA: public housing agency: 

RDD: random digit dialing: 

SRO: Moderate Rehabilitation Single-Room Occupancy: 

Letter March 31, 2005: 

The Honorable Jack Reed: 
Ranking Minority Member: 
Subcommittee on Housing and Transportation: 
Committee on Banking, Housing, and Urban Affairs: 
United States Senate: 

Dear Senator Reed: 

The Department of Housing and Urban Development's (HUD) Housing Choice 
Voucher Program, commonly known as "Section 8" tenant-based assistance, 
is the largest ongoing rental assistance program in the United States, 
serving almost 2 million families with a budget of about $16.9 billion 
for fiscal year 2005. The Housing Choice Voucher Program provides 
subsidies to help low-income families afford rental housing in the 
private market. To determine the amounts of the subsidies it will 
provide to low-income families under the Voucher Program, and for other 
purposes, HUD annually estimates fair market rents (FMR)--that is, rent 
estimates that include utilities. From time to time, public housing 
agencies and others have expressed concern that FMR estimates can be 
inaccurate--often, too low--thereby preventing voucher holders from 
being able to find affordable housing in certain areas. 

HUD estimates FMRs for all bedroom size units for each area in the 
entire United States (typically, counties) in advance of the year 
during which they will be effective. HUD currently uses rent data from 
a variety of surveys--the Bureau of the Census' decennial census long 
form is the major survey used--as a baseline (or benchmark) for 
estimating FMRs throughout the country.[Footnote 1] Between censuses, 
HUD's practice has been to rebenchmark census-based FMRs with data from 
the American Housing Survey (AHS), a Census Bureau survey performed in 
certain metropolitan areas every few years, and from Random Digit 
Dialing (RDD) surveys, telephone interviews that gather rent and other 
data for estimating FMRs for a limited number of metropolitan and 
nonmetropolitan areas annually, conducted by HUD contractors. However, 
a new Census Bureau product, the American Community Survey (ACS), is 
replacing the decennial census long form and will become the major 
source of rent data for FMR estimates in every area. With the ACS, the 
Census Bureau will publish results annually based on 1-, 3-, or 5-year 
averages, depending on the population size of the area surveyed, rather 
than every 10 years. For example, HUD will receive 1-year average data 
(the average of 12 months) annually for areas in which the majority of 
voucher holders reside. 

You asked us to review HUD's process for estimating FMRs and the impact 
that the incorporation of the ACS could have on the accuracy of FMRs. 
Our report discusses (1) how HUD estimates FMRs, (2) how accurate HUD's 
FMR estimates have been, (3) how and when the use of ACS data to 
estimate FMRs may affect their accuracy, and (4) the potential for 
other changes HUD could make to improve the way it estimates FMRs and 
their accuracy. 

To determine the general process for how HUD estimates FMRs, we 
analyzed statutes, regulations, and agency documents and interviewed 
HUD officials. To determine how accurate FMR estimates were, we 
compared all two-bedroom FMRs that HUD put in effect for fiscal year 
2000 with census data for the same year because (1) the decennial 
census rent estimates are considered to be the closest estimates of the 
true value of those rents and (2) HUD estimates FMRs for other bedroom 
sizes as a multiple of the FMR it sets for two-bedroom units. We also 
compared HUD's estimated FMRs in effect during fiscal years 2001-05 for 
selected geographic areas with rents estimated using data from surveys 
HUD and others conducted over this period. After making these 
comparisons, we performed an associative analysis--that is, we analyzed 
specific components of (or data inputs to) the FMR estimation process 
to see how they might relate to the accuracy of FMRs. To determine how 
and when HUD will use ACS data to estimate FMRs and what their 
potential effects on the accuracy of FMRs would be, we compared ACS 
with the other major surveys HUD uses to estimate FMRs, identified 
salient characteristics of the ACS data, and reviewed HUD's plans for 
using ACS data. To determine other changes HUD could make to improve 
its estimation process and the accuracy of FMRs, we analyzed data 
quality guidelines and then assessed HUD's estimation process against 
the guidelines. We also interviewed officials from HUD headquarters and 
field offices, as well as experts and researchers who routinely work 
with housing data sources. Appendix I provides additional details on 
our objectives, scope, and methodology. 

Throughout this report, we refer to the "quality" of surveys or the 
"quality" of data. We use quality as an overarching term for important 
characteristics related to the accuracy, recency, and relevance of data 
sources and surveys. Specifically, for purposes of this report, we 
describe quality data obtained from surveys as: 

* "accurate" when all types of rental housing units have a chance of 
being selected for the survey and the sample size is large enough to 
provide a 90 or 95 percent likelihood that the survey's estimates will 
be within 5 to 10 percent of what would be found if the entire 
population (i.e., all rents) were known;

* "recent" to the extent that the time between when data are collected 
and subsequently used is minimized; and: 

* "relevant" when surveys collect, at a minimum, data on rents for 
HUD's program purposes and, among the survey data sources available, 
HUD chooses the survey that most closely corresponds to the FMR area. 

These characteristics generally match those in data quality guidelines 
used by other federal agencies, and the characteristics of data or 
survey quality required by HUD through statute, regulations, or 
guidance for data submissions. 

We conducted our work in Washington, D.C., between May 2004 and 
February 2005 in accordance with generally accepted government auditing 
standards. 

Results in Brief: 

According to HUD, the typical process to estimate FMRs includes 
developing baseline rents from what it judges to be the best rent data 
available for each area, adjusting those rents to bring them up to 
date, and seeking public comment on its estimates prior to publishing 
them for public housing agencies and others to use. Once HUD determines 
the FMR areas, it uses decennial census housing data when they are 
first released to establish baseline rent estimates, or benchmarks, for 
each. For subsequent years, HUD uses data from other surveys--either 
the AHS or RDD surveys--to establish a new baseline, or to 
"rebenchmark" FMRs for certain areas. To compensate for the time lag 
between when data are collected and when HUD first uses them, HUD 
annually adjusts its baseline estimates in two ways. First, HUD updates 
the estimates to December 31 of the current fiscal year using annual 
percentage changes in rent and utility costs from the local Consumer 
Price Index for major metropolitan areas, or similar information from 
RDD surveys for other areas. Second, to make FMRs relevant for the 
fiscal year in which they will be in effect, HUD trends, or projects, 
the updated figure to the midpoint of the next fiscal year by applying 
a national estimate of annual rent increases between the censuses from 
the decennial census data. After making these adjustments, HUD 
publishes the proposed FMRs in the Federal Register for public comment. 
Although HUD considers all of the comments it receives, it typically 
changes the proposed FMRs only if the comments are supported by data 
that meet HUD's standards. After the period of 60 days to comment on 
the Federal Register ends, HUD still considers other requests and 
submissions throughout the year. 

Over two-thirds of FMRs that HUD estimated for fiscal year 2000, as 
well as those it estimated for areas rebenchmarked after 2000, were 
within 10 percent of the rents indicated by a subsequent quality 
survey, such the AHS. For example, when we compared the fiscal year 
2000 FMRs (which HUD estimated in 1999) with rents from the 2000 census 
data that were collected during the same period the FMRs were in 
effect, 69 percent of all of HUD's FMR area estimates were within 10 
percent of the census figure--an improvement over HUD's 1990 estimates, 
when 39 percent of areas were within 10 percent of the 1990 census. 
When we compared a limited number of FMRs that HUD estimated after 2000 
with rents indicated by data from the AHS or RDD surveys that HUD or 
public housing agencies (PHA) subsequently conducted, a similar 
proportion of FMRs (73 percent) fell in the most accurate range. While 
our associative analysis did not demonstrate what factors definitively 
cause accuracy or how much each contributes, it did show that when HUD 
used more recent, relevant data taken from a higher quality survey than 
some HUD used to rebenchmark in the past, FMR estimates were more often 
within 10 percent of the rents derived from a rebenchmarking survey. 
For example, FMR estimates from areas based on more recent survey data-
-within 1 to 4 years--produced a significantly higher proportion of 
FMRs that were within 10 percent of rents derived from the census than 
FMR estimates from areas surveyed less recently. 

The ACS could improve the accuracy of FMR estimates because it is a 
higher quality survey than some HUD has used in the past and provides 
more recent and local data than are currently available--beginning in 
fiscal year 2006 when HUD first uses ACS data to update FMRs, and 
subsequently in fiscal year 2008 when it will likely rebenchmark FMRs 
in certain areas. HUD will be able to use ACS data to rebenchmark FMRs 
annually (or every 3 or 5 years for areas with smaller populations), 
doing so in generally the same way it used the decennial census to 
estimate baseline rents. Certain challenges related to the manner in 
which ACS data are processed and reported may affect FMR accuracy. For 
example, ACS data are averages of monthly survey data, which may 
"smooth" rental market shifts or trends. According to HUD officials, 
they will begin to address these challenges when the Census Bureau 
releases the fiscal year 2005 data (in Fall 2006), the data collected 
during the first year of full implementation for the ACS. Despite the 
challenges in using the data, neither we nor experts and researchers 
who routinely work with housing data sources identified viable 
alternatives to the ACS. 

Potential exists for HUD to improve its estimation process for FMRs and 
their accuracy because the agency (1) presently does not follow its 
objectivity guideline for ensuring the transparency and reproducibility 
of its FMR estimates and (2) may in the future lack a way to assess the 
accuracy of ACS-based rent estimates. HUD, like other federal agencies, 
has developed guidelines to ensure that it disseminates quality data. 
HUD's guidelines include ensuring the utility (usefulness), integrity 
(protection from unauthorized access), and objectivity (transparency 
and reproducibility) of data. Of the three, HUD appears to be following 
the utility and integrity guidelines as they relate to the FMR 
estimation process. For example, HUD meets its utility guidelines by 
estimating FMRs on an annual schedule and making the estimates public 
and easily accessible. HUD does not follow one of these three--its 
objectivity guideline--because it has made neither the data it uses nor 
its methods for estimating FMRs sufficiently transparent for an 
independent party, such as GAO, to be able to substantially reproduce 
FMRs using publicly available information. Finally, as HUD transitions 
to ACS-based FMRs, it will not only stop using the decennial census 
long form but it will rely less on RDD surveys and the AHS because of 
cost and quality concerns about these surveys. As a result, HUD may not 
have a means to assess the accuracy of future FMR estimates once it 
relies almost exclusively on the ACS. 

This report contains recommendations designed to improve HUD's 
processes for estimating FMRs and their accuracy. We provided HUD with 
a draft of this report for its review and comment. HUD agreed that it 
can better document its methods for estimating FMRs and described 
efforts it has under way to improve the transparency and 
reproducibility of its methods. HUD also requested that we clarify 
certain transparency and reproducibility issues in our report and 
recognize its ongoing efforts. HUD disagreed with our recommendation to 
use state-level ACS data in fiscal year 2006 FMRs, stating that it has 
concerns about the adequacy of ACS sample sizes for the fiscal year 
2006 estimates. We have retained this recommendation because it 
contains a caution that HUD should do so as much as possible, but only 
in instances where HUD determines that the ACS data are sufficiently 
reliable for this purpose. HUD did not explicitly state whether it 
agrees or disagrees with our recommendation that it develop a mechanism 
to assess the accuracy of future FMRs, but it did indicate that it 
recognizes there are areas, such as those with unusual rent increases 
or decreases, that could experience FMR estimation errors when HUD uses 
ACS data for its estimates. HUD also indicated that it anticipates 
continuing to review AHS surveys and making limited use of RDD surveys 
while it explores other long-term alternatives for assessing the 
accuracy of FMRs. Because HUD recognized the challenge we pointed out 
relating to the accuracy of FMRs and stated that it is currently 
exploring ways to address this issue, we have retained our 
recommendation. HUD also suggested a number of technical clarifications 
to our report, which we have made, as appropriate. 

Background: 

HUD estimates FMRs in order to set upper and lower bounds on the cost 
and quality of typical, standard quality units voucher holders rent 
and, in doing so, ensure that the units rented are modest (not 
luxurious), meet the housing quality standards HUD sets for them, and 
are available in sufficient numbers to those seeking to use the 
vouchers. Local PHAs use FMRs to set payment standards, which are the 
basis for determining the subsidies HUD provides to help low-income 
families afford housing in the private rental market under the Housing 
Choice Voucher Program. Specifically, PHAs may set payment standards at 
90 to 110 percent of the FMR for their area and, with HUD approval, 
above 110 percent of the FMR. Because HUD generally requires voucher 
holders to contribute 30 percent of their income as rent, the amount of 
HUD's subsidy (the rental assistance) then becomes the difference 
between the PHA's payment standard and 30 percent of the family's 
monthly income.[Footnote 2]

While FMRs are primarily used in the Housing Choice Voucher Program, 
other programs both inside and outside of HUD also use FMRs. For 
example, HUD uses FMRs to: 

* determine initial rents for housing assistance payments in the 
Moderate Rehabilitation Single-Room Occupancy program;[Footnote 3]

* determine initial renewal rents for units in some expiring project-
based "Section 8" contracts under the Mark-to-Market Program;[Footnote 
4]

* set maximum rents under the HOME Program;[Footnote 5]

* set standard rent ceilings in the Housing Opportunities for Persons 
with AIDS (HOPWA) Program;[Footnote 6]

* make calculations for the "difficult development" areas under the Low 
Income Housing Tax Credit (LIHTC) Program;[Footnote 7] and: 

* review the feasibility of proposed LIHTC projects. 

The Department of Defense (DOD) compares its basic allowance for 
housing (BAH) amounts, which is housing assistance it provides military 
personnel, to HUD's FMRs. More specifically, when DOD determines that 
it is not cost-effective to collect proprietary survey data on housing 
costs, it uses FMRs as a basis for calculating comparable figures. 

Whatever its programmatic use, an FMR must fall within certain 
statutory and regulatory parameters. The U.S. Housing Act of 1937, as 
amended, requires HUD to base FMRs on the most recent available data to 
estimate rents of various sizes and types within a market.[Footnote 
8]HUD regulations and guidance on FMRs further emphasize that rent 
survey data must be the most accurate and current available.[Footnote 
9] HUD specifically requires that the survey methodology provide 
statistically reliable, unbiased estimates of gross rents by, among 
other things, having a large enough sample so that there is a 95 
percent likelihood that the survey's estimates will be within 5 to 10 
percent of what would be found if the entire population (i.e., all 
rents) were collected. HUD also requires that survey samples be random 
and reflect rent levels that exist for housing units of different ages, 
types, and geographic locations within the entire FMR area. Using these 
considerations, HUD's three primary data sources for FMRs are the 
decennial census (long form), the AHS, and RDD surveys. A RDD survey is 
a computer-aided telephone survey of randomly selected households that 
may be conducted by HUD, individual PHAs, or others. 

Finally, FMRs are specifically defined as annual estimates of the 40th 
percentile of gross rents for typical, nonsubstandard market-rate 
rental units occupied by recent movers.[Footnote 10]

Fortieth Percentile of Rents: 

The 40th percentile is the point in a distribution of numbers at which 
40 percent of the numbers are at or below that point; for FMR purposes, 
this is the dollar amount below which 40 percent of the standard 
quality rental units in an area have rented. For example, in the 
distribution in figure 1, $670 is the 40th percentile because 4 of the 
10 rents are at or below that point: 

Figure 1: Example of 40th Percentile of Rent: 

[See PDF for image] 

[End of figure] 

Gross Rent: 

A gross rent is the rent a tenant pays to the owner--sometimes called 
"shelter" costs--plus the cost of utilities (usually, electricity, gas, 
water and sewer, and trash removal charges, but not telephone service). 
If utilities are included in the rent, then the gross rent is simply 
the amount paid to the owner. 

Typical, Standard Rental Units: 

By statute, FMRs are estimates of market rents for typical, standard 
quality housing. HUD has determined that certain rental units should be 
excluded from its data sources in order to meet this definition. 
Specifically, these include rents for units built within the last 2 
years (which tend to be higher priced); units receiving some form of 
subsidy (such as public housing) where the rent does not reflect a 
"market" price; and substandard units--for example, units without 
adequate heating or plumbing--that likely would not meet the housing 
quality standards applicable to the voucher program.[Footnote 11]

Recent Movers: 

HUD has found that rents for units occupied by recent movers (i.e., 
tenants who moved within the past 15 to 24 months) are typically higher 
than what other renters pay. By linking FMR estimates to the rents that 
recent movers have paid, HUD tries to ensure that they more closely 
reflect the rents that low-income households new to the voucher program 
may face when they look for rental housing. 

The Census Bureau is discontinuing the long form and has begun 
replacing it with the ACS.[Footnote 12] Overall, the ACS will provide 
the same type of data as the decennial census long form at the same 
level of geographic area detail, but in a more timely way because it 
will be an ongoing survey (as opposed to one conducted every 10 years). 
Specifically, the ACS will collect data monthly and each year publish 
either 1-, 3-, or 5-year averages (depending on the population in each 
area).[Footnote 13]

HUD Estimates FMRs by Defining Housing Markets, Choosing Data Sources, 
Updating Rent Data, and Evaluating Public Input: 

According to HUD, the typical process it uses to estimate FMRs (rent 
estimates that include utilities) includes choosing what it judges to 
be the best rent data available for each area, adjusting those data so 
that they are up to date, and seeking public comment on the estimates 
prior to finalizing them for public housing agencies and others to use 
(see fig. 2). Once HUD determines each FMR area and receives decennial 
census data or AHS or RDD data, it analyzes the rent data to establish 
a "benchmark" FMR for each area by determining the 40th percentile of 
the rent distribution. Then, HUD annually adjusts the estimates to 
reflect changes in rent and utility costs to compensate for the lag 
between data collection and the period in which the FMR will be in 
effect. After adjusting the FMR for each area, HUD publishes the 
proposed FMRs for public comment. Although HUD considers all of the 
comments it receives, it typically changes FMRs only if the comments 
are supported with data that meet HUD's standards. The public can also 
affect FMRs by (1) requesting that HUD conduct an RDD for the area or 
(2) submitting comments with supporting rent data or information that 
causes HUD to conduct additional research. 

Figure 2: HUD's Typical Process for Estimating FMRs: 

[See PDF for image] 

[End of figure] 

HUD Establishes Areas, Uses Survey Data to Benchmark and Adjust FMRs: 

To ensure that the FMR estimates are useful, HUD's first step is to 
determine FMR areas that they believe correlate with distinct housing 
markets, typically the size of a county (see fig. 2). To determine FMR 
areas, HUD generally uses the boundaries of Office of Management and 
Budget (OMB)-defined metropolitan and nonmetropolitan areas.[Footnote 
14] According to HUD, it may also create new areas that do not 
correspond to OMB boundaries, particularly within sprawling 
metropolitan areas that may have separate housing markets. For 
instance, HUD created a separate FMR area for West Virginia counties 
that had been included in OMB's Washington, D.C., metropolitan area, 
because HUD did not consider these counties to be part of the 
Washington housing market. Although HUD may revise FMR area definitions 
at any time, it typically does so infrequently (not every year when it 
develops FMRs).[Footnote 15] HUD publishes FMR estimates annually for 
356 metropolitan FMR areas and 2,303 nonmetropolitan FMR areas in the 
United States, Puerto Rico, the Virgin Islands, and Guam. 

HUD's second step is to benchmark--that is, estimate baseline rents--
for two-bedroom units by identifying the 40th percentile of the 
estimated rent distribution for each area with the most recent 
available data (for FMR areas for which no new, recent rent data are 
available, HUD skips this step and updates the existing FMR). HUD 
chooses from a variety of data for benchmarking, including the 
decennial census, the AHS, RDD surveys, and traditional surveys from 
the public. According to HUD officials: 

* The decennial census provides the highest quality data to estimate 
FMRs because it provides (1) rent estimates within 1 percent of the 
true value of the 40th percentile of rents in metropolitan areas and 
(2) the most consistent data for all areas to establish a baseline for 
FMRs once every 10 years. 

* Data from RDD surveys have sufficient quality to meet HUD's 
requirements and provide estimates within 3.5 to 5 percent of the true 
value of rents for a limited number of areas, usually metropolitan 
areas. 

* The AHS offers sufficient quality data with estimates within 7 
percent of the true value of rents the survey is measuring and are 
available for a limited number of metropolitan areas every few years. 

According to HUD officials, to be consistent with the definition of 
FMRs, HUD only uses survey data for rental housing units that are: 

* nonsubsidized and of "standard" quality;[Footnote 16]

* more than 2 years old;

* nonseasonal (i.e., occupied year round);

* located on properties of less than 10 acres; and: 

* leased by recent movers (those who have moved within the last 15 to 
24 months). 

HUD adds estimated utility costs to the base rent estimates it derives 
from RDDs because these surveys do not include that information. To do 
so, HUD officials estimate the cost of utilities with PHA utility 
schedules, which include a list of average monthly costs for each 
utility. The decennial census and AHS data include utilities in their 
base year rent estimates. 

The third and fourth steps in the process involve adjusting FMRs. To 
mitigate the time lag between data collection and FMR use, HUD first 
updates FMRs to December 31 of the current fiscal year with information 
about changes in the rent and utility index from the Consumer Price 
Index (CPI) program for specific metropolitan areas or, for other 
metropolitan and all nonmetropolitan areas, with the gross rent "change 
factors" established by regional RDD surveys. To estimate the gross 
rent change factor, or the measure of rent change, HUD conducts 
regionwide RDD surveys in each of its 10 multistate regions (see fig. 
3). 

Figure 3: HUD Regions: 

[See PDF for image] 

[End of figure] 

Once FMRs are updated, HUD attempts to make them useful for the fiscal 
year in which they will be in effect by trending, or projecting, them 
to the midpoint of that fiscal year. To do this, HUD uses a national 
measure of annual rent increases (i.e., average rent increases during 
the 10 years between the censuses, typically 3 percent, on the basis of 
decennial census rent data). 

In the fifth step, HUD also estimates FMRs for other bedroom sizes (in 
practice, one-, three-, and four-bedroom). Because HUD usually lacks 
sufficient survey data to directly estimate FMRs for all unit sizes, it 
typically benchmarks FMRs for two-bedroom units only and estimates rent 
ratios for other sizes.[Footnote 17] According to HUD officials, these 
ratios are based on local rent relationships derived from decennial 
census rent data. Once HUD calculates these ratios, it ensures that 
they are "sequential," which means that FMRs increase as unit size 
increases (e.g., in 1994, three-bedroom FMRs had to be at least 125 
percent of two-bedroom FMRs, and four-bedroom FMRs had to be at least 
140 percent of two-bedroom FMRs). After HUD estimates FMRs for each 
bedroom size unit, it applies a "bonus" to increase FMRs for larger 
units (three-bedrooms or larger) to help ensure that the units can be 
rented by voucher holders. 

HUD Provides Opportunities for Public Input on Proposed FMRs as Well as 
Those in Effect: 

To provide for public input on proposed FMRs: 

* HUD publishes the proposed FMRs in the Federal Register to solicit 
public comments, usually in April or May of each year (sixth process 
step). 

* The public submits comments during the (approximate) 60-day public 
comment period. 

* After the comment period, HUD reviews the responses received and may 
act on some of them prior to finalizing FMRs and publishing them again 
in final form in the Federal Register in September (seventh process 
step). FMRs are in effect for the next fiscal year, which starts 
October 1. 

After the period of 60 days to comment on the Federal Register ends, to 
address situations in which existing FMRs are perceived to be 
inaccurate, members of the public--often, PHAs--also can submit 
information on the existing FMR for HUD to consider. For example, PHAs 
can at any time conduct and submit to HUD the results of their own RDD 
surveys; HUD applies the same criteria to these surveys as it does to 
those that PHAs submit in response to the proposed FMRs in the Federal 
Register. Specifically, HUD requires that any PHA-submitted data it 
uses to change FMRs must be statistically reliable; unbiased estimates 
of gross rents; and, among other things, have a large enough sample 
that there is a 95 percent likelihood that the survey's estimates will 
be within 10 percent of what would be found if the entire population 
(i.e., all rents) were collected.[Footnote 18] Also, PHAs may at any 
time outside of the formal comment process request that HUD conduct an 
RDD survey or submit information about the existing FMR that may cause 
HUD to conduct additional research. 

While the Quality Housing and Work Responsibility Act of 1998 gave PHAs 
the flexibility to set payment standards at 90 to 110 percent of their 
FMRs, they may also request an exception to further adjust either the 
payment standard or the FMR for their area. Specifically, when PHAs 
believe that payment standards at 110 percent of the FMR are 
insufficient to allow voucher holders to successfully lease units, they 
may request from HUD one of two possible exceptions: (1) increase the 
payment standard to exceed the FMR by more than 10 percent or (2) 
benchmark the FMR estimate at the 50TH percentile of rent for the area, 
rather than the 40th percentile of rent.[Footnote 19]

Most FMR Estimates Were Accurate within 10 Percent of the Census or 
Other Rebenchmarking Surveys: 

According to our analysis, more than two-thirds of (1) all FMRs that 
HUD estimated for fiscal year 2000 and (2) a limited number of FMRs 
that HUD rebenchmarked after 2000 were within 10 percent of the rents 
derived from subsequent surveys such as the census, the AHS, or an RDD 
survey. Specifically, 69 percent of all of HUD's FMR estimates for 
fiscal year 2000--published in 1999--were within 10 percent (the most 
accurate range) of rent estimates derived from the 2000 census. 
Moreover, when we considered FMRs by type of area, FMR estimates for 86 
percent of metropolitan areas and 66 percent of nonmetropolitan areas 
fell in the most accurate range in 2000. Similarly, when we compared 
rents derived from rebenchmarking surveys done for 153 FMR areas since 
2000 with the FMR estimates in place at the time of the rebenchmarking 
survey, 73 percent of the estimates were within 10 percent of the rents 
derived from the surveys. FMR estimates were more often associated with 
accuracy when HUD based them on data that were more recent, taken from 
a higher quality survey than some HUD has used in the past, or more 
relevant because the survey covered an area closely matching the 
boundaries of the FMR area.[Footnote 20] Other factors not related to 
the specific survey HUD used to estimate FMRs, such as difficulty in 
estimating utility costs, may also affect the accuracy of FMR 
estimates. 

Over Two-thirds of All FMRs for 2000 Were Accurate within 10 Percent of 
Rents Derived from the 2000 Census: 

According to our analysis, for fiscal year 2000, 69 percent of FMRs 
that HUD estimated for fiscal year 2000 were within 10 percent of the 
2000 census rent estimates, the most accurate comparison data available 
for each FMR area (see fig. 4). 

Figure 4: Accuracy of HUD's Fiscal Year 2000 FMR Estimates: 

[See PDF for image] 

[End of figure] 

FMR estimates that were within 10 percent of the rents derived from a 
higher quality survey, such as a census or RDD survey, could be higher 
or lower (i.e., within plus or minus 10 percent). For example, if HUD 
estimated an FMR of $500 for an area and a higher quality survey of the 
same area found a 40th percentile rent of $550, the difference is 
within 10 percent of the survey as follows: 

$550 (new survey estimate)-$500 (existing FMR) = $50 difference: 

$50 difference/$550 (new survey estimate) = 9 percent: 

In this example, the original FMR was lower than the rent indicated by 
the recent survey, but was within 10 percent. 

The results for 2000 are a significant improvement over results from 
1990 when HUD reported that 39 percent of FMRs were in the most 
accurate range (see table 1). Furthermore, arraying the data by 
population to account for areas where estimates affected more potential 
voucher holders shows that a greater share of FMR estimates were within 
the most accurate range in 2000 than what HUD reported for 1990. 
Considering FMR estimates by type of area, we also found that more 
metropolitan and nonmetropolitan areas were within 10 percent accuracy 
in 2000 than HUD reported in 1990.[Footnote 21]

Table 1: Accuracy of HUD's Fiscal Years 2000 and 1990 FMR Estimates 
Compared with Rents from Census: 

Fiscal year: 2000; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 2%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 8%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 69%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 19%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 2%. 

Fiscal year: 1990; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 25%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 30%; 

Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 39%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 4%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 2%. 

Weighted by population: 

Fiscal year: 2000; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 2%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 4%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 88%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 6%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 1%. 

Fiscal year: 1990; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 5%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 10%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 73%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 12%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 1%. 

Metropolitan areas: 

Fiscal year: 2000; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 3%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 6%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 86%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 4%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 1%. 

Fiscal year: 1990; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 8%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 14%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 71%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 8%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 0%. 

Nonmetropolitan areas: 

Fiscal year: 2000; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 2%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 8%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 66%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 21%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 3%. 

Fiscal year: 1990; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 29%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 31%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 34%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 4%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 2%. 

Sources: GAO analysis of HUD data (2000 figures) and HUD (1990 
figures). 

[End of table]

As table 1 shows, our analysis indicates that in 2000, where FMR 
estimates were higher or lower than the census by 10 percent or more, 
most often the FMR was too low, a different result from 1990 when HUD 
reported that most FMR estimates outside of the most accurate range 
were too high. 

Since the 2000 Census, HUD and others surveyed a limited number of FMR 
areas (153, as of September 2004). When we compared the rents derived 
from these surveys with FMR estimates in effect for these years, the 
outcome was similar to the results we found in our comparison with the 
2000 census--almost three-fourths (73 percent) of FMR estimates were 
within 10 percent of the survey rents. When analyzing the 153 areas, we 
also found a difference in the results shown by rebenchmarking surveys 
undertaken for different reasons. HUD and PHAs conducted rebenchmarking 
surveys for two basic reasons: (1) HUD was adhering to a schedule in 
which it surveyed selected large metropolitan areas on a rotational 
basis or (2) HUD, PHAs, or others received information suggesting FMRs 
were inaccurate (usually a complaint that an FMR was too low) in a 
specific area. As shown in table 2, complaint-driven surveys (RDD 
surveys that were conducted by HUD following a PHA request or by a PHA 
itself) more often found inaccurate FMRs (i.e., FMR estimates that were 
10 percent or more different from the rents derived from the survey). 

Table 2: Accuracy of HUD's FMR Estimates Compared with Rents from RDD 
Surveys (by Reason for Survey, 2001-05): 

Reason for survey: HUD schedule; 
Compared with RDD survey rents--percentage of FMRs that were: Higher by 
20% or more: 0%; 
Compared with RDD survey rents--percentage of FMRs that were: Higher 
by 10% to 19.9%: 3%; 
Compared with RDD survey rents--percentage of FMRs that were: Within 
10%: 87%; 
Compared with RDD survey rents--percentage of FMRs that were: Lower by 
10% to 19.9%: 9%; 
Compared with RDD survey rents--percentage of FMRs that were: Lower by 
20% or more: 1%. 

Reason for survey: By request (HUD surveyed); 
Compared with RDD survey rents--percentage of FMRs that were: Higher by 
20% or more: 2%; 
Compared with RDD survey rents--percentage of FMRs that were: Higher by 
10% to 19.9%: 2%; 
Compared with RDD survey rents--percentage of FMRs that were: Within 
10%: 66%; 
Compared with RDD survey rents--percentage of FMRs that were: Lower by 
10% to 19.9%: 26%; 
Compared with RDD survey rents--percentage of FMRs that were: Lower by 
20% or more: 5%. 

Reason for survey: PHA surveyed; 
Compared with RDD survey rents--percentage of FMRs that were: Higher by 
20% or more: 0%; 
Compared with RDD survey rents--percentage of FMRs that were: Higher 
by 10% to 19.9%: 0%; 
Compared with RDD survey rents--percentage of FMRs that were: Within 
10%: 53%; 
Compared with RDD survey rents--percentage of FMRs that were: Lower by 
10% to 19.9%: 42%; 
Compared with RDD survey rents--percentage of FMRs that were: Lower by 
20% or more: 5%. 

Source: GAO analysis of HUD data. 

Note: HUD estimated FMRs we used in this comparison prior to the public 
comment step that takes place after its estimation process. When HUD 
received the results of RDD surveys prior to the public comment step, 
it used (and published) those rent estimates rather than the initial 
FMR estimate it had developed. As a result, some of the estimates we 
use in this comparison were never published by HUD as proposed FMRs. 

[End of table]

According to HUD, those areas it surveyed because they were on its 
schedule were, like the complaint-driven RDD surveys, not random 
selections. Most often, HUD selected areas from its schedule because it 
had not surveyed them recently, which means that HUD tended to choose 
areas for which the length of time since the last rebenchmarking survey 
was longer. According to HUD officials, choosing areas for RDD surveys 
for this reason increases the likelihood that these surveys would find 
inaccurate FMRs. Further, to the extent that complaints are more likely 
to arise when FMRs are believed to be too low, rather than too high, it 
is not surprising that complaint-driven surveys were much more likely 
to show rents higher than FMRs, rather than lower. 

Quality Survey Data Tended to Produce FMR Estimates That Were Accurate 
within 10 Percent: 

Survey data that had one or all of the characteristics we summarize as 
"quality"--recent, accurate, or relevant--tended to more often produce 
FMRs within 10 percent of another rebenchmarking survey. Specifically, 
our analysis showed that FMR estimates more often fell in the most 
accurate range when HUD based FMRs on survey data that were (1) more 
recent, (2) taken from a higher quality survey than some surveys HUD 
has used in the past, or (3) more relevant because their source closely 
matches the boundaries of the FMR area. 

More Recent Data: 

FMR estimates that HUD rebenchmarked with newer survey data (1 to 4 
years old) were associated with greater accuracy in 2000 (see table 3). 
For example, our analysis found that 88 percent of all FMR estimates 
based on newer data (i.e., 1 to 4 years old) were within 10 percent of 
the census estimates in 2000. 

Table 3: Accuracy of FMR Estimates in 2000 Compared with Rents from 
Census (Based on Age of Baseline FMR Data): 

Age of baseline FMR data: 1 to 4 years; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 1%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 6%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 88%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 5%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 0%. 

Age of baseline FMR data: 5 to 7 years; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 3%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 9%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 67%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 20%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 1%. 

Age of baseline FMR data: No survey from 1990 to 2000; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 2%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 7%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 67%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 20%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 3%. 

Source: GAO analysis of HUD data. 

Note: Areas based on 2000 decennial census data or 8-, 9-, or 10-year-
old non-Census data comprised too few areas from which to calculate 
separate statistics. 

[End of table]

In considering the association we found between recent data and 
accuracy, HUD officials stated that the length of time since the last 
rebenchmarking survey likely affected the accuracy of FMR estimates. As 
our analysis showed, areas for which the baseline data were older 
(including those for which there was no rebenchmarking survey between 
the 1990 and 2000 censuses) more often had FMR estimates that were 10 
percent or more higher or lower than the estimate from a recent survey. 

Data from Higher Quality Surveys: 

When HUD used data from higher quality surveys than some surveys it had 
used in the past, its FMR estimates were accurate more often than when 
it relied on lesser-quality means, such as the traditional surveys some 
PHAs conducted before HUD adopted the RDD survey methodology. 
Currently, HUD uses the AHS or RDD surveys to rebenchmark FMRs between 
the decennial censuses. However, until the mid-1990s, HUD also, on 
occasion, accepted from PHAs and used for rebenchmarking FMRs survey 
data that PHAs collected via less rigorous traditional or telephone 
surveys.[Footnote 22] The AHS and RDD surveys can be considered higher 
quality than the less rigorous ones HUD once accepted because they have 
(1) survey characteristics required by HUD's regulations and guidelines 
and (2) data from a survey closely corresponding to the boundaries of 
the FMR areas. As shown in table 4, the higher quality survey sources-
-AHS and RDD surveys--more often led to FMRs within 10 percent accuracy 
than the estimates based on less rigorous methods. 

Table 4: Accuracy of FMR Estimates in 2000 Compared with Rents from 
Census (Based on Type of Rebenchmarking Survey): 

Type of last FMR rebenchmarking survey: AHS; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 5%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 0%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 95%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 0%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 0%. 

Type of last FMR rebenchmarking survey: RDD-HUD; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 2%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 7%; 
Compared with decennial census rents-
-percentage of FMRs that were: Within 10%: 86%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 5%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 0%. 

Type of last FMR rebenchmarking survey: RDD-PHA; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 3%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 22%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 72%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 4%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 0%. 

Type of last FMR rebenchmarking survey: Traditional; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 2%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 6%; 
Compared with decennial census rents-
-percentage of FMRs that were: Within 10%: 65%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 26%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 1%. 

Type of last FMR rebenchmarking survey: Telephone; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 11%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 28%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 61%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 0%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 0%. 

Type of last FMR rebenchmarking survey: No Survey from 1990 to 2000; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 2%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 7%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 67%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 20%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 3%. 

Source: GAO analysis of HUD data. 

[End of table]

More Relevant (Local) Data: 

When HUD used more relevant (local) surveys to update FMRs--that is, to 
adjust for inflation rather than to rebenchmark or revise the baseline-
-the results were similar: FMR estimates were associated with greater 
accuracy. As shown in table 5, when HUD updated FMR estimates with the 
more local metro-specific CPI--a survey that generally matches the 
boundaries of metropolitan FMR areas--91 percent of estimates were 
within 10 percent accuracy. When HUD used regional RDD surveys--which 
cover much broader areas than the FMR area boundaries--to update FMR 
estimates, many fewer were within 10 percent accuracy. 

Table 5: Accuracy of FMR Estimates in 2000 Compared with Rents from 
Census (by Type of Update Factor): 

Type of update factor: Metro-specific CPI; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 5%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 1%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 91%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 3%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 0%. 

Type of update factor: RDD regional gross rent change factor; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 20% or more: 2%; 
Compared with decennial census rents--percentage of FMRs that were: 
Higher by 10% to 19.9%: 8%; 
Compared with decennial census rents--percentage of FMRs that were: 
Within 10%: 68%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 10% to 19.9%: 19%; 
Compared with decennial census rents--percentage of FMRs that were: 
Lower by 20% or more: 3%. 

Source: GAO analysis of HUD data. 

[End of table]

According to HUD officials, the use of broad factors--that is, factors 
from surveys covering a larger geographic area than the FMR area--for 
updating and trending in the FMR estimation process contributes to 
inaccuracy in the estimates. For instance, the update factors derived 
from regional RDD surveys may not capture changes in the local economy 
within a specific FMR area, such as a large employer leaving town or a 
sizable increase in the housing supply that may affect rents. 
Furthermore, HUD officials stated that the use of a nationwide factor 
for trending FMR estimates--the process of projecting FMR estimates 
into the future year for which they will be effective--may not capture 
local trends. (As previously noted, HUD currently applies to all FMR 
areas a standard trending factor derived from the change in the 
national average rents between the 1990 and 2000 censuses.)

HUD Believes That Other Factors May Influence the Accuracy of FMR 
Estimates: 

In addition to the factors we identified as being associated with the 
accuracy of FMR estimates, HUD officials indicated several more factors 
that might also affect accuracy. Specifically, these officials cited 
(1) general survey error common to all such estimates, (2) the 
characteristics of nonmetropolitan areas, (3) difficulty in estimating 
utility costs, and (4) recent mover rent changes differing from rent 
changes captured by the CPI. 

General Survey Error: 

The data from the survey sources HUD uses are estimates which, by 
definition, can introduce error into FMR estimates. All surveys are 
subject to various types of error, which means that survey data may not 
precisely match the true value the survey is trying to measure. For 
example, sampling error occurs because a sample rather than an entire 
population was surveyed, and, according to HUD officials, census data 
for FMR estimates are generally subject to a 1 percent sampling error 
(in metropolitan areas). While HUD considers census data to be the best 
source for rent estimates (primarily because these data have a far 
larger sample size than any other source used), even the census 
includes some areas with low sample sizes or low response rates. 

Characteristics of Nonmetropolitan Areas: 

Our analysis showed that FMR estimates for nonmetropolitan areas were 
less likely to be based on quality data (more recent, taken from a 
higher quality survey and more relevant) and were also less likely to 
be more accurate. HUD officials told us that nonmetropolitan areas are 
a lower priority for rebenchmarking surveys between the censuses 
because they believe it is better to focus their limited resources (for 
their own rebenchmarking RDD surveys) on the areas where more potential 
voucher holders live (i.e., the metropolitan areas). Nonmetropolitan 
areas were less likely to have a recent rebenchmarking survey 
(sponsored by HUD)--between 1990 and 2000, HUD rebenchmarked 73 percent 
of all metropolitan areas and 31 percent of all nonmetropolitan areas. 
Also, HUD updates almost all nonmetropolitan areas using the broad 
update factors it derives from its regional RDD surveys, meaning that 
these areas' FMR estimates are updated with data that are less "local" 
than what HUD applies to the larger metropolitan areas with local CPI 
rent change estimates. Additionally, surveys of nonmetropolitan areas 
(even the census) often have relatively lower sample sizes than 
metropolitan areas, affecting the quality of the data for 
rebenchmarking FMR estimates there and, as a result, the accuracy of 
these estimates.[Footnote 23]

Difficulty in Estimating Utility Costs: 

According to HUD officials, utility cost data are a source of error in 
all three survey data sources HUD uses to estimate FMRs. For example, 
renters have been documented as unreliable sources of the utility costs 
they pay, yet the census relies on them to report utility cost 
estimates. Utility costs for RDD surveys come from a utility cost 
schedule supplied by the local PHA; however, according to HUD 
officials, although PHAs certify that the data are correct, utility 
schedules can be unreliable and introduce bias into FMR 
estimates.[Footnote 24] The AHS uses a utility estimation model 
(consisting of certain survey variables) that HUD officials believe 
corrects to some extent for the error introduced by relying on tenant 
reporting. Nonetheless, they noted that the AHS model is based on 
survey estimates and thus remains subject to error in ways common to 
all surveys. 

Recent Mover Rent Changes in Metropolitan Areas: 

HUD officials told us that the local survey HUD uses for updating FMRs 
in some metropolitan areas--the metro-specific CPI--may not capture 
sudden changes in rents for recent movers. According to HUD, CPIs 
measure overall rent changes for all renters in a fixed group of units. 
However, rent changes for recent movers can be significantly different 
from changes for all renters. For example, HUD officials stated that 
San Francisco and Boston are among the more volatile housing markets in 
the country and, as a result, among the most difficult for which to 
estimate FMRs. Specifically, in 2000 and 2001, San Francisco's recent 
mover rents increased sharply, then decreased suddenly in 2002. 
However, the CPI for San Francisco, which covers all renters, showed 
above-average but not exceptional rent increases in 2000 and 2001 and 
no change for 2002. 

ACS Could Improve the Accuracy of FMRs by Providing HUD with More 
Recent, Better Data: 

The ACS, which is replacing the decennial census long form, could 
improve the accuracy of FMRs because it is a higher quality survey 
(compared with others HUD has available between the decennial censuses) 
and it provides more recent data that closely matches the boundaries of 
HUD's FMR areas. HUD plans to begin to use ACS data for fiscal year 
2006 FMRs. However, certain challenges that we and others, including 
the National Academy of Sciences (NAS), have identified may affect the 
extent to which HUD can use ACS data to improve its estimates. County-
level ACS data, which will be available each year, could increase the 
accuracy of FMRs because HUD plans to use them to rebenchmark all areas 
more frequently. Because the ACS data are more recent than the 
decennial census data and generally of similar quality and content, HUD 
plans to use ACS data to rebenchmark FMRs in generally the same way 
that it used the decennial census data in the past, but it will be able 
do so more frequently. Certain challenges for HUD regarding the ways 
ACS data are processed and reported may affect its plans for using 
them. For example, the Census Bureau averages ACS data over 1-, 3-, and 
5-year time periods, and averaging could mask sharp trends in rents 
because it can smooth changes that occur within the time period. HUD 
plans to address these challenges after it receives fiscal year 2005 
ACS data--the data collected during the first year of full ACS 
implementation--in Fall 2006. Despite the challenges in using these 
data, neither we nor experts and researchers who routinely work with 
housing data sources identified viable alternatives to the ACS. 

The ACS Is a Higher Quality Survey That Provides More Recent and Local 
Data: 

The ACS could improve the accuracy of FMRs because it is a higher 
quality survey than HUD currently has available between the decennial 
censuses and it provides more recent data closely matching the 
boundaries of HUD's FMR areas. 

Higher Quality Survey: 

The ACS is of higher quality than data sources (RDD surveys and the 
AHS) currently available to estimate FMRs between the decennial 
censuses. According to the Census Bureau, like its long-form 
predecessor, the ACS is the highest quality household survey currently 
conducted by the Census Bureau, and it will provide data more 
frequently.[Footnote 25] The ACS derives similar information as the 
decennial census long form, and its results undergo stringent 
processing by the Census Bureau. Moreover, according to HUD officials, 
the ACS is an impressive improvement over data from any other source. 
For instance, although the AHS is also a Census Bureau product, it is 
similar to RDD surveys because it provides data for only a 
comparatively small number of areas and does so less frequently. More 
specifically, the AHS covers a limited number of the largest 
metropolitan areas every few years. 

More Recent Data: 

Using ACS data to estimate FMRs could improve their accuracy because it 
provides HUD with more recent county-level data. More specifically, the 
ACS will provide data each year that is based on 1-, 3-, or 5-year 
rolling averages (i.e., the Census Bureau will collect data monthly, 
average them over 12 months, and publish new 1-, 3-, and 5-year 
averages each year). Because our analysis indicated that FMRs estimated 
with recent data (i.e., data that are 4 years old or less) more often 
tended to be within 10 percent of the results of a rebenchmarking 
survey, FMRs estimated with annual and 3-year average data could be 
more accurate. Even though FMRs estimated with 5-year average data 
would be based on some data older than 4 years, they could also be more 
accurate than is now the case because HUD could rebenchmark them every 
5 years (as opposed to the 10 years between censuses). 

More Local Data: 

The ACS also will provide more local data--more specifically, state-
level data--that HUD could use to update FMRs and therefore lead to 
more accurate FMRs. Currently, HUD updates FMRs for the majority of 
areas with regional RDD surveys, each of which provides HUD with 
aggregate gross rent change estimates based on data from up to eight 
states. As previously noted, our analysis suggested that when HUD 
estimated FMRs with more local data (i.e., data from a survey that 
closely corresponds to the boundaries of the FMR areas) more FMRs fell 
within the most accurate range. As a result, annual state-level ACS 
data could enable HUD to more accurately update FMRs. Although the 
state-level data do not closely correspond to the boundaries of FMR 
areas, they cover areas much smaller than the currently used RDD 
surveys. 

HUD Expects to First Use ACS Data to Update Fiscal Year 2006 FMRs: 

HUD expects to first use ACS data to update its estimated baseline 
rents when preparing fiscal year 2006 FMRs. To do so, HUD plans to use 
regional-level ACS data, rather than the more local state-level ACS 
data that will be available to it. The state-level ACS data would 
provide reliable data for geographic levels smaller than the areas 
covered by the regional ACS (or regional RDD surveys). However, 
according to HUD officials, they believe they need to obtain and work 
with the ACS data, assuring themselves of its reliability and 
usefulness before they will consider updating FMRs with the state-level 
ACS data. 

The effect of ACS data on FMR accuracy could be most notable once HUD 
begins to rebenchmark--not just update--FMRs with these data, which 
will likely begin with the fiscal year 2008 FMRs. HUD will use the 
first data available under ACS full implementation in Fall 2006 to 
rebenchmark fiscal year 2008 FMRs and plans to use them in ways similar 
to how it had used decennial census data because their content and 
quality are similar to that of the decennial census data. Figure 5 
describes how often HUD could rebenchmark different-sized areas with 
ACS data, showing that, for example, HUD will likely rebenchmark FMRs 
for large metropolitan areas--where the most potential voucher holders 
live--every year. 

Figure 5: Scope of ACS Rebenchmarking as Related to FMR Area Size and 
Housing Choice Voucher Program Data: 

[See PDF for image] 

Note: The most recent available data for population and number of 
housing choice vouchers per FMR area are from fiscal years 2000 and 
2003, respectively. We estimated the "voucher dollar" to approximate 
the relative dollar amounts of housing choice vouchers in each area. To 
do so, we multiplied the FMR (FY 2004) and the number of vouchers for 
each FMR area over 12 months. 

[End of figure] 

Because data developed from a single year of ACS data will be based on 
samples that are approximately one-sixth as large as decennial census 
long-form samples, HUD may need more data points than what the ACS will 
provide for communities with smaller populations in order to estimate 
FMRs. More specifically, according to HUD officials, to obtain a 
sufficient sample of rent data for HUD's program purposes, the agency 
needs data from areas with larger populations--that is, areas that can 
provide more data points--than the ACS will publicly report. For 
instance, in an annual ACS sample from a metropolitan area with a 
population of 100,000, HUD could expect to find in ACS data only 48 
recent movers in two-bedroom rental units, but it needs 200 recent 
movers for its purposes. In order for HUD to obtain its needed minimum 
sample of 200 units, it will likely need to use 1-year average data for 
counties with populations of more than 400,000; 3-year average county-
level data for areas with populations of 133,000 to 400,000; and 5-year 
average county-level data for areas with populations of less than 
133,000. 

In addition, although the Census Bureau will publish 3-and 5-year 
rolling average ACS data every year beginning in 2008 and 2011, 
respectively, HUD may not use these data every year because of concerns 
about their reliability for HUD's FMR estimation purposes. According to 
the Census Bureau, reliable measures of changes in multiyear averages-
-such as what HUD needs in order to estimate FMRs--should only be 
calculated using averages with no overlapping years. The 3-and 5-year 
rolling average ACS data that the Census Bureau publishes every year 
will have overlapping years. For example, in 2008, the Census Bureau 
will publish 3-year average ACS data covering 2005, 2006, and 2007; in 
2009, it will publish 3-year average ACS data for 2006, 2007, and 2008, 
overlapping the previous year's estimate by including 2006 and 2007 
data. For HUD's purposes, a reliable time series of 3-year averages 
would consist of the ACS data that the Census Bureau will publish in 
2008 (2005-07 averages), 2011 (2008-10 averages), 2014 (2011-13 
averages), and so on because these would not have overlapping years. 

ACS Data Pose Certain Challenges to HUD That May Affect FMR Estimation 
and Accuracy: 

HUD's consultant ORC Macro, NAS, the Census Bureau, and we have 
identified certain challenges associated with using ACS data that may 
affect how and when HUD could use the data and improve the accuracy of 
FMRs. The challenges include issues related to the averaging of the ACS 
data, presentation of inflation-adjusted costs (such as rents), 
techniques to deal with missing responses, and reporting differences 
between the decennial census and the ACS. 

Averaging: 

The Census Bureau collects data for the ACS monthly and continuously 
averages them over 1-, 3-, and 5-year time periods. However, this 
averaging could hide rental market shifts because moving averages tend 
to "smooth" changes in data over time.[Footnote 26] For example, if 
from January through September of a given year the rent for an area is 
$800, and from October through December of the same year the rent is 
$1,200, the average annual rent reported by the ACS would be $900, 
which is far less than the current monthly rent of $1,200. As a result, 
the moving averages' "smoothing" effect may hide a turning point, or, 
current prices in the rental housing market. 

Inflation-Adjusted Costs: 

To adjust for general inflation, the Census Bureau will use a general 
adjustment factor rather than an index that is specifically related to 
data items, such as rents or utilities, to present dollar-denominated 
data from the ACS. This could limit the usefulness of the data for 
HUD's program purposes because using a general adjustment factor (i.e., 
national CPI) rather than using an index that is specifically related 
to the dollar-denominated item (i.e., a rent index) could result in a 
less-precise estimate.[Footnote 27] The treatment of dollar-
denominated data is critical to all users of these data, and 
particularly to HUD, which will be using the ACS to determine FMRs 
based on rent data. If HUD had access to the Census Bureau's unadjusted 
annual data, it could then adjust the data pertinent to its FMR 
estimation using rent or utility indexes. We previously raised concerns 
about the Census Bureau inflation adjustment.[Footnote 28] In response, 
the Census Bureau did not provide a rationale for using the general 
adjustment factor, rather than a more specific index, but did indicate 
that the bureau would reconsider its present policy of showing only the 
inflation-adjusted annual estimates. 

Techniques to Deal with Missing Responses: 

A NAS panel and we have raised concerns about how imputation--a 
technique used to deal with surveys with missing responses--could 
affect the accuracy of ACS data, especially in smaller areas. The NAS 
panel that reviewed the 2000 Census raised issues about the potential 
effects of imputation on ACS results. Unlike the process used for the 
decennial census--100 percent follow-up for all nonrespondents--the 
Census Bureau conducts follow-up on only 33 percent of nonrespondents 
to the ACS. The Census Bureau uses the responses from the follow-up 
surveys to attribute a similar pattern of responses to the remaining 66 
percent of nonrespondents. The NAS panel called on the Census Bureau to 
analyze the associated trade-offs in costs and accuracy between 
imputation and additional fieldwork to gather more data.[Footnote 29]

Reporting Differences between the Decennial Census Long Form and the 
ACS: 

In a 2004 study, the Census Bureau found that when the decennial census 
long form and the ACS were used to survey the same area, they reported 
a number of variables differently, including those HUD uses to estimate 
FMRs.[Footnote 30] The variables they reported differently include 
housing occupancy, the year the structure was built, the number of 
rooms, and gross rent. For instance, the study found that for certain 
areas, the ACS reported moderately lower gross rents than did the 
decennial census. According to the Census Bureau, the differences may 
result partly from different survey processing techniques or from the 
multiyear aspect of ACS data. Regardless of the cause, FMRs for fiscal 
year 2008 (the first year of rebenchmarking with ACS data) could show 
bigger changes than would be the case using decennial census data. 
According to HUD, consistent FMRs--that is, estimates that change 
gradually from year to year--are important because wide year-to-year 
fluctuations, especially those changes that lower the FMR, can be 
disruptive to PHAs, which must annually reconsider their payment 
standards any time HUD changes the FMR. 

HUD will address the ACS challenges when it receives and begins to 
analyze 2005 ACS data--that is, the data collected during the first 
year when the ACS is fully implemented--in Fall 2006. HUD may choose to 
participate in an ACS Technical Workshop led by the Census Bureau, 
which may help the agency address these challenges. 

Despite Challenges, the ACS Remains Likely the Best Data Source for 
FMRs: 

Despite the challenges the ACS poses for HUD, neither we nor various 
researchers and industry experts found reason to suggest (1) that HUD 
should not go forward with its plans to use the ACS or (2) that there 
are viable alternatives to the ACS. Other sources of information, such 
as private-market rent data and tax assessment data, typically do not 
contain the information that HUD needs to estimate FMRs. For example: 

* Private-market rent data typically include more expensive properties 
(i.e., luxury units, usually large apartment complexes, in metropolitan 
areas). Most voucher holders do not rent such properties because they 
cannot afford them. Additionally, these data do not include single-
family homes--properties that voucher holders may also lease. 

* Private-market and tax assessment data are typically of lesser 
quality compared with the data sources that HUD generally uses to 
estimate FMRs. Private-market rent data often do not contain a 
representative sample of the full rent distribution in an FMR area. 

* Private-market or tax assessment surveys that include rent data may 
not consistently include questions that ensure the units included 
adhere to HUD's criteria (e.g., rents only from recent movers). 

HUD Did Not Follow One of Its Data Quality Guidelines and May Lack Data 
Sources to Assess the Accuracy of Future FMRs: 

The potential exists for HUD to improve how it estimates FMRs and their 
accuracy because (1) the agency presently does not follow its 
objectivity guideline for ensuring the transparency and reproducibility 
of its data and methods for estimating its FMRs and (2) it may in the 
future lack a way to assess the accuracy of ACS-based rent estimates 
when other information, such as comments from public housing agencies, 
suggests it may need to do so. Various federal agencies, including HUD, 
have developed guidelines to ensure they disseminate quality data. 
Three of HUD's standards--utility, integrity, and objectivity--apply to 
FMR estimation. Although HUD appears to be following the utility and 
integrity guidelines, it did not follow its objectivity guideline--
which calls for the agency to make its data sources and methods 
transparent so the results can be independently reproduced. 
Additionally, as HUD comes to depend less on RDD survey and AHS data, 
it may not have a means to assess the accuracy of future FMR estimates. 

HUD Has Not Followed Its Data Quality Guideline on Objectivity: 

Section 515 of the Treasury and General Government Appropriations Act 
for Fiscal Year 2001 (Pub. L. No. 106-554) directs OMB to issue 
governmentwide guidelines that provide policy and procedural guidance 
to federal agencies for ensuring and maximizing the quality, 
objectivity, utility, and integrity of information disseminated by the 
agencies. According to OMB, information that has been subject to 
independent reanalysis is generally presumed to be of acceptable 
objectivity and therefore reliable to the user. In addition, OMB states 
that an important benefit of transparency and reproducibility 
(objectivity) is that the public can assess how much an agency's 
information hinges on the specific analytical choices of the agency. In 
response to OMB's guidelines, various federal agencies, including HUD, 
have developed similar guidelines for ensuring that they disseminate 
quality information. HUD's guidelines include ensuring the utility 
(usefulness), integrity (protection from unauthorized access), and 
objectivity (transparency and reproducibility) of the data it 
disseminates. 

Based on our review of available information, HUD appears to be 
following the utility and integrity components of its guidelines for 
FMRs. HUD's utility guideline states that the information disseminated 
should be useful--a standard that encompasses accessibility and 
timeliness. HUD follows this guideline by estimating FMRs on an annual 
schedule and making FMRs public and easily accessible by publishing 
them on its Web site and in the Federal Register. HUD's integrity 
guideline states that the information disseminated should be protected 
from corruption or falsification by unauthorized access or revision. 
According to HUD officials, FMR data are kept on an internal server 
with highly restricted access. Furthermore, to ensure the security of 
the system, the officials said they maintain full electronic backups of 
all systems. 

However, we found that HUD does not follow its guideline pertaining to 
objectivity. HUD's guidelines state that it will make publicly 
available the sources, data, and methods used to develop the 
information it disseminates, and that results must be capable of being 
"substantially reproduced." This means that independent reanalysis of 
original or supporting data using the same methods should generate 
similar analytical results. Although HUD generally describes its 
overall methodology for estimating FMRs in publicly available 
documents, the agency has not documented its methodology in sufficient 
detail to permit the results to be independently reproduced. For 
example, although we obtained information on the data and methods HUD 
used to estimate FMRs for fiscal years 2000-05, HUD's process was not 
sufficiently documented to allow us to reproduce FMRs without 
contacting HUD staff to assist us in doing so. In part, this was 
because some of the data HUD used to estimate FMRs, such as utility 
cost data, no longer exist after the agency upgraded the software it 
uses to develop FMRs. Also, HUD did not document some of the key 
procedures, variables, and data it used in estimating FMRs, such as the 
source of benchmarking data (and its rationale for choosing each source 
in any given year).[Footnote 31] Sufficient documentation would have 
allowed outside parties to understand and assess how HUD developed any 
given FMR. For example, sufficient documentation would allow an outside 
party to determine (1) every decision HUD made (such as the FMR area 
definition or survey source), (2) the decision rules it applied in 
making that decision, and (3) the extent to which HUD consistently 
applied these rules. 

HUD's Declining Use of RDD Surveys and AHS Data May Limit Its Options 
for Assessing the Accuracy of Future FMRs: 

HUD officials state that they do not have a plan to assess the accuracy 
of FMRs after they start using ACS data to estimate them, in part 
because they believe they will no longer have a quality comparison 
point or data with which to do so. In the past, HUD assessed accuracy 
by comparing FMR estimates with the rents derived from a subsequent RDD 
survey, the AHS, or a decennial census. However, HUD plans to limit its 
future use of RDD surveys and the AHS because of their concerns about 
cost and quality. According to HUD officials, RDD surveys are very 
expensive (costing upwards of $20,000) and their reliability is 
decreasing. Currently, according to HUD officials, the agency has to 
start with a sample of 97,000 units to obtain a usable sample of 200 
with which to estimate FMRs. Moreover, the response rate for RDD 
surveys is about 40 percent, compared with 90 percent for the ACS, and 
RDD surveys may have nonresponse bias (i.e., people who respond to 
surveys may answer questions differently than those who do not). 
Similarly, the AHS is becoming less useful for HUD's purposes than when 
that survey first began. According to HUD officials, the number and 
sample sizes of AHS metropolitan area surveys has been decreasing over 
the past two decades, and they are not timely for HUD's program 
purposes, thereby making them less useful for estimating FMRs than has 
been the case in the past. Rent data from other sources, such as 
private-market rent surveys and tax assessment records, also would not 
provide HUD with a usable comparison point with which to assess FMR 
accuracy.[Footnote 32]

Nonetheless, HUD's regulations require that the agency allow the public 
to provide comments on proposed FMRs, and its information quality 
guidelines permit affected parties to seek and obtain correction of 
information disseminated by the agency. This extends to the accuracy of 
FMRs. In addition to what its policies may require, even though FMRs 
based on ACS data will most likely be more accurate than previous FMRs, 
HUD officials acknowledge that ACS-based FMR estimates may be 
inaccurate from time to time. For example, FMRs for the smaller areas 
(rebenchmarked every 3 or 5 years with ACS data) may need to be 
assessed within the interval to ensure that they remain accurate 
between rebenchmarkings. Moreover, FMRs for any areas with volatile 
rental markets may need to be assessed with some frequency to ensure 
that they are accurate. However, as previously noted, HUD may lack 
sources of comparable data in the future and may be unable to perform 
these assessments. 

Conclusions: 

HUD's task in accurately estimating FMRs is formidable. It must produce 
estimates for hundreds of areas throughout the country despite having 
few comprehensive, reliable data sources with which to do so. 
Additionally, HUD faces the normal difficulties associated with 
predicting how rents and housing markets will change months (or years) 
into the future. Nonetheless, for those affected by FMRs, such as 
voucher holders, HUD's ability to produce accurate estimates each and 
every year is vital--estimates that are too low make it more difficult 
for low-income households to find housing they can rent with a voucher, 
while estimates that are too high may needlessly waste resources or 
prevent housing agencies from serving more households. 

At the time of our review, HUD could not dispel concerns about its 
process for estimating FMRs because its methodology is not transparent 
enough to allow others--including GAO--to independently analyze its 
rent data and produce similar results. HUD and those who use FMRs would 
benefit from a more transparent methodology because this could enhance 
the credibility of the estimates by clearly delineating the choices HUD 
makes, what alternatives it may have had in making those choices, and 
the decision rules it applied; for example, whether to use OMB's area 
definitions or how much to modify them (and the basis for doing so). 
Making the methodology transparent would also give users more and 
better information with which to consider whether FMRs reliably reflect 
an accurate estimate of the rents voucher holders and others will 
encounter. 

The advent of a new data source holds promise for HUD because a system 
of FMRs that are largely based on the ACS will likely improve the 
quality and accuracy of these estimates. However, the level of 
improvement in the quality and accuracy of FMR estimates depends on how 
HUD uses the ACS data. By choosing to use regional-level data to update 
fiscal year 2006 FMRs rather than the more local state-level data, HUD 
may not be taking full advantage of the new data source as soon as it 
can. 

In addition, as it transitions to the ACS, HUD expects to discontinue 
its use of other surveys like the RDD surveys and the AHS to assess the 
accuracy of its FMRs and, therefore, will not have a means to assure 
itself and others that any given FMR estimate is accurate, particularly 
when it receives public comments or other information suggesting it 
needs to do so. While we agree that HUD is right to be concerned about 
the escalating costs and declining quality of surveys such as the RDD 
surveys, having no reasonable alternative to assess the accuracy of an 
FMR will not likely address the concerns of PHAs with reason to 
question FMR accuracy and may also contradict HUD's own data quality 
guidelines. 

Recommendations for Executive Action: 

To improve the usefulness of its FMR estimates, we recommend that the 
Secretary of HUD take the following three steps: 

* ensure that HUD fully documents its method for estimating FMRs by 
following all of its data dissemination quality guidelines, 
particularly those pertaining to the transparency and reproducibility 
of its methodology;

* use, as much as possible, the ACS data that corresponds more closely 
to FMR areas to update the fiscal year 2006 FMRs; and: 

* develop a mechanism to assess the accuracy of future FMRs, including 
those that are based on the ACS, in instances where HUD learns of 
information suggesting it needs to do so. 

Agency Comments and Our Evaluation: 

We provided a draft of this report to HUD for its review and comment. 
In a letter from the Assistant Secretary for Policy Development and 
Research (see app. II), HUD described our report as a good summary of 
the intent of FMR estimates and the implementation of its methods. HUD 
also suggested certain changes and clarifications to our report. For 
example: 

* HUD suggested that we present population-weighted accuracy estimates 
in the "Highlights" page of our report. We agree that population-
weighted estimates are important and note that we present the 
information in the body of the report rather than the "Highlights" 
page. 

* HUD provided a revised statement describing the process they use to 
eliminate subsidized and nonstandard housing units from the rent 
distribution. As HUD requested, we incorporated the new language in 
footnote 16. 

HUD agreed with our recommendation that it can better document its 
methods for estimating FMRs, but also requested that we clarify certain 
transparency and reproducibility issues in our report and recognize its 
ongoing efforts in this regard. Among other things: 

* HUD noted a distinction between process transparency and 
reproducibility of results, stating that the public's needs are better 
met by providing an overview of how FMRs are calculated and then 
showing the individual calculations for each FMR estimate, rather than 
providing system technical documentation, such as computer programs and 
input data. 

* HUD has sought to make the data and calculation process publicly 
available and transparent. For example, HUD noted that it currently 
posts on its Web site publicly releasable versions of 2000 decennial 
census detailed rent distribution files; FMR history files from the 
Federal Register, including Annual Adjustment Factors; and a summary of 
the general methodology and major data sources it uses to estimate 
FMRs. 

* HUD stated that it provided us with additional information, such as 
computer programs and input data, it used to estimate FMRs, and met 
with us as needed to explain the FMR methodology, including the large 
number of different data sources, decision rules, complex decision 
trees, and complex series of computer programs it uses to estimate 
FMRs. 

We agree that providing step-by-step calculation details for each FMR 
estimate would contribute to process transparency. Moreover, we agree 
that HUD currently makes the major data sources and general methodology 
it uses to estimate FMRs publicly available on its Web site. However, 
as our draft report noted, the current information that HUD makes 
publicly available does not show the individual calculations for each 
FMR estimate and therefore is not sufficient to substantially reproduce 
FMRs, a standard set out in HUD's data quality guidelines. 

With respect to reproducibility, in reviewing HUD's process for 
estimating FMRs, we asked for and HUD provided additional information, 
such as computer programs, input data, and associated documentation. 
Because HUD did not have and could not provide us with critical 
documents, such as a clear step-by-step guide or data dictionary, HUD 
officials met with us to explain the various computer programs and 
variables they used--a step that should not be necessary if the 
objective is for us to be able to independently substantially reproduce 
FMR estimates. Nonetheless, the information and explanations HUD 
provided were not sufficient to allow us to independently reproduce FMR 
estimates. As HUD noted in its comments, documentation of computer 
programs and input data, such as it provided us, are not as useful as 
step-by-step guidelines that clearly detail how it produces each FMR. 
As a result, HUD indicated it plans to consolidate in one place all of 
the information it uses to estimate FMRs and create a new tool, for 
release in April 2005, detailing how it develops each FMR. By making 
this information publicly available on its Web site, HUD expects to 
improve the transparency and reproducibility of its FMR estimates, 
particularly for the users of these estimates. 

HUD disagreed with our recommendation that it use, as much as possible, 
the ACS data that corresponds more close to FMR areas to update its 
fiscal year 2006 FMRs. HUD indicated that many of the annual state-
level rent numbers have a pattern of erratic changes. However, 
according to the Census Bureau, for states with populations of 1 
million or more, annual ACS changes for 2001 to 2004 are generally 
reliable. More importantly, as HUD officials indicated to us during our 
review, a necessary first step in using these data to update fiscal 
year 2006 FMRs would be to assess for each state whether anomalies or 
other concerns might indicate a need to defer in certain instances 
using the state-level ACS data. Accordingly, our recommendation was for 
HUD to use the state-level data as much as possible, recognizing that 
the agency could do so only in instances where the ACS data are 
sufficiently reliable for this purpose, and we have retained the 
recommendation. 

HUD did not explicitly agree or disagree with our recommendation that 
it develop a mechanism to assess the accuracy of future FMR estimates. 
However, HUD disagreed with our draft report's statement that declining 
use of RDD surveys and the AHS may limit its options for assessing FMR 
accuracy. Specifically, HUD stated that even though the ACS will be 
much more accurate than any other survey unless the other survey offers 
more current estimates, ACS rent estimates will always lag by at least 
a year (from the midpoint of the survey estimate); thus, use of 
national rent data to trend the FMR estimate could lead to estimation 
errors in housing markets with unusual rent increases or decreases. 
Accordingly, HUD noted that one of the major challenges posed by the 
ACS is how to identify those areas where the use of regional or 
national trending factors results in estimation inaccuracy, and stated 
that it is currently exploring two alternatives to deal with the 
issues. Thus, although HUD stated that it disagreed with our statement, 
the actions that it intends to take are consistent with our 
recommendation. 

HUD also suggested technical clarifications to our report, which we 
have incorporated as appropriate. 

As arranged with your office, unless you publicly announce its contents 
earlier, we plan no further distribution of this report until 30 days 
after the date of this letter. At that time, we will send copies to the 
appropriate congressional committees and to the Secretary of Housing 
and Urban Development. We will also make copies available to others 
upon request. In addition, this report will be available at no charge 
on the GAO Web site at [Hyperlink, http://www.gao.gov]. 

If you have any questions about this report, please contact me at (202) 
512-6878 or [Hyperlink, woodd@gao.gov] or Bill MacBlane, Assistant 
Director, at (202) 512-6764 or [Hyperlink, macblanew@gao.gov]. Key 
contributors to this report are listed in appendix III. 

Sincerely yours,

Signed by: 

David G. Wood: 
Director, Financial Markets and Community Investment: 

[End of section]

Appendixes: 

Appendix I: Objectives, Scope, and Methodology: 

To describe how the Department of Housing and Urban Development (HUD) 
estimates fair market rents (FMR), we first analyzed statutes and HUD 
regulations, reviewed HUD documents, and interviewed HUD officials to 
identify each step that HUD takes to estimate FMRs, including the role 
that the public has in the process. Further, we also spoke with nine 
HUD field economists for each HUD region--typically, the first point of 
contact for the public--to further understand the role that the public 
can play in adjusting the FMR estimate.[Footnote 33] To identify and 
describe the relevant characteristics of the major data sources HUD 
uses to estimate FMRs, we reviewed agency documents. 

To determine how accurate FMRs were, we compared two-bedroom FMRs that 
HUD had in place for fiscal year 2000--that is, estimates derived from 
HUD's revisions to its baselines and from its update processes--with 
the results of the 2000 census.[Footnote 34] In addition, we compared 
two-bedroom FMRs that HUD estimated for fiscal years 2001-05 with data 
from surveys HUD and others conducted for 153 FMR areas over this 
period. We assessed accuracy by way of a comparison to the decennial 
census or other surveys because our own methodological experts as well 
as others conducting similar research on these issues determined that 
such a comparison is the best way to do so when the true values--that 
is, the distribution of all rents--cannot be known. In conducting both 
of these comparisons, we focused on two-bedroom units because HUD 
directly estimates FMRs for these units from the decennial census and 
its other rebenchmarking surveys. HUD does not directly estimate FMRs 
for other bedroom sizes, making it not possible to do a comparison of 
those FMRs to the results of a survey such as the American Housing 
Survey (AHS) or a Random Digit Dialing (RDD) survey.[Footnote 35]

We performed an associative analysis to determine what components of 
HUD's FMR estimation process may have explained the results we found 
when we assessed accuracy (e.g., whether the estimate was for a 
metropolitan or nonmetropolitan area).[Footnote 36] Our analysis was 
limited to making associations between the components of HUD's 
methodology and the accuracy of its FMR estimates; it did not allow us 
to make a direct causal link between the two because all of the 
information we needed was either no longer available or may not be able 
to be captured by HUD's method for making these estimates. 
Specifically, (1) HUD could not provide all of the data used to 
estimate FMRs from 1990 to 2005, such as utility cost data, because 
these were kept on individual staff's computers and in many cases were 
not transferred when HUD moved its FMR data systems to a more advanced 
server; (2) the lack of transparency we found relative to HUD's 
objectivity guideline for data quality meant that we could not identify 
and isolate specific components of its methodology to attempt a causal 
(rather than associative) analysis; and (3) neither we nor HUD can 
control for factors outside of HUD's estimation process that may affect 
accuracy, such as sudden employment changes that cause an area's rents 
to increase rapidly. 

We present our analysis of the accuracy of FMR estimates in terms of 
the degree (percentage) to which the FMR matched or was close to the 
corresponding survey. For example, for the corresponding fiscal year 
2000 FMRs and census data, we calculated the following for each FMR: 

Survey (census)-Fair Market Rent Estimate = x percent Survey 
(census): 

This calculation produced a percentage that, in this example, we 
characterize as the estimate being within x percent of the census. For 
descriptive purposes, we arrayed these comparisons in increments of 10 
percent because, in terms of the initial FMR, this is the range (90 to 
110 percent of the FMR) in which the public housing agencies may set 
their payment standards without prior approval from HUD. 

To determine how and when the incorporation of the American Community 
Survey (ACS) data might affect the accuracy of the FMR estimates, we 
reviewed agency documents and interviewed HUD officials to determine 
how the agency plans to use ACS data to estimate FMRs. We also analyzed 
Bureau of the Census documents to compare characteristics of the ACS 
data with those of the data sources HUD currently uses (the decennial 
census long form, the AHS, and RDD surveys) to estimate FMRs. 
Additionally, we reviewed research by the National Academy of Sciences 
and ORC Macro, in addition to our own, on the use of ACS data. 

To identify changes HUD could make to improve the way it estimates FMRs 
and their accuracy, we first assessed HUD's process for estimating FMRs 
against its data quality guidelines. More specifically, we analyzed 
each HUD guideline--utility, integrity, and objectivity--and compared 
them with HUD's method for estimating FMRs. We also interviewed HUD 
officials to determine how the guidelines related to FMRs. 
Additionally, on the basis of our analysis of the data characteristics 
we found to be associated with greater accuracy in FMRs (recent, higher 
quality, and more local), we interviewed housing industry experts that 
either routinely work with housing data or are familiar with HUD's data 
needs to identify potential alternative data sources that HUD could use 
to estimate FMRs. We also interviewed HUD officials to determine the 
availability and merits of alternative data sources. 

We conducted our work in Washington, D.C., between May 2004 and 
February 2005 in accordance with generally accepted government auditing 
standards. 

[End of section]

Appendix II: Comments from the Department of Housing and Urban 
Development: 

U.S. DEPARTMENT OF HOUSING AND URBAN DEVELOPMENT: 
ASSISTANT SECRETARY FOR POLICY DEVELOPMENT AND RESEARCH:

WASHINGTON, DC 20410-6000:

March 18, 2005:

Mr. David G. Wood:
Director, Financial Markets and Community Investment:
U.S. Government Accounting Office: 
441 G Street, NW, Room 2440: 
Washington, DC 20548:

Dear Mr. Wood:

Thank you for the opportunity to comment on GAO's draft report on 
Section 8 Fair Market Rents, GAO-05-342. GAO's review of HUD's Section 
8 Fair Market Rent (FMR) system provides a good summary of the intent 
and implementation of the system. There are some items, however, for 
which clarifications or changes are warranted. There are also some 
points with which we disagree for reasons noted.

The major points where clarifications or changes are requested are as 
follows:

1. The title of the "Highlights" report is inconsistent with the 
content. The current title suggests that FMR estimates are inaccurate 
but, as noted below and in the body of GAO's report, this is not the 
case. A more accurate summary of the report would be that "HUD FMRs are 
Generally Accurate And Should Improve With ACS Data, But Calculations 
Should Be Made More Transparent."

2. The unweighted FMR area accuracy measure used in the "Highlights" 
section is misleading. The accuracy percentages reported treat all 356 
metropolitan and 2,303 non-metropolitan areas as equals. Loving County, 
Texas, with a population of 67 is treated as the equal of Los Angeles 
with its 9.5 million people. The most relevant accuracy standard would 
be a voucher-weighted percentage, followed by a population-weighted 
measure. Population-weighted accuracy percentages are provided in the 
body of the paper, but the initial and misleading impression in the 
"Highlights" section will receive more attention. Attachment 1 provides 
data on all three measures.

The voucher-weighted accuracy results of 91 percent overall and 95 
percent for metro areas should be referenced in the summary. It is 
requested that the second paragraph of the "Highlights" section be 
replaced as follows:

"The 2000 Census rent data needed to measure the accuracy of FMRs 
became available four years after the Fiscal Year 2000 FMRs were 
published, and provide the most reliable available accuracy reference 
standard. Ninety-one percent of all Section 8 vouchers in 2000 were in 
areas with FMRs that were within 10 percent of FMRs calculated using 
the 2000 decennial Census. Ninety-five percent of metropolitan area 
vouchers were in areas that met this standard. An equivalent accuracy 
measurement is not available for 1990, but on a population-weighted 
basis 73 percent of all FMRs were within 10 percent of 1990 Census-
based FMRs and 94 percent were within 20 percent. In 2000, 88 percent 
of all population-weighted FMRs were within 10 percent of 2000 Census-
based FMRs and 98 percent were within 20 percent. This is a significant 
improvement over FMR accuracy in 1990. Given that areas considered 
likely to have FMR accuracy problems are given first priority for FMR 
surveys, it is also indicative that about 73 percent of the 153 areas 
surveyed since the 2000 Census had published FMRs within 10 percent of 
the survey estimates and that many of the areas where concerns were 
raised were found to have accurate FMRs."

3. FMR transparency and reproducibility issues raised in GAO's stud 
require clarification and recognition of HUD efforts in these areas. 
There are two approaches that could be taken to making FMRs 
reproducible by interested members of the public. One is to provide all 
of the data and computer programs used plus a detailed explanation of 
system functioning and decision rules. There are statutory restrictions 
on release of some of these data, but HUD has had Census produce 
publicly releasable, close approximations of the data actually used. 
The major deficiency of a systems documentation-based approach is that, 
while any given FMR was developed with a limited number of 
calculations, there are a sufficiently large number of different data 
sources and decision rules to require complex decision trees and a 
complex series of computer programs. Although HUD is willing to provide 
all system documentation and computer programs, and did so for GAO's 
review, its experience with GAO confirms its belief that the public's 
needs are better met by providing an overview of how FMRs are 
calculated and then showing the individual calculations for each FMR 
area.

HUD is in agreement that additional efforts to improve transparency and 
reproducibility are desirable, but requests that GAO's report be 
modified in recognition of the following:

a. There is a significant difference between transparency and system 
reproducibility. HUD has increasingly sought to make the data and 
calculation process as publicly available and transparent as possible. 
Its approach has been to make as much of the underlying data as 
possible available and to provide a step-by-step guideline to show how 
local FMRs were produced. Doing this is much easier than trying to 
provide understandable versions of the computer programs and processing 
stream used to produce FMRs. Documenting the FMR system to meet OMB and 
HUD's Office of Information Technology documentation standards, as was 
done in 2002, was both expensive and staff-intensive. HUD agrees that 
this latter type of documentation is not useful to a typical interested 
party, and believes that providing step-by-step calculation details for 
each FMR area is more useful and provides more estimation transparency 
than trying to expand on system technical documentation.

b. HUD has actively sought to make the FMR process more transparent and 
reproducible. Examples follow:

i. HUD had the Census Bureau prepare special, publicly releasable 
versions of the detailed rent distribution files used to calculate FMRs 
and bedroom rent ratios as well as an explanation of the calculation 
process and factors used. These have been placed on the HUDUSER website 
with file documentation.

ii. HUD provides FMR history files, proposed and final Federal Register 
FMR publications, FMR Annual Adjustment Factors, and a summary of the 
methodology and data sources on its HUDUSER website.

iii. HUD has developed an easy-to-use, Fiscal Year 2005 FMR replication 
tool that permits users to select an area of interest and to see how 
the area's 2000 Census FMRs and bedroom ratios were developed. It 
tracks the year-by-year process to update 2000 Census estimates (or 
more current survey estimates) to produce revised final Fiscal Year 
2005 FMRs. The information provided includes the source and type of all 
update factors used. Historical FMRs and related updating documentation 
will be provided by a related reference tool, although the underlying 
information is already posted on the HUDUSER website. The new reference 
tools are still being tested and improved, but should be made available 
to the public in April 2005. A test version is available for GAO use 
at: http://www.buduser.org/datasets/fmr/computations.btm.

Recognition of these efforts in the GAO report is requested.

c. HUD provided GAO with full documentation for Fiscal Year 2005 FMRs. 
This included all FMR calculation processes, all computer programs, all 
input data, and the step-by-step process that updated Census 2000 rents 
and produced Fiscal Year 2005 FMRs, including the decision rules for 
selecting each of the FMR data sources used. HUD also volunteered to 
answer any questions related to this process and spent many hours doing 
so with GAO staff. Except for Census 2000 base rents (which are based 
on embargoed data), Fiscal Year 2005 FMRs can be reproduced to the 
dollar by following the documentation provided to GAO. HUD also offered 
to show how any given FMR rebenchmarked since the 1990 Census was 
derived, but no longer maintains the mainframe and PC-based mix of 
computer systems needed to systematically replicate all estimates at 
once.

d. HUD disagrees with the criticism on page _47 that its estimates are 
not capable of being "substantially reproduced". The benchmark FMRs for 
any given year are from the Census, the American Housing Survey, a 
Random Digit Dialing survey, or a local survey. The data sources and 
amounts for any rebenchmarked FMRs are specified in the Federal 
Register and in the FMR history file available on the HUDUSER website. 
The FMR Annual Adjustment Factors (AAFs) used to update FMRs each year 
are published in the Federal Re ig ster and posted on the HUDUSER 
website. FMR estimates for any area can be reproduced with these data, 
although the consolidation of all relevant data in one place, as being 
done in the system that will be shortly released to the public, will 
make reproducibility easier for the public.

4. Footnote 16 is misleading because it implies that the Fiscal Year 
2005 FMRs introduced a proxy to eliminate subsidized and nonstandard 
units, whereas HUD has always had to use proxies. This topic is 
relatively complex and is one on which HUD is seeking outside expert 
review to provide additional perspectives. HUD uses the housing quality 
measures available in the Census to eliminate substandard units from 
the rent distributions used to calculate FMRs. Those measures are 
limited, and HUD's objective is to reflect a more rigorous quality 
standard equivalent to Section 8 housing quality standards. A 
statistically process for doing this based on extensive research is 
used that works well for most areas. Moreover, available data from 
various Census Bureau studies suggest that serious housing quality 
deficiencies affect a very small percentage of the inventory and have 
little impact on most FMR estimates. Inclusion of income-based, 
subsidized housing rent charges is a matter of more concern in 
developing FMR estimates. Inclusion of public housing and HUD Section 8 
housing assistance programs could distort FMR estimates, and 2000 
Census data do not identify such assisted housing units. HUD examined 
the adjustment approach used with the 1990 Census data to correct for 
this bias. It also examined alternatives that took advantage of ACS and 
HUD administrative data in conjunction with 2000 Census data, and 
implemented an approach that has the effect of ensuring that a larger 
number of rental units are removed from the bottom of the respective 
local rent distributions prior to estimating FMRs than the number of 
locally assisted housing units.

It is requested that the original footnote 16 be replaced with the more 
accurate statement that follows:

"Prior to 2005, HUD used data on unit quality and assistance from the 
American Housing Survey to generate a proxy for public, assisted and 
substandard housing. This adjustment was constant over the nation and 
did not vary by bedroom size. In the 2000 rebenchmarking, HUD employed 
American Community Survey and HUD administrative data to calculate a 
sub-standard housing adjustment that is tailored to region and bedroom 
sizes. This new proxy allows for larger adjustments in areas with more 
public and assisted housing units and higher housing quality issues. 
HUD still uses information from RDD and AHS surveys to eliminate 
subsidized and nonstandard units from survey data."

5. The discussion of the American Community Survey (ACS) is internally-
inconsistent and is likely to confuse readers. Pages 35 and 36 of the 
GAO draft report state that the ACS is a higher quality survey than 
others available except the Census, that it offers more current data, 
and that "despite challenges in using the data, neither the GAO nor 
experts and researchers who routinely work with housing data sources 
identified viable alternatives to the ACS." Page 44 reiterates this 
conclusion. The report then goes on to raise concerns about the ACS and 
the lack of HUD efforts to develop alternatives to test the accuracy of 
the ACS. In response, the following comments are offered:

a. GAO's concerns about the ACS accuracy apply almost equally to Census 
data-the sample sizes and response rates are so much higher than from 
any other source that evaluating the accuracy of the data using other 
reference information is rarely feasible. The ACS is so much better 
than other data sources that it will become the "gold" reference 
standard once full annual samples are available. The ACS should provide 
annual FMR estimates almost as reliable as those from the Census for 
the largest metro areas and provide highly reliable estimates for all 
other metro areas using either two or three years of data. HUD's 
concern with the wording of the GAO report is that uninformed readers 
may miss the point that the ACS is an impressive improvement over any 
data from any source other than the decennial Census.

b. HUD disagrees with GAO's recommendation to use partial ACS sample 
data at the state rather than the HUD Regional level in developing 
Fiscal Year 2006 FMRs. The ACS sample was not fully implemented until 
the start of 2005, and the previous year samples were much smaller and 
highly clustered. In analyzing ACS data, we have found that many of the 
annual state-level rent numbers show a pattern of erratic change 
suspiciously similar to what one would expect from small samples. We 
have no supporting evidence that instances where unusually high rent 
changes are followed by unusually low rent changes (and vice versa) 
reflect reality rather than sampling variability, which is why we 
believe it is preferable to use multi-state regional rent change 
factors until full sample ACS data for 2005 and subsequent years start 
to become available.

6. HUD disagrees with the statement on page 48 that declining use of 
RDD surveys and AHS data may limit its options for assessing the 
accuracy of future FMRs. First, HUD anticipates continuing to review 
AHS surveys and to make limited use of RDD surveys. HUD's underlying 
disagreement with this statement, however, relates to the relative 
accuracy of the ACS surveys. The ACS surveys will be so much more 
accurate than AHS or RDD surveys for metropolitan areas that, as noted 
in the GAO report on page 44, other available surveys are less reliable 
and therefore cannot be used to evaluate the accuracy of fully 
implemented ACS estimates. Other surveys are likely to be of value only 
if they offer more current estimates. ACS survey data releases will 
always lag by at least a year from the mid-point of the survey 
estimate, and there will be longer lags for all but the largest areas. 
Use of national rent trend information to cover the period from the ACS 
survey to the as-of date of the FMR estimate will work well for the 
large majority of areas, but lead to estimation errors in housing 
markets with unusual rent increases or decreases. One of the major 
challenges posed by ACS data is how to identify the relatively small 
number of areas where the use of regional or national trending factors 
to cover the lag period results in estimation accuracy problems. HUD is 
currently exploring two alternatives to deal with this issue.

Thank you for your consideration of these comments. Please do not 
hesitate to contact me at 202-708-1600 should you have any questions.

Sincerely,

Signed for: 

Dennis C. Shea: 

Attachment 1:

Published Versus Subsequently Available Census-Based FMRs:

[See PDF for image]

[End of table] 

[End of section]

Appendix III: GAO Contacts and Staff Acknowledgments: 

GAO Contacts: 

David G. Wood, (202) 512-6878; 
Bill MacBlane, (202) 512-6764: 

Staff Acknowledgments: 

In addition to the individuals named above, Triana Bash, Tania Calhoun, 
Steve Brown, John Larsen, Marc Molino, Chris Moriarity, Robert Parker, 
MacDonald Phillips, Carl Ramirez, Barbara Roesmann, and Anita Visser 
made key contributions to this report. 

(250205): 

FOOTNOTES

[1] As part of the decennial census since 1960, the Census Bureau has 
mailed separate long-form questionnaires to a sample of households to 
collect detailed information on demographic, housing, social, and 
economic characteristics. 

[2] When households rent units for less than the payment standard, the 
HUD subsidy is the difference between their gross rent and their income 
contribution. 

[3] The Moderate Rehabilitation Single-Room Occupancy (SRO) program 
provides rental assistance for homeless persons in connection with the 
rehabilitation of SRO dwellings. 

[4] HUD's Mark-to-Market Program reduces rents to market levels for 
expiring housing subsidy contracts and restructures existing debt to 
levels supportable by these rents on thousands of privately owned 
multifamily properties with federally insured mortgages. 

[5] HUD's HOME Program helps to expand the supply of decent, affordable 
housing for low-and very-low-income families by providing grants to 
states and local governments to fund housing programs that meet local 
needs and priorities. 

[6] HOPWA addresses the specific needs of persons living with HIV/AIDS 
and their families by making grants to local communities, states, and 
nonprofit organizations for purposes such as facility operations or 
rental assistance. 

[7] The LIHTC Program is an indirect federal subsidy used to increase 
the supply of affordable housing in communities by financing the 
development of affordable rental housing for low-income households. 
Difficult development areas are designated by the Secretary of HUD as 
areas that have high construction, land, and utility costs relative to 
the area median gross income. 

[8] 42 U.S.C. § 1437f(c)(1). 

[9] See 24 C.F.R. § 888.113 for regulations governing the FMR 
methodology. 

[10] Beginning in 2001, HUD set FMRs for 39 metropolitan areas at the 
50TH percentile, because it determined that an FMR increase was needed 
to promote residential choice, help families move closer to areas of 
job growth, and alleviate concentrations of poverty. 

[11] Rents for units on 10 or more acres and seasonal units, such as 
summer rentals, are ineligible for the FMR estimation process. 

[12] The ACS is subject to annual appropriations. Funding for the ACS 
to cover all persons except those living in group quarters (e.g., 
college dormitories and prisons) was approved beginning with fiscal 
year 2005. Funding to cover all persons has been requested beginning 
with 2006. 

[13] The first annual ACS data for geographic areas with populations 
larger than 65,000 will be published beginning in 2006; publication of 
3-year averages for areas with populations of 20,000 to 65,000 will 
begin in 2008; and publication of 5-year averages for areas with less 
than 20,000 will begin in 2010. 

[14] According to OMB, a metropolitan area generally consists of a core 
area containing a substantial population nucleus, and adjacent 
communities exhibiting a high degree of economic and social integration 
with the core. 

[15] In 1994, we reported on a proposal to establish smaller FMR areas. 
See GAO, Rental Housing: Use of Smaller Market Areas to Set Rent 
Subsidy Levels Has Drawbacks, RCED-94-112 (Washington, D.C.: June 24, 
1994). 

[16] Prior to 2005, HUD used information on unit quality and assistance 
from the AHS to generate a proxy for subsidized (public and assisted) 
and substandard housing. This adjustment was constant over the nation 
and did not vary by bedroom size. To estimate fiscal year 2005 FMRs, 
HUD used ACS and HUD administrative data to calculate a substandard 
housing adjustment that is tailored to region and bedroom sizes. 
Specifically, HUD began to use the 75TH percentile of public housing 
rents from its administrative data for each of its regions as a proxy 
to indicate which units are subsidized and substandard. According to 
HUD, this new proxy allows for larger adjustments in areas with more 
public and assisted housing units and higher housing quality issues. 
HUD continues to use information from RDD surveys and the AHS to 
eliminate subsidized and substandard units from survey data. 

[17] The 2000 decennial census produced data sufficient to allow HUD to 
directly estimate FMRs for all bedroom sizes for fiscal year 2005 FMRs 
and update the bedroom ratios. According to HUD officials, they will 
use these new ratios to estimate future non-two-bedroom FMRs. 

[18] In very limited instances, HUD officials will accept data from 
PHAs in areas with small populations that have not followed the 
requirements. According to HUD officials, some areas with small 
populations will not be able to comply due to limited budgets or small 
sample sizes within the FMR area. HUD officials then evaluate the data 
on the basis of their professional judgment. 

[19] In order to obtain an exception to increase the payment standard 
by more than 10 percent, the public must submit documentation that 
demonstrates approval of the special exception is necessary to prevent 
financial hardship for families in the exception area. This 
documentation can include census rent data, locally funded quality 
surveys, lease rates, and success rates. The request must be needed (1) 
to enable families to find housing outside areas of high poverty and 
(2) because voucher holders have trouble finding housing for lease. 

[20] We use "associated with accuracy" because our analysis does not 
enable us to make causal links between survey or FMR area 
characteristics and the accuracy of estimates. 

[21] Most households receiving tenant-based vouchers--85 percent--live 
in metropolitan areas. 

[22] Traditional surveys are surveys of rent data in metropolitan areas 
with relatively low populations in which a PHA or other entities have 
access to all or almost all of the rents in the area--for example, in 
cities or towns that require owners to register rents annually and 
maintain a database of rents. Telephone surveys are generally derived 
from randomly selected lists of residential telephone numbers but are 
not assisted by the use of a computer to track telephone calls and the 
outcomes. 

[23] According to HUD, some nonmetropolitan areas have unusually low 
census data sample sizes and unusually high levels of substandard and 
assisted housing that may distort the accuracy of FMR estimates. For 
the fiscal years 1996 through 2004 FMRs, HUD corrected for low FMR 
estimates that were at or below the cost of operating housing by 
implementing a minimum FMR level for each state. 

[24] According to HUD, a PHA utility schedule is a list of the average 
monthly costs of various types of utilities, such as heating oil, 
electricity, or water and sewer charges, subdivided by the number of 
bedrooms in the unit. 

[25] The Census Bureau reports that the 5-year averages will be about 
as accurate as the long-form data; the annual and 3-year averages will 
be significantly less reliable than the long-form data but more 
reliable than existing household surveys the Census Bureau conducts. 

[26] See ORC Macro, The American Community Survey: Challenges and 
Opportunities for HUD (Calverton, MD: Sept. 27, 2002). ORC Macro is the 
consultant HUD hired. 

[27] See GAO, American Community Survey: Key Unresolved Issues, GAO-05-
82 (Washington, D.C.: Oct. 8, 2004). 

[28] See GAO-05-82. 

[29] See GAO-05-82. 

[30] U.S. Census Bureau, Meeting 21ST Century Demographic Data Needs-
Implementing the American Community Survey Report 8: Comparison of the 
American Community Survey Three-Year Averages and the Census Sample for 
a Sample of Counties and Tracts (Washington, D.C.: 2004). 

[31] For example, if the survey source for an FMR estimate was the AHS, 
HUD's documentation did not indicate what other sources, if any, it 
considered that year and why it chose the AHS over any other available 
sources of data for that year. 

[32] According to ORC Macro, HUD may be able to use special ACS 
tabulations from the Census Bureau to detect shifts in rent trends for 
areas where HUD will use multiyear average data to estimate FMRs. These 
data for each FMR area may not contain enough samples to estimate FMRs, 
but would give HUD an indication that an existing FMR may be 
inaccurate. 

[33] As of December 2004, nine HUD regional field economists managed 
the agency's economic work in the 10 HUD regions because there was a 
vacancy in Region 2 (New York/New Jersey). 

[34] In order to use the 2000 decennial census data we obtained from 
HUD to assess the accuracy of FMRs, we verified the reliability of the 
census data by asking HUD officials a series of data reliability 
questions. 

[35] For non-two-bedroom units in the 2000 decennial census survey and 
153 subsequent rebenchmarking surveys, HUD took the survey results for 
two-bedroom rents and applied a rent ratio that, in HUD's view, 
captured the approximate relationship between rents for two-bedroom 
units and other sizes. For example, through fiscal year 2004, for three-
bedroom units, HUD determined that the relationship between these and 
two-bedroom rents was 1.25, so the three-bedroom FMR would be 125 
percent of what HUD estimated for two-bedroom units. 

[36] When we compared the accuracy of FMR estimates with the two types 
of update factors HUD uses (metro-specific Consumer Price Index or RDD 
regional gross rent change factor), we excluded a limited number of FMR 
areas because HUD applies special rules for updating this group, making 
the update calculations too dissimilar for our purposes. 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office

441 G Street NW, Room LM

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director,

NelliganJ@gao.gov

(202) 512-4800

U.S. Government Accountability Office,

441 G Street NW, Room 7149

Washington, D.C. 20548: