Cover ========================================================================== COVER Report to the Chairman, Committee on Veterans' Affairs, U.S. Senate September 1994 VETERANS' BENEFITS - LACK OF TIMELINESS, POOR COMMUNICATION CAUSE CUSTOMER DISSATISFACTION GAO/HEHS-94-179 VA Customer Satisfaction Abbreviations ========================================================================= ABBREV VA - Department of Veterans Affairs Letter ========================================================================= LETTER B-251769 September 20, 1994 The Honorable John D. Rockefeller IV Chairman, Committee on Veterans' Affairs United States Senate Dear Mr. Chairman: Because of frequent complaints about the quality and timeliness of the Department of Veterans Affairs' (VA) claims processing, you expressed interest in improving the quality of service VA provides to its customers. In fiscal year 1993, VA provided nearly $19 billion in nonmedical benefits to veterans and their families. You asked that we determine how applicants for benefits view the quality of VA's services. This report summarizes the results of our 1993 national survey of 1,400 recent applicants for VA nonmedical benefits. In addition to assessing customers' overall satisfaction with VA's service, the survey highlighted concerns in several key areas, including the time it takes to process claims, how VA communicates with veterans and their families, and the frequency with which applicants have to resubmit documents to VA in support of their claims. The survey also provided information about the extent to which (1) VA denies claims and (2) organizations other than VA, such as veterans service organizations, are involved in the claims process. RESULTS IN BRIEF ---------------------------------------------------------------------- Letter :1 Although most applicants were satisfied, over a third were dissatisfied with VA's handling of their claims. (See fig. 1.) VA has identified customer concerns and is developing approaches designed to address them. The concerns voiced by respondents to our survey have important implications for those efforts to improve customer satisfaction. Figure 1: Applicants' Opinions About How VA Handled Their Claims (See figure in printed edition.) The time it takes VA to process claims was by far the greatest source of applicants' dissatisfaction, according to our survey. VA has recognized the need to speed up its claims processing and has established timeliness goals. However, even if those goals are met, many customers may remain dissatisfied because they believe processing should be completed in less time than the goals call for. For example, initial compensation applicants thought claims should be processed in an average of 9 weeks, but VA's goal for these claims is 15 weeks. Communication with VA was another major concern for applicants. Many customers said they were dissatisfied, whether the communication was by mail, by phone, or in person. For example, 40 percent of those who visited a VA office reported they did not get the information they needed. One veteran's comment may have summed up the issue for many: "It's like you can't go to just one person and sit down and have them explain to you step by step what you need to do for whatever problem . . . ." Applicants most often wanted information about services and benefits available to them, the status of their claims, and the reasons for the decisions on their claims. The need to resubmit documents to VA also inconvenienced applicants. Twenty-two percent of applicants had to resubmit documents at least once. Civilian documents such as marriage certificates were resubmitted most often. Resubmissions cost applicants time and money and can increase VA's workload and processing time. Our study pointed out two other factors that may hold significant implications for VA's efforts to improve customer satisfaction. First, applicants whose claims were denied represented a significant portion--36 percent--of VA's customers. VA knows very little about who those applicants are, why their claims were denied, and what VA could do to help these people. Second, 60 percent of VA customers received service from sources over which VA has no authority, such as state and county veterans offices and veterans service organizations. The extent to which these other sources are involved highlights the need for continued communication between VA and these sources as VA seeks to improve customer satisfaction. SCOPE AND METHODOLOGY ---------------------------------------------------------------------- Letter :2 To obtain the views of VA's customers, we surveyed a national random sample of 1,400 applicants. These applicants' claims were representative of the claims completed at VA's regional offices between April 1 and July 13, 1993, for nine types of benefit claims. Appendix I provides more details on our scope and methodology; appendix II provides a copy of the questionnaire and customer responses. We also conducted two focus groups, one with veterans who applied for disability compensation and one with veterans' survivors who applied for initial death pension benefits. We spoke with representatives of veterans service organizations, including the American Legion, Blinded Veterans Association, Disabled American Veterans, Paralyzed Veterans of America, and Veterans of Foreign Wars. In addition, we telephoned and visited state and county veterans offices in Alabama, Florida, and Georgia. To obtain information about VA's current operations and efforts to improve services, we reviewed VA's policies and procedures, data relative to claims processing and timeliness standards and goals, current customer survey efforts, and documentation outlining VA plans and efforts to address customer and congressional concerns. We also met with VA officials at the Washington, D.C., headquarters and at VA's regional offices in Atlanta, Georgia; Montgomery, Alabama; New York City, New York; St. Petersburg, Florida; and Washington, D.C. This report summarizes responses from applicants for nine types of claims. Appendix III provides more detailed information about, and highlights some differences among, the nine types of claims. Our review was conducted from December 1992 to June 1994 and was performed in accordance with generally accepted government auditing standards. BACKGROUND ---------------------------------------------------------------------- Letter :3 VA's Veterans Benefits Administration is responsible for administering benefit programs, such as disability compensation and pension. Veterans and their families can apply for benefits through any of VA's 58 regional offices.\1 Veterans and their families also may seek assistance with VA benefits from a variety of sources, such as state and county veterans offices and veterans service organizations. Other sources provide assistance with specific benefits, such as funeral home directors with burial benefits and school officials with education benefits. VA has recognized the need to improve services to veterans and is taking steps designed to do so. Much of the emphasis has been on improving timeliness of claims processing and reducing the backlog of claims, which increased from 107,000 in 1988 to over 520,000 in 1993. VA has recognized, for example, that it does not efficiently obtain documents necessary to decide claims, it does not effectively control its records, and many staff are not sufficiently trained. VA has undertaken three long-term initiatives. First, in December 1992, VA awarded a contract to implement a modernization plan designed to improve operations through automation.\2 Second, VA is implementing total quality management and is encouraging each regional office to develop locally appropriate restructuring initiatives to improve operations. VA's third major effort focuses on disability claims, which are usually the most difficult and time consuming to process. In November 1993, the Secretary of Veterans Affairs approved a package of recommendations designed to improve processing of those claims. He gave regions until May 1995 to develop plans to implement those recommendations and make other changes to improve claims processing. To help it develop short- and long-term goals, VA conducted, by telephone, a customer satisfaction survey between October and November 1992. In January 1994, VA published results for five programs: the Veterans Assistance Service, which is responsible for answering customer inquiries by phone or in person; compensation benefits; vocational rehabilitation; and two insurance programs.\3 Although the response rates for these surveys were relatively low, the findings were in many ways similar to ours. VA intends to use survey findings to help establish priorities for future service improvements and to develop goals and standards. -------------------- \1 VA regional offices are located in each state, the District of Columbia, Puerto Rico, and the Republic of the Philippines. New York, Texas, and Pennsylvania have two regional offices each, and California has three. \2 After we pointed to the need for improved planning and oversight of VA's modernization efforts, VA signed an agreement with the Office of Management and Budget to redirect its efforts. See Veterans Benefits: Acquisition of Information Resources for Modernization Is Premature (GAO/IMTEC-93-6, Nov. 4, 1992) and Veterans Benefits: Redirected Modernization Shows Promise (GAO/AIMD-94-26, Dec. 9, 1993). \3 VA surveyed customers about nine programs in all, but does not anticipate publishing results for four of these programs because response rates were too low or surveys could not be completed because of technical difficulties. PROCESSING TIME IS A CRITICAL CONCERN FOR APPLICANTS ---------------------------------------------------------------------- Letter :4 The time VA takes to process claims was the greatest complaint among those customers we surveyed.\4 Fifty percent of customers indicated timeliness should be improved. This was the major concern regardless of the type of claim, whether VA approved the claim, and the applicants' gender, age, or education level. Processing time historically has been a problem for VA, and it is getting worse.\5 VA officials acknowledged this and attributed increased processing time to several factors that are unlikely to change, such as the greater complexity of claims. In October 1992 VA officials set goals for how long it should take to process claims for fiscal year 1993. Those goals generally represent improvement over actual processing times during a baseline period, but the goals do not always meet customer expectations for reasonable processing time. VA's goals for four types of compensation and pension claims were longer than the average time customers said was reasonable for processing claims. For example, VA's goal for processing initial disability compensation claims is an average of 15 weeks. But applicants reported that, on average, initial compensation claims should be processed in 9 weeks. (See fig. 2.) VA's 1992 telephone survey of customers found similar expectations. On average, respondents in VA's survey said they should get a final decision from VA regarding a compensation claim in 10 weeks. Figure 2: VA Not Meeting Customer Expectations (See figure in printed edition.) Our survey also indicated that VA practices may affect applicants' satisfaction level. Those who were not given an estimate of how long it would take to process their claims were generally less satisfied. They were more likely to indicate that VA's processing time was unreasonable, that VA's decision was unfair, and that they were dissatisfied with how VA handled their claims. Over 40 percent of applicants reported VA did not provide any estimate of when their claims would be decided; VA policies do not require an estimate to be given. (See fig. 3.) Figure 3: Satisfaction Levels of Those Who Received Estimates and Those Who Did Not (See figure in printed edition.) However, giving an applicant an estimate of processing time can backfire when the estimate is unrealistic. In that case, applicants' dissatisfaction levels can increase. For example, one compensation applicant said he had been told it would take 5 to 6 months to process his claim, but instead it took about 12 or 13 months. The applicant said that had VA given him a realistic estimate of the processing time, he would not have been so dissatisfied and would not have repeatedly called VA to check on the status of his claim. VA officials agreed that providing realistic estimates may also reduce VA's workload since customers would not telephone VA to ask about claim status until the estimated time had elapsed.\6 -------------------- \4 Applicants for compensation claims were generally most dissatisfied with timeliness and the other aspects of service discussed in this report. But, as the tables in appendix III show, troublingly large portions of applicants for other types of benefits expressed similar concerns about key aspects of service. \5 Compensation and pension claims present the biggest problem. In 1989, we reported that VA took too long to process these claims (see Veterans' Benefits: Improvements Needed in Processing Disability Claims, GAO/HRD-89-24, June 22, 1989). In 1988, initial disability compensation claims--generally considered the most complicated--on average took 21 weeks to process. In fiscal year 1993, they averaged 27 weeks. \6 Our data did not allow us to determine the reasonableness of the estimates VA gave our respondents because we did not have the actual processing time for each case. CUSTOMERS OFTEN DISSATISFIED WITH COMMUNICATION WITH VA ---------------------------------------------------------------------- Letter :5 Many applicants were dissatisfied with their communication with VA. Some did not receive required communications from VA. Others did not get the information they needed whether the communication was by letter, phone call, or personal visit. Poor communication greatly increases the potential for dissatisfaction among customers and increases VA's workload. Many customers did not receive letters that VA is required to provide. Overall, about 34 percent of applicants did not indicate that they were informed by letter that their application had been received, and 38 percent of applicants whose claims had been completed did not indicate they received a letter stating that from VA. Of those who said they received letters from, phoned, or visited VA, over one-third did not get most of the information they needed. (See fig. 4.) Applicants were most interested in VA (1) explaining the services and benefits that are available, (2) keeping them informed on the status of their claims, and (3) explaining the reasons for decisions. One applicant said it would have been helpful if VA had told him what information they used to decide his claim. "That way," he said, "when I appealed it, I would know what to send them--what additional information I needed to send." Other applicants said they would like "an indication of the amount of time it might reasonably take for benefits to reach the person" or "more information on the questions [the applicants] asked--not general information." Figure 4: VA Often Does Not Provide Most of the Information Applicants Need (See figure in printed edition.) For those who sought informational materials from VA, the results were similar. Of the 50 percent of our sample who requested informational materials, more than one-quarter did not get them. Of those who received the informational materials they requested, about one-quarter told us that these materials did not contain most of the information they needed. Despite not getting what they needed, over 80 percent of applicants said that VA employees were courteous. And it appears courtesy helps mitigate applicants' dissatisfaction. One applicant said, "Their phone service is fine. They were helpful in telling me they knew nothing about when my claim would be finished." Likewise, discussion in our focus groups indicated many problems with VA service, but some participants gave VA a high overall rating, citing courteous and friendly employees as the reason. Our survey results also point to the positive effect of courtesy. For example, applicants who contacted VA by phone and reported that employees were courteous were more likely to be satisfied (57 percent) overall with VA's handling of their claims than dissatisfied (33 percent). The quality of VA communications with customers has been a long-standing problem.\7 VA is developing approaches to address some of the problems raised in our survey. For example, VA is testing a variety of software programs to allow written communications, including decision letters, to be more informative and clearer. VA also is implementing a plan to change claims processing; in some regions, changes allow applicants to directly contact the employees most knowledgeable about their claims. This is a significant departure from the traditional way claims are processed. Furthermore, by February 1994 VA had begun sending form letters to applicants every 90 days indicating that their claims were still in process. While these changes are potentially significant, they may not address situations where customers did not get information required or requested or where veterans and their families need to understand benefits and services before they apply. -------------------- \7 Our 1989 report found several shortcomings in VA's communication with customers. For example, we found that VA decision notices did not provide a clear and full explanation for VA actions and decisions. MANY APPLICANTS HAD TO RESUBMIT DOCUMENTS ---------------------------------------------------------------------- Letter :6 Having to resubmit documents to support a claim adds to customer dissatisfaction and the time it takes to process a claim. Twenty-two percent of applicants had to resubmit a document. Applicants who resubmitted documents were more likely to be dissatisfied than those who did not have to resubmit. For example, 48 percent of those who resubmitted records were dissatisfied, overall, with how VA handled their claim, compared with 31 percent of those who did not have to resubmit records. Sixteen percent of applicants had to resubmit civilian documents, such as birth certificates or marriage licenses, rather than military medical or service records. Resubmitting civilian documents involves time and expense on the part of the applicants. The following comments illustrate the frustration some applicants felt. "I have received as many as six demands for marriage certificates and birth certificates all within days from each other . . . . I am afraid to call the regional office anymore because every time I do I get treated like it's my first contact with them, and the barrage of record requests starts all over again." "I hand-delivered records [DD214-personal medical records] and they took photostats and verified them only to have them say 19 days later that they never received them . . . . The confusion in the system leaves something to be desired . . . . I feel I lost to a system and confusion." VA officials recognize this problem and identified some reasons applicants may need to resubmit documents. One reason is that when documents are submitted, they are not matched with the applicant's claim file. This can happen if the applicant does not provide the right information, such as social security numbers, for VA to determine which file the document goes with. Or VA might not be able to find the file. VA's current processing procedures require a file to be handled by many different employees in many different locations, and files are misplaced. VA is beginning to implement a bar code system to allow regional offices to better track the location of claim files within each region. Another reason applicants have to resubmit documents is that they do not understand VA's document requirements. Particularly, some applicants do not understand that VA requires certified, not photocopied, documents. To confirm this, one VA regional official telephoned 12 applicants who had been sent the usual form letter to request documents. Nine of the 12 applicants said that they had not understood from the letter that VA wanted certified copies of those documents, not photocopies. All of those applicants then provided certified copies of the documents. If those applicants had not been telephoned, at least some of them would have sent uncertified documents. If they had, VA's usual procedure would have been to send the same form letter requesting the same documents. Some VA regional offices are changing procedures to allow employees to call applicants to clarify what documents are needed. Moreover, legislation has been introduced to eliminate the requirement that documents be certified, and even without such legislation, VA is revising its regulations to allow photocopies, rather than certified copies, for some purposes, such as showing marriage or death. SURVEY INDICATES OTHER FACTORS TO CONSIDER IN IMPROVEMENT EFFORTS ---------------------------------------------------------------------- Letter :7 Our survey also highlighted two additional factors that could affect VA's efforts to improve customer satisfaction. First, applicants whose claims are denied, about whom VA has little information, constitute a large portion of VA's customer base. And, second, many people other than VA staff influence applicants' satisfaction with the claims process. INFORMATION ABOUT DENIED CLAIMS COULD BE VALUABLE FOR IMPROVING SERVICES -------------------------------------------------------------------- Letter :7.1 Applicants whose claims were denied represent a substantial portion of VA's customers, but VA knows little about them. These applicants were consistently less satisfied and had more difficulty with all aspects of the process. Of those applicants who told us their claims were completed,\8 36 percent had been denied. (See fig. 5.) Figure 5: How Applicants' Claims Were Decided (See figure in printed edition.) Note: "Partially allowed" refers to those applicants who got some of the benefits they asked for, but not all. Figure excludes the 20 percent of our respondents who said their claims had not been completed. Those whose claims were denied represented 45 percent of all applicants who were dissatisfied. Their dissatisfaction levels with timeliness, fairness, and overall processing of the claims were higher than those of applicants whose claims were fully allowed. (See fig. 6.) Figure 6: Applicants' Opinions About VA, by How Their Claims Were Decided (See figure in printed edition.) Denied applicants had more trouble with the claims process than those whose claims were fully allowed. For example They were less likely to have gotten informational materials they requested (61 percent for denied claims versus 93 percent for fully allowed claims). They were more likely to say informational materials did not include most of what they needed to know (33 percent versus 12 percent). They were more likely to indicate the letters from VA did not contain most of what they needed to know (55 percent versus 12 percent). They were more likely to state that telephone conversations with VA did not give them most of what they needed to know (64 percent versus 19 percent). They were more likely to state that visits to VA offices did not give them most of what they needed to know (68 percent versus 14 percent). The areas identified by applicants whose claims were denied as most needing to be improved mirrored those identified by applicants overall. However, they were more concerned about improving explanations of decisions and the helpfulness of employees. (See fig. 7.) Figure 7: Ways Applicants Said VA Should Improve Service (See figure in printed edition.) VA has little information about these denied applicants. For example, until our survey, VA did not know what portion of all applications are denied and, until its most recent customer survey, VA did not include the views and concerns of applicants whose claims were denied in assessing customer satisfaction. VA does not keep data on denied claims that could allow it to assess the implications of our survey's findings or the adequacy of services it provides to these applicants. Data could, for example, allow VA to determine the following for denied applicants: Did those who did not obtain help in filing their claim want such help? The answer has implications for VA's outreach efforts. Did they apply for benefits for which they clearly were not eligible? The answer has implications for changes in VA's informational materials, as demonstrated by one respondent's comment: "The information about eligibility was misleading. More specific [information] would probably have resulted in my not filing a claim." Did those who received letters about denials understand them, and if not, why? The answer has implications for what kinds of changes VA needs to make in those letters. Were their claims frequently denied because records could not be found to support the claims? The answer has implications for assessing, and possibly improving, VA's efforts to assist applicants in filing claims, including assistance in finding other sources of support. -------------------- \8 Eighty percent of respondents indicated that their claims had been completed, that is, VA had fully allowed, partially allowed, or denied their claims. The other 20 percent reported that their claims were still pending, they did not know if their claims had been decided, or, for some other reason, their claims were not final. MANY PEOPLE OTHER THAN VA STAFF ASSIST VA'S CUSTOMERS -------------------------------------------------------------------- Letter :7.2 Many applicants receive assistance from sources not under VA's authority. VA is part of a network that supports veterans and their families that includes state and county offices, veterans service organizations, and others. About 60 percent of applicants receive services from these other sources, sometimes in addition to getting help from VA. VA officials said that applicants seek help from other sources because these sources are more numerous and geographically dispersed than the 58 VA regional offices. Over half of all applicants reported first learning about benefits from someone other than VA or the military. These sources most frequently were veterans service organizations, state and county veterans offices, and friends or relatives. (See fig. 8.) Figure 8: How Applicants First Learned About Benefits (See figure in printed edition.) Notes: Excludes those who answered, "I don't recall." "Other" includes, for example, a friend or relative, funeral home or cemetery staff, and school officials. These and other sources also helped with the claims process, usually in filling out the application or filing the claim. More than half of respondents said someone other than VA was the most helpful. (See fig. 9.) Figure 9: Sources Applicants Said Helped the Most (See figure in printed edition.) Note: "Other" includes, for example, a friend or relative, funeral home or cemetery staff, and school officials. Applicants had high praise for these other sources. "[The veterans service organization] representative crossed off the areas of the form I didn't have to complete. Had he not done this, the form would have been very difficult." "I don't know what I would do without our state service officer to help me. She handles all my VA material and questions." "I didn't fill out these forms. The manager from the shelter care home either did them or had them done." "[The] assistant at school already knew how to fill out the forms." "[The] funeral director filled out all the information. I just answered his questions and signed the papers." Those who found someone other than VA to be most helpful to them were less likely to be satisfied with VA's service overall. On the other hand, those who found VA most helpful generally were more satisfied with various aspects of the process. (See fig. 10.) Figure 10: Applicants' Satisfaction Level, by Where They Obtained Help (See figure in printed edition.) Note: "Other" includes, for example, a friend or relative, funeral home or cemetery staff, and school officials. The reasons satisfaction with VA's service was lower for those who received the most help from non-VA sources are not clear. Were these applicants dissatisfied with VA before they contacted the other sources? Were there problems with the services these other sources provided or with the expectations they gave applicants about VA's service? Did applicants seek help outside VA for the more complex, difficult to support claims that are more likely to take longer and raise frustration levels? The answers to these questions would lead to different actions to improve service and satisfaction. The extent to which these sources are involved highlights the need for VA to fully understand who gets what help from them and why. VA recognizes the important role these organizations play, but some VA officials expressed surprise at the extent of involvement we found. Officials said they make aggressive outreach efforts to these other sources. For example, representatives from some of these sources were part of a panel charged with recommending improvements in disability claims processing. Additionally, VA sponsors conferences attended by state and county officers; however, with tighter budgets some local officers have not been able to attend these meetings. VA officials indicated that the quality of the service provided by these other sources can vary and that sometimes VA does not have enough informational materials to send other sources. CONCLUSIONS ---------------------------------------------------------------------- Letter :8 Our survey pointed to several key problems with the service VA regional offices provide to veterans and their families. VA has recognized those key problems--its own customer survey highlighted similar issues--and is attempting to find ways to address them. But some of these concerns are long-standing and only time will tell if VA's efforts will be successful. Of major concern is that even if VA meets its new timeliness goals, many veterans will not be satisfied. While it may not be advisable to set current goals based solely on customer expectations of what is reasonable--if those expectations are not attainable in the foreseeable future--VA's long-term efforts should be directed at meeting customer expectations. These efforts should include determining what changes will be needed to meet customer expectations. In the short term, our survey suggests that VA may be able to improve satisfaction by better communicating realistic time estimates to applicants. This and other efforts to improve communication could have a major impact on customers' perceptions of VA's service. The substantial extent to which claims are denied and to which service is provided by non-VA sources was not previously known. Though it may not be surprising that applicants whose claims were denied are less satisfied with VA's service, the fact that these applicants constitute an estimated one-third of VA's customers suggests more detailed information about them could be useful in improving service. Similarly, the extent to which other sources provide assistance to applicants strongly suggests these sources can have a substantial impact on customers' satisfaction with the claims process and perceptions about VA. As VA takes steps to change its claims processing system, it will be important for VA to continue communicating and coordinating with these sources, to know what assistance these sources are giving, and to ensure that these sources have the up-to-date information necessary to assist applicants. RECOMMENDATIONS ---------------------------------------------------------------------- Letter :9 The Secretary of Veterans Affairs should direct the Under Secretary for Benefits to set long-term goals to meet customer expectations for processing times, and prepare a plan describing the incremental steps necessary to meet them, provide applicants realistic estimates about how long it will take to complete their claims, and develop data on applicants whose claims are denied so VA can ensure that segment of its customers gets the best service possible. AGENCY COMMENTS --------------------------------------------------------------------- Letter :10 In a letter dated August 3, 1994, the Secretary of Veterans Affairs commented on a draft of this report. (See app. IV.) He stated that VA agreed with each of our recommendations and outlined some actions VA is considering to implement them. Absent more details and time frames, however, we cannot assess the potential of the proposed actions to improve customer satisfaction. The Secretary also pointed out that our report data on the failure of customers to obtain needed information from federal VA employees may be subject to misinterpretation. He stated that his department's customers do not always know when they are contacting a federal VA office or a state or county VA office. Consequently, he believes the failure to provide information to a customer may not have been the federal VA's fault, but rather one of the state and county offices. We agree that some respondents to our questionnaire may not have differentiated between federal and state or county VA offices. VA's comment is particularly pertinent because it confirms the significance of the concern we raised in this report about the extent to which sources other than federal VA offices are involved in providing services and their impact on quality of services and customer satisfaction. Finally, the Secretary expressed concern that our sampling technique overlooked claims that were closed quickly. Our sample was drawn from claims completed between April 1, 1993, and July 13, 1993. VA believes this sampling technique overlooked claims filed after April 1 and closed before July 13. VA believes these applicants would be among those most satisfied with VA's service because their claims were completed so quickly. However, because our sample was representative of claims completed during this period and VA told us this period was typical of other periods, our universe included claims that were completed in a short time. For example, if a claim had been filed on March 30 and closed April 2, it would have been included in our universe and could have been selected in our sample for review. ------------------------------------------------------------------- Letter :10.1 We are sending copies of this report to interested congressional committees, the Secretary of Veterans Affairs, and other interested parties. This work was done under the direction of Flora Milans, Associate Director. If you have any questions about this report, please contact me at (202) 512-7101. Other major contributors are listed in appendix V. Sincerely yours, David P. Baine Director, Federal Health Care Delivery Issues SCOPE AND METHODOLOGY ===================================================================== Appendix I To obtain the views of the Department of Veterans Affairs' customers--veterans, their dependents, and survivors--we surveyed a national random sample of 1,400 applicants. These applicants' claims were representative of the approximately 245,000 claims completed at VA's regional offices between April 1 and July 13, 1993, for 9 types of benefit claims. We included claims for the following: Initial Disability Compensation Benefits. These benefits are paid to veterans who are disabled by injury or disease incurred or aggravated during active military service in the line of duty. Disability and Death Dependency Benefits. These include among others, increased benefits for children attending school after the age of 18, increases for additional dependents, or adjustments in benefits because of marriage, divorce, or death. These actions can be initiated by VA or by the claimant. Initial Dependency and Indemnity Compensation Benefits. These benefits are awarded to the surviving spouses, certain dependent children, and in some cases the parents of veterans who died on active duty or from a disability that was service connected. Surviving spouses and children may also receive dependency and indemnity compensation benefits when totally disabled veterans die from non-service-connected causes. Burial Benefits. These benefits include statutory burial allowances, plot allowances, and in some cases, reimbursement for expenses for headstones, markers, or engraving expenses. Initial Disability Pension Benefits. These needs-based benefits are provided to veterans with limited income who served during wartime and have become totally and permanently disabled from causes not connected to their military service. Initial Death Pension Benefits. These needs-based benefits are provided to the surviving spouses and children of wartime veterans who died from causes not connected to military service. Education Benefits. These benefits are provided to active duty and selected reserve personnel, veterans, and their eligible dependents. Vocational Rehabilitation Benefits. These benefits are provided to eligible disabled veterans who have demonstrated that service-connected disabilities make rehabilitation training necessary to overcome impairments in finding or retaining employment. (Cases in our sample included two types of claims: (1) those filed to determine eligibility for vocational rehabilitation benefits and (2) We also included Reopened Claims for Compensation and Pension Benefits. These include, among others, claims for benefits VA previously denied, increased disability compensation benefits, additional disabilities, and benefits granted through a successful appeal to the Board of Veterans Appeals. We did not include as part of our customer survey two other major nonmedical benefit programs offered by VA. The home loan guaranty program was not selected because the national database of those who take advantage of this benefit does not contain sufficient information on each loan to allow us to draw a valid national sample. Veterans life insurance programs were not selected because claims processing is limited to only two regional offices: Philadelphia, Pennsylvania, and St. Paul, Minnesota. To determine our universe, we compared VA's work-in-progress database\9 on April 1, 1993, with the same database on July 13, 1993. Claims that were in process in April that did not appear in July were considered completed claims, and we drew our sample from these claims. VA told us that this period was representative of VA's workload throughout the year. We sent questionnaires to random samples of applicants for each of the nine types of benefit claims listed previously.\10 The survey was administered anonymously, and individual responses could not be linked to the actual respondent. The questions we asked were based, in part, on discussions with VA officials, veterans service organizations, state officials, and veterans and their families. The final questionnaire was sent to VA for comments. We tailored the questionnaires only to enable us to ask the same questions to all nine groups (for example, applicants for compensation benefits or burial benefits). A sample questionnaire showing aggregate responses and percentages is included in appendix II. Table I.1 shows the universe of potential claims, the sample size, and the number of questionnaires received by strata. Table I.1 Universe of Potential Claims, Sample Size, and Questionnaires Received by Strata Type of benefit claim ----------------- ---- ----------------- ----------------- ----------------- Initial 37,5 200 141 134 disability 25 compensation Disability and 22,6 200 144 100 death 54 dependency Initial 5,07 100 88 81 dependency and 4 indemnity compensation Burial 16,7 100 71 58 12 Initial 9,96 200 145 132 disability 7 pension Initial death 6,88 100 78 71 pension 7 Education 29,4 200 135 124 36 Vocational 38,6 100 68 45 rehabilitation 08 eligibility and rehabilitation benefits Reopened 77,9 200 155 146 compensation and 77 pension Total 244, 1,400 1,025 891 840 -------------------------------------------------------------------------------- \a Universe: Numbers reflect total number of claims completed by VA during the period from April 1 through July 13, 1993, for the nine types of benefit claims included in our review. \b Respondents: "Total" represents the number of questionnaires returned that were not blank or did not indicate the applicant had died. "Recall" is the number of questionnaires returned on which the applicant recalled having had contact with VA concerning the claim in our sample. This represents the number of usable responses. We obtained an overall response rate across all nine strata of 73 percent. Response rates for individual strata varied from a low of 67.5 percent for education claims to a high of 88 percent for initial dependency and indemnity compensation claims. Because these samples are representative, the statistics we cite based on the survey are estimates for the population of all veterans, their spouses, and their dependents who had claims completed by VA from April 1 through July 13, 1993. We calculated sampling errors for estimates from this survey at the 95-percent confidence level. This means that the chances are about 19 out of 20 that the actual percentage being estimated falls within the range defined by our estimate, plus or minus the sampling error. Generally, the sampling errors did not exceed 5 percentage points when addressing all strata combined. To determine if the nonrespondents were significantly different from those who responded to our questionnaire, we obtained information about those who did not respond. First, we developed a geographic profile and found that the geographic distribution of the nonrespondents was similar to the distribution of the respondents. Second, because claim status (for example, approved or denied) was strongly correlated with satisfaction, we determined the disposition of the nonrespondents' claims. We found that the percentage of denied claims was similar to the percentage of denied claims for those who responded. Our limited analysis found nothing that would indicate that their responses would be significantly different from those who returned our questionnaire. Finally, about 10 percent of those who responded to our survey indicated that their claims were still pending, even though VA's data indicated that the claims had been completed. We included their responses because these applicants had been able to complete the questionnaire and had opinions about VA's service. Since our questionnaire was administered anonymously, we could not follow up on any individual cases. However, we believe some of those who told us that their cases were pending had in fact appealed VA's decision on their original claim. Other reasons for this response could be that the VA data on the disposition were inaccurate, the claimant never received a letter on the disposition of the claim, the disposition letter was still in process, the claimant had received conditional or partial approval of the claim, or the claimant had more than one claim in process. (See figure in printed edition.)Appendix II -------------------- \9 VA uses the work-in-progress subsystem to track all claims in process at any given time. In a recent report on claims processing, the VA Inspector General commented that the subsystem's data are generally reliable enough to be useful in controlling and monitoring work flow. Although the Inspector General's report found about one-third of the records it reviewed were inaccurate in some way, these inaccuracies did not affect the integrity of our sample. \10 If the same applicant was selected in more than one strata (that is, the claimant had more than one type of claim in process), we sought information on only one type of claim. SUMMARY OF RESPONSES TO GAO'S SURVEY OF APPLICANTS FOR VA BENEFITS ===================================================================== Appendix I (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) ESTIMATES OF APPLICANTS' VIEWS BY CLAIM TYPE =================================================================== Appendix III The tables and figures in this appendix provide information by individual types of claims included in our survey for the major issues discussed in this report. In many cases the differences between claim types are not statistically significant.\11 However, some clear patterns emerge. Most notably, applicants for compensation-related benefits were more likely to be dissatisfied and had more difficulties with the claims process. For example, applicants for initial disability compensation or those who had reopened compensation and pension claims were the most dissatisfied with VA timeliness, fairness, and overall handling of the claim (tables III.1, III.2, and III.3); had the largest difference between their expectations for timeliness and the goals VA set (table III.4); and were less likely to obtain the information they needed from VA through letters, telephone calls, and personal visits (tables III.8, III.9, and III.10). Conversely, applicants for burial and education benefits were generally more satisfied and had fewer problems with the process. Our survey was not intended to provide a full explanation for why applicants for some benefits are more satisfied than others. However, disability compensation claims tend to be more complicated and take longer to resolve than other types of claims and, therefore, may be more likely to cause frustration and dissatisfaction. In contrast, education and burial claims are relatively straightforward, take less time to resolve, and may therefore be less likely to cause frustration. Furthermore, applicants for burial and education benefits were the least likely to be denied and, as discussed in the report, being denied is highly correlated with dissatisfaction. This appendix also provides information by individual types of claims on whether applicants received estimated processing times, required letters, and informational materials from VA (tables III.5, III.6, and III.7). Other tables cover the resubmission of civilian records (table III.11); claims fully allowed, partially allowed, and denied (table III.12); how applicants first learned about VA benefits (table III.13); and sources applicants found to be the most helpful in filing their claims (table III.14). Figure III.1 highlights what applicants said they would most like VA to improve. Table III.1 Applicants' Opinions About Time VA Took to Process Claims (Question 58) (Percentages) About as reasonable as Type of claim Reasonable unreasonable Unreasonable -------------------- ---------- ------------ ------------ Initial disability 41 14 45 compensation Disability and death 44 11 44 dependency Initial dependency 51 19 30 and indemnity compensation Burial 64\a 9 26 Initial disability 51 8 41 pension Initial death 50 17 33 pension Education 57 13 30 Reopened 39 14 47 compensation and pension All claims 47 12 41 ------------------------------------------------------------ Note: Percentages may not add to 100 due to rounding. \a Sampling error exceeds plus or minus 12 percentage points. See figure III.2. Table III.2 Applicants' Opinions About Fairness of VA's Decisions (Question 59) -------------------- \11 As discussed in appendix I, the sampling errors when discussing all types of claims generally did not exceed plus or minus 5 percentage points. However, sampling errors for individual types of claims differed. (See fig. III.2). Data for vocational rehabilitation claims are not shown in the tables and figures in this appendix because generally there were too few usable responses to allow meaningful analysis.