UNITED STATES OF AMERICA NATIONAL TRANSPORTATION SAFETY BOARD WASHINGTON, D.C. ********************************************************** IN THE MATTER OF THE INVESTIGATION OF * AMERICAN AIRLINES, INC., FLIGHT 1420, * Docket Number McDONNELL DOUGLAS MD-82, N215AA * SA-519 LITTLE ROCK, ARKANSAS, JUNE 1, 1999 * *********************************************************** Arkansas Excelsior Hotel Bill Clinton Ballroom Three Statehouse Plaza Little Rock, Arkansas 72201 Thursday, January 27, 2000 8:30 a.m. Board of Inquiry HONORABLE JIM HALL, Chairman Board of Inquiry THOMAS HAUETER, Deputy Director Office of Aviation Safety JOHN CLARK, Deputy Director Office of Research and Engineering BARRY SWEEDLER, Director Office of Safety Recommendations and Accomplishments BEN BERMAN, Hearing Officer Office of Aviation Safety Technical Panel GREGORY SALOTTOLO GREGORY FEITH EVAN BYRNE MARK GEORGE CHARLES PEREIRA LAWRENCE ROMAN DAVID TEW DONALD EICK Public Information Officer PAUL SCHLAMM Office of Public Affairs Washington, D.C. Parties to the Hearing LYLE STREETER, Air Safety Investigator Accident Investigation Division, AAI-100 Federal Aviation Administration RONALD J. HINDERBERGER, Director Airplane Safety Boeing Commercial Airplane Group ROBERT W. BAKER, Vice Chairman American Airlines, Inc. CAPTAIN CHRIS D. ZWINGLE Special Assistant to Chairman National Safety and Training Committee Allied Pilots Association KATHY LORD-JONES National Safety Coordinator Association of Professional Flight Attendants ROBERT KUESSNER National Weather Service DEBORAH H. SCHWARTZ, A.A.E. Airport Manager Little Rock National Airport J.T. CANTRELL, Training Chief Little Rock Fire Department I N D E X WITNESS: PAGE: Operations David C. Gilliom, Manager 374 Flight Standards Division - Western Pacific Region Interview by David Tew Robert Baker, Vice Chairman 443 American Airlines Interview by Evan Byrne/Gregory Feith Meteorology George Wilken, Science and Operations Officer 542 National Weather Service North Little Rock Forecast Office Interview by Donald Eick Dr. Wes Wilson 516 MIT Lincoln Labs Interview by Donald Eick Dale A. Rhoda 677 MIT Lincoln Labs Interview by Donald Eick Rainer Dombrowsky, Chief 729 Surface Observation Branch National Weather Service, Headquarters P R O C E E D I N G S 8:30 a.m. CHAIRMAN HALL: We will reconvene this public hearing of the National Transportation Safety Board. This hearing is being held in connection with the Accident Investigation involving American Airlines Flight 1420, a McDonnell Douglas MD-82, Registration N215AA, that had an accident here in Little Rock on June 1, 1999. I would ask our hearing officer, Mr. Berman, if he would call the next witness. MR. BERMAN: I call Mr. David C. Gilliom. Whereupon, DAVID C. GILLIOM having been first duly affirmed, was called as a witness herein and was examined and testified as follows: INTERVIEW BY BOARD OF INQUIRY BY MR. BERMAN: Q Good morning, sir. Would you please state your full name and business address for the record? I'm sorry. Flip the switch on your mike. MR. BERMAN: Stand by one second. CHAIRMAN HALL: We've got assistance coming here. MR. BERMAN: Go ahead. THE WITNESS: Thanks. I'm David C. Gilliom. I work for the Federal Aviation Administration, and my business address is the Western Pacific Regional Office in Los Angeles, California. MR. BERMAN: Thank you. BY MR. BERMAN: Q And what is your -- your present position at the Federal Aviation Administration? A I'm the Manager of the Flight Standards Division for the Western Pacific Region. Q How long have you held that position? A For eight years. Q Would you please describe your duties and responsibilities as Manager of Flight Standards Division? A Yes. I would first distinguish -- excuse me -- that the field divisions are responsible for the operational aspects of our programs, and I bring that out because it's obviously distinct from what's done in Washington, which is the policy and rulemaking piece of it. So, as an implementing body, I oversee the functions of 17 field offices, and I ensure that those offices comply with internal guidance, and that the resources that are distributed in my region are used in accordance with agency priorities. I'd also point out that I chair a national committee. So, this is slightly different from the implementing role, and that committee has provided sponsorship of the ATOS System. Q Thank you. And could you please summarize your education, training and experience that you used to qualify for your position? A I've been in aviation for 32 years, with the FAA for 22 years. I have a military background, a concurrent career, with the Air Force, 25 years on active duty and as a reservist, hold an airline transport pilot certificate with 727-737 ratings, although I haven't been an active pilot for eight years. I also have a Master's degree in Business Administration. Q Thank you very much. MR. BERMAN: Go ahead, Captain Tew. INTERVIEW BY TECHNICAL PANEL BY MR. TEW: Q Good morning, Mr. Gilliom. A Good morning. Q Could you tell us what your role was in the development of ATOS? A Yes. I have participated in the development of ATOS since its inception, primarily in the oversight of the activities that occurred to develop the nuts and bolts of the system. I was involved with the 90-day safety review in 1996. Some of the recommendations that came from that review led to our internal look that eventually led to the decision that we needed to move forward from where we were in relation to our regulatory oversight systems. MR. TEW: Okay. For the audience's benefit, I should point out, we did yesterday, we'll do it again today, ATOS is the FAA's Air Transportation Oversight System. It's the new oversight system that is in place, and that's what we're discussing when we say ATOS. CHAIRMAN HALL: And it might help the audience if we had an explanation of what precipitated the 90-day review in 1996, you know, and your look at the system. THE WITNESS: The 90-day safety review was directed by at that time Administrator Hensen following the ValuJet accident, in which Flight Standards was under certainly some scrutiny regarding its current oversight system, and as a result of that review, there -- which was quite brief. It was advertised as a 90-day review. Actually, the primary work took place in about 45 days. So, it was very compressed and obviously not enough time to do a thorough system development. But as a result of that review, there were several recommendations that involved changes to the oversight of Flight Standards, and also changes within our own internal systems. It was an opportunity to look inward, to make sure that our own safety culture was an adequate one to support the activities that we're responsible for. Many of those recommendations were further developed as we worked with Sandia National Laboratory subsequent to the review to develop the Air Transportation Oversight System. We chose Sandia as a consultant because of their eminent expertise in the systems safety arena. CHAIRMAN HALL: Not because of the weather? THE WITNESS: Well, obviously not because of the weather. That's correct. BY MR. TEW: Q Could you tell us how much ATOS has cost to develop? A How much it has cost? Q Yes. A Oh, I don't have precise figures in terms of developing a system of its magnitude. The actual nuts and bolts of it have not been that expensive in terms of -- of buying outside consultancy help and facilitation help and documentation help, the kinds of things that the agency's not prepared to do internally. Much of the manpower to develop it came from our own resources. In fact, the subject matter experts that we used were, for the most part, line inspectors. We wanted to use their expertise in developing the new system. So, I can give you a rough estimate. I would guess from 1996 to '98, which is the initial development period, we spent roughly $2 million on development costs outside of our own internal investment, and again I'm talking about the costs of salaries for the people who were involved with the development and travel. But money that was spent on contractors, in the neighborhood of $2 million, and we still have expenditures on-going because there's extensive development continuing with ATOS as -- as we further develop the sophistication of the system. Q Was the ATOS System intended to replace the old PTRS System? A It definitely is intended to replace that system for Part 121 air transportation air carriers. It's not implemented to that extent. I'm not sure if it's been made clear that the system has only been implemented for 10 existing carriers. There's still a 140 121 air carriers that are operating under the National Program Guidelines and PTRS. Q At these 10 carriers, did the PTRS System stop when ATOS was implemented or is still being run in parallel? A It stopped to the extent that we use PTRS to record surveillance activities. There are still certificate management functions that occur that are reported through the PTRS activity, and I would say that there has been some transitional problems in terms of inspectors getting used to the new system. So, some functions that are borderline between certificate management and surveillance, and I would mention specifically initial operating experience, in some cases, that's still been entered in the PTRS Recordkeeping System. CHAIRMAN HALL: We might again observe for the individuals that are observing and keep them informed, this system was evaluated initially by the Board as a result of our accident that occurred in Miami with FineAir. It was the Board's recommendation at that time that the system be extended from the major carriers to all the Part 121 fleet. THE WITNESS: Yes, sir, and we have developed a plan to do that. At this point, we're somewhat reluctant to move forward until we've finished complete development of some of the process modules in the system and also have the resources available to move forward. CHAIRMAN HALL: Might finish staffing the ATOS for American first, and then we'll move forward. THE WITNESS: Well, that's certainly an important issue, and staffing is -- is one of the things that this system, once it's developed fully, will help us be very clear about what the needs are, and in fact, I think much of that is in place for American today. There's a new Comprehensive Surveillance Plan, as was mentioned yesterday, that was developed just last week for American. That plan is very quantifiable and should identify resource shortages, particularly in the geographic inspector arena, and there are policies and procedures to identify those needs. CHAIRMAN HALL: Dave, I don't want to be interrupting, but I would ask, what was wrong with the old system? Why do we need a new system? Just simply put, what were the deficiencies that you found, and what are the things you're trying to give attention to with this new system? THE WITNESS: Well, quite simply, the old system, which was very good and certainly established an excellent record of safety for the industry and FAA, had reached a plateau. There was really no way to make further safety gains with it. It was a system that was based on professional expertise of individual inspectors. It really wasn't a system so much. It was designed to inspect in safety. So, we attempted to do that by observing as many activities and surveillances we had resources to do. Actually, the data that was produced in terms of taking actions on that data, the predominant actions we took were enforcements, and I believe it was mentioned yesterday that that approach has limited effectiveness. The change to the ATOS model is an effort to move away from a system that's totally reliant on numbers of inspectors to again attempt to inspect in quality to a system that is focused on data and risk management and uses system safety principles not only to develop the FAA's system but to look at the air carriers' operating systems to ensure that those systems also are built around good system safety concepts. CHAIRMAN HALL: Wasn't it correct, David, that as a result of ValuJet, maybe there was a feeling that there weren't enough resources for -- that the major carriers basically had their safety act together, and that there weren't enough resources being expended on some of the new entrants and some of the other carriers? THE WITNESS: That -- that is a fact. I believe the FAA's situation is not different from other agencies. We probably will never have all the resources that we would certainly like to have and -- and could use. Our challenge is to build systems that leverage those resources and allow us to do our job with -- CHAIRMAN HALL: What has actually happened to the number of inspectors over the last 10 years, to your knowledge? THE WITNESS: Following the 90-day safety review in the ValuJet incident, we had an opportunity to substantially increase inspector resources. Due to budget contingencies last year, we were not able to do that, and this year, I believe we will be, at least at present, able to replace inspectors that we lose. But we're not in a growth mode, certainly. BY MR. TEW: Q You just mentioned that you just developed -- not that you developed, that a Comprehensive Surveillance Plan was developed for American, I believe? A Yes. Actually for all 10 of the carriers under ATOS. Q Okay. During testimony yesterday, I believe that it came out that there was not an analyst working at American, I believe. A That's correct. Q And therefore the data was not being analyzed, so it couldn't be used in a development of this. How -- the POI of the American unit said that he developed this comprehensive plan based on his experience, I believe. How can you develop an effective comprehensive plan for ATOS without the aid of an analyst? A The experience, I believe, that was referenced yesterday would be in the context of experience that came from ATOS data. The fact is that these comprehensive surveillance plans are developed using very detailed tools. There's a system safety analysis tool and an air carrier analysis tool that involve extensive processes to analyze the conditions that -- and the environment that face a particular carrier, in this case American, and from that, the plan is developed. So, it's not done on the basis of subjective expert opinion as somewhat the old plan is under the National Program Guidelines. This is completely different. It's very systematic. It's very detailed. It doesn't have to do with the analyst function. I do have one illustration that actually might be helpful in pointing out how the system works and where analysis comes into play. It might help understand while it's important to have analysts in these offices, we can still do what we're doing, which is an improved surveillance process, without the use of the analysts. CHAIRMAN HALL: Well, now you realize if you tell Congress that, they'll cut all your analysts? THE WITNESS: Well, I wouldn't attempt to say that this system is valid as a system without analysts. We do need the analysts. What is important to -- to point out is -- CHAIRMAN HALL: I mean I just -- I'm just asking because I mean either they're part of the system or they aren't part of the system. As a taxpayer, I'm not interested in funding accessories. THE WITNESS: They're a hundred percent part of the system, and what was said yesterday about the ability to analyze the data that comes from this system is limited without them. I would also point out, though, that -- that among the activities that is on-going to -- to enhance the system is further development in the process module that deal with the analysts. So, we're working on the procedures that the analysts will use right now. CHAIRMAN HALL: Have you got a diagram you want to put up? THE WITNESS: Yes, I think it would be helpful. CHAIRMAN HALL: Victor? THE WITNESS: This diagram shows the system that Sandia Labs helped us design, and again we used our own line inspectors basically to help develop this model. It has eight processes, and they're represented by the boxes, thank you, and if -- if you -- CHAIRMAN HALL: Can people in the back see the writing? Make it out? THE WITNESS: Well, -- CHAIRMAN HALL: So, you might read it. THE WITNESS: -- I'll read it. You know, just beginning with this first box, it says, "System Configuration". That, we would call a process module. There are eight of them around the circle, obviously linked in a closed loop system. So, the first one, System Configuration, deals with the initial certification of operators. It's our certification activity, and it also deals with how our office or how the FAA's internal structure is prepared to deal with a new operator. CHAIRMAN HALL: For the benefit of the audience, and I think people have grasped this, but basically -- and put the chart back up, Victor. Basically, what you're talking about is how we spend our tax dollars through our Government to oversee the aviation safety system, is that correct? THE WITNESS: That's correct. When -- CHAIRMAN HALL: And this is a -- you know, the latest of how you all have put together an effective way to do that. THE WITNESS: That's correct. In fact, I would advocate it as such a major step forward, that it truly is the leading edge of a regulatory system. So, I believe it puts us in a posture again to be making a world standard or setting a world standard. But going quickly around the system as it's designed and represented by this model, after initial certification, the next process is certificate management. It's this process that develops the Comprehensive Surveillance Plan. CHAIRMAN HALL: And every major carrier, every Part 121, has a certificate, right? Is that -- THE WITNESS: That's correct. CHAIRMAN HALL: That gives them the right to -- THE WITNESS: That's correct. CHAIRMAN HALL: -- use the skyways that belong to all of us? THE WITNESS: I believe it will be clear once I have described briefly this entire model, that its application to new entrant carriers, and again when -- when you think about how this came about after ValuJet, that's one of the major things we wanted to deal with, improved quality of our certification services, that it will be clear this model really is -- can be very effective, will be very effective for new entrants and the remaining carriers that are not under it today. In the second module, as I said, the Comprehensive Surveillance Plan is -- is developed. The third module deals with the resources required to carry out that surveillance. So, this is where we're ensuring we have enough people, that they're properly trained, that there's enough money to go out and do the actual work. The fourth module is surveillance implementation, self- explanatory. It's where we carry out the Comprehensive Surveillance Plan. The fifth module deals with reporting. This is the module where we have moved away from PTRS, which is basically a work program tracking system. That's the old model. This one is -- is in truth much improved from a PTRS activity. It allows us to enter much richer data fields. I should point out that in surveillance implementation, one of the criticisms of the Board and others with which we agreed was that our old system did not require the extensive use of checklists and job aids to complete an activity. We could call an activity complete no matter how rudimentary our surveillance work had been for that activity. Under the current system, there are extensive job aids which must be completed before either an element performance inspection or a safety attribute inspection can be called complete. The data is reported and entered in the fifth module. In the sixth module, evaluation. We do something with the data we've never done before, and that is we evaluate it to make sure that it is complete and accurate, and that it meets the standards that we're looking for with this type of data. The seventh module, after the data's been validated, it's then analyzed to determine what it's showing us. From the analysis, we are looking for root causes, trends, other things that would be of use to bring about changes or to confirm that in fact the carrier's continuing to operate in accordance with its design systems, and that those systems are well-designed. And from the analysis, then clearly there could and will be actions that -- that come forth, even if that action is -- is simply to document that we have confirmed the carrier's doing what it said it would do and doing it well. It is a closed loop system. So, from what we've learned from our data collection and analysis, if there are changes that are required, we go right back into Module 1 to change operations specifications or whatever else might be required to bring about the necessary changes. CHAIRMAN HALL: And as it pertains to just American Airlines, the loop right now has problems that Number 3, Number 4 and Number 7? THE WITNESS: Number 3 and -- Number 3 is certainly a problem in that that deals with staffing issues. CHAIRMAN HALL: Hm-hmm. THE WITNESS: However, there are clearly-developed policies and procedures to identify those staffing needs, and I would expect, now that there is the first -- actually the second surveillance plan in place, that as resource needs are identified, that they will be elevated. Last year, I can say in my own region, Western Pacific Region, when the American Certificate Management Unit requested new inspector resources, we were able to provide those in Western Pacific. I can't speak for other regions. So, there is a process in place to deal with the needs, and again if that were to happen this year, I have no doubt that I would be able to meet the requirements of any ATOS carrier in terms of surveillance needs. CHAIRMAN HALL: I'm just asking as an observation today, do we still have -- we still have some challenges -- THE WITNESS: Yes, sir. CHAIRMAN HALL: -- in 3 and 4 and 7? THE WITNESS: I would say there are challenges in those modules. I would also point out -- CHAIRMAN HALL: I'm in the position, too. I sometimes don't have enough resources at the Board, and I don't like to ever admit I don't have enough resources in terms of being able to do the job. I want to be able to say yes, we're able to do the job. But the truth of the matter on that situation is you got 3, 4 and 7 for you to really have your loop closed like you want it closed, and you've designed it, need -- need attention. THE WITNESS: That's correct. I would also point out that the predominant work that has been done on this system to date deals with surveillance. We are working currently on the certification activity. I'm sure the Board is aware that we have a new Certification Standardization Evaluation Team, CSET. CSET is part of this model. Its function is primarily in that first system configuration module. They are using procedures in a much more standardized way than we have used them in the past, but they still predominantly are our old procedures. So, we have extensive activity. It will take time to finish on-going in that module now to convert or re-engineer our certification activity to comply with the system safety concepts. CHAIRMAN HALL: Good. BY MR. TEW: Q Just for clarification, the Chairman mentioned that there were deficiencies in three of the modules. There's eight modules here in this closed loop. Could you tell us how many have been fully developed of the eight? A The modules that have had the most work are all of those dealing with surveillance. So, it would be 2, 3, 4 and 5. Now, again, the problem that we're having with Module 3 is not so much that there aren't processes that have been developed; it's the ability to have full resources to carry that work out. Again, it gives us tools, I think, to better use the resources that we do have, but as the Chairman pointed out, and as I said earlier, government agencies will always be more in need of resources than are generally available. Q So, we really haven't achieved this closed loop yet then, I believe? A We have not. I think it was a valid point, for example, that there's not an analyst in every office. I mentioned that that's one of the modules that we've done rudimentary work on. We need to do much more in that module. When that work is completed, there will be -- the analysts would be better used, not to say that they couldn't function in the system as it's designed today, but that's clearly an area where there's still need. Q How do you develop an analysis module in this work? How do you develop this without the help of the analysts? Wouldn't they be an aid? A We -- well, there are two ways to provide that expertise. We do have a function in the Headquarters organization that is staffed with operations research analysts. I believe there are three or four analysts on board in that Headquarters staff. We have the one analyst hired in Southwest that gives us a core group of analysts as subject matter experts to use internally. Also, we are using contractors to help with this activity. We're continuing to use Sandia to help with this activity. All of those outside sources can provide expertise and in some cases an external input which is very important to have. So, that's where the expertise would come from. Q So, you're using staff analysts and then outside help? A That's correct. Q Is -- is -- is this effective in analyzing this data then? Because this is -- A Well, when -- when you asked me the question about developing the analyst module, we're -- when it comes to analyzing the data, which is an operational concern, we're still somewhat deficient in that area in that again we only have the one analyst on site at Southwest. This group in Washington has a limited capability to help with analysis in CMTs. However, I think as -- as Corky Valentine testified yesterday, although there isn't an analyst present at American Airlines, clearly the data which he said was much improved from the PTRS data is being looked at by Corky and others. It is being analyzed, certainly to the extent that we analyze PTRS data in the old system. So, there is a rudimentary activity going on in that process module. There are actions coming out of that analysis. There are changes to comprehensive surveillance plans in terms of targeting that are occurring. So, again it's a rudimentary function. We need to take it much further, but -- but it is occurring, and -- and there is a difference between the development activity and the operational activity. CHAIRMAN HALL: Well, David, as you know, they said -- they had told us in correspondence that this thing would be up and running. Has the budget cuts -- if you hadn't had the budget cuts, and you'd had the staffing, would this thing be ready to take off the shelf now? THE WITNESS: Yes, but I don't want to under-play the fact that -- two facts, actually. That there's extensive work to be done with initial certification. We need to develop better analytic procedures. We need to develop better implementation procedures, and this model is designed for continuous improvement. One of our past mistakes is to fix -- CHAIRMAN HALL: That's life, though, right? THE WITNESS: It is life, but -- but we, unfortunately, didn't learn the lesson as well as we might have in that we have always tried to fix it once, once fixed, keep it in place. That's not the intent here. We realize that we need to continuously revise this system through a continuous improvement process, and that will be on-going throughout the life of the system. CHAIRMAN HALL: Good. THE WITNESS: And I think we're in a much better posture to do that than we were with the old system, which was quite linear. It was not a closed loop system. CHAIRMAN HALL: But the reality of the old loop system was that we used to have days that we -- we tried to -- we had enough inspectors, and the Government -- the way the Government approached inspecting has changed, and a lot of that's changed because of the realities of Government being cut as well as all the modern philosophy about things, in my personal -- now, that's my personal opinion. As we sit here trying to do the job of regulating an industry that has grown in about 10 years with essentially the same resources, you're put in the situation of trying to look at different ways of accomplishing the same goal with the same manpower. THE WITNESS: That's largely true, but I would point out that the value of this system as it matures is that it does allow us to leverage resources through better targeting, and -- CHAIRMAN HALL: I mean the Board's on record. We -- we think it's a good system. We just want to try and get it -- the loop closed. BY MR. TEW: Q You said this system was -- it was put in place. It wasn't finished, and it was designed that -- to be improved as it goes along. But, you know, four of the eight modules, I believe, were not developed to start with, and none of the analysts were in place. Do you feel possibly that it might have been put into place just a little bit premature as a replacement for the PTRS System? A Well, again, all the modules were developed to an extent. Four of them were worked on more extensively than others, and the remaining four had rudimentary improvements. Even the initial certification module, which -- which we're re-engineering now, was substantially improved when we put the CSET team in place over what we had before. So, all the modules were developed to an extent. We did have a complete system. We did make a conscious decision to implement that system, knowing that there was further work needed in some of the modules, simply for the reason that after working on it for two years, and government things tend to become either another study that gets put on a shelf and never implemented or you move forward, we were very convinced that what we had done in the surveillance arena was a substantial improvement over what we had before, and I think again that came out in testimony yesterday, that the data coming from the system is much richer than the data that we were getting from the old system. Q How were the airlines and industry representatives involved in the development of ATOS? A The airlines and industry were -- were not involved to any great extent in the initial development of the system. We realize that industry input is important. We have sought it. Certainly once the system was initially implemented, we've had meetings with ATA to discuss the system, to make sure it's understood, to get some feedback about its shortcomings from that perspective. CHAIRMAN HALL: You might explain, David, who ATA is. THE WITNESS: The Air Transport Association, which represents most of the major carriers in the 121 arena. So, we are getting that input now, and we certainly will get the input as we develop the initial certification activity. BY MR. TEW: Q You just mentioned a minute ago that you -- this is -- this program is the leading edge of the -- of a regulatory system -- new regulatory system. I understand that a significant number of the questions in the inspection reports are not based on the FARs or any regulatory basis. How can you collect data and analyze data that is questionable or -- and non-regulatory and expect to achieve compliance? A I don't believe the data is questionable. I believe the data are legitimate areas for FAA to -- to be concerned. The Board and others have pointed out that simply looking at the regulatory requirements for compliance falls short of what's needed to take the next step in safety in terms of looking at corporate cultures, safety attributes, a number of concepts that are standard in the safety business, whether it's nuclear safety or safety in airlines. These are all legitimate areas for the FAA to collect data in order to target its resources. When it comes to insisting that certain changes be made, clearly we're limited to what we can enforce to what is written in our regulations. However, that doesn't limit what we can look at and what we can bring to the carrier's attention as a concern. One of the things that's very important about this system, and I believe it came out to some extent yesterday, is that one of the action items that can occur is a system analysis team or a SAT that did occur with the American unit, and it would correct some perception that I believe existed yesterday in that regard. A system analysis team is not an audit team. In fact, it is a team that undertakes a problem-solving activity. It's not something that can be done unilaterally by FAA. It necessarily involves -- requires the involvement of -- of other parties to do it properly. We can't problem-solve for American, for example, and in that regard, I believe that the other two activities which were mentioned probably were legitimate audits, as I understand, done by outside auditors hired by APA and by American to -- to do an audit of a certain function for which it wouldn't have been appropriate to include FAA's team members, although I do understand it's not my -- the certificate clearly isn't in my region. I do understand that there were extensive briefings from those audit efforts to the CMU or CMO as it occurred, but I think it is important certainly to distinguish again that this SAT or system analysis team is a problem-solving activity that looks at specific trend data or any set of data that may indicate a problem or a concern, and the attempt then is to identify that -- that risk or hazard, to mitigate it, eliminate it or in some way manage it, which is what occurred here. The recommendations that came out of that SAT team, I believe 17 of them have been implemented already by American, which is -- obviously without some follow-up implementation, it's not an effective process. I believe that does prove that as a tool which is part of ATOS, there are clearly good things happening that -- that are making real changes. Q Thank you. Under the old system, you could put an airman's name into the system and check/see how many inspections he's had, what areas he was inspected on, and you could do the same thing with an N number of an airplane. You could put it in and check on a particular airplane and see what oversight had been done on that. Why does the ATOS System not allow this? A Well, to -- to say that it doesn't allow this, I'm not sure it's quite accurate, and I -- I believe that we -- we still are capable of deriving from the system inspection work that we have done involving a particular aircraft or a particular airman. But I would point out again that we're trying to move away from this technology that won't take us any further in terms of looking at individual pieces of the system for a quality control-type activity, that -- for example, it was mentioned perhaps nobody's looking at every piece of American's system wherever they operate. Well, that -- that well might be true, and we do that with some confidence. If we're sure that American's system was properly designed, and by what we have looked at, that American is complying or continuing to -- to operate with what was designed initially as it was designed. In other words, they're keeping integrity in their systems. So, we may not have to look at every airman or every check airman or every facility in order to have confidence that the system that American designed and is -- is practicing is in fact being carried out the way it was designed. Q Thank you. I understand that inspectors are having difficulty in interpreting the safety attribute inspections and the element performance inspection questions, and in fact, inspectors are responding differently to the same questions. If this is so, how do you achieve quality data, and how do you achieve standardization? A Through lessons learned, and again this is a continuing improvement process. We did implement ATOS, and since implementing it, we've -- we've learned several things about shortcomings in the initial training and the design of the initial questionnaires and checklists that we're using. We have revamped and redesigned both the training program and the checklist for EPIs, and we intend -- and -- and the procedures as well. We intend to conduct seminars in the near future with all of the CMTs, the complete units, all the inspectors involved, in other words, to ensure that there's an understanding of how to use those new tools. So, we're moving forward in all the areas you mentioned. Q Okay. Under the -- the General Accounting Office did an audit, and they mentioned in there that the training was ineffective. Is this what you're saying? A That's correct. In fact, the training that was offered was developed and provided in a fairly special environment in order to quickly train the inspectors that would be involved with these 10 carriers or the initial cadre. Since that time, the training has been moved to our Academy in Oklahoma City, where it's now resident, and there's been several -- there have been several changes made to the curriculum. My understanding from talking to our people who have come through that new program is that it's a very good course. Q Also according to the same GAO report, the guidance for planning and performing inspections was not very effective. Has this guidance been reviewed and improved? A The -- yes is the answer. The -- the job aids that are being used to do element performance inspections have been revamped, field tested, and they are ready now for implementation. That should occur within the next month or so. There was a decision made that the guidance used on the safety attribute inspection is adequate, but there's further explanation needed in terms of how to apply it, and that will be accomplished through the seminars that I mentioned previously. Q Under the ATOS Program, since the inspections are kind of specifically targeted, what method is there for identifying and reporting problems or areas of concern? A The -- there are two methods actually. Because these inspections are very detailed, they take time, and our reporting process is -- is one that allows entry of data as the inspection is on- going. Initially, CMUs, certificate management inspectors, could not access that data easily. That's another correction that we've made. So that as an inspection activity is on-going, the -- the information is now available to our principal inspectors. At the same time, we believe that with this concept, there is the capability for much improved communications between the geographic inspectors and principal inspectors. The good old phone still is -- is a -- is a very usable system. Now that we don't have a 190 geographic inspectors doing generic work on American Airlines but rather have a much smaller team, working almost full time on American, it's easier for that team to communicate to the principals when they find immediate problems that need attention. Q Once again referring to the GAO report, could you explain why an airline's screening, boarding and cabin briefing procedures are given more weight than cockpit procedures in determining the number of inspections that an ATOS team is to conduct? A I'm not sure that I would agree that they're given more weight than those other factors at all. In fact, I believe the CMT has a great deal of latitude on where to place its emphasis. It could be for any given carrier, if the data show that the training and cockpit procedures are being adequately executed as designed, but cabin safety procedures are not, that we would focus our resources in those areas, but there is flexibility intentionally built into the system, and targeting is to occur based on the data that we're deriving from the system. CHAIRMAN HALL: How does the system deal with that manual situation, where the manufacturer makes the change and half of your major carriers implement it and the other half don't? Do you all communicate on that and try to -- is there any policy or -- THE WITNESS: There are -- there are things that the system can do to deal with that issue. I would point out, outside of ATOS, that the agency recently has made available through a web site all changes of that nature, changes to AFMs and bulletins from manufacturers, and we've sent instructions not just to ATOS principals but to all of our principals telling them to check that web site in order to determine whether there are applicable changes that are recent, and, if so, to take those changes to the operator to ensure that the operator's aware of them. We still have some work to do in that area. Again, because this is a closed loop system, when we find those kinds of problems with one carrier -- one of the modules that I didn't point out, it's a ninth module on the schematic that I displayed, is the policy module. That's really the headquarters work function. But the system allows us to -- to insist actually that we provide feedback to that policy group when there are problems that need dissemination to a broader audience or that need the attention that only the policy group can deal with. So, again, I think that the system is in place. It's a matter of refining it and making it work to our advantage. CHAIRMAN HALL: And under ATOS, how are you going to evaluate training and oversee training? THE WITNESS: Carrier training? CHAIRMAN HALL: Carrier training, yeah. THE WITNESS: The -- the system is -- is designed to do inspections in 88 different elements of air carrier systems. The system has defined some generic air carrier systems as part of the model. There's seven air carrier systems that we're looking at, 14 subsystems, 88 system elements. We do element performance inspections in each of those areas. In fact, one of the major air carrier systems is training. So, all the subsystems and elements under that activity have specific protocols that we use to -- to examine how the training's been put together and how well it's being executed. CHAIRMAN HALL: And you have -- we heard yesterday that your inspectors have average of 38 years of -- 36-38 years with the service, with the FAA. So, I guess the majority of that 38 years, they've been using the old system. So, how are you going to train these gentlemen or ladies with all this experience to do this new job without having gaps and -- and problems? Life's full of problems, though. THE WITNESS: Well, -- CHAIRMAN HALL: Disregard the problems, the gaps. THE WITNESS: When -- when you're speaking of gaps, I'm assuming that it's in reference to bringing on new people or these -- CHAIRMAN HALL: Bringing on your new system. I mean it -- obviously this event occurred during the transition of these two systems. Was that an impact or not an impact? Whether this was an audit or a problem-solving thing, ATOS is designed, I believe, to prevent incidents such as this occurring. THE WITNESS: Absolutely. CHAIRMAN HALL: Not evaluate them after they occur, and what we have here is we've had -- whether it was an audit or problem-solving, is it 13 or 17 different items that -- that were noted by the carrier, the union and -- and everyone, that were of an urgent - - we were told of an urgent situation. That's my -- my concern -- observation last night was if I was in the situation of government oversight, that's -- that's a reflection and a concern to me. How did that occur? THE WITNESS: Well, I think it's a very good point, and clearly the FAA's very reliant on the expertise that our inspectors bring to the agency, and we have focused in the past on enhancing that type of traditional expertise. Clearly we have a need with this new model to focus also on training that develops skills, such as auditing skills, system safety skills, and risk management skills. These are not skills that are particularly common to mechanics, pilots and other people that operate in this industry. So, we have a clear requirement ahead of us to identify those needs and to develop appropriate training courses that can provide for those needs. That's -- CHAIRMAN HALL: And attract people to Oklahoma City to take the courses. THE WITNESS: That's true, also. CHAIRMAN HALL: Go ahead, then. BY MR. TEW: Q All the data that's been collected so far, has it been entered into the system? A Yes. I say yes in that one of the things that -- that has always been a problem, and I believe we have the ability to -- to work on quite effectively with this system, is that inspectors take corrective action on site without entering data into the system, and therefore valuable information is lost to us. Again, this system demands that inspectors use protocols that require documentation before an inspection can be called complete. That forces the issue of data entry much more effectively. It's not to say that we're still not losing some valuable data, and I think that as we move forward with some of these safety programs that have recently been approved, particularly the partnership programs, gives us the ability to collect data in a much more extensive and valuable way. Q Well, following up on the Chairman's question, I understand that a number of inspectors are having a good bit of trouble working with the system, entering the data, that it's not really user-friendly. A Well, two things about that. First of all, the system uses a web-based technology, and we are -- our data entry was dependent upon accessing the database through the Internet, through modems, when we initially implemented the system. So, the speed and user- friendliness was not so much system design as -- as how well the Internet service provider provided that service. We now have moved to the database inside the FAA's fire wall. It's on the FAA Intranet. It's accessible through our wide area network. The speed of the system has substantially improved from the people I've talked to. Those types of problems have -- have been virtually eliminated. In terms of the ease of entering data and the ease of what we're doing, I would be the last to say that ATOS was designed or that it will make the inspector's job easier. In fact, it's tougher. It's tough work. It requires a great deal of expertise. It's -- it's not easy to -- to do what we're asking, and yet the only way to improve the data quality is to ask people to do this work and do it well. That's one reason, also, that it's not so well accepted. Change is very difficult to bring about in any organization, such as ours, or anyone else that has an organization of comparable size or complexity. But this -- this is tough work. Q Okay. Thank you. The FAA spent $95 million on the Safety Performance Analysis System or SPAS, as it's called, so that it would analyze key aviation data, identify trends and potential safety concerns, and target inspection resources accordingly. Why isn't the ATOS System linked up with this existing Flight Standards Aviation Risk Analysis System? A SPAS is -- is an excellent system, still being used with a 140 carriers that are not under ATOS and will be the primary analytic engine for the analytic module process once that process is developed and once SPAS has been re-engineered, so to speak, to use ATOS data. That work is currently on-going. The initial interfaces should be done this fiscal year, and within two to three fiscal years, SPAS, as I said, will become the primary analytic engine for ATOS. Q Since it doesn't currently link with SPAS, how do you ensure that the information gathered by the ATOS is sufficient to allow the agency to perform critical trend and safety analysis, also since the analysis module is not developed? A Well, again what -- what is lacking for principal inspectors, such as Corky testified yesterday, is an analytic tool, either through an analyst or through a tool such as SPAS. So, the analysis that's being done has to be done based on what he and his staff are able to do simply by reviewing the data and identifying trends using manual techniques. Excuse me. So, there -- there clearly is work to be done in that area yet. We do have plans to do it. It's on-going now. Q ATOS was originally designed that all new entrant airlines would come under ATOS. Is this being done? A It's not being done. Again, as I mentioned, we're reluctant to move forward in expanding the application of ATOS until we've done two things. One, finished the work that is on-going now to make the remaining four process modules more sophisticated in their activities, and, two, till we have a fair comfort level with the resources that are needed to move forward with the system. Q You touched briefly on this. Has FAA upper management received any complaints from inspectors in the field and the principal operating inspectors? What are the complaints, and what is being done to respond to them? A Well, I believe we have touched on this extensively actually. We have certainly received many complaints. As I said, some of those complaints are based simply on not wanting to do something different that's harder work. The other complaints that -- that truly are valid have all been, I think, dealt with quite effectively, either through the improvements that we have made in terms of job aids, automation, changes to procedures, some of the educational changes that I've talked about, or they have been duly noted as changes that need to be made as we re-engineer our initial certification activities and continue to develop the analysis, evaluation and implementation processes. Q Have you uncovered any major safety defects as a result of the ATOS Program? A By that, do you mean have we discovered safety defects in air carrier operations? Q Yes. Have you uncovered any major safety defects in the operations of the airlines themselves? A Yes. I mean in -- in the regard that the data and the surveillance that we have done have identified problems that need to be fixed, yes. Again, I believe that you can go to any of the 10 initial cadre carriers, the Certificate Management Units, and find examples where there have been problems that led to system analysis teams and eventually actions that were taken to correct problems. Q Thank you, Inspector Gilliom. MR. TEW: Mr. Chairman, I have no more questions. CHAIRMAN HALL: Well, I guess I see Mr. Feith turning his microphone on. Do you have questions, Mr. Feith? MR. FEITH: Yes, sir. CHAIRMAN HALL: Please proceed. BY MR. FEITH: Q Mr. Gilliom, you were talking about the fact that there are no analysts in all of the offices, and that that function has now fallen on to basically the POIs and their staff. Given the level of detail now that you're receiving through this program as far as the information that's being gathered, how much time in your view do you see the POIs and their staff having to take to analyze this data, and doesn't that compromise their ability to be out in the field doing the surveillance work and doing the inspections rather than being in the office trying to analyze that data? A I think there are two -- two important things in this regard. First, the time that's spent is no more than the time that was spent under the old system. So, again, while we don't have everything we need for this new system, we -- we -- we certainly have no less than what we had before. Secondly, the inspection work or the surveillance work that -- that you're referencing really shouldn't be done by the principal inspectors. They're there to manage the certificate activities. When they need inspection work done, they should rely on the geographic inspectors who are there to do that work, and in fact, when we finally have the training in place, as I mentioned, the audits, system safety, risk management training in place that's needed, that area of expertise of doing that surveillance will be highly specialized. It's not something the principals should be focused on doing. Q Okay. Now, two things. First off, you've got more -- more information. So, how can you be taking the same amount of time as in the old system, evaluating new data that has more detail to it and still be able to do it in the same period of time? A Well, -- Q And, second, -- and then, second, you were talking about that the POI relies on the geographic inspectors, and as we heard yesterday from Mr. Valentine, he's got a geographic inspector sitting in Las Vegas where he really needs the inspector in New York because that's a hub, that's a big operation, and he can't rely on anybody there because he doesn't have anybody there. A Right. They're both important points. First of all, the data may be more in quantity. I couldn't verify that. What was testified to yesterday is that it's much improved in quality. That quality improvement could in fact make it easier to analyze now. I'm not here to say that that's ideal in terms of that we do need analysts for this system. So, that's a clear need that is ahead of us. But the -- the enhanced data should be easier to work with in the long run. Secondly, the -- the workforce that is in place today to do the surveillance work may not be in a most efficient place to do that work. That's for a reason. We had to use the people that we were using before. The person, for example, doing operations work in Las Vegas for American Airlines is the same person who is there doing that work predominantly under the NPG and PTRS activity. We couldn't just eliminate that person from the system. We had to keep using that person, even though he's not in the most efficient place to do the work. When he moves on, the position will not exist in Las Vegas any longer. It will go to in fact wherever American Airlines Certificate Management Unit tells us they want it. If that is in New York, we'll do our best to fill the position in New York. In my region again, this is my personal experience, we have moved many of our geographic inspector positions from where they began to where they are now most needed. Second point in this arena is that that person in Las Vegas is not limited to Las Vegas as he used to be under the old system. It used to be a very confined geographic area in which that person operated. That person now has the authority to travel wherever he needs to travel to do the work under the Comprehensive Surveillance Plan, including offshore, because there's a fair amount of international work that needs to be done. Q Okay. Following up on that, we heard yesterday in previous testimony that there are travel restrictions because of budgeting. A Yes. Q So, now you've just told me two different things. A I -- I don't believe -- I didn't intend to do that. There -- there are always travel restrictions in relation to budget. However, we this year have funded the Certificate Management Units, each of them, with twice as much travel funding as they had last year. It's roughly a $100,000 per unit. We have the capability now of costing this activity much more accurately than we've had in the past. We're going to continue to try to use the existing resources to serve the priorities that are specified by the agency, and clearly surveillance of Part 121 operators, whether they're under ATOS or not, is a high priority for us. Q Do you have funds in Western Pacific Region to do all of the things you need to do with your people? A Well, you asked a very sweeping question, and my answer will be more specific. I believe we have the funds to do what's needed under this ATOS model with the United America West certificates which my region's responsible for. I think that will be true across the board for the 10 certificates we're dealing with. Notwithstanding that, again I don't believe government agencies will ever have all the resources that they think they need to do the work that's there. So, we're prioritizing in Western Pacific Region to make sure that we do the first things first. Q Okay. And I'm not going to belabor this, but I just want to get clarification. Let's say towards the latter part of the year, you see a projection that you're going to run short of funds to fulfill the mission. How hard is it to get additional funds to complete the work necessary for that particular year or is that going to be sacrificed? A My experience with the agency is that when a true need is identified, such as you're talking about, it gets met one way or another. Usually it means by creating pain in another part of the agency. Within the things that I have control over in my own region, if I see that situation arising, I'll be moving resources from General Aviation activities to serve this priority, and, similarly, the agency will attempt to cope with that. Again, we're dealing with some tough situations in the long run, and as I pointed out at the beginning of this, there's a limit to -- to what I can provide valid testimony to in regards to resources and probably have reached that point. Q Let's see. In this -- in this transition period from old to new, and the fact that the training had been identified by a GAO report, and you've acknowledged that the training was basically insufficient for the inspectors that are using this system. A It was -- I -- I can understand why you would use that word, and certainly others have. It's not how I would characterize it, but it needed improvement which has occurred. Q Well, it's our understanding, we've -- I mean prior to coming to this hearing, we -- we have talked to a variety of people, and they told us that it was ineffective, that it didn't give them the level of knowledge necessary to start this new system. They had a lot of problems with it. It was very time-consuming in entering this information into the system and basically bogged them down, and it was their opinion that they weren't able to do the job that they were being paid to do because of this time-consuming transition, and I know that there's been changes that have been evaluated throughout this past year that the system's been in place, but have those changes been filtered down? Have the inspectors received additional training or recurrent training on these changes that are continually being -- A Well, again, I believe the answer is yes, and if you look at the testimony from yesterday, I think there was a clear distinction drawn between the first activity to develop the initial Comprehensive Surveillance Plan a year ago versus what occurred last week, which was an enhanced process. I think it went much more smoothly in all of the CMOs, and, frankly, I think we got much better plans out of it. So, yes is the answer. We have made improvements that have been effective that are in place. Q Okay. Thank you, Mr. Gilliom. CHAIRMAN HALL: Okay. We'll move to the tables now. The National Weather Service? MR. KUESSNER: No questions, sir. CHAIRMAN HALL: Little Rock National Airport? MS. SCHWARTZ: No questions. CHAIRMAN HALL: Little Rock Fire Department? MR. CANTRELL: No questions, sir. CHAIRMAN HALL: The Federal -- excuse me. I'll hold the Federal -- Boeing Commercial Airplane Group? MR. HINDERBERGER: No questions. CHAIRMAN HALL: American Airlines? MR. BAKER: We have no questions, Mr. Chairman. CHAIRMAN HALL: Allied Pilots Association? MR. ZWINGLE: No questions, sir. CHAIRMAN HALL: The Association of Professional Flight Attendants? MS. LORD-JONES: No questions, sir. CHAIRMAN HALL: And the Federal Aviation Administration? MR. STREETER: A couple, sir. INTERVIEW BY PARTIES TO THE HEARING BY MR. STREETER: Q The -- I will, first of all, start off with the ATOS data. Is that the only data that is planned to feed into the analysis process? A No, it isn't. We in that process want to look at frankly the entire world of data that's available. So, not only FAA data, such as accident and incident data, enforcement data, other things that come from FAA databases, but in the long run, the system is really designed to use the wealth of data that the carrier's generating, and I believe through our new partnership programs, there will be a variety of data inputs from that activity. Any data is valuable data, and as we develop the analysis module, we will be, first of all, determining exactly what questions we need to answer in order to effectively manage the certificate, and any data that helps us answer those questions, we would hope would be available and used in the process. Q I believe you mentioned that the Comprehensive Surveillance Plans were finished for most of the ATOS carriers within the last week or so. Do you view that as a static document then for the remainder of the year? A No. The system and those documents are intended to be flexible. It's quite different from the old system, and for that reason, it's unlikely that any of these surveillance plans would be in a strict sense 100 percent completed in any given year because the intent is, as we learn from the surveillance we're doing, to shift or retarget our resources from areas where there are not problems to areas where there are more likely to be problems. Q Okay. So, then, if you saw a surveillance plan that had been retargeted one or more times during the year, would you consider that a normal -- A That's excessive. Q Okay. Thank you. I have no further questions, sir. CHAIRMAN HALL: Mr. Sweedler? MR. SWEEDLER: Yes, Mr. Chairman. I just have one question. INTERVIEW BY BOARD OF INQUIRY BY MR. SWEEDLER: Q I think the testimony has been that only one of the ATOS carriers has an analyst. Could you give us some idea of what the planning is to add the other nine analysts, and how long it might take to get them on board? A Planning to date was to hire at least four more analysts this year, and that's on hold due to a hiring freeze that is currently in place. Analysts are new to us. So, these are not internal hires. These have to be outside hires, and the population that's available to fill those positions is still somewhat of an unknown, and until we know exactly who we're dealing with, it will be hard to determine what training may be necessary, and again this is one area where we're still doing extensive work, and we know that's ahead of us. But our plan at least would be to move forward to that extent this year. Q So, not only do you have the problem of trying to fill the positions that you're not usually familiar with, but you have the hiring freeze to deal with? A That's true. Q Thank you, sir. CHAIRMAN HALL: Mr. Berman? MR. BERMAN: No questions, Mr. Chairman. CHAIRMAN HALL: Mr. Haueter? MR. HAUETER: I have a couple. BY MR. HAUETER: Q As I mentioned yesterday, under the NASIP Program, which no longer exists, there was an internal audit function where FAA inspectors in different regions came in and looked at another region. How is that being handled now under ATOS? A Well, first of all, I would correct something that -- that I think was -- was said yesterday about NASIPs. They have not been eliminated. They've always been available on request to -- to be accomplished, and, so, the program is -- is still there. What has been decided and what is on-going is that the NASIP Program, like our old surveillance program, is not adequate to do the work that needs to be done in the future in order to make the advances that we'd like to make. So, it also has been redesigned in terms of -- of along the same system safety lines, so that when we do a NASIP, it's not simply a compliance inspection but it will look at the very things that the Board, I think, has rightly pointed out need to be looked at in terms of safety cultures and management structures and the systems of the carrier. Also, the program will be turned internally to look at FAA's own capabilities and how well we're doing our job in terms of our own culture and structure and compliance with our own procedures. So, it's -- it'll be a two-pronged approach in the future once it's redesigned, but it still is there. We don't intend to eliminate it. I also would point out that Certificate Management Teams have two components, the Certificate Management Unit are the principal inspectors who are there to manage certificate activities, and the geographic arm of the team which is doing surveillance. By design, those geographic inspectors, while they are team members, do not report to the same supervisors or managers. Those people do come from other regions. They are independent. We designed it that way as a control, so that those people shouldn't feel pressure to not enter honest data as a result of their surveillance. So, in fact, we have a full-time look now by those people from inspectors who are outside of the region, and hopefully we will keep some level of discomfort between those two elements of the team in order to maintain that objectivity. Q I guess a follow-up there. Do you have any idea when the NASIP Program will be redesigned and available again? A I -- I -- I know that it's an activity that is on-going. I don't think it's so extensive that it won't be done this fiscal year, but it's not a program I'm working directly with. So, I don't -- I don't know the dates. Q Okay. If a geographic, you know, inspector were to note problems in how a principal operations inspector was running the shop or something else, how would he get that information up higher in the FAA? A Beyond the Certificate Management Team? Q Right. A Well, first of all, because of this separation of powers, so to speak, there's nothing in the system that can override whatever data the geographic inspector wants to enter. So, even with the evaluation function that is there to validate the data, there's -- there's no way that whatever that person has put down into the system, that it can be eliminated. In fact, to some extent, that person, if he makes an error, it's fairly difficult to correct that error. It can be corrected, but there will be a record of the change. So, there's -- there's that type of integrity built in. There's always -- there are processes in place to elevate concerns outside of the CMT. Part of those are covered by our collective bargaining agreement, and again we're looking at our own safety culture to make sure that there's no impediment to raising safety issues to whatever level's necessary to address them. Q Will ATOS be customized for each carrier? A It is customized for each carrier. I mean that really is part and parcel of the whole concept. Before, our geographic inspectors were very generic. Some of them were -- were trying to look -- do inspections on 20 or more carriers, couldn't be experts in the systems of that carrier. Now, they're working on a single carrier, and the design is to make them experts on that carrier's systems. Q Okay. We had, you know, mention yesterday that the inspector had no choice over where the geographic inspectors would be. Who makes those decisions? A Well, that's a misperception that was generated again by the way we put the system in place initially. We needed to use the inspectors that had been doing the work before where they were. We acknowledged from the beginning that that was not the most efficient choice, but it was necessary at the time. We acknowledged from the beginning that it would change over time. So, when these people leave their jobs for whatever reason, they're not replaced where they were. Instead, the Certificate Management Unit and that unit alone has the authority to request where the new position will be, not who the person will be but where it will be, and the design is to get it in the most efficient place to do the work. Q It seems like it could take quite some time if you're waiting for a period to resign or retire for that to happen. A In some cases, that may be. One of the unfortunate problems we're dealing with is that with the geographic inspectors, there's a high rate of turnover, too high a rate of turnover. So, some -- the -- as I said, when we make an investment in the inspector, and it's a new system, it will be quite an investment. So, we need actually to focus on reducing the rate of turnover. What we obviously haven't done a good job on is communicating that CMUs do have control over where those positions are, and perhaps this hearing will help correct that to some extent. Q But I guess, you know, how much control do you have when you're waiting for somebody to retire before you can make the switch? That seems -- A To some extent, it's a problem that can and will be addressed. We have some union concerns there that are legitimate in terms of fairness to employees, and those are problems, I think, that we have to work. But, once again, where a person is doesn't limit where that person can do the work. Q Okay. And I guess also from the testimony yesterday, it indicated that he believed he's about 40 percent under-staffed, and that's a huge number in my opinion. A It is, and I'm sure quite valid from -- from where Corky sits, and maybe -- maybe quite valid period. Every -- every inspector you ask will have, including myself, a different opinion about what the staffing needs are. What the ATOS model does is help us become much more explicit about these needs, much more -- make the numbers much more quantifiable, and I'm looking forward actually to -- as the system is applied, how -- how that piece of it works well for us. Q Okay. Thank you. CHAIRMAN HALL: Mr. Clark? BY MR. CLARK: Q There were three studies, one on SAT and one by Allied Pilots and American, that came up with 89 recommendations. How is that type of information or recommendations or resolutions of those recommendations transferred to other airlines or to other CMTs or CMUs? A It's -- it's an area of challenge, I think. Excuse me. It's an area of challenge not only for FAA but for industry and everyone else, how to share best practices and problems. It's one that we're working on. I frankly believe this model is -- is a good model for sharing the data. As we've talked about it for the most part throughout this hearing, we've talked about ATOS as it applies to American Airlines or to another individual carrier, but if you look at again that schematic, it is a model that -- that -- that integrates all those activities and permits data to flow into again that policy module in a systematic way. So, I think ultimately, we have now a structure and an infrastructure in place that can better utilize data, such as you're talking about, and spread it than we've had in the past. But there are a lot of problems involved from proprietary issues to litigation issues to other things that impinge on the industry and the FAA and all of us involved in dealing with this issue and how well we can do it. So, we need to keep working on it. Q Okay. Is -- is there something -- we hear about ATOS under development, and is there something functionally in place now that can distribute the lessons learned or at least those 17 recommendations that have been adopted by an agreement with American and the FAA? Have those been distributed or is there -- the model is a little nebulous. Is there -- what is the -- what shows up on somebody's desk that says this information has been transferred? How does that get done? A The -- well, again, of those recommendations -- and I'm not familiar with each of them individually, some of them may be proprietary, and ceratinly that would be a concern in terms of sharing them. Otherwise the ATOS activity at this point only would provide an opportunity for those recommendations to be surfaced through the ATOS CMO which is the Headquarters group that provides support to this activity, and -- and through that group disseminated to other ATOS carriers. Part of the -- the analysis function and the implementation function is to identify best practices so that they can be shared. Again, I think we have rudimentary elements in place at this time that need further development. Q Okay. Thank you. CHAIRMAN HALL: Well, David, I have had the honor and privilege of being the chairman of this agency since 1994 and being on this Board since the Fall of '93. So, most of your testimony I have seen this process take place, and, of course, I have my own observations as to why this -- this took place, but clearly after ValuJet, there was an attention on some of the new entrant carriers, feeling that the major airlines basically had their act together, and we needed to do a better job, and this system was designed. Hopefully it's going to do a better job on all carriers, but I think what we've seen in the discussion today is that, you know, you have a very experienced workforce that's been working with the system that by and large has done very well historically, as you pointed out in your opening comments, and that now you're putting a new system in, and I'm still concerned about the gap that's taking place and the resistance that we obviously are seeing, and I'm sure you're aware of, which is just human nature. All of us don't like change. It's difficult, and if you've been doing something, particularly if it's something you think is effective, somebody comes along and wants you to do it differently, you -- you resist, and there, we've got to have some training and the system. The only thing that's been sort of discussed, and I think you're aware as well as I am or much, much better than I am, are the problems and the challenges. You know, I observed two things that I want to be sure of. One, I see all these people in Washington go up to Congress and say, you know, less is more, and many times, less is less, and when it comes to safety, less means -- can mean the loss of lives and serious injuries to individuals that are lying on the Federal Government to provide safety oversight of the aviation system. But the main concern I have is -- gets to what Mr. Clark said. I guess we got a free audit here out of the airline union, which we appreciate, and American Airlines, looking at the system with some recommendations after an accident, and, you know, we live in a great country. We have very -- these fatal accidents, such as this, are rare in the system. We killed over 400,000 people on our highways last year, and a lot of the -- a lot of the good -- last year -- last decade. I apologize. 40,000 a year over the last 10 years. It's the Number 1 killer of kids. One of my interests has been to try to get a lot of the technology and practices in aviation on our highways. But one of the reasons aviation is so safe is because of the detail we go into safety and the culture we have as demonstrated by this hearing and everyone's participation. But you need to see if you can't figure out how these 88 items get transferred within the system. Just to say it's not -- I understand your answer, short answer is there's no way to do it right now, but I would suggest if you -- if you can go back and do that, and I'll try to get -- I'd like to ask Mr. Haueter -- I'd like to send a letter to Ms. Garvey on the basis of this hearing and ask her to -- to see if you all can't make that a priority because there are -- it's important information. There are a number of factors that we're going to be looking at, have looked at and will look at in this hearing that are important to the safety of each and every flight that takes off every flight, and -- and -- and obviously an oversight, a close look at the safety system of American, we went through. There was another major carrier that had five accidents in the earlier part of my service on the Board, and they had to do a very serious internal look at themselves and learned a great deal and improved greatly, and that information needs to be transferred through the -- through the system. Now, you have been an excellent witness, and I compliment on your service to the Government and the military. You don't have quite as much as what, 57 years as one of our previous witnesses has, but it's obviously excellent service, and I think service to our nation should be acknowledged and recognized, and I appreciate that. I'd like to present you an opportunity as I have with all the witnesses, and I think Mr. Tew now is not going to be stepping on the Chairman's question here, and I appreciate that. Do you have any additional suggestions or thoughts for the Board in light of this accident or your experience that might be helpful to aviation safety to prevent an event such as this from recurring? THE WITNESS: Thank you, Mr. Chairman. The only thing that I would like to leave, I guess, as a thought is that in my opinion, this move to a new air transportation oversight system from a regulator's point of view is very important, and it's very easy to derail a process like this. One of the things that Sandia did for us when we were initially designing this system and considering designing it was a study of why we had been so unsuccessful at making changes with the old system, and they looked at our entire environment, the whole environment that includes not only the regulator but the industry and various oversight groups, and determined that -- that from their observation of that phenomena, that that environment's immutable. It really is very change-resistant because the overall interplay of all the players is that the static status is nice, and clearly I think we're dealing with that internally in making these changes. It's very difficult. We admittedly have some rough edges. We're working to smooth those out, but it's going to take a great deal of fortitude, I think, to -- to see this through, and I -- I hope that we have the opportunity and support to do that because in my personal opinion, this is the one thing that as a regulator, if we don't do, we will never progress, and while I would not say that the system we have in place is -- is everything that it should be because it is one that is designed for future generations and improvement, over time, I think it will be again the leading edge worldclass system, and I'd like to continue participating in that development. So, thank you for that opportunity. CHAIRMAN HALL: Okay. Well, thank you very much, and we -- as I said, the Board's on record in favor of ATOS, on record in the favor of expansion of ATOS, and I will individually try, and the Board will, to get you the necessary resources to implement the program. I understand you have to -- but it's clear to me when you have a closed loop, and you don't have the resources for three boxes in the loop, your system is not going to perform as -- as designed. Well, we've had -- I appreciate this first witness. We're going to take a break now before our next witness. I would like to make a very important announcement. The City of Little Rock and its Mayor, Jim Daley, have provided coffee outside the ballroom compliments of the citizens of this great city. The Mayor properly observed yesterday that the Federal Government with all its wonderful rules and regulations is unable to provide funding for coffee for you all, and, so, the City of Little Rock has rushed in to the -- to the gap, and there's coffee outside the door compliments of the City of Little Rock. I hope that will in no way bias anyone in their opinions or later proceedings in this hearing, but the coffee is there. So, we'll stand in recess -- MR. STREETER: Mr. Chairman? CHAIRMAN HALL: -- until -- MR. STREETER: Mr. Chairman? CHAIRMAN HALL: For 15 minutes. MR. STREETER: Mr. Chairman? CHAIRMAN HALL: Yes? MR. STREETER: Over here, sir. CHAIRMAN HALL: Yes, sir? MR. STREETER: I would ask that Mr. Gilliom be released as a witness so he can return to his duties in Western Pacific Region before our weather arrives. CHAIRMAN HALL: Oh, I forgot. The National Weather Service is also providing five to 10 inches of snow later today, we are told. We're told. Now, you are excused to return, Mr. Gilliom. THE WITNESS: Thank you, Mr. Chairman. (Whereupon, the witness was excused.) (Whereupon, a recess was taken.) CHAIRMAN HALL: We will reconvene this public hearing of the National Transportation Safety Board, and I'll ask Mr. Berman if he'll call our next witness. MR. BERMAN: I call Mr. Robert Baker. Whereupon, ROBERT BAKER having been first duly affirmed, was called as a witness herein and was examined and testified as follows: INTERVIEW BY BOARD OF INQUIRY BY MR. BERMAN: Q Good morning. A Good morning. Q Would you please state your full name and business address for the record? A My name is Robert W. Baker. I office at American Airlines Headquarters, Dallas-Fort Worth, Texas. Q And by whom are you employed? A American Airlines. Q What's your present position, and then, please, your position at the time of the accident? A I am currently Vice Chairman of AMR Corporation, the owner of American Airlines and American. At the time of the accident, I was Executive Vice President of Operations. Q Of American Airlines? A Of American Airlines. Q Not AMR. Okay. And how long have you held these two positions? A I've been in the Vice Chairman's role as a part of the Office of the Chairman for approximately three weeks. I was in charge of American's Airline Operations under various titles since 1985. Q Thinking back to the Executive Vice President and similar positions at the airline, could you please summarize your duties and responsibilities in those positions? A I'd be glad to, and in a few moments, I'll show you an organizational chart that will give you more detail. Operations within American Airlines encompasses all of those functions and responsibilities having to do with the actual operation of the aircraft and our airport functions. That would include the Flight Department, the hiring, training and day-to-day operation of the flying part of the company, Maintenance and Engineering for our fleet of aircraft, some planning functions in the crew area, dependability monitoring, the Operations Center, including the dispatch function, crew schedule, and then other support roles, such as security, corporate real estate, safety, and our cargo operations. Q And also please describe your responsibilities as Vice Chairman of AMR. A That is a relatively new concept at American Airlines. It is an attempt to -- to create a little different structure as we go forward. The chairman of the corporation, Mr. Don Carty, and I will co-manage the airline. I will be principally responsible for continued oversight of the operating side of the company. He will be more involved in the finance, marketing and external affairs. Q Thank you. And could you summarize your education and training that qualified you for these positions? A I received a Bachelor's in Economics from Trinity College in Hartford, Connecticut, in 1966, and an MBA from the Wharton School of the University of Pennsylvania in 1968. Q And do you hold FAA airman certificates? A I do not. Q Thank you very much, sir. A Thank you. INTERVIEW BY TECHNICAL PANEL BY DR. BYRNE: Q Good morning, Mr. Baker. A Good morning. Just one follow-up question as far as your background and current duties and responsibilities. Do you have the opportunity to ever observe flight operations from the flight deck during your daily activities? A Yes, I do. I travel two to three days a week to carry out various assignments. In that context, I have a personal policy of spending approximately 50 percent of those travel days on the jumpseat in our aircraft. Q Thank you. Before -- what we'd like to talk about today are your internal oversight programs in terms of flight safety, and also your response to recent accidents and incidents at the company. But before we do so, I know you have a presentation prepared, and would you please start that presentation by going through the management structure and hierarchy as it relates to flight operations and flight safety at American Airlines? A Thank you very much. I very much appreciate the opportunity to participate in this process. Let me -- before showing you some organizational charts, which I think is the most efficient way to describe the way we are organized and some changes that we have recently made and are evolving toward. I think it's appropriate that I begin by trying to describe the culture within the company and the way in which the management team at the senior level attempts to run the company. First of all, I think it is fair to say, and our reputation is widely known, as a company that is highly analytical and highly data rich. We -- we very much like to deal with problems in an analytical and a data-driven sense. We have lots of feeds of data that we believe tell us where our successes and failures lie. Secondly, we cherish very much the notion of innovation, whether it be in the marketing area or the operating area. We are prepared to spend the resources and human capital to create constant innovation in an attempt to find better ways to do business and to serve our customers. The third point that is important is that we have the corporation under constant evaluation. Throughout the staff groups and as a group of senior managers, we sit once a week and evaluate the operation of the airline from almost every conceivable direction. We are not the least bit reluctant to identify our failures and to then use that same analytical approach to designing the solutions and fixes. We are very hard on ourselves on a routine basis. We believe very much in constant and continuous change as a means of keeping up with the business that is operating in an environment that changes rapidly throughout our network. One item that I hope to demonstrate to you this morning is something a bit new for us, and that is we are moving very quickly toward more of a third party evaluation. We think we're pretty tough on ourselves. We do a lot of self-evaluation. We think it appropriate, particularly as the company gets larger, that we bring to bear the resources of third parties to help us evaluate our performance. One thing in the safety area that -- that guides us throughout the operating departments is that we never feel we achieve perfect safety. That is our goal, but we never get there, and if we ever believe that we've gotten there, we probably have a very serious problem, and that's what keeps us moving and bringing change to -- to the corporation. Let me now show you -- and I apologize if -- if these are hard to read in this large room. I'll show you a series of organizational charts that represents American Airlines in the past and at the point of the accident, and then I will highlight some changes that we have and are in the process of implementing at the corporation. Let me say at the beginning that this appears to be a stacked chart, but in reality, all of the people on this chart report directly to the chairman, president and CEO, Mr. Don Carty. If you were to draw this on a piece of paper, it would be a single horizontal string of positions reporting to -- to the chairman's office. These are all of the typical functions of -- of an airline and a big company. Marketing, Government Affairs, General Counsel, and then some of our subsidiary efforts and in the computer business, American Eagle, our regional airline subsidiary, and here is -- is the Operations side of the company, showing myself as -- as being in that role. This basic organization has been in place at American for many years. Dropping now to -- to the Operations organization, specifically it is, as I indicated, all-encompassing to the -- to the airline operations, but more specifically, starting at the left, we have Maintenance and Engineering responsible for the maintenance of -- of our aircraft. That is done both in line maintenance stations and in our two major maintenance bases, one at Tulsa, Oklahoma, and one at Alliance Airport in Fort Worth, Texas. We employ approximately 10,000 people in this Maintenance and Engineering function. The second function are those Planning and Performance Evaluation functions that deal primarily with our crew resources and also manages our dispatch and crew schedule function. The Vice President of Flight and Chief Pilot is responsible for the hiring, training and day-to-day operation of our -- of our flying. The President of Cargo is a self-contained separate company in charge of all aspects of the cargo business, which, to put it in perspective, is roughly a $1 billion business for American Airlines. We have on the far right shown a VP of the Reno Air Integration. In 1999, we purchased the assets of Reno Air and integrated them into American. This function was principally the place in which we -- we focused that integration, coordinated it across all the company's departments. Then Corporate Real Estate on the bottom left, responsible for the real estate and building of our facilities, purchasing, responsible for everything that we buy, including aircraft parts. Managing Director of Security, both internal and external security, and, finally and importantly, in the far right, the Managing Director of Safety. This function includes the safety processes and the environmental oversight at American Airlines and the subsidiary companies. Q Mr. Baker, just a quick question. A Yes. Q Pardon me. With those nine operational units underneath you, would you describe for us your relative amount of time in oversight or your -- where you allocate your responsibilities? Is it equally across those nine operational units or bias towards one versus another? A Well, I -- first of all, it is a plate full. I can assure you every one, I think, in the airline business works extraordinarily long hours. It is to some degree a reactive environment. Clearly, the -- the areas that get the majority of my attention are maintenance and flight. Those are the two functions that probably between them get 60 percent of my attention. Q Thank you. A This chart now reflects the changes that we have most recently made. The important pieces are that I have been elevated to the Office of the Chairman. Mr. Carty and I will share the management of the company, and as you -- as you will see in a minute, the important operations functions report directly to the Office of the Chairman, and I will retain primary operational responsibility. Again, these are -- the stacking of these boxes are not intended to imply that the upper row is more important functionally than the bottom. They would all be horizontally. I have been replaced in the Executive VP of Operations role, and a new function at American at the very senior level of the corporation. We have now taken safety, security and our environmental assets and put them together in one department reporting directly to the Office of the Chairman. We think that is an important step that we have been considering for -- for a number of months, and we have now implemented that organization, and let me show you what that organization looks like. This is the top line, if you will, of this new organization. On the far left, the safety functions, then security, environmental, and those are the three functions that existed but not under a single leadership prior to this. The two new areas are on the right side of the chart, labeled Operational Audits and Compliance. In these areas, we intend to substantially pick up, if you will, our auditing processes by third parties who are not responsible for the operation of particular operational functions throughout the company. All in all, I expect as we -- we populate this organization, we will be adding on the order of 20 to 25 additional people to these functions. In going now one step below that, this shows you that far left box, the safety function. Here, we have taken each of the areas that are in the operation. We have assigned safety responsibility to a member of the safety staff, also focusing on those interfaces to the FAA in terms of airworthiness on the far right, and the flying operation on -- on the far left. So, this is a highly- integrated but extensive attempt to provide more safety emphasis at the company. One additional change that I believe is important for this context is in our Flight Department itself. These are the basic functions of the Vice President and Chief Pilot in Flight. The change we have made is an important one on the left side of the chart, in which we have separated very distinctly from an organizational point of view the function of doing the training through the ground school and -- and simulator to the Flight Standards Group or the Checking Group to ensure that the training has been done according to the syllabus and done in a standardized way. In the past, those functions have been combined within the training organization. We think separating them will give us an improved level of objectivity and enable us to -- to enhance our level of standardization. That, I believe, is an important change for our company, and that about does it for the organizational changes. Q Thank you, Mr. Baker. A couple follow-up questions. What was the fundamental reason for these organizational changes at the high level? A Well, at the very highest level, the creation of the Office of the Chairman was aimed at a number of -- of the objectives. The first objective is to provide some additional focus in some areas, one of those being the combined safety, security and environmental. Clearly, it was time to -- to provide more resources there and to focus them at the top of the corporation. Secondly, we have an obligation to our shareholders to prepare for successorship, and if you could understand all of the details of -- of the people we put into the various jobs, what we are essentially doing is preparing for the successorship to Mr. Carty and I when we retire down the road. So, this was an opportunity to do some training while we're still here and to bring the younger men and women forward in the organization. So, with those two thoughts in mind, that's how the organization evolved. Q Were these changes proposed or considered before the Little Rock accident? A Yes, they were. Particularly in the safety area, they were under consideration and evaluation, but we had not acted on them. Q And why hadn't you acted on them? A Well, we had not reached a consensus that any particular change was -- was appropriate or meaningful. We've spent a good deal more time on that and reached that conclusion and have now made those changes. Q And when you say "we", could you give me a picture of who you're talking about? Who are making these decisions? A Well, the decisions are made essentially by Mr. Carty and I with the concurrence of our Board of Directors, since our board is required to approve all officerships in the corporation. Q Let me turn to the Flight Safety aspect of the management structure, and this touches more -- gets away from box diagrams and into philosophy. Does American Airlines have a formal written statement outlining corporate safety policies? A Well, I'm not sure we have a statement that -- that outlines them for the entire corporation, other than safety being the Number 1 priority of all of our responsibilities, and I believe that is continuously stated in public documents for both our shareholders and our employees. As you get down into individual departments and through our internal audit function, we do have statements of that philosophy. Q And as far as the Department of Flight Safety is concerned, at the time of the accident, what was its scope or reach? What was its mandate? A Let me go back and -- and also indicate that at the time of the accident, the Flight Safety responsibility was in the Flight Department itself. Under this organizational change I've just reviewed, the direct reporting relationship will be to the Vice President of Safety, Security and Environmental with a dotted line to the VP of Flight and Chief Pilot. So, we are making that -- that change as well. The mandate of -- of the Flight Safety Department was (1) to investigate all events and incidents for what we could learn. That is a very important ingredient of our management style, to learn from our shortcomings and hopefully translate them into changes. Secondly, to make suggestions and input to flight management and -- and to myself as to things that they observed needed to be changed or -- or looked at. Q Why does the dotted -- why does the straight line reporting stop at the Vice President level and not continue on to the - - or bypass the Vice President and go directly to the Chairman level? A Well, we -- we felt, and -- and with the help of some outside folks who came in and looked with us at the way we were organized, which I'll review in a few minutes, that a third party relationship of -- of the safety -- the Flight Safety function reporting to the VP of Safety was a better approach than to the Vice President responsible for producing the product, if you will. So, again it is an attempt to separate functions and responsibilities of providing the product from assessing its -- its quality and its compliance, and, so, we thought separating those would be an appropriate step to take. Q What criteria do you use to evaluate the effectiveness of the Flight Safety Department? A Well, we have a number of management tools that we use for all of our senior people in evaluating their performance and in running a particular function. So, we will utilize those tools. We use such things as 360 reviews. Obviously we -- we will look at the output, if you will, of -- of that function. Are they probing deeply enough? Are they coming up with issues that we need to deal with? We want to get to an aggressive approach of problem finding and solving, less reactive types of things. So, those are the -- the -- the -- the points that we will be looking for as this organization matures and develops over the next year or so. Q And who is doing this review or oversight of the Flight Safety Program? A Well, the Vice President of Safety, Environmental and Security and myself will be personally involved in -- in that organization's maturing and its performance. Q Who had direct oversight of that program at the time of the accident? A At the time of the accident, our Safety and Environmental functions were a department in my organization. That department was managed by Mr. Tommy McFall, who reported directly to me. Q And does the Flight Safety Department or did the Flight Safety Department at the time of the accident have a formal means to communicate safety-related information to pilots and other operational personnel at the company? A Oh, yes. I -- we -- we have a premium, if you will, on communications. We -- we emphasize continuously the importance of -- of communication, horizontally and vertically. I think we have a culture that -- that cherishes the flow of information. As I said, we are data-driven. So, I don't think we -- we have a problem of communicating issues, findings and problems. Q Mr. Baker, before we proceed in your presentation, I'd like to, with the Chairman's concurrence, pass the microphone to Mr. Feith for further questions. CHAIRMAN HALL: Okay. MR. FEITH: We'll keep going. BY DR. BYRNE: Q I guess no further questions. So, if we could at this point transition from the management structure and hierarchy to specific oversight programs and responsibilities that the company has, and continue your presentation at that point. A Thank you very much. As I indicated in my introductory remarks, we are a company and a group of -- of managers that believe in getting all of the data we can possibly get. In fact, there is no such thing as too much data. We value input from all sources, and we want to use that data to identify problems and hopefully help us articulate solutions that pinpoint the problems very carefully. We do like to learn from events. There is no point in having things happen unless you learn from those events and use that knowledge to try to -- to prevent their reoccurrence. I will attempt in this to not use jargon. So, I'll define some of these acronyms as we go forward. We have a number of programs that help us provide this data and this input to our management process. The first is a program called AASAP, which stands for the American Airlines Safety Action Program. This is a program conceived at American Airlines in 1992 by one of our captains. We worked with it internally in a small group, and in 1994, the program was actually put into place in the Flight Department at American Airlines, and I will go into more detail on this program in just a moment. A companion program is FOQA, which stands for Flight Operations Quality Assessment, and that is another form of data- gathering, again looking for problems, looking for trends, that point us at difficulties that we can then correct and deal with. Next is a system that we call FlightNet Reporting. This is a more traditional feedback system from our line pilots. In essence, it's often called a debrief in aviation terms. We get approximately 18 to 21,000 debriefs a year from our cockpit crewmen. There are certain mandatory debriefs for some 21 events that happen on the airline. We want to know about these events, so that we can go understand them, see what drove these events to happen. A couple of examples. We want to know about diversions. We want to know about engine shutdowns. We want to know about bird strikes, and there's some 21 of those mandatory things that happen in the airline every day that must be debriefed through this FlightNet reporting system. Again, though, this is an opportunity for our line crews to tell us of problems and things that bother them as they fly the airline. We have, and I spoke briefly about our internal and external audit program. We have had an industry-leading internal audit program for some 10 years. Every six months, we look at the output of that internal audit process. It is a true compliance to the FARs' approach, and we have this internal audit process operating in all of the operating departments of the company, and, of course, in the maintenance area, we have a very extensive inspection and quality assurance organization of several hundred people constantly looking and back- checking our maintenance organization. I will review with you some of our external audit processes in just a minute. Another important data generator, if you will, is the Air Carrier Voluntary Disclosure Program. This is an opportunity under the FARs for an air carrier or its personnel to step forward and disclose to the FAA a discrepancy or an operation that -- that may in fact be a violation, so that we can get all of the facts on the table and review exactly what happened. Next is the American Airlines and the Allied Pilots Association safety review process, and I will talk further about that in a minute, but this is an organized interface between the pilot union and the company again to -- to pull up to a level of action any input that we can get from our line pilots. Finally, the ATOS Program. We spent a lot of time on that this morning, and I won't go into that any further. We are very, very hopeful that ATOS will mature very quickly. We look forward to mining that data. We think it's very important, and we're very anxious to get on with that. Q Mr. Baker, were all six of those programs in place at the time of the accident? A Yes, with one exception, and I'll touch on that. The safety review process. Q Thank you. A Now, specifically to -- to AASAP. As I indicated, this is a program that was developed at American. It started in the Flight Department. It now involves not only our pilots but our mechanics and dispatchers as well. It is a true accident prevention program. It operates on the basis of the following principle, and that is that the best safety information is the information that we don't know, and, so, any way that we can create to bring information that otherwise would not be known to those who can effect change, to bring that information to the surface is very, very valuable, and that's what AASAP's principle concept is all about. Since the inception of the program, we have had over 20,000 reports submitted to AASAP. These are all in a database which is continuously updated. This database provides us trending information on particular subjects. It feeds, if you will, our training syllabus and our other functions of managing our pilots. Every six weeks, we take the data from the reports that have come in, and we circulate those reports and descriptors of incidents to all of our line pilots because we believe that line pilots, too, can learn from the other -- from the incidents and -- and events that others have been involved in. It is a self-reporting program. It is intended, as I said, to identify and correct safety concerns. Once a report is submitted and accepted into the program, and not every report that is offered is accepted -- if a pilot - - I'll use a pilot example. If a pilot has intentionally violated the rules and the Federal Air Regulations, that report is not acceptable for consideration under the AASAP Program. Secondly, if the FAA is aware of the event and the incident from other known and public sources, an AASAP report is -- is -- is not appropriate and is not accepted. We have found in the five or so years of this program that we have been able to -- to get a handle on improvements in all operational areas. We have characterized the -- the database and tried to create the top 10 event categories for our own focus in our training and in our surveillance of the airline. Operational distractions are Number 1. Rushing to comply, Number 1. Automation dependency, Number 3, and situational awareness, Number 4, just to give you a flavor of the types of things that have come out of our database work and our trending. Let me now give you some specific examples of some improvements that have come directly out of the input of the AASAP Program. First, we've been able to make operating manual changes across all of our fleets. One of the problems we dealt with early on in AASAP was to try to understand why our pilots, particularly flying the Super 80 aircraft, were having difficulty maintaining the assigned altitude given to them by Air Traffic Control. We found two -- we made two changes. First, we found a mechanical problem with the altitude knob on the -- on the glare shield that moves the assigned altitude into a window, if you will, for the crew to -- to observe, and, secondly, we adopted a -- a new approach that we call point and shoot, and that is when the crew is given a new altitude assignment, the pilot not flying will put it in the box. The pilot flying will physically point at it and affirm that he understands that there has been a change in the assigned altitude. We have put that across all of our -- all of our fleets. We have likewise adopted that point and shoot approach to changes in our FMC computers on board the airplane, which is crucial that both crew members understand the changes have been made in the FMC input. The second example is a change in our policy on position hold. We had an event in which one of our aircraft was cleared into position hold, and another carrier's aircraft flew very closely over the top of the airplane. We believe after looking at this in some detail, because of the report that we got in AASAP, that we were being held too long in a position hold and not given clearance. So, we now forbid our aircraft to take a lengthy assignment in the position hold at our airports. We think that's a prudent thing to do. The third example is the situation in which we have an engine failure. At slow speed, the rudder will be confined. There is a technique to kick the opposite rudder to free that rudder. This is a manual change that has come directly out of AASAP. We've made policy changes for both pilots, dispatchers and mechanics, including some additional work and guidance on thunderstorm avoidance, decision-making and crosswind approaches. In the area of training procedures, we've done a lot more work because we now understand more clearly the issues of distractions and so forth, human factors and safety. We've used the AASAP incidents as feed to our loft design, so that we actually fly things that our pilots experience, and we use the items that we learn through AASAP as -- as briefing items. In the air traffic area, we feed these -- these topics to -- to ATC by way of the FAA member of the evaluation committee. We were able to deal with a runway incursion problem in El Paso that was observed by several of our crews. That brought about some additional painting and identification and the threshold of two convergent runways. We have identified problems at particular airports on the arrivals in which our enhanced GPWS suggests that we are being brought in too low for the approach, giving an alarm. This was the case in an airport at Middletown. Finally, we identified a very dangerous situation that we worked with Air Traffic Service to modify the air space in an area that was being used by parachute jumpers, which is a very incompatible use of air space. But we would only know about those almost-kinds of situations by having the ability of our crew members to step forward, and, finally, we do on occasion determine and detect that additional proficiency for individual crew members is an appropriate strategy, and we take action on that basis. I indicated that I would talk more about the flight safety review process. This is the interface of American Airlines and the Allied Pilots Association. There are two committees that are very important to -- to providing data and helping us manage our operation; that is, the Training Committee and the Safety Committee. These have been very -- historically very strong relationships with the company, with open doors. I don't think it's any secret that we have had some economic differences with our pilots union. Unfortunately, this drove, if you will, a closing of some of those doors. The relationship deteriorated. The flow of information was reduced. We were not getting the benefit of -- of the APA Safety Committee and Training Committee's input to our process. We believe that that is an essential part going forward. As the senior leader in the Operating side of the airline, I am committed to restoring that relationship to a very positive role. I believe that the union's leadership and board are as well. They have assured me they are. We are rebuilding that consensus, and I expect that as we go forward, this is going to be a tremendously-important and positive process for American Airlines and for our crew members. So, I think we are -- we're going to put the past behind us and go forward and mine that data very -- very well. That is all that I have to offer on our systems, Mr. Byrne. Q I'd like to follow up on two general areas. One is on the reporting systems, the AASAP Program, and the second area is the internal audits. As far as the AASAP Program is concerned, how many personnel from American specifically are assigned to the AASAP Program? A One part of my description probably didn't -- didn't explain very well how it works. When a -- when an AASAP report is -- is put forward, there -- there is a three-party meeting that takes place every week. It involves the FAA, our CMO office, the Allied Pilots Association, and American Airlines, and the three parties look at the submitted report. They decide who are the appropriate people at APA or FAA or the company to go and investigate the details or to take action to -- to change or correct things. So, that meeting takes place, and those are three individuals that probably spend on the order of half a day a week. In addition, within the company, we have about two and a half people that are full time kind of running the program continuously. Q In terms of the volume of reports that come in, Captain Griffith testified in July that approximately 3,500 reports come in specific to the AASAP program on an annual basis, is that correct? A That's correct. Q And for the FlightNet System, which encompasses, I guess, both AASAP and the OF25 Program, he testified between about 18,000 and 21,000 reports come in annually? A That's correct. Q And could you just back up and describe the OF25 Program? A The OF25 Program is what I described as the FlightNet Reporting System. It is a more traditional debrief system for our flight crews to use. As you indicated, 18 to 20,000 per year. It encompasses or solicits general comments about the operation, but it also is a routine reporting mechanism to report to the company some 21 specific events that may or may not occur out in the day-to-day operation. Q Are flight diversions part of those OF25 criteria? A Yes, they are. Q What happens to OF25 reports that are in that category of flight diversions? A We look at those reports for the purpose of understanding why the diversion took place. Was it handled correctly? It is much more an informational thing than an action item, but we want to know what the frequency of our diversions are. For instance, if we were to see a large number of diversions to one small city, we'd worry a lot about that because of the overload on that small city and their ability to do a good job, both for the crew and -- and our passengers. So, it's really kind of a visibility kind of a reporting system, so that we know what's going on in the system, and we trend it. Q And my understanding is the AASAP Program's synonymous with the OF25 Program when a captain submits a report of diversion. Is that -- how is that information held? A That's held in the OF25 system, I believe under the cockpit crew member's name. Q How long is it held in the OF25 system? A I do not know. Q Are captains tracked for diversions? A Absolutely not. Q Moving on -- moving back to the staffing and how the AASAP reports are evaluated, with 3,500 reports coming in, and recently I read in a newspaper article, approximately a 150 reports are evaluated each week by the ERT, Event Review Team, how do you ensure that specific nuggets of information or safety-critical items are not being missed? A I -- I think the number of unduplicated reports suggests a somewhat lower number. It's not unusual, and I'll just use one example of, let's say, an altitude excedance, that we will get an AASAP report from both the captain and the co-pilot. So, within the 3,500, we really have two about the same incident. A lot of the reports fall into known categories of altitude excedances, misreads back leading to -- to difficulties, and we can categorize those fairly quickly. The unique and -- and different ones come out of that reporting flow pretty -- pretty quickly and are dealt with by the staffs that are assigned those reports to come back to the ERT with feedback on what happened and what we're going to do about it. So, while we have just the three kind of clearinghousing all of those reports, there's a large staff that gets the reports all over the company, including our Airport Managers, who have an obligation to come back and -- and tell us of the investigation and the results. Q And you mentioned the top 10 issues that are coming out of the AASAP Program. Would you elaborate on what is meant by "rushing to comply"? A Rushing to comply, as I understand the concept since I'm not a pilot, I'm a little less able to articulate that -- that concept very well perhaps, but it is the -- the concept that -- that a crew member or a crew is asked to do something, and without thinking it all the way through will just simply execute the request. It's termed "rushing to comply". Q And who's asking the crews to do something? A Well, my understanding is it's typically a combination of Air Traffic Control perhaps giving them instructions or (2) just the normal course of operating the flight gets into this rush to comply process. Q And you mentioned one of the products of the AASAP Program was new thunderstorm guidance procedures or new thunderstorm guidance. When did that come into place? A That is in -- in work at the present time. Q So, it was not in place at the time of the accident? A I don't believe so. That -- that isn't to say we didn't have any guidance, but we're redoing that guidance. Q I'd like to turn now to the Internal Audit Program. In general terms, not specifically related to any accidents or incidents, how frequently are these -- is this an on-going evaluation program? Are they time based or event driven? A This is a continuous internal audit process. It is a true audit in the sense that what we are looking at is -- is compliance to the FARs and our operating specification as approved by the FAA. We are looking very narrowly at are we doing what we said we were going to do in our ops spec, and are we within compliance of all the FARs as -- as we know them? That process cycles through every operating department every six months. We then pull the results of that together. We sit down at my level and review each department's performance. We then go to our Certificate Management Office at the FAA, and we share our findings with them as to what we have found and what we're going to do about it. So, it's continuous in six-month increments. Q And that's each department, each operational unit, Flight Operations? Training and Standards is conducting their own audits? A Dispatch, Weather, Flight Training. Maintenance has a huge part of that process. Our Flight Service people that manage our flight attendants. We've tried to get that into as many of the areas of the company that have an operating and safety impact. Q Who is the point person within each department that's conducting these audits or responsible -- A We have a coordinator assigned in each department to make sure that the -- the analysis and the audit is -- is conducted, conducted properly, and then assembled into a -- into a report. Q At what level is that coordinator? A Usually a manager or managing director, one step below the vice president level. Q The scope of these audits, how often does it go beyond regulatory compliance? A These particular audits are not intended, and I don't believe they -- they go beyond regulatory compliance or compliance to our operations spec. We use other processes to get at some of those issues. Q And the other processes are? A External audits, general management reviews and performance reviews. Q Who conducts the external audits, other than the FAA? A Various folks, and in a minute, I -- I will kind of give you some detail on that, if I might. Q Okay. DR. BYRNE: I guess at this point, I'd like to pass the microphone to Mr. Feith. BY MR. FEITH: Q Well, it looks like it's still morning. Good morning, Mr. Baker. A Good morning. Q You had talked about several things, and let me just back up to one point that you had just made regarding the -- I believe it was the guidance about weather that you had just spoke of, and that it was being revamped, and you said that that had come from the OF25 and AASAP input? A Yes. Q Okay. Can you be specific as to what kind of reports came from the AASAP Program or the OF25 Program that necessitated that guidance change? A I don't think I'm qualified at this point to give you a detailed understanding of specifically what drove it, but a collection of -- of those input prompted the ERT to suggest to our Flight Management that -- that some modification was appropriate. Q As far as that same program, the OF25 and the AASAP Program, do you have any understanding about any other anomalies or deficiencies in the organization with regard to crew pairing? We had looked at the crew pairing issue with having the chief pilot with a new first officer. Have you seen any kind of crew pairing issues come through the AASAP Program? A No, I have not. Q How about with any other training issues, such as FMC training, or any particular training with the MD-80 aircraft? A Not with the MD-80, but as I indicated, one of our top 10 items that have come out of the trended data in AASAP is what we call "automation dependence". That relates more to the FMC- driven aircraft, 757 and above. But I have -- I'm not aware personally of any direct input relative to the Super 80. Q And then using the OF25 and the AASAP Program, you're using that system to monitor trends and that kind of thing, to identify deficiencies or anomalies that you can take corrective action with. Given the fact that the Chairman had made a statement earlier regarding American Airlines, and during his tenure, American has had several accidents and some serious incidents, have you seen anything that -- that would lead to a systemic problem from your perspective through that program to identify the specific target areas that needed to be corrected? A Well, I think the -- I think it's fair to say that the top 10 trended issues coming out of AASAP in particular represent the most rewarding areas for us to concentrate on to avoid a systemic problem. That's where our resource ought to be -- be focused and changes made as we understand that there are better ways to do things. I think it's very, very helpful to us to be able to get out of that data those kinds of trended areas, and we work those very hard, and we'll continue to. Q As far as -- can you give me a characterization of your relationship with the FAA? We -- we heard the FAA's perspective yesterday, and the fact that they had some manpower issues and that kind of thing. But could you just briefly describe your relationship with the FAA? A Well, the FAA is a critical part of -- of our Aviation Safety Program. We take the view that the FAA has the resources and, most importantly, the expertise to bring to American Airlines additional input on things we do well and things we don't do so well. So, we have a very open door with the FAA. We -- we -- we listen attentively to what they say. We are absolutely committed to -- to ATOS as a process that can help us provide more input and more data to identify problems. So, the -- the FAA is really a member of the Safety Team at American Airlines in every respect, and they're very important to us. Q During the course of the investigation of not only this accident but previous accidents, typically we're made aware via different sources about concerns that on-line pilots have within the organization. We've talked about fatigue in this hearing and in this accident, and I just wanted to know if you're aware of any AASAP or OF25 Program issues regarding to crew fatigue, and, if so, what measures were taken to accommodate those or -- or fix those? A I don't think I can -- I can comment as to the presence of fatigue within AASAP or the OF25. Fatigue is -- is something that any one who runs big machinery in challenging areas needs to pay attention to in our view. We have recently reaffirmed our fatigue policy that permits any one of our crew members to take him or herself off duty because of fatigue with no consequences adverse to that employee, and we think that is a sound policy, and -- and it has been used, and that crew member will never hear from us. Q As far as taking this one step further in a different vein, we know that the Department of Transportation does track on-time performance of the carriers, and given the fact that this is a very competitive market, do you see, especially now in your new position as Vice Chairman, the fact that if we have a late-arriving airplane, and the consequences and the publication of that on-time performance, do you see that driving any of these flights to be completed? A No, and if you look at the -- the incident particulars at American Airlines since 1995, in almost every case, these operations were woefully late before they ever got in the air, and -- and, so, that the dependent -- the arrival dependability was -- was certainly not a factor. I don't believe that American's professional pilots are ever going to make a trade-off of being on time versus the safety of the operation, and we certainly would not condone that trade-off or that approach either. Q Given the facts of what we know thus far about this particular flight and some of the things that had been said early on in this investigation by, I guess, some American -- well, one particular American Airlines person who is no longer at the airline, can you just characterize for me and for the rest of us what -- what your assessment may have been as to the captain flying into Little Rock that particular night under the conditions that -- that we're talking about here? We had a two-hour late flight, and going into an area where weather is a -- is a very big consideration in the decision- making process, and this captain's continuation into an area of weather that, based on some testimony and witness interviews, was very questionable at the time. A Mr. Feith, I -- I am not a pilot. I have not been trained on the Super 80 or nor American's procedures. It would be grossly inappropriate for me to register any kind of opinion on Captain Bushman's decision-making that evening. I remain very optimistic that through this process that we're involved in here today, that we, too, will get a better understanding of all the factors involved in this tragic event. But I'm -- I'm not qualified to make that kind of a judgment. I'm sorry. Q Let me clarify then. Given the fact that in the position that you held prior to now being Vice Chairman but being the Vice President of the Operations side of the house, and given that this falls under your control, the purview of your control, as far as the feedback that you've received since the accident, don't you have enough information to make a judgment or at least an opinion about the captain's decision to fly into Little Rock that night? A I -- I certainly have gained a clearer understanding of some areas in which American Airlines will -- will address and improve. The human decision-making, I -- I have no opinion at this point in time. Q Okay. Thank you, Mr. Baker. BY MR. EICK: Q Mr. Baker, I have a few questions for you. Have any of your managers in the Systems Operations Center brought to you any reports or requests for additional support equipment regarding weather or reported any other deficiencies that they believe would affect their operation role of safety and efficiency? A No. We are -- we have been very aggressive in innovating, if you will, in the -- particularly in the weather area. We have stepped forward on several occasions with our own funding for a joint program with the University of Oklahoma and close-in weather forecasting, called HUBCAPS, which has been very successful. We have cooperated with the FAA on a program called ITWIS. As I think you heard yesterday, the airlines, including American, has a standing and very strong request that we get Doppler in our Operations Center. I'm not aware of any request for technology or -- or data that -- that we haven't fulfilled. Q And that request for Doppler data specifically is the Terminal Doppler Weather Radar? A That's correct. Q Does the American Systems Operations Control Center have access to the Internet at their work stations? A I can't be absolutely sure. I believe so, but I'm -- I'm not absolutely sure. Q One of the follow-up questions related to that is, there are some weather support pages on the Internet, and one of the questions related to that is, has American Airlines gotten FAA approval for the use of that data? A I don't know the answer to that. I'd be glad to furnish it, though. Q I think that's all the questions I have. Thank you, Mr. Baker. A Thank you. MR. EICK: Mr. Chairman? DR. BYRNE: Mr. Chairman, I still have one or two more questions in this area, then we'd like to let Mr. Baker continue with his presentation, with your concurrence. CHAIRMAN HALL: Well, that's fine. We're an hour and five minutes, and -- but I want to be sure we take as much time as -- as we need with Mr. Baker, but let's -- let's move. DR. BYRNE: We'll expedite. BY DR. BYRNE: Q Mr. Baker, how do you evaluate individual bases? I'm sorry. Individual bases? A Yes. Q Crew bases? A I am not directly involved in the -- in the management of the bases. Our Flight Department structure deals with that. So, I'm not qualified to answer that. Q And in terms of the internal audit program, would you characterize or describe for us its relationship to voluntary disclosure? A There's a relationship, although of a minor nature, that goes like this. If, in the process of conducting the audit, we were to find something that was clearly a violation, in addition to reviewing that with our Certificate Management Office, we may voluntarily disclose it under the FAA's program. Q That's all I have in this area. If you could continue with your presentation now, addressing American's actions following incidents and accidents? A Thank you, and I'll try to expedite this, Mr. Chairman. CHAIRMAN HALL: Oh, no, no. This is what we've been waiting on. THE WITNESS: I'm feeling ill. American obviously takes its performance and particularly the fatal accidents quite seriously. As I've -- as I've tried to describe this morning, we are constantly trying to improve that margin of safety at American Airlines and throughout our operation. The safety performance of American and the airline industry in this country is at an extraordinarily high level, but -- but we can never assume that we have reached a satisfactory level until we never ever have a fatal accident again. We were very proud of our performance for the period from 1979 to 1995, in which we operated the largest fleet of jet aircraft in the world without ever having a fatal accident, and then our corporation literally was rocked by two accidents, first in '95 and then in 1999, here in Little Rock. We have tried to work very hard to understand those events and to garner from that as much as we can to avoid a recurrence, and I'd like to review the specific approaches -- CHAIRMAN HALL: Mr. Baker, the only -- and I -- first of all, let me applaud you all on your record, but is there some reason the Roselawn accident is not listed on there? THE WITNESS: No. That is -- that was an American Eagle event. Simply a bookkeeping issue of American Eagle versus American Airlines, but -- but it had -- CHAIRMAN HALL: It had your paint on the skin of the plane? THE WITNESS: Absolutely, and we bear full responsibility, and we worked that -- that event. CHAIRMAN HALL: And you all have done, you know, a lot of things that were -- that the Board had concerns about, the FAA had concerns about. American's been very aggressive in following up on those, and I compliment you, but I just had noted that was not on the list. THE WITNESS: Thank you. Following the Cali, Colombia, 757 accident, we thought it would be appropriate to -- to do an intraspective assessment of our entire operation. We set out to do a comprehensive flight safety review. It involved the Allied Pilots Association and a lot of their folks as resources, the FAA. We undertook an 18-month review of every single part of American's operation that we could look at, looking for suggestions and opportunities to improve our operations and be safer. We even went to each of our crew bases and held meetings with our line pilots. We got some 3 or 400 suggestions from those line pilots of things to look at. Over the next several months, we boiled all this down into some areas in which we were going to focus, and those are in the human factors area. Division-specific operations, which is a little bit technical, but our pilots are qualified to operate in certain parts of our route system which represent different challenges that need special training. For instance, those that operate into Central and South American need some training in addition to what a pilot flying in the domestic United States would receive. We examined advance technologies that would be available to help us improve our safety margin. We looked at our structure and our procedures, and we examined the concept of FOQA as a -- as an additional process. Let me give you now some direct examples of things that came out of that lengthy and comprehensive process. The first thing we stumbled into was a piece of technology that had been developed by the Allied Signal Corporation, which is name-enhanced, Ground Proximity Warning System. This is an on-board device available for our large commercial aircraft that couple the location of the aircraft relative to the ground and presents to the crew a real-time database of the terrain in and around the aircraft. This had been a conceptual idea that had been developed by Allied Signal, but they could never frankly get anyone very interested in it. We got interested in a big way. We helped them certify that technology, and American Airlines led the industry in committing to equip every one of our aircraft, and we're getting to the end of that installation process now. The second important thing we discovered in examining the accident in Cali, Colombia, was that perhaps our crew members needed some additional work in how to recover an aircraft that got itself into an unusual attitude, very nose high, very nose low. There are specific techniques of flying an aircraft out of those precarious positions. While lots of our pilots come out of the military where that is standard training, to an increasing degree, we have regional airline and corporate jet pilots joining us. So, we committed to a one-day course for every pilot at American and an on-going program on a recurrent training basis of these technologies and approaches to maneuvering aircraft out of unusual positions. Coupling an approach event at Hartford and Cali, we put together a non-precision approach working group to try to see if we couldn't come up with a standard approach to non-precision landings. There were some runways and airports that -- that we frankly evaluated and excluded as acceptable to American, and we have been working with the industry to bring this approach across the entire industry. Finally, we have asked Jepsen, who is the primary supplier of the reference materials that our pilots use, the so-called plates or charts, to put the terrain in and around airports on those charts. That's not completely done, but we've made very good progress there. Following the tragic accident here in -- in Little Rock, Arkansas, we took a little different approach to again see if we couldn't organize an approach to -- to learning from -- from what had happened. We had used basically internal resources with the help of the FAA in 1995 and '96. We thought it was appropriate to bring some outside third parties into the process, and, so, we embarked on a -- on a three-phase, if you will, approach or concurrent evaluations during the period August through November. The first of those was an evaluation of our flight training, our flight organization, and fundamentally the way we were running the Flight Department. What we were seeking here was the input of other professional pilots who had managed airline operations at other places. If the safety business were simply a case of plugging in 10 specific things that we all agreed on, we would have done that long ago, but in actuality, airline safety is a process of improvement continuously of the margin of safety, and we wanted the benefit of outside eyes looking at how we did business. So, we -- we started with -- with that evaluation. Almost concurrent with that, the Allied Pilots Association leadership came forward, indicated that they would very much like to participate with the company in a similar approach. We agreed with that. We need -- we saw no problem with another set of eyes, and, so, we launched a second vendor, again experienced airline command pilots, looking at the operation, almost the same subjects that the first group had looked at, but a second set of eyes were certainly welcome. Finally, we started the third leg of this concurrent evaluation process with the FAA and the APA in what is known as the "systems analysis team". That was a process involving all three parties. They took a 120 days. The basic process is to look at what had happened during the last several years at American Airlines in various events, why did those things happen, and then, finally, what are the intervention approaches that the airline and the FAA and APA could embark on to try to prevent a reoccurrence? Out of that process, and I -- and I -- and I need to stop here to -- to make sure that the record is absolutely clear. These were not compliance audits. These were evaluations, and these were really management consulting assignments to American and to the APA as to how we might do business differently in a positive way. We have taken all three of those efforts and unduplicated the recommendations which were in many cases very similar, and we've ended up with some 85 unduplicated recommendations. Interestingly, over half of those 85 recommendations came out of the joint FAA/APA and American SAT Program, a very productive part of the -- of the three evaluations. American and -- and the APA have sat and taken those 85. We have put them in -- in a priority as to both time line and the importance or safety impact, and we produced from that effort, which no one can probably read, but from left to right, across the bottom, are time lines. The first group is the first quarter of this year. The middle group, June of this year, and then, finally, a group of -- CHAIRMAN HALL: Victor, can you put the slide back up while he's talking about it? Thank you. THE WITNESS: Finally, on the far right, a group that will extend from July of this year to January of the year 02. The most important are on the top. The lesser impacts are on the bottom, and, so, now we have an agreed-upon priority of -- of how to proceed. We have submitted that list to the CMO. They have reviewed it. They are comfortable with our implementation plan, and in fact, we have taken this work and gone to Washington with our CMO individuals and presented this to the Flight Standards Group in Washington, and certainly following this hearing, Mr. Chairman, it is our plan to -- to share the detail of this with your staff as well as we did back in '95 and '96. So, we have -- we have learned some more things. I think it's fair to say we've got some work to do. We will be a better airline for it, and we think we've done this in a very straightforward participative way, and I'm now prepared for questions. BY DR. BYRNE: Q Mr. Baker, a couple of follow-up questions. The audit following -- in 1996, following the Cali and the Hartford accident, the Special Assessment Team -- seven areas that were evaluated. One of them you had up on one of your slides listing Operation Structures and Procedures. Did that -- did the team that was responsible for looking at that area also evaluate corporate culture or organizational influences on flight operations? A You're going back a little further than my memory can take me. I don't believe that it -- its emphasis was in the cultural area. There were some structural issues, but I -- I would frankly have to refresh myself to give you an accurate answer. Q And concerning the audit after Little Rock, Mr. Valentine yesterday expressed some concerns about the rapid expansion and the large volume of training on-going at the airline. Were you aware of those concerns from the POI before yesterday, and how are you addressing those concerns? Were they part of the audit? A Yes. I personally and the rest of the Flight staff has had an on-going dialogue with Mr. Valentine on that subject. He has raised some very specific issues that we are reacting to. We think it is -- it is clearly manageable. We -- we have put on the necessary resources to ensure that the training is -- is done properly and -- and completely. I think the FAA has -- has done some of the same but perhaps needs additional help in that area. I would point out that while we added two fleet types, new fleet types, to American's fleet in 1999, we are also in the process of retiring two fleets that will decomplicate the airline considerably. These are first-generation jet aircraft, the 7-2 and the DC-10. Secondly, back in the '80s, American grew at an even greater rate in terms of the number of new aircraft and pilots we took in. This is not a new phenomena in the industry, and, finally, the growth on a relative basis in the industry is often much higher than it is at American. Q Thank you. BY MR. FEITH: Q Mr. Baker, in looking at this one slide that you had highlights by category, one of the -- MR. FEITH: And I don't know, Victor, if you could put that last slide back up? BY MR. FEITH: Q In the lower left-hand corner, there is -- one of these objectives is to improve chief pilot communication skills. Can you tell us what that is, and what prompted that? A I'm not sure I recall what prompted it, but what we have planned there is to send our chief pilots who run our bases to formal leadership education school on communications, so that they are good at communicating to their line pilots, standing on their feet to express themselves. They asked, I believe, for this kind of training. We have started that training already and have several graduates. Q Was there a problem? Apparently -- apparently there was or they wouldn't have asked for this, but do you have any kind of specifics you could give us? A I do not. I believe it came out of the -- the notion that some of them felt less competent than they wanted to be on their feet, and if we're -- if they're going to lead people, they need to be confident. Q Is this a requisite now for new chief pilots being selected in the position then? A I think it will become part of -- of the training that -- some of which or most of it will be given after they're selected, but we think this is important training. Q The term "corporate culture" has been brought up. Can you just briefly describe for me what your belief is about corporate culture? A Corporate culture to me is -- is an articulation of the environment in which a company or a department or any kind of an organization operates on a day-to-day basis. It is really the atmosphere that the leadership of -- of that organization sets by example. It has various characteristics as to whether people feel good working there, whether they feel comfortable bringing problems forward, whether they feel they're being compensated fairly for -- for their effort and within the competitive marketplace. So, it's all of those factors that lead up to the basic question to an individual, and that is, is this a good place to work, and do you want to be here tomorrow? I think it's many, many factors, but -- but those are just a few. Q Do you feel that you in your position and in your previous position were plugged in enough into the system to get a good flavor for the corporate culture that existed at American Airlines not only this past year but over the last five-six years? A Yes, I do. Q Have you seen any kind of change? Has it basically stagnated? Is everybody happy? A Well, -- CHAIRMAN HALL: Well, now, Mr. Feith, hell, everybody isn't happy at the NTSB. MR. FEITH: Yeah. Well, you didn't ask about the corporate culture at the NTSB. CHAIRMAN HALL: Well, -- MR. FEITH: I'm just seeing if there's a better place to work. CHAIRMAN HALL: Well, now, if you're going to do that, you need to do that on your own time. THE WITNESS: You and I will talk later, Mr. Feith. MR. FEITH: Okay. THE WITNESS: I -- I think the answer to that must be taken in its proper context. The airline business is a tough business. We don't make any money. This is very cyclical. If you look at the industry over the last 50 years, we basically have been a break-even. Many years, we make a lot of money, and a lot of years, we lose a lot of money. We uniquely combine large numbers of capital assets called airplanes that cost lots and lots of money with a very high labor concentration. That's a very unusual mix in corporate America. That means basically that we must run that business carefully and diligently, and we're working on one percent kind of margins. That makes it a -- a challenging business for all of us to be in, and we have to keep, of course, the priorities straight of safety, customer convenience and the other earmarks of our product. That -- that, I think, causes the employees of an airline to be -- to have to be aggressive, to be intelligent, to work hard, and in our -- in our environment in which we're constantly re-evaluating and making changes, it -- it's a fairly turbulent environment. I don't think there's anything peculiar about American Airlines in that regard, but I think that's a fair assessment of the industry. BY MR. FEITH: Q Well, I wanted to get -- get that -- that flavor only because I know that Mr. Carty had made a statement in a speech regarding corporate culture at American Airlines, and he had said that one of the strategic objectives of American Airlines was to create the industry's best corporate culture. My question then is, what tools or what strategies are going to be used to improve a corporate culture that we know is volatile in any organization, NTSB, American Airlines or anywhere else, but how is that objective going to be attained, especially -- and the second part is what tools are you going to use to effectively evaluate management in attaining that goal? A Unfortunately, there are no quick, easy answers to those issues. The things we are paying attention to in hopes that they will deliver a -- a positive culture for our employees which then quickly reflects to our customers, and if our customers are happy, our shareholders are happy. Some of those issues start with the leadership. Who are we selecting to lead the company all the way through the management structure? Do they embark upon their jobs with the right attitudes and expectations of their people? Are we giving them the proper leadership training or are we simply giving them a badge and throwing them out there, saying you're now the boss? I think the -- the discussion we had about our chief pilot communication skill is an example of the recognition that we've got to have the right skills in the hands of our -- of our managers. We've got to deal with the policy issues of compensation and the way we treat people aggressively but clearly with equity. People are very sensitive to whether they think they are being treated equitably, one to another group. So, it's a constant process of evaluation, hiring the right people, giving them the training, deciding on policies that affect people in a humanitarian and equitable fashion, and then continuing when things go wrong to deal with those objectively and -- and forcefully and aggressively, and I think if we do some of those types of things and insist upon it all the way up and down the line, we're going to have a very positive culture at American Airlines. Q Thank you. Switching gears back to the specific area of flight training and personnel performance, with regard to trend monitoring for deficiencies in individual crew members' performance, does American Airlines have a system in place to do that? A Well, it comes in a number of flavors, I think. AASAP is clearly an opportunity for -- for that kind of input, and we -- and we do get that input. The evaluation during recurrent and upgrade training is -- is another process that is in place along with our check rides. So, I -- I think we have all of the traditional tools, plus AASAP in place, to deal with that. Q And in keeping that type of record, the training records and things like that, do you record negative as well as positive or just positive as far as check ride passing and things like that? A I'm going to have to defer that. I don't know the detail of the recordkeeping system. Q Okay. The reason I bring that up is looking at getting meaningful qualitative data to make assessments as -- as management, looking for trends, given that you have a very complex AASAP Program and OF25 Program, is American -- do they have the proper tools in place to be able to make those evaluations? A I think so. I think we do as much or more than -- than we are required to do under -- under the FARs, and we do as much or more than -- than our -- than the other big airlines. We are always open to suggestions in that area and opportunity to learn. Q And this may be a difficult question to answer, but you had said that you -- you believed you were fairly plugged into the culture that exists at American Airlines. What types of feedback do you receive in making sure that the pilots that don't have the feeling of subtle pressures that they have to complete flights, that they -- you know, that they have to fly on the edge of fatigue and things like that? We've received, as the NTSB, excuse me, numerous people calling us or providing us letters that while we understand your position, and -- and yours is the company, I mean we continue to get this feedback that people are flying tired, that American Airlines pilots are still flying on the edge of fatigue, and that management isn't listening to these pilots who say, you know, management's saying one thing for the cameras, but in fact, this is the reality. How plugged in are you, and what kind of feedback, I mean, are you getting that -- that's different than what we're getting because we're basically at opposite ends of this table and trying to address this problem? You said that you are revising the fatigue issues at -- at American. Can you just -- I don't know if we've described that or not, but if you could just briefly describe what you're doing to correct that? A Let me go back over that, if I might. Q I -- I -- pardon my ignorance if I've -- if you've already done it. I -- I just don't recall. A Well, very simply, we -- we revised and re-emphasized our fatigue policies. As you know, the reserve rest issue in the domestic operation has been one of controversy over the last eight months. The industry is now working toward compliance with -- with the FAR. Substantial impacts on the airlines, and while we transition to that, which requires us to -- to add considerable number of personnel, we put into place a fatigue -- a no-fault, if you will, fatigue policy, and we have said to our pilots in no uncertain terms, you feel you are unable to perform because of fatigue, then you're off the trip by your -- by your own request with -- with no recourse from the company, and -- and you'll never from us again. The proof of the fact that I believe that the notion that American Airlines pilots are routinely operating in a fatigued fashion is simply that the number of people availing themselves of that process has not increased. Q Okay. Let me just take a word that you said, and that is no-fault policy. Was there a fault policy? A No, but -- but, clearly, people that -- that would -- would fly part of a sequence and then decide that they were too fatigued were -- were asked by their -- by their base management what the issue was and what the problem was and how did it happen and so forth. We -- we've abandoned all that inquiry, and if they go fatigued, they go fatigued, and that's the end of the story. Q Does this carry on to the other side of the cockpit door, to the cabin crew as well? A I'm not familiar with what fatigue policies the flight attendants operate under. Q Well, if I butcher this question, then Don will bail me out, but you had made mention about -- and in one of your -- in this last slide you had talking about the revised thunderstorm avoidance policy, and Captain Lewis talked about this yesterday, and that was an increase, one day check -- for the check airman of one-day training on airborne weather radar and the interpretation of the results. Was that based on this accident at Little Rock that that increased training occurred, and who does that apply to? Captain Lewis had talked about check airmen. Shouldn't that training be filtered throughout the entire flight organization? A Absolutely, and that is clearly our plan. As I think you heard Captain Lewis or -- or one of the other witnesses say, our principal radar training has to be done in the aircraft to get the true techniques of operating airborne radar down properly. So, we started with our check airmen who are our principal teachers, if you will, in flight as they do line checks and upgrade training and so forth. It is clearly our intent to move on to all of our captains and first officers with this radar training. It is something that we have talked about for a couple of years. I think the situation here in Little Rock clearly made us simply make the decision and commitment to go forward immediately. Q Switching gears again, originally it was brought up during the early stages of this investigation about crew pairing with the management pilot flying with a new first officer. Do you have any opinions on that? A No, I don't. Q Okay. That is all of my questions. I know that Dr. Byrne has one last question. CHAIRMAN HALL: Well, let's move on, Dr. Byrne. What is your question? DR. BYRNE: I'll make it quick. BY DR. BYRNE: Q Yesterday, we spoke about procedures and techniques and the differences between the two. The Safety Board raised the issue of the differences between procedures and techniques in its 1993 -- or its report on the 1993 DC-10 landing accident at Dallas. How can American ensure that techniques used by individual pilots are equally safe and equally tolerant of error? A Well, that's a very far-reaching question, and I'm not sure I'm -- I'm the one that ought to answer that. It's much more in line with someone more familiar with the specifics of flight training than I. But I -- I think from my -- my perspective and my focus in the corporation, that's got to be an item that we insist that our flight management pay attention to, but I'm not qualified to -- to deliver any information on the specifics of how you would do that. Q Thank you, Mr. Baker. DR. BYRNE: Mr. Chairman, this completes my questioning. CHAIRMAN HALL: We'll move to the tables. Little Rock National Airport? MS. SCHWARTZ: No questions, Mr. Chairman. CHAIRMAN HALL: The Little Rock Fire Department? MR. CANTRELL: No questions, sir. CHAIRMAN HALL: Federal Aviation Administration? MR. STREETER: No questions, Mr. Chairman. CHAIRMAN HALL: The Boeing Commercial Airplane Group? MR. HINDERBERGER: No questions, Mr. Chairman. CHAIRMAN HALL: The Allied Pilots Association? MR. ZWINGLE: We have no questions, Mr. Chairman. CHAIRMAN HALL: The Association of Professional Flight Attendants? MS. LORD-JONES: Actually, I have one. INTERVIEW BY PARTIES TO THE HEARING BY MS. LORD-JONES: Q Mr. Baker, throughout your testimony today, you have mentioned the AASAP Program. Lately, there has been references to the AASAP Program as an immunity program or a whistle-blower program. Would you mind commenting on that for us, please? A Just two short weeks ago, President Clinton held a news briefing in the White House in which he endorsed and -- and the FAA announced their intent to execute an advisory circular that will permit the AASAP concept to be implemented at the rest of the U.S. industry's airlines. It has taken us five years to get to that point, and we are extremely pleased that he took that action. Unfortunately, the media in reporting that activity suggested that perhaps this was an immunity or -- or a whistle- blower program which -- which I find a very disturbing and -- and inaccurate characterization. This program is intended to get information out in the open that otherwise we would know very little about. The ultimate immunity in the safety business and aviation is silence, and without AASAP and its companion programs, FOQA, that we will be developing, we will have to live under silence, and that's the ultimate immunity, not this program. MS. LORD-JONES: I have no further questions. Thank you. CHAIRMAN HALL: Very well. The Allied -- let's see. No. I'm -- I'm -- back to American Airlines. No. The National Weather Service? I apologize. I lost track. MR. KUESSNER: No questions, sir. CHAIRMAN HALL: Okay. American Airlines? MR. McFALL: No questions, sir. CHAIRMAN HALL: Very well. Mr. Sweedler? MR. SWEEDLER: Yes, Mr. Chairman. INTERVIEW BY BOARD OF INQUIRY BY MR. SWEEDLER: Q Mr. Baker, I have one question I'd like to pose to you. There was some discussion yesterday about the possible use of additional data to look at potential problems before they become accidents. I was very impressed with your AASAP Program and the use of this program by -- now by the rest of the industry, hopefully, but as you -- as you're, I'm sure, aware, there are some airlines, primarily in Europe, that use recorded data as part of this program to sort of augment what you -- what you have in AASAP, and, for example -- I'll just take one airline as an example. KLM has a program that they've devised between -- with -- with the cooperation of -- of their pilots obviously, where every time a large aircraft returns to Amsterdam, the data that's recorded is downloaded and is run against a program just to see if there are omissions or -- or deviations from standard procedures, and anything they learn from -- from -- from that program is obviously disseminated to the pilots for -- so they can learn from this, and they've come up with some good examples similar to what you have, and what I'm -- what I'm asking you is, have you considered using this to augment your AASAP Program or if you -- if you haven't, can I sort of ask you to take a look at this and see if it's something that you could be utilizing? A Thank you very much. We -- we consider AASAP really to be a Phase 1 of a FOQA program as they're commonly called. We, too, share your enthusiasm for the benefits that can come from on- board recorders that are occasionally dumped and so forth. We and the Allied Pilots Association have committed to each other to work in that direction as rapidly as we can get there. It is not as easy a subject in our environment as it is in Europe to get into place, but we share the enthusiasm and the importance of it, and I have no doubt that American Airlines and the Allied Pilots Association are going to get that done. Q Thank you, sir. MR. SWEEDLER: I have no further questions, Mr. Chairman. CHAIRMAN HALL: Mr. Berman? MR. BERMAN: Thank you. BY MR. BERMAN: Q Mr. Baker, we are looking forward to meeting the new Vice President of Safety, Mr. Hern. We'd like to know him, and then hopefully we'll never have to speak to him on an urgent basis. Could you please also fill in for us the names that were on those boxes going down through the Flight and Operations and Maintenance Safety people working for Mr. Hern? A We'd be glad to -- to fill those boxes in as we go along. That organization, as you can imagine, because, as I indicated, it represents a large expansion of resource, is yet to be populated, but we'll give you those names, and as that organization evolves, just how it lines up. Be glad to. Q Okay. Are any of them populated right now or is it -- A Yes. On the -- on the Safety side of that organization, I believe that Captain Scott Griffith will be the principal leadership player there. Q For Flight Safety? A Yes. On the Operational Audits, Mr. McFall will lead that group. The Security Group is currently managed by Mr. Larry Wonsley, and he will continue to do that. We have not selected an Environmental or Compliance leader at this point. Q And Maintenance also is open? A Maintenance? Q I think you had a Maintenance Safety Branch somewhere there. A Airworthiness managing director will be on the Tulsa staff principally. I do not have that name. Q Okay. Thank you. Does American Airlines track flights that are close to the 15-minute DOT limit for timeliness, the ones that are kind of in danger of falling across and being classified late, and does American inform the pilots when flights are in that status? A Not on a real-time basis. As we go through the month, which is the DOT recordkeeping window, and we have flights that are performing woefully below the performance expectation, we try to work with the stations to get the airplanes out on time. We let the crew member know that this is a DOT trip that hasn't performed very well, but on a real-time basis, as an aircraft flies this afternoon, we do not. Q So, you do let the flight crew know, say, on their release for the day that -- that a certain flight is a DOT-critical flight? A That -- that it is a trip that has not performed. We -- we -- we call it the "shame list", -- Q Hm-hmm. A -- and trips that are on the shame list, which is an evolving list every day, we will let the crew know that the trip is on that list and -- and for their information. Q And what do you expect them to do as pilots when -- A I think -- Q -- they see that? A -- it's just part of the situational awareness. We don't ask or expect them to do anything in particular but just be aware that it's -- it's a trip that hasn't performed well. Q Okay. Let me switch gears a little bit and talk about the decision you've made to separate training from flight standards. Prior to the Little Rock accident, had you taken a look at how other airlines organized themselves in this regard, identified any different practices or best practices in the industry? A It -- it is very complicated to try as hard as we try to -- to examine how other airlines do their business. We -- we -- we spend a lot of time doing exactly that. The problem we run into is we don't -- we see a box that is labeled similar to one we have, but the actual functions of those are very different carrier-to-carrier. So, yes, we have looked at the other carriers. We see a mix of approaches to how the Flight Department is organized and what apparent authority they have. We try to learn from that, but it is a very difficult process, in spite of our efforts to -- to know exactly how our principal competitors are -- are organized and doing business. Q Hm-hmm. A But we have seen different approaches to -- to training from executing the training syllabus to -- to checking and standardization. We were once organized as we are now with the separation. So, this is really a return to that approach of many, many years ago, long before I came along. Q Had this change been discussed after the accidents preceding Little Rock and before the Little Rock accident? A I believe so. Q And why wasn't the action taken before Little Rock? A Whenever you make those kinds of -- of large changes, it has lots of impact on -- on people and the way things happen. So, we -- we -- we are careful and prudent, and we try to build a consensus before we make that big a change. We were -- we simply weren't able to -- to get to the point of actioning that concept, but we are now of the view that that's a good thing to do, and we're going to do it. Q In the audits that you conducted and that the other parties conducted with you after this accident, did you get any specific data to feed into your organization's decision-making apparatus that would indicate to you that there was a specific reason to separate these functions? In other words, was there a problem identified? A Well, let me recall to you that -- that the -- two of the three evaluation approaches were -- were a process of garnering the views of very accomplished and senior aviation consultants. In both cases, they made recommendations in the direction that we have taken, but they didn't necessarily, as most consultants do, they don't justify their views. They just give them to us. But we took those views. We thought about them. We had thought about them in the past and became convinced that this was the right time to make such a change, and we've done it. Q Okay. Sir, you mentioned that distraction and rushing to comply were the top two kind of high-volume mass findings coming out of the mass of incidents reported in AASAP. A Correct. Q Certainly without prejudging anything, those are -- those are issues that need to be considered in this accident as well. What is American Airlines doing about these -- these high-volume issues that churn up -- I'm sure pop up across the AASAP Committee's desk many, many times on each meeting? A Well, they become briefing items. Let's take distractions as a good example. We -- we brief our captains. We have a magazine that -- that we put out approximately quarterly called "Flight Deck" in which we try to communicate with our captains and first officers about important subjects. That is an avenue for us to deal with that subject on our check rides, in our recurrent training, and -- and then, quite honestly, whatever we can do to eliminate the sources of those distractions. Frankly, and this is not a secret to anyone, the conflict that occasionally occurs in a large company between a union and management is -- is a generator of -- of distraction. So, to the degree that we can, along with APA, control that and keep it out of the cockpit, that's good. Q Hm-hmm. A So, it's a constant process, I think, through a lot of different avenues. Q I hear what you're saying, and I guess while the mass volume of reports that involve these factors may make it appear like those are the ones to treat quickly in AASAP process, maybe the message is actually the opposite, and these -- these things that keep repeating again and again and aren't -- aren't the rare unique finds are -- are perhaps a place to focus. Also, you're raising the -- the communication issue between pilots and management raises the next issue I was going to bring up. Was that specifically or particularly an acute problem at the time of the Little Rock accident? A I don't believe so. I frankly don't recall the source of that particular recommendation, but I don't believe so. Q Hm-hmm. Okay. Thinking back to the question I asked you a few minutes ago about the DOT time and informing the pilots when their flights are chronically a late on, I'm having trouble trying to identify exactly what you would expect a pilot to do or -- or what a pilot could do that -- that might not, you know, be subliminally or inadvertently giving them the message that they ought to rush to comply during that flight. What -- what's your thought about that? A Well, I think it is -- it is simply an element of situational awareness of many factors that we inform our crew members of. If -- if you look at the paperwork we give to a crew member, it is a lot of information about a whole plethora of subjects across a wide range that we think as a command pilot they ought to have the benefit of. This is just simply one of those -- those factors. We have never trained nor told our crew members or captains to do anything in particular but be aware of the fact that this airplane has -- or this particular flight has -- has not performed very well, and if you had a choice of -- of being able to help the dependability or not, we want them to know that. Q Has American given any thought as a corporate philosophy to -- to insulating its pilots from certain information that might have only potentially negative safety implications and no positive ones? A I'm not opposed to that concept. I -- as I sit here, I'm not quite sure how you do that, but -- Q Well, -- A -- conceptually, -- Q -- for example, not telling them which flights are chronically late? A Well, you first have to agree that that's a negative, but I take -- Q I said potentially negative, but I won't say it's a complete negative one, yeah. Was this flight, the accident flight, a DOT-flagged flight? A I don't -- I don't know. Q Okay. A That -- certainly that evening, it was over two hours late before it ever left DFW. Q Yes. It certainly was not going to cross the boundary to be on time -- A No. Q -- no matter what the crew did. Just in the global sense, based on the findings of all the post-accident audits, do you feel there is possibly a level of complacency in the Operations Departments at your airline that somehow managed to survive the -- the previous accidents and get up to the one we're talking about here? A I don't believe there -- there is a complacency issue at American Airlines. However, that is -- that is an issue that requires constant attention and focus at every airline. These -- these aircraft that we fly today are -- are so sophisticated and reliable, 99 percent of the time, the operation works very smoothly. So, complacency is a risk that we need to focus on every day, but I don't believe I've ever seen anything to suggest that complacency is a bigger problem at American versus anywhere else. Q Hm-hmm. We certainly applaud all the findings, all the hard work in these audits and the actions that the company's taking to -- to resolve the problems that were found, but the question still does remain, why wasn't the organization fired up in the same way before the accident and able to prevent it? A Well, I wish I -- I wish I knew what we could have done precisely to have avoided each of the events that -- that we've had. I can -- I can only tell you with some conviction that every one of them is unacceptable, and that we are committed as -- as a leadership team and every employee at American Airlines who pay a big price when these events happen, that we're going to work as hard as we know how with -- with the best of help that we can garner to not have another one. I don't want to do another one of these hearings, sir. Q Do you think you'll have the same number, volume and -- and level of activity of auditing every year on into the future as you did after this accident? A I think we're going to do more. As I tried to -- to show -- to show you, we are moving toward less internal auditing and more third party auditing for the purpose of creating an on-going process of -- of objective assessment of our performance against our -- our own rules and our own standards and those that are expected of us. Q That certainly gives the FAA and ATOS and anyone else looking at your company from outside a thing to monitor and watch on into the future, and it will be -- A And I think ATOS is -- is potentially going to make a dramatic contribution to aviation safety. Q Let me ask you just a very specific final question that was just passed to me. Point of clarification. Is there a new thunderstorm avoidance policy or are we talking about the training that is being given to check airmen? A Well, the policy is the same, and that is we do not enter areas of -- of thunderstorms. The training is in evolution, but I don't believe that it has actually been put into place. I'm probably not the right one to be precise on that, but that's my understanding. Q Okay. Thank you. No more questions. CHAIRMAN HALL: Go ahead, Tom. BY MR. HAUETER: Q I just have a couple. To follow up Mr. Sweedler's question about the FOQA Program, does American now routinely download quick access recorder information and review it? A We download the recorders in response to either an AASAP event in which it would be appropriate to understand the movement of the aircraft and so forth and any -- any events, such as a tail strike, turbulence. So, we have half a dozen or so event characteristics that drive us to pull the data. We do not in -- in a programmatic sense, as Mr. Sweedler indicated the Europeans do, pull recorders looking for excedances. We have -- we have not reached the point yet with the Allied Pilots Association where that -- that approach is -- is underway. Q Okay. And you made mention that you fly in the jumpseat a fair amount. Do you find the pilots to be very forthright and talking to you about their problems concerning the line or are they intimidated by your position in their cockpit? A Well, I don't think I'm a very intimidating guy, and they seem to tell me an awful lot of things that -- that they'd like me to hear. I -- I am very fortunate in that over the many years I've been with the company, I think I have developed the confidence of a lot of those folks, and they don't hesitate to write, call me at home, call me in the office, send me e-mails, and they talk freely on the jumpseat. So, I -- I usually walk -- my wife always complains that I litter the top of my dresser with little pieces of paper of all the things people tell me as I journey around the company, things that need attention, things that need changing, things that are problems, and our cockpit crewmen are professionals who -- who believe very strongly in what they're doing, and they want it to be right, and they want it to be safe, and they -- they tell me a lot. Q Thank you, sir. CHAIRMAN HALL: Mr. Clark? MR. CLARK: No questions. CHAIRMAN HALL: Mr. Baker, first, let me just observe and thank you for your presence. The audience may all know or may not know, but you are certainly respected as a leader in the aviation industry, an effective spokesman for your airline and someone who is called upon and very knowledgeable in the safety area, and the Board has long felt that the safety culture of an organization depends on leadership from the top. You need to have the culture, safety culture up and down the organization, but clearly the individuals in charge of the resources of the organization are the ones that can have the most impact in setting the tone for the organization, and I think it speaks in my personal opinion very favorably of American Airlines that you, as the Vice Chairman of the whole operation, have taken this time to come and -- and participate in these hearings, and I hope they will be of -- that what you learn here and see here will be of continued benefit to you in your responsibilities. I also applaud you on trying to develop new leadership at American. That's one of the situations in the Federal Government we're constantly challenged with, is trying to bring on new and able public servants to fulfill those responsibilities. Tell us a little about -- you put up a slide and talked about the growth of the system from 1979 to 1995. Can you tell us - - can you characterize that in terms of the number of either flights or the number of pilots, mechanics? How much growth was there through that period? THE WITNESS: Let me take you back to approximately 1982 when the first substantial growth at American Airlines and for that matter the industry took place. In the early '80s, American Airlines was fundamentally a domestic airline, an airline that employed approximately 34,000 employees, an airline that operated 240 or 50 jet aircraft. We saw as deregulation had occurred in the late '70s that the only way that we were going to survive the almost perfectly competitive business that the airline industry is, that we were going to have to grow, both domestically and internationally, and we set about that through the middle and late '80s and arrived at the Desert Storm period of 1990-91 having grown to an airline of something over 600 aircraft and probably 70 or 80,000 employees. We did not grow during those Desert Storm years when the economy was lacking. Our competitors did start to grow before we did, and, so, we again started to grow in 1998, and, today, we sit at approximately a 100,000 employees and 750 jet aircraft. We operate internationally to Europe, South and Central America, the Orient and North America. CHAIRMAN HALL: And how many pilots and how many mechanics do -- do you have that information on the top of your head? If not, I -- THE WITNESS: We've grown from approximately 7,000 pilots. I believe we're just over 10,000 pilots today. We are hiring pilots at the rate of 80 per month. That is the maximum that we can train effectively with our resources, and, frankly, that's -- that's as many as I want to bring on at one time. Mechanics have likewise grown commensurate with the fleet. They don't grow quite in the same relationship because the new aircraft requires fewer man hours in its initial years than an older one, but as a rule of thumb, Mr. Chairman, we apply approximately 14 man hours of mechanic time for every hour that an aircraft flies. Now, that's a blended number of the very oldest airplanes in the fleet and the brand-new 737 we took last week. As those airplanes mature and get more hours, the maintenance work goes up, and that, of course, is one of the reasons we're anxious to retire our older fleets, the 727 and DC-10. CHAIRMAN HALL: Now, does that -- do those numbers include the American Eagle system? THE WITNESS: They do not. CHAIRMAN HALL: And could you give us an idea of how big American Eagle is, and I believe it's under the umbrella there? It's under you and Mr. Carty? THE WITNESS: It is, indeed. CHAIRMAN HALL: Are those subsidiaries wholly owned by AMR? THE WITNESS: Yes. There's a fairly long history there that I won't take a lot of time on, but American Eagle is a large regional airline that operates basically in Puerto Rico, Florida, New England, the Chicago/Upper Midwest, the DFW area and the West Coast. It is the by-product of several years of merger, acquisition and consolidation of smaller regional airlines, but, today, with the exception of the operation in Puerto Rico, it operates as a single certificate, operating two aircraft, three aircraft types, the regional jet, the Saab and the ATR. It is about 220 aircraft in total. I do not know the employee numbers. CHAIRMAN HALL: The reason I ask that is that we have a growth in this country of the feeder system, of which American Eagle is an important part of that community, similar to the one we are meeting in today, and you had put up the slide that got my attention in regard to the major accidents, and I accept the fact that you all view Cali and this particular accident as sort of maybe major accidents that prompted you to action. During my tenure at the Board, as Chairman, the Board has, in February of '94, dealt, as you know, with the Saab 340 incident at New Rhodes, Louisiana. I was the member on scene in Marsville, North Carolina, which was an American Eagle Jetstream, 3379, with 15 fatal that occurred in December of '94. The Board has looked at in Grandby, Connecticut, a collision you're very familiar with with the major -- with American Airlines, the MD-83, that occurred in November of 1995. As I mentioned, of course, I was the member on scene at the American Eagle accident in Roselawn that we are familiar with with -- with a number of fatalities. In addition, the Board has investigated incidents that met the category for our investigations in American 1340 in Chicago, American 903 in Palm Springs, and just recently have finished an investigation on a DC-10 in Antiqua. So, the -- when we talk about the number of events, those are in addition to Cali and to the incident here in Little Rock, and -- and without getting into the details of the -- of the accident, I think and I feel that that has probably helped prompt the corporate leadership to take the action that you all have taken in these areas, and -- and I -- and I applaud that. I just have a couple of questions in regard to this organization and this information. How does this apply to American Eagle? THE WITNESS: American Eagle operates on its own certificate. It is what I would characterize as a sister company. We are increasingly sharing both resources and -- and human capital back and forth. Everything that we do at American, we will share with American Eagle and likely implement in an appropriately-similar way. CHAIRMAN HALL: And I mentioned that because I fly, you know. I'm one of your frequent fliers, and I have your miles, and when I buy a ticket on your airline, I don't usually pay any attention, and I may go out, and it's an American Eagle flight or it may be, you know, an American flight, but I really don't know usually till I get to the gate, and the colors are the same. The uniforms are the same, and I'm just wondering whether the safety system that's in place is the same. THE WITNESS: I think we share the responsibility for both certificate operations. We are committed equally on both sides and understand the public's perception, and, frankly, they deserve to have the best that we can provide on -- on both airlines. CHAIRMAN HALL: But as Vice Chairman, both of them are under your direction? THE WITNESS: They are, indeed. CHAIRMAN HALL: And do you jumpseat on American Eagle as well? THE WITNESS: Yes, I do. CHAIRMAN HALL: That's good. The issue of the chief pilot, would you kind of explain very briefly again for me, and I don't want to try to cover any of the ground that's already been covered, what is the role of the chief pilot? THE WITNESS: Are you referring to the corporate chief pilot or a base chief pilot? CHAIRMAN HALL: Well, the gentleman that was in this particular accident flight in Chicago, I guess we were referring to him as the chief pilot. What -- what distinguished him from the rest of the pilot community? THE WITNESS: Of the almost 10,000 pilots, they are assigned to nine crew bases across our domestic system. All of our crew bases are within the United States. Our two largest crew bases are at Dallas-Fort Worth Airport and Chicago, which are our two large domestic hubs. In Chicago, I'm going to guess that we probably have on the order of 2,500 pilots who are assigned to that base. In addition to those pilots, we have a management structure assigned to that base for the purpose of supervising and providing those interfaces to the corporation that every employee deserves in the way of personnel issues and paycheck issues and communications conduits and so forth. Captain Bushman was one of the team of pilots who had been hired to serve as a supervisory pilot at the Chicago base. He and his other colleagues in that role worked for a base manager and chief pilot who's the boss in Chicago. CHAIRMAN HALL: Okay. But he was -- in other words, you have 10,000 in American Airlines, the pilots you're responsible for. How many at American Eagle? THE WITNESS: Probably on the order of 3,000 pilots. CHAIRMAN HALL: So, an additional 3,000, but here, obviously the chief pilot is somebody who's picked out for above- average ability, leadership skills, communications skills? THE WITNESS: Yes, sir, exactly. CHAIRMAN HALL: You talk about a -- the organization being data-rich, and I -- and I think that's -- and I applaud that, and I think that's excellent. I do wonder, how does that -- we saw some specific questions, I think, that have been raised by this accident in regard to training, and how does that data -- how is that data richness used to evaluate the effectiveness of the -- of the training program? I'm thinking specifically of, you know, here we have the issues of reading radar data, understanding weather, understanding undefined terms, such as bowling alley and blanketed. How do you use that data to -- to get inside that system to evaluate the training since we've seen that we've had some -- I think the FAA is in the process of trying to go from one system of oversight to another, and the gap that's left, we're looking really to the carrier. THE WITNESS: If -- if we collect and mine the data properly, we should -- what should pop out of that data are the non- standard and the aberrational factors in our operation that we can then attempt to deal with by either changing the procedure, changing the training or getting a higher level of compliance to those. You can -- you can fail for any one of those three problems. I'm not sure that we have as -- as robust a data richness in the -- in the training area as we need. We do an awful lot of data collection on the pilot's view of the training process, and, of course, we do the line checking and so forth. Are there better ways beyond AASAP to do it? I think we have to continuously look for those opportunities, but the data isn't worth an awful lot unless you can mine it up into action to change the training, change the compliance, and so forth. CHAIRMAN HALL: Well, just -- and this is just a suggestion on my part. That may be an area that you look at in terms of -- of your data -- data program. Now, I have -- first, let me applaud you on your no-fault fatigue program. But also I have been told, and I guess learned from personal experience, that usually the last person that would know that they are fatigued is the person that is fatigued, and there is sort of a system among many of us that that's a sign of weakness, and therefore we don't -- if somebody says you look tired, well, no, I don't look tired. I'm fine. There are times in your system that, you know, there's weather problems, there are other things that -- obviously delays increase the work in the cockpit, work on the crew, and is there anything being done in addition to the no-fault fatigue policy, which I -- you know, again we applaud, to try to either train the individual or provide some oversight in -- in key and difficult situations that you aren't over-stressing particular individuals in -- in the process? THE WITNESS: Well, I think our fatigue countermeasures program is a good start. Whether it's exactly targeted, I think, deserves a review. It is intended to give our cockpit crew members some advice on -- on how to manage themselves as they cross time zones and attempt to sleep and eat and -- and get exercise and so forth. I'm not sure as I sit here because I'm not intimately familiar with the content of the program, that it stresses enough the self-evaluation, and it's okay to say I'm fatigued. Maybe there's an opportunity to do some more there. CHAIRMAN HALL: Okay. Well, let me just say again, and this is my final comment, how much I appreciate you being here, how much I appreciate American Airlines providing all of the witnesses that you have in a very cooperative fashion, and I know that you share with me the -- the wish that we won't have any future hearings in regard to your all's operations, and I -- and I'm -- feel very comfortable with your commitment, and -- and I think again you are in a position, you and Mr. Carty, being responsible for the resources over both American Airlines and American Eagle, to be sure that the safety needs are -- are funded and given the priority, and I compliment you on the new structure and attempts to do that. As I have done with all the other witnesses, is there anything else that you additionally would like to add in this forum, in this setting, at this time for the benefit of this hearing, which, of course, is to focus on this accident, to try to come up with recommendations to prevent a similar occurrence? THE WITNESS: Thank you very much, Mr. Chairman. Let me just roll through a couple of subjects that have occurred to me over the last couple of days. I believe if you look at the record, that the problem of runway overruns is -- is getting worse. The data I have looked at suggests that it has gotten worse in 1999 versus '98. Much like runway incursions, I think we ought to look to see if there's some systemic runway-airplane combination problem that deserves our collective attention. Second, I believe that we can never do enough to improve weather analysis and reporting, both to the dispatcher and to the flight crew in a tactical sense. Third, programs like AASAP, which have the opportunity of allowing dedicated airline people to surface the issues to all of us to action, need to be encouraged, need to be fostered, and need to be implemented as quickly as we know how. The airline industry, the FAA, the government, is too slow in terms of time to market. There is technology available to help us. There are changes that we all have identified through this process and others, but we take too darn long to go from let's do it to getting it in place, and we've got to learn how to do that a lot better on the operating side of -- of aviation. We talked a lot yesterday about money. This is a business that takes a lot of money, and for anyone to believe that the FAA or an airline is going to be able to do the safety job for the American public without adequate funding for equipment and people, that's just not going to make it. So, we've got to have adequate money. We've got to have a healthy industry to be able to afford to do the things that need to be done. And, finally, Mr. Chairman, a pet peeve of my own is that we, in my opinion, are on the brink of taking one of this country's greatest attributes, and that is commercial aviation, and putting it in jeopardy for the simple reason that we are not attracting the young men and women to this business, both on the government side and on the airline side, and with the down-sizing of the military, that's going to get worse. We've got to figure out how to make this an attractive place for men and women to work. We can only make this business work if we have the very best people that this country can offer, and I think we -- we've got to go forward and figure that out as a national priority, not only for aviation but the other -- the other businesses that operate in challenging environments. This is a business that needs commitment and professionalism, and we need the young men and women to get started now. Mr. Chairman, I'm getting old, and I'm not going to be here much longer. The young folks have got to come along, and we've got to do everything we can to encourage them. Thank you. CHAIRMAN HALL: Well, thank you, Mr. Baker, and thank you for all your comments, and we've spent a long time with this witness, and I appreciate the time you've given us. We are going to take our lunch break. I have a note here from the Mayor. Now, don't -- he's not buying lunch. But he does say we have an hour. If the hotel restaurants are crowded, the River Market area is two blocks east on Markum, and the large market hall has many food vendors that provide delicious food from Arkansas. So, we would -- who do we need to meet with? If we could -- Mr. Clark would like to meet with each one of the coordinators. We'll stand in recess until 2:00. (Whereupon, at 1:00 p.m., the hearing was recessed, to reconvene this same day, Thursday, January 27th, 2000, at 2:00 p.m.) A F T E R N O O N S E S S I O N 2:00 p.m. CHAIRMAN HALL: We will reconvene the hearing of the National Transportation Safety Board. This is being held in connection with the accident of American Airlines Flight 1420 that occurred June 1st, 1999, at Little Rock, Arkansas. I'll ask our hearing officer, Mr. Berman, if he'll call the next witness. MR. BERMAN: Thank you, Mr. Chairman. I call Mr. George Wilken. Whereupon, GEORGE WILKEN having been first duly affirmed, was called as a witness herein and was examined and testified as follows: INTERVIEW BY BOARD OF INQUIRY BY MR. BERMAN: Q Good afternoon, sir. Could you please state your name, your full name, and your business address for the record? A My name is George Wilken. I work for the National Weather Service in North Little Rock, Arkansas. Q And what's the business address of that office? A It's 8400 Remount Road. Q In Little Rock. What's your present position there? A I'm a science and operations officer at the station. Q Did you hold that position at the time of the accident? A Yes, I did. Q Thank you, sir. How long have you been in that job? A I've been seven years as a science and operations officer. Q Would you please give me a brief description of your duties and responsibilities in that job? A Briefly put, I work in Training and Technology Transfer. Just to embellish that a little bit, I'm responsible for when our people come back from schools on technical matters, such as Doppler Weather Radar. I additionally train them, and there is on-going training at the station for that as well as the other systems on the station. Q Thanks. Could you please give me a summary of your education and training and experience for this position? A I hold a Bachelor of Science in Meteorology from Northern Illinois University. I've also done extensive work for a Master's in Computer Data Analysis. I've worked on virtually every weather radar that ever has been since the 1960s, was in the Air Force and spent two years as a weather instructor. Q Okay. Thanks. Do you have any FAA airman certificates? A No, I do not. Q Do you hold any National Weather Service certifications that we should know about? A The several certifications in various things, including radar. Q Okay. Thank you very much. MR. BERMAN: Mr. Eick? INTERVIEW BY THE TECHNICAL PANEL BY MR. EICK: Q Good afternoon, Mr. Wilken, and thank you very much for being here to help us answer the questions regarding the weather radar products. First question I have for you, though, are you not also one of the local presidents of the American Meteorological Society? A I am the president of the local American Meteorological Society. Q All right. Well, I also understand that you have a presentation for us involving the weather radar products, regarding the evening of June 1st, but before we go into that, I would like to ask you some background information regarding the weather situation and your role at the Weather Service. Can you first tell us something about how you got involved in recreating some of these radar products we're going to be seeing today? A Yes. As a science and operations officer, part of my duties are to critique and sort of do quality assurance on our both service and products. When we have a significant event such as this aircraft accident occur, I immediately sit down the next day or the closest day to that and review all the imagery to see if we've done our best on that particular event. So, the morning after this event occurred, I did just that. I sat down and reviewed all the imagery, including some of the products I had been working on insofar as critiquing them for use as radar products, and we'll see those later, and that's primarily a combined shear. Q What was the staffing on the evening of June 1st? This happened late at night. Was the National Weather Service Forecast Office up to full staffing? A Yes. Normally, if we have a severe weather event, our staffing normally would be two forecasters and one public service person, that's a hydro-meteorological technician on duty. That's at all times of the day. If we do have severe weather, we would staff according to how extreme or how busy that event might be. On that particular night, we additionally had a meteorologist watching the radar particularly, and we had another person, an HMT, hydro- meteorological technician, who was doing weather radio and some of the warnings. Q What type of products does the National Weather Service Forecast Office in Little Rock issue aimed at aviation? A We issue primarily two products. Those are the TAFs or the terminal area forecasts, and the TWFs, which are the transcribed weather forecasts. Q Now, I also understand that a weather warning was issued from your office that evening or several -- A Yes. Q -- weather warnings? A Actually two for Pulaski County, which, of course, this was located in the accident, that is. Q Could I bring your attention to Exhibit 5-A and on Page 24? Are these the weather warnings that were issued? A Yes. Q Can you describe for us why they were issued, and what criteria the National Weather Service was using? A I think as we'll see on the imagery that will be presented in a little while, the weather warnings are issued primarily based on some criteria that we've established, both at our office and in the Weather Service at large, and those would be reflectivity characteristics, physical characteristics of the imagery that we see on the radar. So, on this particular night, we were looking at a very strong line of thunderstorms moving towards Central Arkansas from the northwest, and we saw some very strong to severe cells located within that line. We had some indications and some reports of some wind damage being created as the line passed a particular area. So, that, in combination with the radar echoes and how strong they were, we put out warnings based on those factors. Q How are those warnings distributed? A The warnings are distributed in a couple different ways. We do distribute them to the Air Traffic Control Tower at the Little Rock Airport by a direct line. We also distribute them by our NOAA Weather Wire, which is essentially a computer-driven system that provides that to the press. We also distribute it by NOAA Weather Radio, which, of course, is a voice-type distribution. Q Was the severe thunderstorm warning and amended terminal forecast provided to the Air Traffic Control through that direct link? A Yes. Q Does the National Weather Service provide any other advisories, such as high wind alerts, potential thunderstorm impacts, snow advisories, for the Little Rock Airport on a regular basis? A Not for the Little Rock Airport on a regular basis, but in some extreme events, like tornadic events in particular, we have placed a phone call over to them in 1997 to let them know that there was a tornado headed their way, and the Control Tower people probably should consider evacuating the control tower. But on a thunderstorm situation, as we saw with this line, that did not really prompt us to do anything special. Q Did your role at the Weather Service Office deal anything with the Memphis Center Weather Service Unit? A Yes, we do. We interact with them quite often. At -- the Center Weather Service Unit was closed. So, we couldn't interact with them at that particular time. Q But normally you would contact them to advise that a severe thunderstorm warning or a major event was occurring in your region of responsibility? A We would talk to them not on a case-by-case basis necessarily, but if something was very, very unusual, we would certainly talk to them. Q Is there a direct link also to the Center Weather Service Unit or just a dial telephone? A It would be a dial telephone. We have them on our speed dial at the office. Q All right. These radar products you're about to show us, are these products unique to the National Weather Service, North Little Rock Office, or are these also available to other National Weather Service and FAA Aviation Centers? A The WSR-88D, which is the Doppler Weather Radar, has a suite of products indicated which is available to all offices and all the users. Q All right. Did you have availability to lightening data? A Yes. Q Was it useful in tracking and identifying any severe signatures with the thunderstorms? A Very much so. We just recently acquired a new computer system called AWIPS, Advanced Weather Interactive Processing System, and was placed in our station roughly about two months before the event occurred, and that has live lightening data on it that we can access. So, our forecasters had become accustomed to looking at that lightening data in conjunction with severe weather that was occurring. Q Now, the radar data you're going to show us is a different type of system than airborne radar, is that correct? A Yes, it is. Q The airborne radar, such as what, 1420 was using was an XBand Radar with three centimeters and about a three-degree beam width. What is the characteristics with the radar you're going to be using? A Well, our -- our radar is much more powerful, has a very narrow beam and a very much longer wave length on it. It's a 10- centimeter radar, commonly called an S Band Radar. It has 750,000 watt output, and it has one degree -- approximately one degree -- it's a little less than one degree beam coverage. So, what it essentially does is it puts out a stronger amount of power in a much narrower or smaller cross-section. Q Now, the WSR-88D or the NEXRAD Doppler Radar, it also performs what is called a volume scan strategy. Can you explain that to us, please? A Yes. In fact, I think we could probably look at one of the -- I think it's Page 9, I believe, is the revised Page 9 that shows the diagram which is toward the -- roughly about the middle or bottom of the page. It talks about the volume scan strategy. There was a revised single page put out for this. Basically, the volume scan strategy is when the radar will gradually elevate on a kilt-by-kilt basis. In this particular case, we were in Volume Coverage Pattern 21, which dictates we would see nine elevation cuts in six minutes of time. So, the radar will make a 360 at the first elevation cut, then jump to the next one, make a 360 and so forth, all the way up to the top. It will start at about a half-degree elevation at the bottom level and up at about 19 and a half degrees at the top. Q Can you describe what the scale of intensity or reflectivity is? A The reflectivity -- you're talking about the decibels? Is that what you're talking about? Q Yes, Mr. Wilken. A The decibel intensity that we see on radar and will see that again on the pictures when we look at them a little bit is a reflectivity value, that the higher the value basically the stronger the storm or actually the stronger the rainfall. The radar does not really see storms. It sees rain, but we can also interpret some of the reflectivity returns as being those of hail or just having a strong thunderstorm in progress. Q In Exhibit 5A on Page 10, are -- is this a correct representation of the reflectivity scales with the appropriate rainfall rates? A Appears to be, yes. Q All right. And if we can go with your presentation, and can you show us something on the composite reflectivity images that you've brought? A Yeah. Q Can you also show snow? A Well, I should put in a little commercial right now because I talked to some people earlier saying that snow would start after 12 and before 2 p.m., but I won't go into that. What you're looking at on the right-hand side of the picture here, this is a reflectivity, up through this area, is the reflectivity scale. As you look at the top, the five units of reflectivity or five dbzs, as they're called, extend up to 75 dbzs at the upper limit. Now, we can look at the picture here. It's going to be a little hard to see. Maybe it's a little muddy. Little Rock is located right about here. The airport symbol, if you can see it, it's not real visible, is located almost immediately east of the Little Rock indicator. So, what we're looking at is a thunderstorm complex line as it was approaching Little Rock area. You'll notice some blowing up to the northern end of this line. Also notice some fairly high reflectivity values that are located in two different areas. This was a thunderstorm that initially had passed over Little Rock Airport which is where my little red dot is, and the second one is the thunderstorm that was approaching the airport about the time the aircraft was approaching. You'll also see that, as mentioned, we have some high reflectivity returns, 60 to 65 dbzs, which is considered in our office, especially in the 65 dbz range, to be indicative of large hail. In addition, when we look at some of the other photographs or pictures of the base reflectivity a little different than this, you'll notice also that we can see some reflectivity aloft. Now, what we're looking at here, I might clarify a little bit, is composite reflectivity. It is a composite of all the elevation angles of the radar, constructing the highest reflectivity at all the elevation angles and putting them on one picture. So, what the radar operator normally does is to look at this picture and sort of get an idea of where the worst or highest reflectivities are, here and here, and then start to look aloft to see which of those reflectivities are suspended aloft perhaps to produce a microburst or some other wind event down at the surface. So, this is sort of a first look product, I guess you would say, just to get a general idea of what was going on. MR. EICK: All right. If I can interject just a point of information, I should explain to Mr. Chairman and everyone in the audience, this exhibit that Mr. Wilken is presenting is in Docket 5D, and you've been provided those. Docket Attachment 5D will include the radar products that Mr. Wilken is presenting. MR. CLARK: Mr. Wilken, can you tell us the time of that particular view? The accident occurred about 11:50 to 11:51. THE WITNESS: Right. This first one is the composite reflectivity at 0445 or 11:45 local time. MR. CLARK: That would be five minutes before the accident? THE WITNESS: Yes. MR. CLARK: Okay. And how much did that composite -- did that cover a six-minute period? THE WITNESS: That covers a six-minute period, yes. Now, I might add, also, that the base products, as they're called, that is the base reflectivity, velocity and what we call spectrum width, products are available immediately. The post-process products, like the composite reflectivity, usually do not arrive immediately after the volume scan of six minutes, but you have to require some processing by the radar computer. So, this would be available probably another six minutes down the line. MR. CLARK: That 11:45 time, is that when the scan ended? THE WITNESS: That's when it began. MR. CLARK: So, that piece of data was scanning weather from 11:45 till 11:51, and you would have had that approximately six minutes later? THE WITNESS: Right. MR. CLARK: Okay. THE WITNESS: Looking at the next picture here, which would be the composite at 0451 Greenwich or Universal Time or 11:51, about the time the aircraft touched down, this is a little more difficult to see here. It's even a little muddier than the other one. I was hoping -- the reason we chose to show these or at least I wanted them shown is that I felt they were a little bit clearer than what we had looked at earlier on some of the other imagery. I believe LIT is right in this area here. The airport is located right about where my dot is. You can see an airport symbol. The airport symbol does not necessarily show exactly where the runways are. It just indicates like any type of icon that that is located in that general vicinity. I have put the rivers -- you can barely see the river going north, the Arkansas River -- here we go. That's good. The airport symbol is right here. The river north of there. The aircraft ended up right about in this area. You can see a very large again strong echo, 60 dbzs. The one to the east has broken up a little bit, but essentially what we had working at the airport about aircraft touch down time was this very large, very strong thunderstorm. BY DR. BYRNE: Q Can you estimate what reflectivity is over the airport at that time? A It looks like right at this point, you do have some 60 dbz right over the airport. So, that would be in the upper limit or the old -- what we used to call a VIP-6 category at that time. Q Okay. Now, airborne weather radar usually has three contours or three levels of reflectivity. With Level 3 activity, the strongest that they're going to show, at 40 dbz, on your scale there, can you show us in your color scale where 40 dbz would be painting red or the maximum intensity on the airborne radar? A Well, we have -- we'd have to go back up a little bit to -- you look back again at that -- I think the best thing to do would be to look back at the scale that we had in there. I forget which page that was on. 20 something. Q Page 10 on 5A? A Probably 10. Okay. Okay. If we look at the scale, the VIP scales we're talking about, what is -- is what the National Weather Service used to use on the predecessor to the 88D, and they go from a 0 to a 6, basically a 1 through 6, but the zeroes are pretty much non-existent for any real significance. The 88D levels, of course, have much more or many gates to go through, and if you look at the third column from the left, if you look at 40 to 44, the Level 3, this is what we're looking at right here, up in the green and blue. So, a lot of these surrounding echoes, surrounding much stronger echoes, all through this area are about the 40 to 44 dbz rating, while toward the airport at that particular time, as I said, we had a 60 and a 55 to 60 dbz. So, that would put us, of course, across the line into the extreme category. MR. BAKER: Mr. Wilken, while you're pointing at things, could you repoint to the river and the airport? We're a little confused. THE WITNESS: Okay. The river runs right up through here, across over this, and we can see the northern bank real good here in the east and west. The airport symbol is right where my dot is. MR. BAKER: What is that straight line between the airport and the river? THE WITNESS: This line right here? MR. BAKER: Yeah. THE WITNESS: That's the movement of the storm itself. The radar will put a movement of the storm, estimate where it's going, and how fast it's going to get there. So, where you see a little X in a little circle, that's where the storm is now, where the storm centroid is at the present time, and it will subsequently plot points along the line. You can see this storm supposedly had moved from a different direction, but the radar sometimes will combine two storms and give an erroneous direction, but this particular direction appeared to be the same as the environmental winds at that particular time, northwest to southeast movement. MR. BAKER: Thank you. THE WITNESS: We can probably pan in a little bit on that one as well. BY DR. BYRNE: Q What is the time on this image, too? A This is the six minutes after the last one. So, it's 0457 Greenwich or 11:57, some -- six minutes after roughly, after the aircraft touched down. The river again is located right about in through here. The airport symbol right about there. You can see that there's still some 60 dbz located over the airport, over the area we're talking about. Characteristically, we would expect to see very heavy rain, hail, at least half inch or larger, and probably some gusty winds from those photographs. Again, we see tracks of the storms, the lines coming through here, and you can see there's another track here of this storm. The track here that's pointing down to the southeast. Q Okay. Go on to base reflectivity. A Now, the difference here between the composite and the base is the fact that the base are basically one elevation cut of reflectivity. This one, it's hard to see up here in the corner, but it's 0.5. So, it's a half degree elevation cut. You can see it there. You can see the time 0445 Greenwich or 11:45 local time. The radar also will tell you the max dbz rating on the radar. It won't exactly tell you where, but it will at least tell you that you've got that to look at, and, of course, you can look at the scale and see that 61 would be somewhere in this gape between the 60 and 65. We can look at the center of the picture now. Just pull it up just a little bit. Okay. You can pan in just a little bit. Okay. That's good. Here again, the river curling up. Little Rock is down here at this time. The airport symbol is sort of masked by some of the imagery right in this area. Q Now, this image is at what time, did you say? A 0451, 11:51 local time. Q Okay. A And basically it was painting the thunderstorms again at -- it had passed over the airport, at that time was only about a 40 to 45 dbzs, but you can see 55 dbzs are pretty widespread to the northwest as this next storm moved in. MR. FEITH: Mr. Wilken, just a question. With that 60 dbz rating, would you expect wind shear in any of that? THE WITNESS: Not necessarily. It would have to -- we would have to look up in the thunderstorm, and we'll do that in a few minutes, to see if we had any suspended reflectivities. If we do have some suspended reflectivities, we could very possibly get a microburst which would produce wind shear. BY DR. BYRNE: Q Now, at this time, 04 -- is this 0445 or 51 there, George? A This is 445. I think the first one, I think, we looked at. Q Okay. So, at this time, the aircraft is still out to the south of the airport. A I am looking at 0451. You can see it up here. Okay. We can look toward the center of the image, please. Little Rock again right about in this area. The airport again masked a little bit by some of the imagery. You'll notice that although the 55 dbz has somewhat diminished at this time, we do see the start of the 60 dbz right in the middle of the 55, which is usually the way the operator would want to see it. He would want to see a high reflectivity surrounded by somewhat lesser and not just have what we call an outlier, where you have a high reflectivity with no supporting high reflectivities around it. So, this is showing some development or strengthening of the storm as it's moving toward the airport. I'm going to go up at the same time, 0451 Z, on base reflectivity on the next picture, and this will be at 1.5 degrees. Okay. We can go toward the center of the picture. You'll see that we elevate a little bit. We start to see more white. It's going to be -- okay. Little Rock is right in this area. Can't quite see the airport symbol. It should be right about in this area here, I believe. Should be just about due east. I don't really see it. Should be right about in here, I believe. But as we're going up, you can see that there is more in the way of 60 dbz reflectivity that's suspended a little bit higher in the thunderstorm at this particular time. We can go on to the next one. Going up approximately one more degree, to 2.4, at the same time frame but toward the center. Okay. The airport symbol is right here. Little Rock, of course, downtown area is here. You can see very large area, and that's significant when you see a very large area, even more so at this higher altitude than the lower one we just saw. So, this is what we call suspended reflectivity -- high reflectivities, which are something that the radar operator would look for to sort of contemplate maybe a microburst possibility. And the last one will be about another degree up at 3.3 or 4. Sorry. Once again, it's very highly reflectivities with suspended up inside the thunderstorm at about the time the aircraft touched down. Here is the airport symbol. Little Rock is over here. So, basically, at 0451, we saw some very strong suspended high reflectivities in the thunderstorm as it was moving toward the airport. MR. ZWINGLE: Mr. Chairman? May I ask Mr. Wilken -- Allied Pilots Association. Mr. Chairman? CHAIRMAN HALL: Yes. I'm sorry. MR. ZWINGLE: That's all right. May I ask Mr. Wilken to clarify the -- the elevation of those -- the 2.5 and -- and 3.4 degree -- THE WITNESS: You mean above ground level? MR. ZWINGLE: Yes, sir. THE WITNESS: You can look back -- again a follow- along. You can -- there is a chart on Page 9 at the top of the page of your hand-out. MR. ZWINGLE: 5A? THE WITNESS: 5A. I'm sorry. Yes. At a half degree elevation, you're looking at the center of the beam about a little under a thousand feet at 1.5, center of the beam about 1,700-1,800 feet. At 2.4, you're looking at about 2,500 feet. 3.3 at about 3,100. That's the center of the beam, and that's what the radar will show you. The beam width is such of one degree or thereabouts would show you that the lower level is somewhat lower than the center. For instance, at .4, .5, you're looking at about 624 feet for the bottom of the beam, and for 3.3, the bottom is about 2,800 feet. So, you can match those with what we see up on the screen. We're going to look at a four-panel base reflectivity once again at 0451 Z, and basically this is a four panel of the four panels we just saw, but you can take a quick look at the comparatives between one panel to the next. The next one. It's not going to be real easy to see, I know, but basically we went from about a 55 dbz echo. This is the first time at 0451 -- excuse me -- these are bottom. They're half degree echo at 0451. As we start to go up, within the thunderstorm, we start to see more and more 60 dbz in the third quadrant. See more and more 60 dbz echoes showing up as we go further aloft, and that is significant. That is very significant to look for the possibility of a microburst because you have basically a high water load very high in the cloud, maybe some hail contained with that, and it's forming a very dense pocket which can very rapidly rush toward the ground and produce a microburst, and that would be something -- a very strong concern for the radar operator. We also have -- the next group is a time series which basically shows the movement in toward the airport on a one-by-one basis. Let's see. Can you pan out to see what we have here? I think we might have got off track here a little bit. Let's see what we're at. Okay. This is 0457 Z. I think I'm one ahead of myself, and this is the correct one. At 0457 Z, this is just at the half degree level. Once again, what we saw earlier was about a 55 dbz at that level, and now we're seeing definitely some embedded 60 dbz echoes. The airport -- I think it's right over the airport at that time. Little Rock is over here, and this is the airport symbol right in this area. Okay. We can go to the next group. This will be two full hard copies, one partial hard copy, showing the general movement. This is going to be hard to see on the screen because they're a little smaller, but basically the airport or LIT is right about in this area. Oops. Getting up on me there. Let's see. Lost it now. I had it. Okay. I think this is LIT right here. The airport symbol right about in this area, I believe. I can't quite make it out myself. But basically at this time, if you can show me the time up here in the left corner -- oop. Go down a little bit more. Over to the right a little bit more. A little more. Okay. Quad 1, which we're looking at here, a max of 63 dbz somewhere around the particular picture, but this is -- let's see. I lost the time on this one. Okay. Go back to the picture, if you would and go toward the bottom of the picture a little bit. Oop. Nope. Just to that particular quadrant. There you go. This is at 0410. So, this is well before the time when the aircraft was approaching the field, but if you'll pull it up back to the middle again, once again Little Rock is right about here. The airport would be right about in this area. I think I see the symbol vaguely right about in there, but you can see off to the west that that particular time, we did have some echoes. They weren't real strong, but they did have some configuration. You have a little bit of what was referred to earlier as a loop, a line echo wave pattern, which is showing right in this area. Characteristically, you would see some wind damage or some problems develop right to the east-northeast of that center of that line echo wave pattern, but at this time, we had just generally a line of thunderstorms that is gradually moving itself toward the Little Rock area. You go to the right, Number 2 Quadrant there. Basically let's run through these fairly quickly because it just shows basically the line movement. We start to see some red marks. It's hard to see from where you're at, I'm sure, but there's some 55 dbz start to show up within the areas of thunderstorms. Okay. If you go to the bottom left quadrant, you start to see a little bit of blowing occurring which is indicative of some strong winds moving across, and indeed when the first thunderstorm moved across the airport, there was some wind at that time, not as significant as it became later, but certainly enough to be concerned about. Go to the bottom right quadrant. It's a little harder to see, but basically there was some blowing in this area again. Little Rock is right here. You can see just the outer edge of this enhanced area of echoes. You still see some 55 dbz. If we go to the next -- BY DR. BYRNE: Q Mr. Wilken, could I just interject? On Exhibit 5B, on Page 18 through 20, are these not approximately the same type of -- A Yeah. They are, and I think probably it would be easier to see them in the hand-out than they would be on the screen. So, I - - I would opt to maybe let the audience look at those and maybe we'll go on to the next series. Q All right. Well, if you'd put your last slide back up, let me just ask you a question. For everyone, Exhibit 5B, Pages 18 through 21, show the same presentation. Can I ask you to characterize how are those thunderstorms organized? Is that a scattered area? Multi-cellular storm? Is it a line activity? A We would probably call this a solid line because there are very few breaks in the line. There's at least nine-tenths coverage of the stronger echoes. As we look -- I'm looking at the 0410, Figure 24A, right now, and looking through this, pretty much all through the first page, I don't have my page number real handy, but it goes from Figure 24A through C, pretty much a solid line indicated on that. You're looking at 24D through 24F. We see a few breaks starting to develop toward the middle or at least to the southwest of Little Rock in the line, especially as we get toward 24F. You see a few breaks developing down to the southwest, but essentially it's still pretty much a solid line of coverage. Q And this is before the accident time? A Yes, it is. And you can look at the times coincident with those images and see the times that were incurred there, and as we get on to Figure 24H would be about the time of the aircraft touching down. You can see that there is pretty strong coverage right over the Little Rock and just immediately east of Little Rock which would encompass the airport. Q All right. A Okay. We'll go on to the next series. Okay. Q We can go right on to the exhibit on Page 14, Mr. Wilken. A Thank you. Q The radio velocity? A Okay. That's where we're going to, yes. What we're going to look at now is a little more abstract, and it's going to require a little more concentration when I'm trying to interpret this. This is the velocity picture from the Doppler, and this is why the Doppler is the Doppler, because it can do velocity and look at velocity at various elevation angles as we saw in reflectivity. This particular shot is a half a degree elevation angle up from the ground at 445 Greenwich. Now, the difference here in the scale is that this scale is broken essentially in two. We have the zero velocity which we call the zero isotope. As we go to the negative numbers, we're looking at wind velocities or the cool colors, as we call them, moving toward the radar. We're looking at the warm colors, the reds and yellows. We have wind moving away from the radar. So, if we could sort of center this again? Oop. Yep. There you go. Okay. Little Rock is right here. The airport symbol is hard to see, but it's right in this area. What this is depicting is that you have some warm colors showing here. So, you have some wind blowing away from the radar. The radar is up in this area. So, the wind is actually coming from the northwest to southeast, and we know that that first thunderstorm passed. It probably put out a gust front with it, and actually what we're seeing down here where the red and the greens come together is an indication of the gust front. We actually have winds toward the radar down here in the greens, winds away from the radar in this area. So, you have sort of a convergence line which would be very indicative of a gust front. So, we did have a frontal passage of some sort, that is, at least the gust front, move through the airport before the aircraft was approaching the airport. This is at 445 again, I believe. Yes. Go on to the next one. The next picture here, 0451 Z again, about six minutes down the line, after the elevation again. We're looking at the same type of situation. Warm colors away, the cool colors toward the radar. If you could center the picture again? Little Rock is located right here. Airport symbol right about here, and, once again, you can see there were some fairly strong winds. These would be up in the roughly 40 knot to 45 knot range on the color scale. We see something ominous developing up to the northwest with this new thunderstorm because we're starting to see some colors that are indicative of perhaps a microburst development, and, of course, about this time in reflectivity, we're looking at 60 dbzs located about the same area. Q So, at this time period, the microburst is located where in respect to the -- A It's not fully developed yet, but it's located northwest of the airport, the airport being here, and you can see the wind flow right now is moving -- this is moving toward the radar at this particular time, actually moving this way, where this is moving away from the radar. So, the environmental winds are the winds generally over the airport at this particular time would be essentially from the northwest where you're starting to see some movement in toward this particular cell now, coupling up with some other winds, again moving in an opposite direction. So, this was located right about on the river, just to the northwest of the airport, which was located right about here. If we can go to the next one? This is 0457. This is six minutes after the aircraft touched down. Once again, half degree elevation. We can look toward the center of the picture once again. What we're seeing here now is a -- looks like a microburst right just about over the airport. The center of the microburst would be right in this area. You have winds moving to the southeast, to the southeast, winds moving to the northwest here. So, I can compare this to like a drop of water in a puddle where the microburst would come down, hit the ground, and it would generally spread out in all directions. So, you're seeing a very strong footprint here of a microburst that occurred and did produce that very strong wind gust at about that time. So, this and the instrumentation at the ground level were very well correlated as far as what was happening. Q One question I have for you real quick, Mr. Wilken. A Sure. Q How much meteorology training did it take you to interpret all these radar products? A The base education for one of our operators is four weeks of concentrated study at our Radar School in Norman, Oklahoma. Once you do that, you've started, but you've only started. After that, it's a matter of sitting on the radar, and I'm sort of the, you might say, the check pilot for our people on the radar where I sit and look over their shoulder and make sure that they're doing what we think they should be doing as far as quality assurance goes. So, training is on-going. It's never stopped in terms of looking at different imagery. Our people review imagery which is kept on archive disks. We have various drills that we make up for them. We have seminars at least twice yearly for them in terms of radar, and, of course, Arkansas is a very strong state in terms of severe weather. So, it's really our duty to keep very vigilant on this, and I'm happy to say we've been awarded two Department of Commerce Silver Medals for our work here in Little Rock, one in 1977 and one in 1999. Okay. This is basically another four panel which will show the same radio velocities we just looked at for the various time frames with the 0440 Z added in as the first frame. You can go to your Exhibit 28 in your hand-out and look at this. I think you'll probably get more or better resolution by looking at your hand-out perhaps than looking at this. But you can see what happens. If you'd just pan back once, we can just do a very quick look at this, and our operators -- one thing bad about a hard copy is that you don't see movement. The radar has the capability to produce time lapse, and we usually have or always have, I should say, one screen running in time lapse to see the progression of events because it's a very strong indicator as to what we need to do in terms of putting out warnings. So, if we had this in time lapse, you would see basically one picture, but you would see the progression of events. For instance, the velocities in the area of the airport there which were produced by the gust front going through which is this feature right here, and you can see the gust front moving along with time. As we get to the third picture here, we start to see something very ominous south of the radar, northwest of the airport, in terms of a developing microburst, and then once we get to the 0457 time or the 11:57 time, the microburst indeed does drop down to the surface and produces 70-80 knot gusts at the airport at that particular time. We're going to jump over the Exhibit or Attachment 49, which is actually the bad wind display, on the last couple time periods of the bad wind display. We did not really have much occurring or being detected at the low levels. The bad wind display is a product that is sort of an averaged wind around the radar. So, any extremes that do occur will be averaged out to some degree, and in this particular time, we did some northwest winds at about a thousand or 2,000 feet at about 30 knots, but I don't feel they were representative of what was going on because of the averaging. So, we've chosen to bypass that one for now and show you the combined shear product. MR. BAKER: Mr. Wilken, before you go to the -- too far into the winds, -- THE WITNESS: Yes? MR. BAKER: -- does that time lapse technology also have a delay process in it? Earlier, you were saying that your imagery was approximately six minutes ago, plus delivery time. THE WITNESS: Now, the base reflectivity is delivered immediately when that particular elevation cut is completed. So, as soon as -- MR. BAKER: But that takes about six minutes, I think you said? THE WITNESS: No. MR. BAKER: Okay. THE WITNESS: As soon as that cut is completed, all nine cuts take six minutes. So, the first cut would take a portion of that, one-ninth of whatever it is, and then you go to the next cut and so forth. So, a half degree elevation, the base reflectivity would be available pretty quickly after the elevation scan or the volume scan started. So, the cuts we are looking at on base reflectivity would be available immediately after that one elevation cut was completed. At that time, the radar would inject that into the time lapse, and you'd have it immediately available. It's just a matter of tenths of a second because the radar will automatically add those to the time lapse as they accrue. So, you keep seeing an extension of each time lapse by one picture every time it's completed down the line. Is that clear? Okay. Okay. I'll go back to combined shear. This is a little bit more of an abstract product. It looks more like modern art perhaps than it does radar, but I found it to be a very useful product, and I'm perhaps the only one in the whole country that's doing any work on this product. We do have some products on the radar which are not used very much, and people have chosen not to really look at them for whatever reason. I found that the combined shear product, although it does take a very intense computer operation, is very much a resource hog on the computer to some degree, and you can only measure it at one elevation angle selectable, we chose 1.5 degrees because over time, we have seen that this is the most representative elevation angle to use. So, in this particular one at quarter till the hour, 11:45, once again, -- now, here we're looking at units of shear, and shear has no units. So, it's sort of a contradiction, I know, but basically I call them units of shear. So, the further up the scale we get, in a paper I wrote, basically shows that about 70 to 80 units of shear is very indicative of something, strong shear, at that particular location. So, at this point in time, we're looking at the airport here as it's moving along, Little Rock here, and we're looking at some -- just some low-level units of shear right over Little Rock and somewhat higher shear value over the airport at that time, not seeming to be very indicative of anything serious at this particular time. Up to the northwest, we see again the cell developing up in that area, and as I mentioned earlier, and we saw in both reflectivity and velocity, it was developing into a microburst. So, this was showing some shear at this particular time. I think that's what Mr. Feith was talking about earlier. This was showing some indications that it was becoming the area to watch insofar as bad thunderstorms go. Do you want to go to the next one? You can see that the thunderstorm obviously is approaching the airport again. The airport symbol is very clear at this particular time, 0451, about the time the aircraft touched down. It did not look like there was any real strong shear over the airport, but certainly immediately to the northwest, the shear was really starting to develop as this downburst started to develop, and I think you'll see something very dramatic in the next picture. We see our 70 units of shear have come into play. The airport is right here, indicative of the downburst, the splaying out of the winds from the center of the storm, showing the shear that it produced as it moved out, the very strong shear over the airport itself. So, this product, you can see, is a very good product, not in real time, again this is a product that is post-process, but certainly in some kind of a forensic-type case like this one is, this is a very good product to look at. MR. BAKER: What time is that one? THE WITNESS: This is 0457. Again, these are all at 1.5 degrees. BY DR. BYRNE: Q Are there any other severe weather products on the WSR-88D that proved useful in forecasting thunderstorms during this evening of June 1st? A Well, there are several that our people normally look at. We look at the echo tops product quite a bit. Of course, we use the velocity products quite a bit because we're always concerned about rotation in the storms because we find that a large part of thunderstorms, that is, a large percentage of thunderstorms, have rotation in them. What the public hears in terms of tornado warnings are just a very small subset of the storms that we see that have rotation. There's a series of four products called a "severe weather product series" which will provide a percent chance of severe weather at a location. Those are used to some extent, but we have not really found a good threshold to use on those yet. So, they're in less use. There are some products that we use called "VIL", vertically integrated liquid. VIL is a product we look at both for microbursts and hail, and I designed a chart that our people use during the day. It's called the VIL of the Day. It's correlated with a 500 millibar temperature to give you a number that you need to look for for dime-sized hail or larger. That's in use at our office every day. So, there are several more things that we look at. Of course, we look at all elevation cuts. It's very important that we look at every elevation angle to see what is suspended aloft. If we look at the top of the storm, we can look at the storm top divergence which would produce an idea of whether or not we have strong winds or large hail developing. Q How did the radar products with respect to where the activity was located, the timing, match up with the severe weather reports reported across the area? A Matched up quite well. The gusts reported at the airport was very coincidental with the time of the microburst recorded by the 88D. Q Mr. Wilken, my last question I have for you is I'm going to draw your attention to the poster in front of you, which is in Exhibit 5B on Page 12, which is the base reflectivity image at the 0.4 to 0.5 degree elevation scan at 0451 Z. A Okay. Q Or at the approximate time of the accident or specifically within the minute after the accident. Do you recognize this product? A Well, the color scheme's a little different, but it looks like it would be equal to our reflectivity product, I would say. Q Well, this is an eight -- magnification of eight times, and the color scale has been changed to more approximate the -- the airborne radar with about 40 dbz showing up on this as dark orange going to red. So, we're looking at -- would you agree that these are the same products that you have presented -- A Yes. Q -- previously? A The imagery is the same, the color scheme is different. Q Now, in -- on this image, we also have the aircraft track. If you can look at that track, and the Little Rock Airport, the geographic center of the airport, is at the crosshair with Little Rock, and you'll see the track going in. Is there any -- what approximate reflectivity would you say was along the track at that period of time? A Well, at that time, it looks like you were probably running in the 50 dbz or higher at that time. There's some pink indicated just to the west or over the airport, I should say, at that time. So, you're looking at somewhere between 50 to 60 dbzs. Q And that would relate to what on the Scale 1 through 6 with the Weather Service? A You'd be a 5 to a 6 on your VIP scales. Q Mr. Wilken, I thank you very much. DR. BYRNE: Mr. Chairman, any questions for Mr. Wilken? CHAIRMAN HALL: Do the parties -- we'll move to the party tables. The Little Rock Fire Department? MR. CANTRELL: Thank you, Mr. Chairman. INTERVIEW BY PARTIES TO THE HEARING BY MR. CANTRELL: Q Sir, in your opinion, approximately how much rain fell between 2345 and 15 minutes after midnight or could you estimate? A I can't really estimate. I don't have those pictures with me, but the radar does do an estimation on it. I know it was a very heavy amount at that particular time. I think at one time, if memory recalls, it was about a third of an inch a minute, about three-tenths of an inch a minute. Q What impact on visibility would rain of that magnitude have, sir? A Well, it would draw down. Of course, you wouldn't have fog coincidental with the rain. So, your visibilities would drop dramatically at that time. Q On field visibility at the time, could you estimate that, sir? A I don't have any way to do that right here, no. Q All right, sir. Are you familiar with the properties of FLIRR, forward-looking infrared radar? A Not really, no. Q All right, sir. A radar question in general. Does infrared radar have an effect where it is masked or blinded by heavy rain, in particular cold rain, or would you know, sir? A Infrared radar, you said? Q Yes, sir. A I really don't have any knowledge about infrared radar. So, I couldn't answer that question. Q All right, sir. No more questions. Thank you very much, sir. CHAIRMAN HALL: The Federal Aviation Administration? MR. STREETER: Yes, sir. BY MR. STREETER: Q If you could help explain a little bit more on the product you're using here. I'm not familiar with your radar. I am familiar with the air traffic radar and with in-flight radar systems. Are you -- are the -- are the graphics you're presenting here, is this an example of what your radar operator sees when he's sitting at the -- at the display? A Yes, sir. It's a one-to-one relationship, right. Q Okay. And I guess the part I don't understand then is you talk about the -- I believe it was 9 -- 9 build-ups in a six-minute period. How -- how is the picture evolving over that six-minute period? Is he just getting a different -- a different elevation on each sweep? A Yes. You're getting an elevation cut as it's completed in the base reflectivity. The composite is one that has to include all the angles. So, it would have to be made up after the entire volume scan completed. Q Okay, sir. A So, we're talking about base and composite as two different things. Q Understood. So, the base would be what he would actually see then on a single sweep sitting at the -- at the display? A That's correct. Q Okay. And then this product is the composite made up after the entire volume has been swept for a particular period? A That's correct. Q Okay. I understand. Thank you, sir. You did also mention that your operators undergo four weeks of intensive training on -- on the basic equipment and so on before they start working this. Is it safe to assume that they have a scientific education prior to doing that? A All our people are graduate meteorologists, yes. Q Okay. Are you -- are you familiar with the air traffic control radar similar to what they have over here at Little Rock Tower? A Minimally. Q Okay. Are you familiar enough to -- to say what kind of picture it would have presented in this type of weather situation that night? A Well, from past experience with the FAA-type radars, it would depict precipitation, but it probably would not be a very strong presentation. Q So, the controller would not have seen anything at all like this type of return? A Well, certainly not like we have -- CHAIRMAN HALL: I don't think Mr. Wilken is going to be able to answer that question. MR. STREETER: Okay. I'm sorry. I see what you mean. I didn't mean to word it that way, sir, and it's a good point. Thank you. I'll disregard that. BY MR. STREETER: Q Are there any other weather radars in the Little Rock area besides -- A We have three -- three television stations. Each have their own radar, -- Q Hm-hmm. A -- which are a shorter wave length and a much less powered radar than we have. Q How would the -- how would their displays compare to what you're seeing here? A They have a different processing system; that is, theirs is processed a little differently than our Doppler is. Their presentations are made more for the tv audience rather than for the professional. Q Okay. Understood. And then the last question is, are you familiar with on-board weather radar systems at all? A Not at all, no. Q Okay. That's all I have, sir. Thank you. CHAIRMAN HALL: The Boeing Commercial Airplane Group? MR. HINDERBERGER: Mr. Chairman, we have no questions. Thank you. CHAIRMAN HALL: American Airlines, Incorporated? MR. BAKER: Thank you, Mr. Chairman. BY MR. BAKER: Q Mr. Wilken, do you have a responsibility to communicate severe weather warnings to special groups, such as air traffic control towers or en route centers or the airlines? A Well, would you define responsibility? Q If you were to see with all of your equipment a significant weather phenomena, are you expected to tell the air traffic controllers at Little Rock Airport or the en route center or airlines through any mechanism of that weather? A We do have a direct line to the air traffic control tower, and that provides them instantaneous bulletins of severe weather, severe thunderstorms, tornado warnings, the watches, all aviation- type things like SIGMECs and AIRMECs. So, we do have that facility to them. Q And do your -- do your personnel on duty routinely notify the airport? A Not by telephone, not routinely, no. In the case of 1997, when we had a tornado approaching the airport, that was over and above what we would suspect or -- or call on. So, we called on that particular occasion. In the case of thunderstorms, we'd be making an awful lot of calls because Arkansas is very ripe with thunderstorms all through the year. Q And in your view and experience, having seen this type of weather, would you have expected the -- the person on duty to call the tower at the airport? A Not as a matter of routine, no. Q On your equipment, are there audible or visual alerts that are generated to notify you that particular threshold values may have been reached that would result in a warning of any type? A Yes, sir. Q Were they sounding that evening? A Yes, they were. But I might add that we expect our operators to beat the radar; that is, not have the audible alert sound before the particular signature has been identified. That is one of our prime training tools that we deal with. That is, we expect them to have that knowledge and foresight to be able to see that event taking place before the radar does. Q Within the National Weather Service Severe Weather Warnings, do the terms "microburst" or "wind shear" -- are they used routinely? A Yes. Q Okay. Do other groups have access to the same WSR- 88D information as is received in your office? A They do, but they are provided that through vendors for the most part. There are three, what we call, NIDS vendors at the present time, which produce radar information that is given to whoever wants to subscribe to the service. That is taken off of our radar and provided through the vendor to a tv station that may not have their own radar or wants to have the extra-added attraction of having ours on to look at as well. There are several of those vendors on the Internet. Very few of them offer radar in real time, however. Q Do they routinely issue warnings? A The Weather Service is solely responsible and charged with issuing warnings. Q Going back to -- the evening -- given -- given the lead times and so forth of your equipment, at what point would the individual on duty have been able to determine that a gust front was in fact going to impact the Little Rock Airport? A Well, a gust front is a feature that usually moves along in advance of a line of thunderstorms. So, the lead time should be relatively long on that. You could probably look upstream, that is, to the northwest in this case, see that the gust front had developed and was moving in a particular direction, and almost with unerring accuracy predict when it would arrive at a particular point. So, the lead time could be anywhere from, you know, depending on how long you wanted to make it. You could do it 30-40 minutes out in time without a problem. Q Do I understand from your previous question -- answers that you probably would not have as a matter of routine warned the control tower of an approaching gust front? A No, we normally do not. That's a standard feature with a lot of thunderstorms. So, we'd be calling the control tower a lot of times, and I'm afraid they might get a little tired of hearing us call them really. Q Okay. Now, -- now, going to this -- this picture, this large display with the reflectivity, as I understand it, this was taken at a snapshot point in time of 51 after the hour. A Well, snapshot. Of course, you would have to qualify that by saying the complete rotation of the radar 360 degrees. So, that would involve probably about a little less than a minute of time, something like that. Q Okay. And -- A Starting time at 0451. Q And -- and the aircraft track has then been imposed on it. A Yes. Q But the -- would you agree that when the aircraft was, for instance, flying the arc to the northeast of the airport, it would have been a different time than this picture was taken? A I can't really determine it because I didn't make up those pictures, really. Q I see. Did -- did you -- just for clarification, did you tell us that the center weather service would have gotten this information if they had not been closed? A They are on a circuit connected to us by computer, just as all the weather service facilities are. So, they would have seen everything we issued, yes. Q What -- what would you say is the -- the relative frequency of Level 6 thunderstorms in the Little Rock area in a year's time? A The frequency? Q Is it frequent, infrequent, once a year, twice a year? A I'd probably have to put it on more of a percentage basis. I would say probably about maybe 20 percent of the time. Q That you have thunderstorms that rise to the Level 6? A Yes. Q Thank you very much. CHAIRMAN HALL: Allied Pilots Association? MR. ZWINGLE: Yes, sir. BY MR. ZWINGLE: Q Mr. Wilken, with regards to the composite image that has been displayed to the public for the duration of your presentation and to which Mr. Baker referenced, is there not -- I can't see it from here, but is there not a time stamp on the aircraft track or a series of time stamps on the aircraft track that would indicate that -- that the track of the aircraft, the time element involved in the track of the aircraft is not equivalent to that of the radar display? A I would think it was. Like I said, I -- we didn't make up these pictures. Mr. Eick, I think, and his group made them up. So, I can't really directly relate to that, but that would seem logical, yes. Q Okay. Is it possible, given the elevation of your scan, of your radar scans, that the aircraft flew this track underneath the weather depicted and clear of clouds? A It's hard to tell on a plan or map-type picture which you're looking at. You would have to look at a vertical cross-section with the aircraft track superimposed on that to see that feature. Q But is it possible? A It's possible. Q Thank you. Would you define the meteorological term "microburst"? A A microburst is just a sudden down rush caused by the suspended -- usually a large area of rain, suspended about 10,000 feet or higher in the thunderstorm, and when that load exceeds the up drafts in the thunderstorm, it will immediately drop through the thunderstorm and splay out at the surface. Wind gusts have been clocked at well over a hundred miles an hour in some of them. Q And are you familiar with the operational definition of the term "microburst"? Operational meaning that which is relevant to the operation of aircraft. A No, I'm not. Q Okay. In your presentation, you made reference, and correct me if I'm wrong, I may have misunderstood this, I believe you said with reference to the VIP intensity chart or conversion chart, that it used to be called Level 6? A Yes. Q Has that changed? A Well, we use the smaller range gates of the 88D, which provide a lot of levels, and one of what we used to call the VIP levels. So, in one VIP level, you may see several gates or intensity levels in dbzs for this radar. So, we don't look at -- for our purposes, we don't use VIP levels any more. That was a WSR-57 technology that was replaced in our office in 1993. We've now gone to the smaller range gates which provide a little bit more definitive intensity values for our people to look at. Q Is this chart then for an Exhibit 5A, Page 10, still relevant for our use, given what you just said, for the purpose of the investigation? A 5A, Page 10. Q It's the end of the SVIP DBZ Conversion Chart. A Yeah. It's relevant because in the first column, you have the VIP levels which the weather service used to use. You can convert the third column over, the precip mode, in dbzs, which is now -- you can see several range gates -- excuse me -- several intensity gates in each of these or at least quite a few of these that are gathered into one of these VIP levels. So, you can equate, for instance, if you're looking at a 26 dbz echo, you can equate that out to the left as being a very light or VIP Level 1. Q But this chart is still valid, still current? A Still valid, yes. Q Still relevant? Okay. MR. BERMAN: Sir, can I interrupt you for just a second? I think we'd like to let Mr. Eick give an explanation of this chart because it is an NTSB-produced product. He can tell about how it's produced, what the markings are about the flight track of the airplane and describe it in general for the people in the back who can't see it. MR. EICK: The chart that you're seeing right there as well as a lot of the exhibits in Exhibit 5B from Page 11 on were obtained by getting the Level 2 archive tape from the National Weather Service and putting it on an HP work station back at NTSB Headquarters and using Motif Interactive Radar Analysis Software to reapply the radar data to the image. In our software, we are able to manipulate the data, such as centering it on the accident site, which we did here. We were also able to put the range rings in nautical miles, centered on the center of the airport, and we were able to change this color scale to whatever range we felt necessary to highlight the activity. The track information was obtained from our ATC group, from the National Track Analysis Program or NTAP. So, it is the history of the flight track, and that is corrected. It is not instantaneous. It is the track history with the end of the track being the final resting location. The radar data that you are seeing here is the volume scan that started at 0451 or 51 and going through. The exact image was taken. This volume scan was completed at 0451:59 and is the -- that's when that volume scan was available, and that is the presentation with track. That is the lowest reflectivity, 0.4 degrees, closest to the time of the accident. MR. BERMAN: So, Mr. Eick, the airplane's position at the time this radar picture was taken is basically the end of that track at the accident location or close to it? MR. EICK: That is correct. MR. BERMAN: And how many minutes are elapsed between the time the airplane's in that tear-drop maneuver out to the east of the airport and the accident time? MR. EICK: At the top of the tear drop is at 0441:04. So, if you go back in the other images that are already in the docket, you will see, if you wanted to match the track to the time, there are specific images to match the time period. That's why you'll see the same track on different images, just so you can find the position of the aircraft. At 0451 is the closest to the time of the accident, and it shows for our purposes the closest time to the accident here. MR. ZWINGLE: Thank you, Mr. Eick, Mr. Berman, for that clarification. I think that's very important. BY MR. ZWINGLE: Q Mr. Wilken, the -- I'm sorry this is not included in any exhibits. So, I certainly understand if Mr. Wilken does not want to comment on this. However, in the Airman's Information Manual and in the Air Traffic Controller Handbook is contained a glossary which is known as a Pilot Controller Glossary and is the only tool that I have as a pilot to interpret NWS VIP Doppler Weather Radar Information. Six levels are identified. Level 1 being weak, Level 2 being moderate, Level 3 being strong, Level 4 being very strong, Level 5 being intense, and Level 6 being extreme. In consideration of the terminology of the identification of your weather warning product being the severe weather warning, where does the term "severe" integrate? A Actually, you can have severe anywhere from a Level 3 on up. Q Okay. Thank you. Does the WSR-88D Doppler Radar provide precipitation totals over a specific geographic area? A Yes, it does. Q And how small is that area or how small can that area be? A Well, the area can be as big as the state of Arkansas. Q And how small? A Well, you can narrow it down to just, you know, do an 8X magnification like we did, and you can see right in the local area of Little Rock or North Little Rock or whatever. Q Can you tell me how many pixels cover the Little Rock Airport? A Pixel is usually about a half a nautical mile on the side. I'm not quite sure how big the airport is, but I would guess two to three across and maybe three up and down, north-south, something like that. Q Okay. Thank you. Are you able to compare the product of the WSR-88D with the product of the Terminal Doppler Weather Radar? A No, I can't. Q The precipitation intensity expected or experienced from a Level 6 thunderstorm, could that cause flooding, serious flooding in a community? A Of course, it depends on the time that rain was occurring, how long it occurred. A very quick, what I would call a, burst of precipitation might produce, you know, quarter of an inch of rain, which wouldn't be a big problem. You have other things to consider, like antecedent conditions, like if the soil was saturated and other things like that. If the soil was saturated, it would flood, of course, much more rapidly, depending on the time scale of the rain itself. So, there are a couple other things to consider which we do consider in the course of looking at flash flooding. Q But in the general community, it is possible? A It is possible, yes. Q Thank you. My last question. In consideration of the intensity of the weather, the intensity of this thunderstorm in terms of precipitation and wind velocities, would it have been reasonable for someone with knowledge of this, the existence of this weather, to have warned these pilots, the crew of 1420? A Well, it goes back again to what we normally do, and normally that would not -- Q I'm not asking for you. I'm saying someone. A Someone? Q Someone with the knowledge of this information. A Well, I -- I think the answer -- it sort of answers its own question. I think it's sort of yes, really. Q Sort of yes? A Well, I mean it's yes. Q Thank you very much. No further questions. CHAIRMAN HALL: The Association of Professional Flight Attendants? MS. LORD-JONES: We have no questions. Thank you. CHAIRMAN HALL: The Little Rock National Airport? MS. SCHWARTZ: No questions, Mr. Chairman. CHAIRMAN HALL: The National Weather Service? MR. KUESSNER: I have no questions. CHAIRMAN HALL: No questions. Now, Mr. Wilken, I just want to clarify one thing before we go to the Board of Inquiry. Do you think you do a better job in accurately forecasting than the three tv stations here locally? THE WITNESS: Considering they have our forecast to look at, and we don't have theirs, yes. CHAIRMAN HALL: Mr. Sweedler? MR. SWEEDLER: I have no questions, Mr. Chairman. CHAIRMAN HALL: Mr. Berman? INTERVIEW BY BOARD OF INQUIRY BY MR. BERMAN: Q Sir, was the -- was the gust front that was shown on your radar, was that generated locally by the thunderstorm that was to the northwest of the airport at the time of the accident or was that part of a general frontal cold front passage or larger weather scale feature? A I don't recall exactly. You can look back toward the front of the weather and look at the synoptic conditions at that time, but just looking at the radar alone, it appears that that was generated locally because it fits pretty much the model of what it would look like flowing out from a thunderstorm. Q Okay. And to what degree is your -- your Doppler Radar a real-time product? I heard you say that it takes one minute to make a single scan, nine minutes to make a compilation. Is there any other processing or -- or work required before that picture could be shifted to -- to a pilot if the communication means were available? A The base reflectivities are available -- all the base products are available instantaneously. The reason I showed, and we used what's called storm relative motion, is that if you look at a base velocity product, you'd be looking at all the velocities, including the environmental velocities that would somewhat mask some of these features. The SRM, the storm relative motion, subtracts the environmental velocity and gives you basically a snapshot of the velocities at that point in time. So, I think it's well worth the time for the processing to take place to have that product available because that's what we very often use for putting out warnings, especially tornado warnings. Of course, when we look at tornado warnings, although we didn't have that problem here, we would look at a couplet developing; that is, what we call just a tornadic couplet where the wind is in-bound/out-bound in very close proximity to one another, and that is something that storm relative velocity shows very easily. Base velocity has trouble showing that. But all the base products in short are available as soon as that elevation cut is completed. Q And about how much delay would be required for that summary product that you mentioned? A The SRM would be about a six-minute delay on that. Q Okay. A That's the length of the volume scan, basically. Q Okay. Thank you. No more questions. CHAIRMAN HALL: Mr. Haueter? MR. HAUETER: I have no questions. CHAIRMAN HALL: Mr. Clark? BY MR. CLARK: Q Earlier, you were looking at several elevation slices of the radar data, and you pointed out an area that you interpreted to be a developing microburst. Do you have -- is your operator on duty looking for those specific events as they occur on the slice-by-slice basis? A Yes. Our people are basically -- it's the old story of having a property of an aircraft and a jet. In the old technology, the 57th Radar, we had the prop-driven. It was a three rpm radar. It required strictly manual technology to recognize features which were few and far between. This radar produces a full suite of products every six minutes. So, the radar operator has to keep well ahead of that power curve and keep with the radar in examining each volume scan, each sweep of the volume scan, as it occurs, and once again, we expect our people to recognize features before the radar does. I set up all the thresholds in the radar for various things, like precipitation, wind speed, reflectivity values and whatever, but before the alarm goes off on the radar, we would fully expect the operator to have that well in hand by that time, even two volume scans before, and part of that is because we're using time lapse to watch trends as well, and the trends are very important in radar to get an idea of what's going on. Q Okay. That particular area where you identified the -- the potential emergence of a microburst, if we're looking at two scans previous, is that -- do you mean that's 12 minutes earlier? A Yes. Q And that particular event should have started -- the signature should have started appearing about that time, the 12 minutes earlier? A Well, when we start to see a color change -- and we developed the color scale, and that's one of the other reasons why I wanted to use our pictures from our radar. The color scale we have developed has better and quicker recognition to it. We have light colors within dark colors and so forth, so that you can really pick out the features as they're developing, and that's another thing that our radar operators have the advantage of, is that specific color scale, which is not in use by every office. Every office can choose to use the one they want. We find this color scale is very good for that and will allow us very quick recognition of a particular feature developing. Q When your operator starts recognizing that a -- a potential microburst is developing, how long does it take him to get that information to the tower? A If he puts out a warning, that information will go instantaneously. A warning could be put out in less than two minutes. Q I'm not familiar. Was a warning put out for that particular event? A There was a warning put out for something before then, about 18 past the hour, about 11:18, 0418. 0418, there was a warning issued for the approach of the line because we had had some reports of wind damage with the line as it was moving across. So, Pulaski County, including the atmosphere, was included in that warning. Q Okay. And that particular warning was 18 minutes earlier. What about the specific event that you pointed out that -- that the time line you were showing was radar data from about the time the airplane touched down. Was there a warning put out associated with that? A We already had a warning in progress. So, there was really no need to put out another one because that would tend to confuse everybody that, you know, got the product. The warning that we had put out earlier at 18 past was in effect for one hour. So, that would encompass the time the aircraft touched down. Q So, even though you see a specific -- a specific event developing directly over the airport or in direct proximity to the airport, you don't put out an additional warning for that specific event to the tower? A Not necessarily an additional warning, but we would put out something. We have severe weather statements that we put out to track particular events as they're occurring, and they're issued between the warning times. So, this warning was issued about 18 past the hour. We probably would have issued two or three severe weather statements by the time the aircraft touched down. Q Was there any severe weather statement put out that -- one particular event that you pointed out to us, the developing microburst or the potential, was there a statement put out associated with that event? A Not particularly for that event, just generally for the wind speeds that were producing downed trees and other things as the line moved through. Q Is that -- it appears, though -- you heard earlier -- I assume you were here yesterday. The -- you heard about the tactical and strategic weather dissemination or tactical operations, for example. It sounds to me that you don't consider your weather product to be directly involved in a tactical operation. A Well, our weather product is -- is put out for aviation and public, but our focus is more on public really on that particular product, although watches that are put out earlier, preparatory to having warnings issued by the local office, do include aviation weather. Ours again is worded more for the public rather than the aviation community. Q Are there -- your weather products go to other places. Are they more in tune to supplying more direct weather information for the aviation community or is it your office that does that? A I don't quite understand what you're getting at. Q Well, it seems like the -- from what you described, you have very pertinent information that could be used almost immediately at the Little Rock Tower, and it doesn't appear that that very specific information is being relayed directly to the Little Rock Tower. Is there -- are there other people that use your product to provide that information or is that solely your -- A I'm not aware of anyone else providing aviation information to the tower, if that's what you mean. Q Right. Okay. All right. Thank you. MR. KUESSNER: Excuse me. Chairman Hall? CHAIRMAN HALL: Yes? MR. KUESSNER: Weather Service here. I know this is unusual, but I -- I'd like to ask some follow-up questions based on some of the questioning from John Clark and the representative from APA. CHAIRMAN HALL: Sure. Do any of the parties have objection to the National Weather Service following up? (No response) CHAIRMAN HALL: No objection. Proceed. INTERVIEW BY PARTIES TO THE HEARING BY MR. KUESSNER: Q Now, you said that your warning, the severe weather warning was essentially geared toward the public sector, is that correct? A That's correct. Q Are there other products that -- that you put out that are geared specifically for aviation? A Well, yes. We have the Terminal Forecast or TAF as it's called, and then the Transcribed Weather Broadcast, the TWB. Q Now, you were here when we heard the testimony of the first officer on Wednesday. It seems like an eon ago. A Yes. Q And you heard him acknowledge that he received a Terminal Forecast for Little Rock, -- A Yes. Q -- which included a forecast of thunderstorms and wind gusts of 40 knots for the time that they had intended to land at the airport, is that correct? A That's correct. Q Also, you heard that the first officer acknowledged receiving a convective SIGMEC which included the airport? A Yes. Q And that convective SIGMEC included advice of a severe thunderstorm with wind gusts of 70 knots possible and hail to two inches possible, is that correct? A That is correct, as I remember that specifically. Q Is that an advisory that is geared and aimed at the aviation community? A Yes. Q Also, did you hear testimony that the crew in this case received an observation of gusts of winds of 45 knots, gusting to 45 knots, at the field? A Yes. Q And also receiving information that it was raining heavily at the airport, and the visibility had dropped down to one mile? A Yes. Q Thank you. CHAIRMAN HALL: Very well. Well, since we're covering what you heard yesterday, did you hear the terminology "bowling alley" and "blanketed"? THE WITNESS: Yes. CHAIRMAN HALL: Are you familiar with those or -- THE WITNESS: I can't say I am, no. CHAIRMAN HALL: -- are those commonly used? THE WITNESS: No. That's the first time I'd heard either term. CHAIRMAN HALL: And, so, you wouldn't have any definition as to what either one of those mean in weather terms? THE WITNESS: Neither are in my glossary of meteorological terms, no. CHAIRMAN HALL: Well, okay. Very well. Mr. Wilken, you've been an excellent witness, and we would like to give you the opportunity if there's anything else in addition you would like to add for the benefit of the Board or that you think might be something the Board should look at in terms of preventing an accident similar to this one recurring. THE WITNESS: Well, I think we at our office and in the Weather Service at large always try to do things better than we did them before, and that's part of my job, to do that quality assurance, and I think we are very open to suggestions as to how we can do things better. So, I feel that if -- when the Board puts forward their decision on how things worked on this particular accident, we would be very interested in seeing that report and seeing any recommendations from the Board. CHAIRMAN HALL: Well, you have an important public position, and I certainly appreciate that attitude. That's excellent. You're excused, sir. (Whereupon, the witness was excused.) CHAIRMAN HALL: Mr. Berman, would you call the next witness? MR. BERMAN: I call Dr. Wes Wilson. CHAIRMAN HALL: We'll get our next witness started and go 15-20 minutes, and then we'll take a break. We'll take a break at the top of the hour. So, wherever we are when we get there. Whereupon, DR. WES WILSON having been first duly affirmed, was called as a witness herein and was examined and testified as follows: INTERVIEW BY BOARD OF INQUIRY BY MR. BERMAN: Q Good afternoon. Would you please state your full name and business address for the record? A Wesley Wilson. I work for MIT Lincoln Laboratory in Lexington, Massachusetts, 244 Wood Street. Q And what's your present position at the Lincoln Laboratories? A I'm a member of the Technical staff. Q How long have you held that position? A I've been at Lincoln since 1990. Q Please describe for us your duties and responsibilities as a member of the Technical staff. A Well, I've been -- I have a fairly long history of working in aviation weather. I was part of the original -- before I came to Lincoln, I was a member of the original group of scientists who did research in wind shear and helped develop the FAA Wind Shear Detection Systems, and I have worked on a variety of aviation weather products while at Lincoln. I am currently the lead of the FAA Ceiling and Visibility Product Development Team. Q So, just to also clarify, you're mostly involved in aviation research for most of your job or all of it? A Right now, I do a little bit of operational weather research for the Navy, but mostly it's FAA aviation weather research. That's correct. Q Thanks. And also please summarize for us your experience and training that brought you to your position. A Well, I have a Ph.D. in Mathematics from the University of Maryland a long time ago, 1964. I was a professor for about 23 years, then I worked at NCAR for almost 10 years on aviation weather, and now I've been at Lincoln for 10 years on aviation weather. Q Okay. Thank you. Do you have any FAA airman certificates, pilot experience and the like? A No, I do not. Q Okay. A I guess one other thing I should say, I am the source scientist on the LWAS Algorithms and on parts of the TWR Wind Shear Algorithms. Q Thank you. Do you have any National Weather Service certifications as an observer or -- A No, I do not. Q -- anything like that? Okay. Great. Thank you very much. MR. BERMAN: Mr. Eick? INTERVIEW BY THE TECHNICAL PANEL BY MR. EICK: Q Dr. Wilson, good afternoon. I understand that you have a presentation based on our exhibits on the LWAS data or the Low- Level Wind Shear Alert System at Little Rock? A Yes, I do. Q Would you like to proceed with your presentation? A Sure. CHAIRMAN HALL: How long's that take, the presentation take, Doctor? THE WITNESS: It's about a 30-minute presentation. CHAIRMAN HALL: Why don't we do this, so everybody will be focused on it? Let's take our break now, and we'll come back so you can start your presentation with a fresh crowd. THE WITNESS: Sure. CHAIRMAN HALL: We'll break for 15 minutes. (Whereupon, a recess was taken.) CHAIRMAN HALL: Could I ask everyone now to, you know, to please take a seat, and if you have a private conversation, please take it outside the room. We'll reconvene the hearing. Please proceed, Mr. Eick. BY MR. EICK: Q Yes, Dr. Wilson. Will you please present your presentation for us, please? A Sure. Okay. I've been asked to give a little background, theoretical and historical, on the LWAS System, just so we can have a conceptual understanding of what the system did on the evening of June 1st. CHAIRMAN HALL: You might explain just briefly for everybody what LWAS means. THE WITNESS: Okay. LWAS is Low-Level Wind Shear Alert System. It is a -- it is a system of anemometers at an airport, a network of anemometers in an airport, and we'll see some planned views in a little bit, which is one of the FAA systems for the detection of wind shear. I think as we go through the talk, you'll see there's quite a history. This has been around for awhile. I'm going to start by talking about the characteristics of wind shear, then a description and history of the LWAS System, and there are really three or actually historically four different LWAS Systems. I'll talk about its general capabilities that have been established through field experiments and then finally the -- discuss what happened on June 1st. Wind shear -- well, there are a lot of things that could be called wind shear, but there are two dominant types of wind shear that are considered to be the prominent hazards to aviation. They are the gust front wind shear and the microburst wind shear, and we've, of course, already heard both of them discussed a little bit today. I've drawn some cartoons here. In the gust front wind shear, we have a pool of dense air which comes from some far-away event, which is spreading outwards because of just buoyancy. At the leading edge, there would be a strong -- usually a strong surge, and then as that pushes into the still air, we have an area where there is a masking or a -- what's in fluid dynamics called convergence. The microburst wind shear is a falling column of air, hits the ground and splashes, and as it flows outwards, we have in the horizontal wind field a loss of mass or a divergence. The -- if we're thinking of detecting this kind of wind shear with anemometers or a series of anemometers, we have to think about what are the signatures we might expect to see. If we have two anemometers, then the convergence signature would be wind flowing into this area to create a mass gain or a very strong inflow and a weak outflow would give the same resulting convergence. If we had three anemometers forming a triangle, we either could have inflow everywhere on the triangle or we could have very strong inflow coupled with weak outflow. In this situation, we get the convergence signature. What that's trying to do is identify that we are right here ahead of the gust front. Sometimes we either have only one anemometer or the anemometers are so far apart, that only one of the anemometers can see the event. In that case, -- oops. Oh, I'm sorry. Back to my face. In that case, we'd be concentrating on trying to see when the surge goes across the anemometer, and we would look for what's called an anomalous wind. What this would be is a wind that is suddenly very unusual compared with the expected wind in that region. So, we have to have some other reference which has given us what the expected wind would be, and then we suddenly see a departure, and that would be the anomalous wind signature. For the microburst detection, we do essentially the same thing, but now we look at two anemometers and the line segment between them and look for an outflow or a mass loss, could be shown by a weak inflow and a very strong outflow. On a triangle, we look for the mass loss in the same general scenario of a net resulting mass loss. Again, if we only had one anemometer or our anemometers are very widely spaced, we might look for just a surge at the anemometer. So, the anomalous wind is again an indicator -- in the case where we don't have a dense network, the anomalous wind is another indicator. That should be a warning to us that when you see the anomalous wind, you don't know what is causing it. You just know you have wind shear, and, unfortunately, an anomalous wind could also be caused by a wind shear that is not an aviation hazard, such as a dust devil. So, when we use this particular detection technology, we have to be very careful or we will be giving false alerts or what the pilot will operationally perceive to be a false alert. Let's press forward. I want to now talk about how this -- general ideas about wind shear can be and have been applied to the building of -- design and building of the LWAS System. The LWAS design and indeed at sort of the same time, the TDWR design came out of a series of meetings that were conducted by the FAA, known as the Wind Shear Users Group. That group was composed, of course, of people from the FAA, scientists from several research laboratories. Some of the pilots organizations that are seated here today were members of that. The airlines were represented. ATA was there. NASA was there. It was pretty much everybody that had a stake in aviation safety and wind shear were brought together in one room, and together, we designed the system that FAA eventually made -- imposed the requirements for. The basic precepts that came out of that were, first of all, we had to minimize false alerts. The alerts that were given by the system had to be believable. Of course, the purpose of it was to detect hazardous wind shear, hazardous to aviation. From the research and the scientific community brought in the fact that the scale of the phenomena was a couple of kilometers, and therefore if we wanted to detect it best, our sensing system should reflect or respect that scale. When we took that knowledge to the LWAS, the -- what came out of that group was a requirement that the station spacing should be two to two and a half kilometers. The reason for that was that the research had shown that if you had that kind of a station spacing, then the divergence and convergence became very reliable indicators. Reliable means two things. First of all, it's able to detect the event, and, secondly, if it makes a detection that the -- that the event really is a hazardous wind shear. Unfortunately, we also looked at situations where the station spacing was larger, and in that case, we knew we could detect wind shear with -- with the anomalous winds, but that the detections would be less reliable. So, false alerts became a problem. At that point, a notion of persistence was introduced which said that the shear had to be detected for several repetitions before the alert would be given to pilots. In the LWAS System, those repetitions are every 10 seconds. So, this delays alerts 40 to 50 seconds. Okay. Quickly through the history. The first LWAS was not designed at this time. Indeed, it was designed 10 years earlier and was already operational at the airports. It used only the anomalous winds alerts scheme. It had wind shear alert by sector in the northwest, east, whatever, of the airport, and it used six sensors. A centerfield sector sensor and boundary sensors around. This system was designed to detect gust fronts encroaching on the airport. Coming out of that work with the user group, an algorithm -- we designed an algorithm which was based on the divergence/convergence technology. This is the algorithm that the FAA selected to go into the LWAS System. NE is for Network Expansion. This is in some sense the Cadillac of LWAS Systems. Unfortunately, they turn out to be difficult to install because to get full coverage of where the airplane is in hazard, it has to be put on off-airport property which has caused a lot of complications with getting real estate rights on the land that the airport doesn't own and control. The system was designed and specified in the late '80s, and yet the first system wasn't deployed until 1996. This was because of the requirement or need to procure a lot of hardware and to do these complicated installations. The savvy folks at the FAA figured out back in the mid- '80s that it was going to be a long time before they saw these. They recognized that wind shear was happening. Airplanes would be flying, and, so, they asked could -- was it possible to take some of this technology for the NE and to apply it to the existing hardware of the old LWAS to get some microburst detection and to reduce false alerts to hazardous wind shears, and that became the LWAS-2 System with what's called the Wind Shear Algorithm, and because we had to fit the old hardware, we have the sparse station spacing, and we can only get wind shear alerts. This system never says the word "microburst" even though internally to the algorithm, it detects microbursts. The reason for that is the display has no way to issue the microburst message. So, this is a hardware constraint. The reason this is very important to us today, this is the LWAS that is in operation at Little Rock today and was in operation on June 1st. Again, just very quickly, what would the ultimate LWAS look like? It would have the close station spacing. It would be able to say wind shear alert or microburst alert based on what the algorithm detected. It would use the divergence or convergence as the signatures. A microburst is what is -- what the message is given when there is a very strong divergence. Otherwise, it would be a wind shear alert. However, again at the guidance of the user group, the FAA accepted the ruling that the weaker alert, the wind shear alert, is given only if we actually measure a head wind loss or gain along the runway, and the reason for that is that means that if a pilot ignores the alert and lands, he will likely encounter and feel a loss or gain as he flies into that approach, and it was felt that pilots didn't want to hear alerts about events that weren't going to actually affect the performance of the aircraft. So, this was part of the guidance of the user group, part of the direction from the FAA. So, the LWAS NE is designed to -- with that extra caveat in the alert. LWAS-2 -- and I've decided to show you a map now. This is the Little Rock Airport. These are -- the red numbers are the six locations of the LWAS stations. The blue letters are the sectors where the wind shear alerts are given. So, a pilot would be given wind shear alerts centerfield, wind shear alert north, wind shear alert northeast, depending on the circumstance, and those alerts are based on a new analysis which involves both a divergence estimate, and I'm showing here some typical edges or a triangle formed by these stations. Because of the fact these stations are so far apart, these are fairly long edges and triangles, and it's possible that a small microburst could just fit in between and not be detected at all. Because of that problem and the gust fronts, we also do an analysis for anomalous station winds, and the -- if an anomalous wind is detected, and it is persistent for five scans, then an alert is issued. So, this is the basic philosophy of the system that is in operation at Little Rock today. Just to give you a comparison, the FAA is already procuring an improved system which would be just on airport. It would not go out far beyond the runways to give the full extension capability of the NE. This is the restricted system, and what it will be is on airport sensors but using the advanced algorithm techniques, and that is the coming technology. I think the first article should be delivered this year, and they will be installed this year and next. At Little Rock, that means sensors will be added in these four locations indicated by the red dots, and you can see how that will reduce the size of the triangle, and I'm going to ask you to flip backwards and forwards just once or twice. Backwards, we see the sparse geometry. Forwards, we see how much denser it gets by adding these few extra stations on the airport. CHAIRMAN HALL: How many LWAS-2s will get that? All of them? THE WITNESS: All of the LWAS-2s will be upgraded. CHAIRMAN HALL: And how many are there? THE WITNESS: This is -- or they'll be decommissioned and replaced by ASR-9 WSP. CHAIRMAN HALL: Okay. THE WITNESS: So, some will get the radar, some airports will get this. When -- but there will be no LWAS-2s left after this procurement is complete. CHAIRMAN HALL: Okay. THE WITNESS: Okay. Let's pop forward. I want to talk a little bit about the performance statistics. Now, this is not what happened at Little Rock. This is just how well the systems work in general, which was the basis for the FAA's decision to deploy these systems. This is based on meteorological field experiments where other sensors, like radar -- extra radars were brought in, so that we could accurately estimate what the true wind shear circumstance was. Okay. I've got the three LWAS systems here, the NE, the RS, which is the restricted system, and the LWAS-2, the one we have today. My measures of scale are the probability of detection, the probability that wind shear alert exists -- is issued when there is wind shear at the airport. We have either generic wind shear or what is the probability that the alert will be issued when there's a microburst at the airport, and then we also have to worry about the false alerts. I'm using here the probability of a false -- that an issued alert is false. So, this is the percentage of issued alerts which are not appropriate. For the notion of wind shear, I have two notions. One is wind shear along a runway, and the other is just wind shear in some direction at the airport. So, the "all" indicates wind shear in some direction. This requires that there be a head wind loss or gain on one of the runways. The LWAS NE, the so-called -- I call it the Cadillac LWAS, detects three quarters of the wind shear, virtually all of the microbursts, and has a very low chance of issuing a false alert. The LWAS RS is just as good, but it only covers 60 percent of the area because it doesn't have the runway extensions. So, these numbers are simply 60 percent of these numbers, and the reduction there is because of the lack of coverage, not because of lack of algorithm skill. The LWAS-2 has the same coverage as the RS but has the much sparser stations and the older algorithm. So, it drops in prediction -- yeah. The detection capability rather substantially. It does successfully reduce the false alerts, if we use the criteria of false alerts in all directions. Since it has no test for along runway alerts, it must issue alerts for shears that are not along the runway. If you score it against the requirement of a shear along the runway, it has more false alerts. This is a training problem because the pilots who ignore LWAS alerts today will not see the head wind loss or gain and will perceive that the system is giving false alerts even though there is a wind shear at the airport. We would hope, of course, that pilots don't test the system, especially when they have guys in the backseat, but if they should inadvertently go through when a wind shear alert is being offered, they may perceive a false alert when the system is in fact correctly detecting wind shear. MR. CLARK: Excuse me. How would these numbers change if we're dealing with events on the runway? THE WITNESS: Well, this -- MR. CLARK: Part of the numbers are biased for -- THE WITNESS: These numbers are for the winds along the runway. MR. CLARK: I meant on the runway, around the runway. Part of the -- the far-right numbers, I understand, -- THE WITNESS: Okay. MR. CLARK: -- were extended off of the runway, and our event was -- THE WITNESS: Okay. All of the -- the LWAS NE and LWAS RS alerts are runway-specific. They are only issued to the -- the alert is issued only to the runway where the shear is present. The LWAS-2 alerts are issued by sector, and the sector in some sense is associated with the runway, but the -- because of the sparsity of the situation, there's less precision in the LWAS-2 alerts. Okay. I think what we came for was to find out what happened on June 1st. Was the system credible? First of all, what was the meteorological situation? How did the system measure it, and was that a credible interpretation of the meteorology? So, let's pop on. I think I'm going to talk to this one backwards. The first issue is did the system measure the wind correctly. The second issue is after it measured the wind correctly, did it make a correct interpretation of the wind shear circumstance? In this course of upgrading a system to RS, the FAA has sent out teams to do surveys of every airport and, where necessary, either move existing sensors or raise the height of the pole of the sensor so that the sensors will measure the true winds. This had already been done at Little Rock. So, because we knew that they had just improved the system siting, we expected to see good performance. The FAA Program Support Facility in Oklahoma City took the data recorded for the month before the accident and analyzed it statistically to see if there was any evidence of sheltering in the winds from that recorded data set. They're using statistical software that I helped design, and I trained them in the use of. They concluded that there were no problems. The system was performing properly and well. I reviewed their work, and I concurred. So, I think we don't have to worry in this case with the sensors not being properly located for the use intended. The sensor data, the LWAS wind data, are archived. I have access to those, and I've analyzed them in ways that I will describe, and I also have access to the archived LWAS data, display data, what appeared on the tower displays for the controllers, both the winds and the alerts that they saw, and they are in this exhibit. DR. BYRNE: They're in Exhibit 5B, beginning on Page 27, and up to the accident time on Page 30. THE WITNESS: Okay. I am going to show you why this is not a perfect or complete data set from a scientific viewpoint, and, so, we have to either admit that there are gaps in knowledge or make some guesses, and I think it's not appropriate to guess. So, I will simply tell you what I can determine with some credibility from the data I see, and in that sense, what I'm going to be doing -- conducting is what I call a subjective analysis. A subjective analysis is when human experts sit down and ponder the data and try to figure out what was going on. In this subjective analysis, I was assisted with -- by several of the scientists at Lincoln Laboratory, a couple of whom sit down at Memphis, Eric Proteus and Mark Eisenbinger, and review microbursts and wind shear events performance day after day, all summer long. They've been doing this for several years. So, I've had some help from some really good experts in this. But that's what this line and this line amount to. Best estimations of what went on by human experts. I also have access to a simulation of the LWAS-2 algorithm. Now, by this, I mean I have a software program that executes the LWAS-2 algorithm, and it's accurate to the best of my ability to determine that I have good software. I must be very careful and point out this is not the official FAA software, and therefore there are some differences in how it performs, some of which I can sort of explain, some of which I cannot explain. The FAA software has gone through acceptance tests and a lot of scrutiny that my software has not gone through. So, when in doubt, I tend to believe that software a little better. Unfortunately, the LWAS-1 and LWAS-2 do not have with them what's called event reconstruction, which is the ability to run the actual software and put hooks into it to see what it did and why it did it. LWAS NE and LWAS RS will have event reconstruction capability delivered with the system. But we are handicapped here. That's just not a capability that comes with this system. I've tried to describe what was going on here in sort of a pseudo-time line. Across here, we have the time. This is approximately the last few minutes of flight, and the time when the aircraft was on the runway. From what I've heard here, I gather I have this slid about a half a minute early. The 88D took a scan at about 4650, six minutes later, would be 52 and 58. That would be the velocity fields. The reflectivity fields are a minute earlier. There is a six-minute step between here. The LWAS is 10-second data. So, it's very fine time sliced through the event. It is unfortunately on a very restricted patch of real estate, and with these sensors that are a little bit too widely spaced to be appropriate for fine details about what happened to wind shear. So, our data set is a little bit inadequate on two ways. The radar is very precise but porous in time. The LWAS is very fine in time but doesn't have the spacial resolution that we would like to have. So, we'll do the best we can. The LWAS System. I've put across here the gust front that we -- that was discussed earlier, went through. LWAS issued wind shear alerts on that gust front up till about, I think it is, 11:36 or 37. Those alerts started in the north, and as the event moved across the airport, moved to centerfield and then to the south. There was a secondary front around 11:40 which lasted -- this caused alerts which lasted for a few minutes, and with wind shear alerts here primarily in the north and northwest. It went quiet for a little while. Around 11:47, there was a wind shear alert issued for just a little over a minute, and this was issued the alerts at centerfield north and northeast. Then those alerts stopped. There was a wind surge that I've been asked about. So, I'm going to specifically talk about this wind surge at the northwest sector. I put question marks here. The operational LWAS did not issue alerts on this wind surge or at least the archived data set does not indicate that the alerts were issued. My running of the software does indicate for several scans that there was an alert level. So, we have a discrepancy here that I cannot explain easily. I won't even try. They're just different. And then, finally, at about 11:52 -- now, everything sort of had stopped with the plane by 11:51. By 11:52, a microburst impacted the airport in the northwest, and then LWAS in a fairly timely way issued a wind shear alert for that. Remember that's all LWAS can say is wind shear alert, and that wind shear alert went on for 13 minutes. So, we're going to look at the data. I'm going to take you through the data step-by-step. This is what's happening event- wise. After we've gone through the data, I'm going to put this back up again, so that we can look again at what we just saw. So, I'm going to tell you what -- I've told you what I'm going to tell you. I'm going to tell you, and then I'm going to tell you what I told you. Let's move forward now. We got the base data from the NEXRAD. Eric Proteus from Lincoln ran this analysis for me. This has somewhat finer scale of data resolution than we were looking at before, and also I've zoomed in so we're looking fairly closely in a fairly tight region around the airport. What we -- this is the reflectivity over on the right. Reds are my top level. So, this is more like VIP levels. We see this squall line down here, a little shower down through this region, and then this intense rainfall up here indicated by the red. On the other side, the right side here, I have the velocity field. The blue colors, the cool colors, are the winds towards the radar. The warm colors are the winds away from the radar. Here's that old gust front that went by, oh, 10 or 15 minutes before. The time here is 11:46, and the -- I have put on here the LWAS winds, rough indications of the LWAS winds around the airport, and the radar says -- the -- yeah. The radar indicates the winds blowing away from the radar. The LWAS winds indicate the wind's going away from the radar. So, everything's in agreement here. I would point out that the action line, the storm and the other stuff, is still separated from the airport by some amount here. I also want to point out something else, that with the data delays that were discussed in the previous discussion, that the - - at least six minutes would pass before the National Weather Service meteorologists could see this. Even if you had -- and this capability doesn't exist today, except among scientists. Even if you had really good data communication and other things, a delay of 12 to 15 minutes, especially if you went through a vendor, it will certainly be 15 minutes, is what you would expect. This is the picture that would be available to the user community six minutes after the crash, even if we had all this stuff hooked together, and we were delivering stuff to the pilot just because of the delays that are inherent in the system, and as we go forward, you're going to see a lot happens in this next 12 minutes. This is the reason why the FAA has built their own wind shear systems, the TDWR, etc., because these delays from this kind of sensing system are just intolerable. So, let's step forward. This is six minutes later, and this -- we did see this. The rain has mostly abated, except for this very sharp shaft right of the northwest part of the airport. So, this is the rain shaft that people have been talking about. This is the wind field. The wind field here shows a very weak divergence. The divergence here is across the beam of the radar. The LWAS winds show a fairly strong divergence. The radar can't see it, but the microburst is in fact starting at this time, 11:52. The radar is not picking it up, and as we move to the next time now, this -- the -- this is the time that was gone through with some scrutiny. The precipitation pattern has broadened, and the microburst is now clearly visible. The aviation definition of a microburst is at least 30 knots over no more than four kilometers. This one shows 50 knots over the four kilometers. This is clearly a microburst. LWAS also gets that 50 knot number. As we go -- I'm going to go into the LWAS winds now, and it's worse than it looks. What we're going to see is an awful lot happened as is revealed by the LWAS wind field that's even subscale to these radar pictures, and remember the -- the end user who's buying this product is still looking at that first picture. All this stuff has happened, and now a lot more -- I'm going to show you a lot more has happened, that LWAS correctly analyzes and was not even available to the radar meteorologists. So, we're back to 11:46, the beginning of the event. What we see here are nominal 20 to 30 knot winds, gusty winds, around the airport. As we step forward, same sort of story. I want to bring your eye to the fact that in these three stations, I suddenly have a fairly strong outflow and a weak inflow. This is at the vergence signature we're looking for, and indeed LWAS started making the detection, but because of persistence hasn't issued an alert yet. Let's see. Can we pop forward? There we go. The winds are now getting strong, and actually the detection's started in between these two time scans, and at this point, LWAS is making that wind shear alert that we saw the system issued. So, it issued this alert because of this divergence, and the alert is located at centerfield northeast and north. That's the alert that was given to the pilot. That situation persists. That starts to weaken now. The system has a linger of the alert which holds the alert for awhile after the event. It doesn't jump to an all clear until it's pretty sure it really is all clear. So, the alert persisted, and this is the last pole in fact when that alert is issued. I want to stop here and point out that while this is going away, right at -- the next 10 seconds, this alert is gone. This sensor starts showing winds. Now, this is the alert that wasn't issued. That's a station anomaly. We step forward. That gets to be a pretty good gust over there. It now weakens and goes away. Now, this is -- in some sense -- I'm sure this is not a dust devil, but this would be the kind of a signature you would see from a localized feature that was not a hazard to aviation. I can't tell you what this is because whatever it was was happening outside. It influenced that one sensor and then stopped. Now, one issue about something like that, oh, wait a minute, there's a lot of real estate in here. Is that really something dangerous that's crawling across the airport that's going to cause a problem, and we just can't see it because of the sparse station spacing? That's a real concern. I've looked at the data, and sometimes that's what's going on, but then you see it later. This time, I don't believe that was what's going on. I really believe this was a local event out in that sector that didn't go forward. Let's look at the situation. We're down to our 20 and 30 knot winds, still down with our 20 and 30 knot winds, and now, oops, what's that? Well, we've certainly got a surge down here by the runway now. However, it's coupled with an almost stopping of the wind up here. What's happening, I believe, and my colleagues who've examined this case with me believe, is that a microburst has come down right here. It's sending an outflow in this direction, an outflow in this direction, which is enhancing this wind, causing this large number and canceling out the wind that was coming in with that northwest flow. It was actually -- LWAS -- actually 40 seconds later, not 30 seconds later, which is the time I've got here. LWAS issued a first alert on this because it did correctly detect this divergence over several poles, and I've drawn now a picture of my best guess of where this microburst was. Remember, we saw the bright spot in the reflectivity field at 11:52, that red dot right on the end of the runway, and this is what the LWAS winds are at exactly the same time that we're looking at that red dot. Stepping forward by 30 second steps -- and I will assure you I have looked at the 10 second data. There aren't any surprises that I'm not telling you about. I just didn't want to keep you all afternoon looking at pictures of wind arrows. So, I've stepped it up to 30 seconds, and we see that this thing is there. It stays there. It's not moving. It's pretty well planted right there on the northwest part. It does begin to expand, and now, boy, we have a really classic picture of a microburst as seen by anemometers here. It has now expanded to enclose the whole airport and keeps doing this. This -- I jumped forward two minutes. This matches that last radar scan where the radar showed us the same picture with the microburst. So, what I've done with a few sensors is taken you through what I saw in the wind data that I had available to examine, and, so, now I'll wrap up and tell you what I conclude from all this. First of all, let's go back to the time line. What we've done is we've looked at this 12 minutes here sort of carefully. We found out why this wind shear was offered. We have an explanation for that wind surge that didn't issue a wind shear alert, and then we found the microburst and saw the microburst outflow. What I'm -- my stated conclusions about this are, and I'll emphasize the word "issued" here, I have looked at all of the alerts LWAS issued and all of the data, and I don't think there's any false alerts in there. The issued alerts, I believe, were correct on this day. That's not always true with LWAS, but this particular day, I think the issued alerts were very credible and gave a good interpretation of what was going on. This wind shear alert at 11:47, this is the alert that was given to the pilot, I believe, this alert was a proper wind shear alert, and it was properly located for the -- where it was located in the alert messages. The simulation indicates an additional wind shear alert. It was fairly brief, but it was an alert that my running of the software indicates would have passed the persistence test, and that alert was not issued. Then there was a microburst alert. The -- we saw that situation. The alert was very timely. It started in the northwest sector. It moved slowly to the southeast, which is mostly the enlarging of the system, and I will repeat, I don't think that northwest surge was associated with this microburst. So, this was not -- that was not some early warning about the microburst. That was something else, then this microburst came down, and the main reason I believe that is because once the microburst came down, it didn't move. It stayed planted, almost rooted where it was. It's hard to believe that something would come screaming into the airport from the northwest, stop on the end of the runway and sit there for five or 13 minutes. So, that's the basis for my conclusion that these were two different meteorological events that were just going on. Remember, this is a very chaotic wind field. There was a lot of stuff going on that day, and, finally, of course, we saw that the LWAS system this time actually nailed the detection of that microburst. It's not always that good, but we'll take credit for it. I guess I'm done. BY DR. BYRNE: Q Dr. Wilson, can you tell us something about the comparison of the simulation software that you mentioned in the operational alerts? A Okay. The simulation -- okay. The operational system is doing a bunch of stuff. It is poling the stations. It is bringing the data in. It is running the algorithm. It is driving the displays. It is archiving the data. A lot of things going on. My software is simply software which executes the wind shear algorithms from a data file. So, my computer is not loaded with other tasks. The -- there is also an issue that part of the -- what we do to minimize false alerts is we do a lot of data filtering and stuff. The data filtering is something that is an on-going thing which lasts, well, for as long as the system has been running, although the actual filters that are being used pretty well cleanse themselves certainly within an hour or so. I don't have a very long data set here. My data set starts just about 20 minutes before the first gust front comes across. So, my filters aren't spun up to the same level that the filters are spun up in the operational software. So, there are minor differences. What I found when I went through this thing very carefully was that my software seems to be a little bit more aggressive about issuing alerts than the operational system was. By aggressive, I mean that at each of the sectors, I had somewhere between -- over an hour and a half period, I had something like 15 extra alerts. Those are 10 second alerts per sector. It was always the case that my software issued alerts that the other software didn't. So, for some reason, my software was a little bit more sensitive, and since I don't have access to the operational software, I can't make a comparison to tell you why. I can just tell you it's always a little bit more aggressive. Usually this shows up as my software issues an alert one or two poles earlier than the operational system, and my software holds the alert one or two poles towards the end of where the operational system stops, and over these several different wind events, that adds up. When we have a short event like that surge at the north end, if you knock two poles off the front and two poles off the back, and you require a five pole persistence, the operational software might very well have seen that thing for three or four poles, not quite gotten to the persistence, and then turned it off because of this difference in the software -- in the sensitivity of the two performances. But I'm speculating. I don't know. The only way I could know would be if I could get inside of the operational software to look at it. But there are differences, and my software is more aggressive. That I can say. Q Can you comment on the crosswind shear that the aircraft might have encountered on 4 Right? A We see -- we definitely see from when that microburst came down at about the time when the aircraft was rolling down the runway, the LWAS sensor at centerfield surged to 41 knots. That certainly was more crosswind -- now, he -- he expected some crosswind there. He'd been getting reports on crosswind, but he -- that is more crosswind than was being shown in the system for the several minutes before and which certainly he had received information about in any message. I don't know if he was already rolled past there because I don't know the detailed time line of that aircraft. The other problem we have is because of those sparse sensors, there could have been other surges, even stronger surges, down that runway that were between the sensors, and we just don't have information. So, I'm speculating. If that's an important issue, I would presume that someone has analyzed the flight data recorder, and they can tell you much more about what happened to that plane than I can from the data that I have. Q Later this year, the Little Rock Airport is scheduled for their upgrade to the LWAS RS or relocation and software -- A Yes. Q -- upgrade with nine sensors. A Yes. It's going to be 10, it turns out. Q We have some improvements. Would there have been any major differences in the advisory with the RS versus the performance in the LWAS-2 system that occurred on June 1st? A I don't think so, and the reason I don't is because this -- this microburst was -- well, I can't -- well, you said any differences. I was thinking of the microburst. This microburst started with a fairly large footprint and happened to fall conveniently between the sensors in such a way that the alert seems to be extremely timely from the LWAS-2. The LWAS RS would get a lot of alerts that the LWAS-2 wouldn't get, but on this particular microburst, I think they'd have both shown the same thing. There is a possibility, in fact even a probability, that the LWAS RS, unless it got the full strength of that microburst, might initially have thought it was a wind shear and detected it better but canceled the alert because it didn't have a measure of headwind loss and gain along the runway. That microburst is far enough off, it probably was not causing a headwind problem on that -- or a loss -- a headwind loss down that runway. So, LWAS RS might have in fact done a little worse. Q All right. If there was a Terminal Doppler Weather Radar installed at Little Rock, would we have seen any major differences in our air traffic control support or any advisories? A It depends on where it was located. If it was located where the NEXRAD is, because of the non-alignment with the radar beam, it probably would have been later than LWAS on this particular alert. If it had been located looking down the runway or up from the south, sort of the southeast, it probably would have given a wonderful picture of that microburst, but that's sort of a crapshoot. Q All right. Well, Dr. Wilson, I would like to thank you. With your presentation, you've answered all my questions, and I'll turn it over to Mr. Hall for your handling. CHAIRMAN HALL: Okay. Well, we'll go to the tables for questions. I would ask -- Dr. Wilson, you probably don't know what an LWAS costs, do you? THE WITNESS: Well, I know some old numbers, but they're fussy, but I think part of the problem is that the system is procured, and then the FAA independently does the real estate work, and I'm -- you'd probably have to actually do some snooping around to get the whole bundle together. CHAIRMAN HALL: Well, that's just -- my only point is obviously the purpose of the system is to provide safety protection -- THE WITNESS: Right. CHAIRMAN HALL: -- at the airport, and it's paid for by the taxpayers. It's not a cheap affair, I would assume. THE WITNESS: Oh, it's not cheap. It's well over a million dollars a system. CHAIRMAN HALL: Right. And, so, federal aviation -- so, the performance of it is something I wanted to put it in context. We're very interested in the performance of it. It's there for safety, for the safety of flight, and it is an expensive item. Federal Aviation Administration. INTERVIEW BY PARTIES TO THE HEARING BY MR. STREETER: Q Doctor, if you could clarify one item for me, and I believe this was -- the wind event. You were talking about where the wind shear alert was not issued, and you were -- A Yes. Q -- discussing the differences between the software, and I think that was one that was at 11:49:10. A Yes. Q When you say the wind shear alert was not issued, for clarification, you mean the LWAS system did not issue -- A The LWAS system did not put -- put an alert on the display -- Q Okay. A -- or at least -- let me be very careful. What I have access to is the archive of the displays, which I think is the official record. It may not be the correct record. However, I also -- I guess what I know about this, other than LWAS, is what I read in the papers. The archive does not show an alert, and I read in this morning's paper the cockpit voice recorder transcript, and there was no -- I looked very carefully at what alerts did the pilot receive, and he got the alert for the one at 47. He did not get a subsequent wind shear alert. So, my belief is that the controllers did not see an alert, and my archive is correct. Q Okay. So, based on what you saw in the CVR transcript and your archive data, it does not appear that the LWAS system alert -- A It's the newspaper version. I always mistrust papers. Q Oh, okay. A But, yes, I believe -- I believe that the issue -- sorry about that, folks. Q Okay. A It was certainly not in the official transcript. Q Okay. Thank you very much, sir. CHAIRMAN HALL: Boeing Commercial Airplane Group? MR. HINDERBERGER: Mr. Chairman, Boeing has no questions. Thank you. CHAIRMAN HALL: Okay. American Airlines, Incorporated? MR. BAKER: Mr. Eick, is Dr. Wilson's presentation in the exhibits? MR. EICK: No, it is not. It was just prepared on Monday and finalized. So, once I do get it, I will include it and forward it out to the groups. It is not in the attachment right now. Just the tower display of the low-level wind shear alerts are in an exhibit to be. THE WITNESS: Yes. I have it here. I will hand it to him today. MR. BAKER: We would like to request -- and Dr. Wilson identified the subject -- that the Weather Group and the Performance Group correlate Dr. Wilson's data on wind direction and velocities with the DFDR, so we really nail down once and for all where the airplane was and what it was subjected to, so we don't speculate as to what really happened. CHAIRMAN HALL: I think that's reasonable. That will be done. MR. BAKER: Thank you. CHAIRMAN HALL: Allied Pilots Association? MR. ZWINGLE: Mr. Chairman, I'd like to comment first that I think Dr. Wilson's testimony was critical and an excellent presentation. BY MR. ZWINGLE: Q Dr. Wilson, you began to clarify, and I'd just like you to complete, if you would, the notion of alerts, that just because the wind shear -- I'm sorry -- the LWAS system alerts and displays in the tower cab, that does not necessarily mean the alert was forwarded to the pilot, is that correct? A What I said was -- that's all a matter of -- of somebody's record because everything's recorded. I said I have not reviewed that. So, I don't know. Q Okay. But your -- your references to alert meant that the LWAS system alerted in the tower cab? A Well, if we're going to pick straw -- pick hairs here, my reference is that the archive of what the computer says it sent to the tower cab is my alert. I wasn't in the cab. Q That's fine. A I didn't see what the controllers saw. Q I understand. That's fine. That's accurate. Would you describe the two-minute averaging function of the centerfield wind, and how this wind value compares to reality? A Okay. I certainly will, but I want to point out that that number, while it is of -- it's an ICAO requirement, and it's a very important number, it has nothing to do with the functioning of these algorithms. The two-minute average wind at centerfield is simply the running average, and, so, in two minutes at 10 seconds poles, there are 12 samples of the wind taken each 10 seconds, and they are simply averaged. Every time you step forward 10 seconds, the oldest value is dropped off, the new value is put on, and the window slides forward, and you compute what's known as a running average value. That -- the reasons -- well, if you put one anemometer out, what it measures is representative of exactly what happens at it. It may not have anything to do or it certainly decorrelates with what happens as far away as you are or as far away as the end of the runway is. The reason -- there was research conducted by ICAO back in the early '70s, excellent research paper, which I have read, in which it was established that the most representative wind for the entire airport is a 10-minute running average, but, of course, the 10- minute running average has a very slow time response when there is a shift, and, so, ICAO settled on the two-minute average as being a compromise between representative of the whole airport and having a fairly rapid time response when there's a wind shift. That is the reason why the original LWAS was designed with boundary sensors, so you could pick out those perhaps non- representative areas. Q In consideration of the fact that there are variations in the LWAS equipment, LWAS-2, LWAS NE, etc., what sort of variations and warnings do we get from these equipment? A The LWAS-2 says wind shear alert. It's just like LWAS-1. Wind shear alert by sector. The LWAS NE, the LWAS RS and the TDWR all have identical messages which will either say microburst alert on a runway, expect a certain amount of loss, or wind shear alert on a runway, expect either a gain or a loss, depending on what the system has detected. Those -- that was the main or one of the really important things that that wind shear working group did, was it brought all of these systems into a standard message. Q Okay. Depending upon the size of the airport and, of course, the geographical size of the airport and the type of system installed, should wind values, in your opinion, should wind values sensed at a location closest to the landing runway or operational runway for take-off or landing be reported to the pilot advice centerfield winds? A I guess I could be a little snide and say I was brought here because I have expertise, not because I have opinions. Q In your expertise then. A That's really not for me to say. I believe that the centerfield wind is sort of an ICAO standard, and, so, there's -- Q Okay. A -- some hills to climb to change that. But, yes, certainly we could have opinions about what might be more useful to pilots. Q Would -- would perimeter winds be of value to me, -- A They -- Q -- absent -- absent alert? A Well, certainly they -- they -- oh, yes, and -- and a lot of pilots, if they have any concern, pilots are always free to ask for them. Controllers have them on their displays, and pilots can ask for them. Q In your expertise, do you think that aeronautical charts provided to pilots should identify the vintage of the system, the type of system, LWAS-2 versus LWAS NE, etc.? A I don't have expertise in that. I thought something about that was on the Jepsen charts, but I don't know all the detail. Q Okay. And are you familiar with -- with the description of the LWAS system in the Aeronautical Information Manual? A No, I'm not. Q Okay. Thank you very much. CHAIRMAN HALL: Association of Professional Flight Attendants? MS. LORD-JONES: We have no questions. Thank you. CHAIRMAN HALL: National Weather Service? MR. KUESSNER: No questions, sir. CHAIRMAN HALL: I see your prediction outside is continuing to provide accuracy. I think it looks like about three to four inches now on that shelf. The Little Rock National Airport? MS. SCHWARTZ: No questions, sir. CHAIRMAN HALL: Do you know how much the LWAS system cost for your airport? MS. SCHWARTZ: Not exactly, sir. CHAIRMAN HALL: If you have that, if you could provide that for the record, we would appreciate it. The Little Rock Fire Department? MR. CANTRELL: No questions, sir. Thank you. CHAIRMAN HALL: Mr. Sweedler? MR. SWEEDLER: I have no questions, Mr. Chairman. CHAIRMAN HALL: Mr. Berman? INTERVIEW BY THE BOARD OF INQUIRY BY MR. BERMAN: Q Sir, I was interested in your statement that a side gust or a crosswind gust coming over a runway that -- that might affect the performance of -- of an airplane that's landing on a slippery runway wouldn't necessarily be flagged with an alert on the -- on the newest LWAS systems, and that was an explicit choice made by -- by a committee. A Yes. Q Were you a member of that committee? A I was. Q Okay. Could you give us a brief summary of the -- the thoughts behind that, and were there any counter arguments? Was there unanimous agreement reached on that? Give us a little flavor of that. A That committee had met for three days or about 60 to 80 people. I don't remember exactly which one. The committee sort of grew with time. It got to be popular, and more people wanted to be part of it. It was a big group. It's hard to call a committee. Almost every decision that that -- and this committee met several times. It was a lot of the same people. Almost every -- by the way, the NTSB was represented on that -- at that. I think Greg was -- Greg, weren't you part of that or was that -- blame it on Bud. Greg and Bud were there. That's right. I got this now. Almost every decision that was made by this committee was unanimous. This was at the end -- this issue was the last issue considered. It was the end of the third day. There was a lot of fatigue, and this one cut almost right down the middle. I can remember a member from the FAA who was -- literally had tears coming down his face screaming "you're going to kill people", and there were people in the other side saying "we just can't have this system sitting out there giving all these false alerts. Nobody will believe it, and then we'll kill people", and I think it was more out of fatigue than good sense that the committee stopped at that point, and the 53 percent side won, and that became the rule. People commented that we ought to revisit it, but it never was. The group never met again, and it stopped there. Q And what year was this? A '88. Q Or millennium, if you don't remember the exact year. A Approximately late '80s. Q Okay. A There are still people -- I was just called last -- oh, six months ago by somebody from NCAR saying we really ought to revisit that. Q We -- we won't -- we won't ask how the NTSB voted because we don't traditionally vote. Well, seriously, we don't -- we don't -- we -- we function usually as observers at meetings like this and don't vote. But can you -- and you've said that an FAA -- at least one FAA representative was opposed to this decision. A Paul O'Brien from the Tech Center. He's retired now. Q Can you -- can you characterize the positions of yourself and other people at that time or -- and tell me how you feel now about it. A You know, it's hard to tell what's conservative in this circumstance because of the issue that if you -- if a system alerts a lot, users may start to ignore it, and yet it's clear that we can design situations where it might be really critical information, and maybe the reason we couldn't figure it out was because at that time, we didn't have the right people in the room to make the decision. Certainly nobody could explain to that group of people why they should become -- come to consensus. Maybe they need some aeronautical engineers or somebody. Q And how do you feel about it now? A Still bothers me. Q You -- you believe it should be revisited in a committee? A Revisited, I think -- okay. You're getting opinions now, not expertise, but, yeah, the -- it bothers me because I think of a scenario, and, you know, I look at the data here. I wonder if I maybe saw it. Q Okay. Thanks very much. No more questions. CHAIRMAN HALL: Mr. Haueter? MR. HAUETER: No questions. CHAIRMAN HALL: Mr. Clark? BY MR. CLARK: Q Regarding the crosswind warnings that weren't programmed into the system, does your computer program analyze that and/or for the data we have, would we have generated some type of crosswind warning? A No. Well, let's be -- this only comes up for systems that are able to estimate a runway header -- headwind gain or loss. That would be the NE or the RS system. The unique -- in order to make a reliable estimation of the headwind change on a runway, you need a lot of sensors. The LWAS-2 doesn't have enough sensors to reliably make that estimation. Since it can't estimate the headwind change, it doesn't even try and just issues all the alerts. Q Okay. The other thing. You made several comments somewhat apart. One is that the sensors were too far apart in one sense, but then the whole airport had recently undergone a survey. Does that -- would the survey have identified that -- A No. Q -- the sensors were too far apart? A Okay. No. The sensors in the LWAS-1 were specified to be one at centerfield and others at sort of the end -- the far ends of the airport property in all directions. A little bit beyond the runway ends, but certainly not everywhere where the airplanes were low. The LWAS-2 was required to be a software upgrade. There was no money in that procurement to pay for new poles or more poles or anything. So, LWAS-2 got what was there. The system had already been in operation for about 10 years at that point, and, so, what we were forced to do was come up with an algorithm that would do the best possible job, given that we had that geometry to deal with. Q Okay. A This was intended to be an interim system, just a bridge, until the NE could be procured, and then we got into money problems because the NE turned out to be expensive, and, so, the bridge has lasted a lot more years than were intended. I think we started with that in the first presentation this morning. We've been revisiting the fact that we're in transition more than we're ever getting anything done. Q Okay. What -- what is in the program for Little Rock now? Do we -- A The RS system -- as I said, they've already installed the new poles. They are waiting for the software to be completed and tested. The factory test will happen some time in the next few months, and then they will install the new software there. Q Okay. All right. Thank you. A I'm optimistic when I say few months. That probably means 18 months, but in the next year or two, the -- I mean I could -- there's also a schedule of what airport gets what. CHAIRMAN HALL: Well, I'm -- let me try to put this in sort of bites that I can understand. You have a system at Little Rock. It's your LWAS-2 system. THE WITNESS: Yes. CHAIRMAN HALL: It has five poles, five sensors? THE WITNESS: Six. CHAIRMAN HALL: Six, but I saw -- I think -- didn't I -- I thought you said there were five. THE WITNESS: There's the centerfield -- CHAIRMAN HALL: You said at the airport, that there were five operational now. Are there six or five? THE WITNESS: There's six. CHAIRMAN HALL: Six? THE WITNESS: I have the data from six. CHAIRMAN HALL: So, there's six? THE WITNESS: There's five sectors. Five alert sectors. MS. SCHWARTZ: Mr. Chairman, I would defer to the FAA as it is their equipment and installation. CHAIRMAN HALL: Okay. But you all thought there was five, but you -- there are six? They're saying there's six. Okay. So, there's six of them. We can only find five poles. We went -- I went to look. THE WITNESS: There was two in the south. There's one in the north, one in the northeast, and one on the northwest, and one in the centerfield. CHAIRMAN HALL: On that hill over across the river, right? THE WITNESS: I have not -- I got in after dark last night. I would have done a survey, but -- CHAIRMAN HALL: Well, go out there and count and make me feel better. THE WITNESS: Well, they gave me data from six poles. CHAIRMAN HALL: Well, that's -- that's what I want to find out about. So, you got the data from those poles, and that system, that present system is scheduled to be upgraded, right? THE WITNESS: Yes. I showed you the map of where the new poles -- those -- CHAIRMAN HALL: Right. THE WITNESS: -- poles will be kept, and the new poles will be stuck in. CHAIRMAN HALL: And -- but the new system -- tell me why the new system is better than the old system. Essentially, as I understand it, it gives greater coverage and doesn't have these holes -- THE WITNESS: Okay. It will not give greater coverage. It will fill the holes, fill the coverage -- CHAIRMAN HALL: Fill the holes. THE WITNESS: -- area, so that smaller events can't slip through the cracks. CHAIRMAN HALL: And does that mean more warnings? THE WITNESS: Yes, it should. It should mean -- well, I showed you the performance statistics. CHAIRMAN HALL: Right. THE WITNESS: It will probably increase the number of microburst warnings by about 50 percent correctly. CHAIRMAN HALL: Okay. And I guess, is that -- that gets into something I guess we'll get into with the FAA or should have or maybe you have some experience -- is anybody training the pilots so they know we have -- as the gentleman with Allied Pilots said, here's a new system. It's going to be more sensitive. You're going to see -- experience more warnings. So, we don't have it desensitized -- the pilots being desensitized -- THE WITNESS: Actually, the pilots have already in a sense been trained because when the new system is in, the alerts will look just like TDWR alerts. CHAIRMAN HALL: Good. Okay. Now, -- THE WITNESS: So, it will be transparent to them what system is giving them the wind shear alert. CHAIRMAN HALL: But the new alerts will not have the crosswind alerts, right? THE WITNESS: That's correct. CHAIRMAN HALL: But the old system does have some crosswind -- THE WITNESS: That's correct. CHAIRMAN HALL: -- alerts? And -- THE WITNESS: Well, the old system doesn't know enough to know. It just gives the alerts. CHAIRMAN HALL: But there was then another crosswind, I guess, approximately 40 miles an hour. THE WITNESS: It alerted -- that's the one -- it did alert on that. It delayed the required persistence count of 50 seconds. CHAIRMAN HALL: Okay. THE WITNESS: But it did alert on that, yes. CHAIRMAN HALL: And, now was that consistent because the forecast was gusts up to 40 miles an hour? THE WITNESS: Yeah. The LWAS system certainly measured gusts that supported the Weather Service forecast. CHAIRMAN HALL: And you've developed this software, and is it -- do you think that the system itself -- and I'm looking not just at Little Rock, but -- but nationally is being maintained properly and kept current, so it's -- it's a useful tool it's designed for because it's -- it's a big investment? THE WITNESS: Well, the FAA has a steady program for making improvements. The procurement of the RS and the resiting of the poles was part of that. I see steady progress. I do work sort of closely with the people. They -- they turned the technology or the care of the system from the Tech Center over to Oklahoma City. When they did that, I made several trips out there to train them in the technology and in the -- how -- methods for using -- maintaining the technology. The FAA funded that, of course. I think -- I am not -- would not be critical of how the FAA maintains this system. Obviously every door knob could be polished more, but I've seen good conscientious work in the maintaining of this system, and the system did check out well. CHAIRMAN HALL: And here it was good performance? THE WITNESS: Yes. A lot of that has to do with the people at the airport, the facilities people at the airport. If they don't stay on top of it -- so, I think you have to give a lot of credit to the -- to the people at the airport to keep it running. CHAIRMAN HALL: Well, very good. Very good. Well, Dr. Wilson, you've given a very -- a most interesting presentation on a very important safety system that we've got a big investment in. So, -- and I guess the bottom line is, here it performed pretty much as designed. THE WITNESS: That's my conclusion, yes. CHAIRMAN HALL: That, I understand then. I would like to give you an opportunity. Is there anything else the Board should be looking at? You've raised one issue here on the crosswind component. Is there anything else that you think -- and I'm not putting words in your mouth about whether there's something we should or shouldn't look at, but are there any things you -- you've studied this particular event, and any information the Board should consider in trying to make recommendations to prevent another occurrence similar to this? THE WITNESS: You know, I almost -- I can just say you can pull my transcript from Charlotte and reread it because I think you asked the same question there. CHAIRMAN HALL: Well, we -- maybe I ought to do that. THE WITNESS: My -- CHAIRMAN HALL: If staff had been sharp, they would have that for me to have already read and be prepared on. I remember now. THE WITNESS: My focus is -- is -- CHAIRMAN HALL: Please cover those points again then, because, I mean -- THE WITNESS: This is just a point of -- CHAIRMAN HALL: Charlotte was a windy -- you know, for the audience, you might tell them a little bit about the Charlotte accident. THE WITNESS: That was an accident which clearly was a microburst. The pilot flew into a microburst, and the issue there was -- or one issue -- there were many issues, of course. One issue there was that he made a decision to go around, and the LWAS issue -- system issued its first alert on that microburst 10 seconds after he changed frequency, and they weren't able to get the word to him, and he flew the plane believing he was flying to avoid a heavy rain shaft and never knew that he was fighting with the wind shear, and he did not use the -- I believe he did not use wind shear escape maneuver in that circumstance. Clearly, the information could have meant something different to him. I guess what I said at the last meeting, and I can make it a little more colorful now because I heard a cute quote last summer, and I'll prefix this by saying my focus for the last 20 years has been aviation weather and a lot of time hazardous weather. I think there's a lot of good stuff out there, and I know that there are other safety issues and other priorities, and you guys have to balance them all. I also want to make it very clear that I am not in any way trying to prejudge this case or say what did or didn't happen with respect to this airplane and this pilot. My optimism is that I see a lot of good stuff happening. The FAA has a good weather research program going on. It's an on- going thing. They have built the ITWS System. They have efforts going to improve it. That's now in procurement. A lot of good stuff. AWIN. NASA is working on the AWIN Program, trying to figure out how to get information to pilots. I maliciously took the time to show you, even if you had a 10-minute delay between when the radar saw something and when you got it to the pilot, how out-of- date things might be, and I think communication systems is not rocket science. There's no risk in developing a communication system. There's some bucks involved. Some of them are probably federal bucks to turn on the pipeline of information, some of them are going to have to come from the carriers to turn on the bucket to receive the data. So, there's bucks involved both ways, and there's a lot of things that are competing for those bucks. I understand that. I'll close with a quote from Patton. I guess my charge is stay the course, don't give up on funding safety stuff. Less is less to quote our esteemed chairman. Patton's comment was weather's not important until it's important, and then it's too late. CHAIRMAN HALL: Well, thank you very much, Dr. Wilson, for your testimony, and, Mr. Berman, if you'll call us the next witness. (Whereupon, the witness was excused.) MR. BERMAN: I call Dale A. Rhoda. Whereupon, DALE A. RHODA having been first duly affirmed, was called as a witness herein and was examined and testified as follows: CHAIRMAN HALL: Mr. Rhoda, welcome. Dr. Wilson does not follow the MIT football team. Do you follow the MIT football team? THE WITNESS: I'm not sure we have a football team. CHAIRMAN HALL: Oh, you do, definitely do, definitely do. Please proceed. MR. EICK: Good afternoon, Mr. Rhoda. MR. BERMAN: Excuse me. Mr. Eick, let me ask him the basic questions. MR. EICK: Oh, I'm sorry. MR. BERMAN: Sorry about that. INTERVIEW BY BOARD OF INQUIRY BY MR. BERMAN: Q Please state your full name and business address for the record. A My name is Dale Rhoda. I'm an employee of MIT Lincoln Laboratory at 244 Wood Street in Lexington, Massachusetts. Q Okay. Thank you. And your present position at Lincoln Laboratory? A I'm also a member of the Research staff there. Q How long have you held that position? A I started working at Lincoln Laboratory in the Fall of 1989. Q Could you please tell us your duties and responsibilities in that position? A Hm-hmm. In those 10 years, I've contributed in different ways to the development of and testing of the Terminal Doppler Weather Radar and then the Integrated Terminal Weather System, which has been referenced several times today, the ITWS, looking at various -- doing various tasks. I was the field site manager for the Dallas-Fort Worth Prototype of the ITWS System for a couple of years. I've wrote some software in the TDWR System, and since about 1994, I guess, one of my areas of expertise has been combining flight track data along with weather information from these FAA weather acquisition systems. Q Okay. Thanks. And can you please give us a summary of your education and training and prior experience to qualify you for the current position you have? A Hm-hmm. I have a Bachelor's Degree in Mathematics. I have a Master's Degree in Public Policy. Before my time at Lincoln Laboratory, I was employed in another atmospheric research lab at MIT, also, writing software to analyze Doppler Weather Radar data. Q Okay. Thank you. Do you have any FAA airman certificates? A I do not. Q Okay. Very good. MR. BERMAN: Go ahead, Mr. Eick. INTERVIEW BY TECHNICAL PANEL BY MR. EICK: Q Mr. Rhoda, you -- we have added Exhibit Number 5C, based on your research in thunderstorm penetrations in the terminal area. Can you tell us something about your research? I understand you have a presentation to show to us. A I do have a presentation which describes that research and summarizes what's in that report which is in the deposition. Q Please continue. A Thank you. The work that I'll describe this afternoon was performed by myself and my colleague Margo Povlok in the Weather Sensing Group at Lincoln Lab. We've heard several things about airline policies about flying around when there are thunderstorms in the terminal area, and one of the results of our study was that we were able to look at the practice as well as the policy, and that's what I'm going to talk about today. I have a brief introduction where I will review the roles and responsibilities of the players in the decision to -- whether to penetrate or deviate storms in the terminal area. I'll go over briefly what we refer to as the Level 3 rule of thumb that we keep hearing about at various FAA facilities and that we hear about from pilots and read about in pilot publications as well, and then I'll clarify which of the several precipitation scales that we've talked about today, I'll be using for the rest of my presentation. Then I'll go over the NASA-funded study that we performed that's described in that report that we just mentioned. Why did NASA ask us to do the work, what did we do, how did we do it, and what did we find essentially? First, I'd like to just, as part of the introduction, briefly review who the players are that helped make decisions about penetrating or deviating around storms in the terminal area. What are their sources of weather information, and what are their responsibilities, and it's a lot of information on the slide, but I'll just go through it quickly. First, of course, the pilots -- the decision whether to penetrate or deviate around a thunderstorm ultimately lies with them. They have several sources of weather information. Two sources that are available to them are the ATIS, which I believe stands for Automated Terminal Information System, but I could be wrong, but that's a voice recording that's updated at least hourly that describes the current conditions at the air field, the current runways that are active, any significant weather information, and it's updated if the weather changes within the hour, that is updated as well. That's available to them on a particular radio frequency at each terminal. TWIP stands for Terminal Weather Information for Pilots. That's only available currently at the airports that have terminal doppler weather radars, and that's a simplified message that describes -- it's a text message which pilots can request over their ACARs, describes the storms within 15 miles of the airport, their direction from the airport, and their overall direction of motion. So, that's available at the TDWR airports. It's not available at Little Rock. Commercial aircraft, of course, have airborne radar on board. Pilots observe visual cues when the visibility is such that they can. CHAIRMAN HALL: There is no terminal doppler weather radar here at Little Rock. Is there any planned or scheduled that we're aware of? THE WITNESS: Not that I'm aware of. CHAIRMAN HALL: Are you all -- THE WITNESS: That's an FAA question, really. CHAIRMAN HALL: -- in line for the terminal doppler weather radar system here? MR. STREETER: As soon as you asked that question, my resource is not at the table. I can find that out for you and let you know. CHAIRMAN HALL: When your resource returns, let me know. MR. STREETER: We'll do that. CHAIRMAN HALL: Okay. MR. STREETER: Sure. THE WITNESS: Pilots can observe the visual cues outside the cockpit window, and then those methods -- those first three methods or particularly the first two methods, I like to think of those as being active information acquisition, where they make some sort of request, and they get some sort of reply. So, they tune in to the ATIS frequency. They request a TWIP message or they manipulate the controls on their airborne radar. The other sources of information are a little more passive. You can look out the window. You don't have to change what you're doing. Simply the information comes in to your eyes, if you look for it, as well as monitoring the radio frequency for either wind shear alerts from the various wind shear detection systems or from PIREPs or pilot reports. Sometimes the preceding pilots will give a ride report or describe the conditions along the route of flight. Another way of acquiring information is the pilots can ask the controller to solicit information from the preceding pilot, if they are not already giving that information or if the pilot recently changed to the current frequency. Moving along to the airline dispatchers, as we discussed before, those dispatchers have weather data from various vendors and others. Typically, the dispatchers rarely initiate contact with the pilot once the pilot has entered the TRACON, but in some cases, particularly when the pilots call and ask the dispatchers for their opinion, they will use the weather products that are at their disposal to give advice about whether or not a diversion is going to be necessary, but not about individual deviations and penetrations around individual storms. The air traffic controller in the tower has the responsibility of reading the centerfield wind and has the responsibility of reading wind shear alerts when they are available from LWAS or TDWR. At the airports where the controllers have access to a six-level precipitation product, the controllers in the tower have limited access to that product. They see it on a monochrome screen. So, they see one of the six levels in a light green and another of the six levels in a bright green, and those controllers do not have the responsibility to relay PIREPs, but they do do so on a time-available basis, but they're not required to do that. And then, finally, the TRACON controllers that are talking to the -- to the planes when they're a little further out, they -- at those airports where they have a six-level weather product, where they have an ASR-9, they can see again the six-level precip on a monochrome display in light and bright green. They do not have any access ever to the wind shear information that the controllers in the tower have, and, so, the controllers that are directing the planes to the airport, if the tower controllers are reading wind shear alerts, the TRACON controllers do not hear those wind shear alerts, and they don't have any display that shows those alerts to them. And, finally, at the bottom of the slide that we were just looking at, TRACON controllers also may relay PIREPs from the preceding planes. Before we did the study that I'm going to describe in just a minute, we did a literature search and really, to the best of our knowledge, there hadn't been any other studies where people collected weather data and flight track data to see what sorts of decisions the pilots make in the terminal area, but there are some things written in the literature. This is a quote from a magazine. This quote was written by a pilot to other pilots describing the weather products that are available to air traffic controllers, and this is very typical of the things that we would hear from pilots, and that we've heard in the various places around the country where we have been testing and developing, for instance, the TDWR and ITWS. When we visit and spend many hours in TRACONs and towers developing these products, we generally hear this sort of Level 3 rule of thumb, where the controllers have access to the sixth level VIP weather product. They say that generally speaking, since that's the only presentation they have, generally speaking, the pilots avoid storms that are Level 3 and higher. Here's another quote along those lines, and that was really -- we weren't able to find an instance where anyone had taken a look to validate that rule of thumb, but that's a very widely-quoted rule of thumb at FAA facilities across the country. So, just to review, we've talked about a couple of different precipitation scales today, and the remainder of my talk, I'm going to use this sixth level precip scale which the National Weather Service gentleman said is the old way of doing things at the NWS, but I assure you it's the current way of doing things at the FAA and probably will be for some time. Rather than breaking precipitation up into 16 levels, 16 or more levels, the FAA systems detect precipitation on the six level scale. You'll see some adjectives over there to the left from the AIM that was mentioned earlier. To put those -- these precip levels in layman's terms, I've listed what they would correspond to roughly in terms of rainfall in inches per hour, if you want to think about this in terms of watering your lawn. There will be -- I'm going to show a couple of movies in a minute of planes flying around weather in the Dallas-Fort Worth TRACON area, and the colors that I'm going to use for these six level weather product are the same colors that are used on the TDWR depiction and will be used on the ITWS depictions as well, and they're listed here, where the greens are Level 1 and 2, and anything yellow or orange or red is Level 3 and higher. And then on the far right, I've listed what a typical airborne radar display would look like with three colors. Level 1 generally shows up as green, Level 2 is yellow, and everything higher as red. Now, there are some four-color airborne displays. My understanding is that with the four-color displays, Level 5 and Level 6 would show up in a magenta as well. There may be some variation, but that's sort of the general color scheme. With that background out of the way, I'd like to describe the study that we did that was written up this past June. NASA AIMs is developing some air traffic control decision support tools for use in the terminal area. The one that I'm specifically going to talk about is called "the final approach spacing tool", sometimes abbreviated as FAST. That tool knows about all the traffic that's arriving into the airport area and aids the controllers by suggesting for each plane which runway the controller should land that plane on and what sequence the plane should be landed in in order to achieve a very efficient throughput in busy traffic periods. The way that it does that is by relying on a set of routes that it assumes each plane will fly from a particular arrival fix to a particular runway. As long as the plane does fly the route that the system knows about, then the system has some very sophisticated aerodynamic information, information about the performance of each different type of aircraft and can estimate the time it will take the plane to fly from its current point to the runway touchdown point. Therefore, it can set up a list -- a sequence list, and it can distribute the aircraft evenly among the runways and achieve a very efficient throughput. But all of those calculations rely on those, quote unquote, time-to-fly estimates which rely on knowing the route that the plane will fly in the terminal area. On days when there are thunderstorms in the terminal area, those form effective roadblocks. The routes change and meander as we will see in a minute, and the computer no longer knows what routes the planes will fly, and it's not able to make time- to-fly estimates, and with the current version of FAST, it's simply turned off on storm days because it doesn't know how to deal with weather. So, we were asked by NASA to think about some future generation of FAST that might be able to, first of all, predict the location of the weather in the future, but, more specifically, the piece of the puzzle we were working on is saying if you knew -- if you know what the weather is, how do you know which of those storms are going to be more or less likely to cause requests for deviations, and, so, we wanted to measure all the available weather variables at a particular airport in this study, which I'll describe in a minute, and we wanted to do an initial investigation to see whether or not we could combine those variables into a measure of the probability that a plane would deviate around a particular storm. That was the goal of the study, and in looking at that, basically efficiency-based study, it's also -- our conclusions or our findings are also applicable to terminal area safety, and also in the days of the future when controllers are not looking at monochrome displays but color displays, what weather products should the controllers be looking at? Is the six level VIP scale the right one to help them anticipate pilot deviations or is there some other product that would be more helpful? Our approach was to collect 60 hours of weather data and flight track data in the DFW terminal area. Because FAST is only concerned with aircraft that are arriving into the terminal area space, for this study, we restricted our attention to arrivals only. So, we did not look at pilot encounters with thunderstorms for aircraft that were departing, only arrivals. In that 60 hours of stormy data, we identified every encounter where an aircraft, arriving aircraft had a thunderstorm in its nominal flight path. We split those into penetrations and deviations, and I'll describe how we did that in a minute, and for each of those, we extracted the weather and the flight variables for each of those encounters. So, we had then a large database of the types of weather that the planes did fly through, and the types of weather that the planes specifically avoided, and we analyzed that statistically, and I just want to point out that the data we looked at were nine stormy days in the late Spring and Summer of 1997, and while there were some fairly dramatic penetrations of thunderstorms, these were not days that anyone read about in the newspaper in terms of there being any accidents or, as far as we know, any -- any injuries. They were nine Dallas thunderstorm days. The way we were able to collect the weather data and the reason -- one of the reasons that we did the study in Dallas is that our group at Lincoln Laboratory was testing our prototype of the integrated terminal weather system which is shown in a block diagram here. I've included this slide for two reasons. First, to show you the weather sensors that we used to collect the weather data I'll describe in a minute, and, second, to make the point that on these days, this was not a pristine environment for an experiment about behavior in the presence of thunderstorms because we were providing real-time weather information via this prototype system to the traffic managers in the towers, the TRACON, the en route center at Fort Worth, and as well as the major airlines operating in and out of Dallas-Fort Worth at their dispatch offices, either via web sites or dedicated phone lines. So, the people thinking about traffic management on these nine days that I'll describe had the best available weather information. They had false echoes removed from their displays. They had excellent weather information with which to make their decisions. The sensors that we used to collect the weather information were the three radars shown across the top, the National Weather Service's NEXRAD located in Fort Worth, two Terminal Doppler Weather Radars, one that looks for wind shear over Love Field and the other that looks for wind shear over DFW, three airport surveillance radars that cover the DFW TRACON air space. In our study, we did not look at AWAS data or LWAS data, but we did look at data from the National Lightening Detection Network, and that's a network of sensors all across the country that detects every cloud-to-ground lightening strike in the country, and we didn't look at any weather data from aircraft, but we did in fact have flight track data in the terminal area. Here's a list of the sensors again, and I listed the variables that we looked at in our study. So, the airport surveillance radar gives you a six level precipitation product. We computed another variable which we call weather coverage with that product. We -- it's simply what percentage of the air space is covered in rain. What percentage of the air space has light rain over it, and what percentage of the air space has heavy rain over it? From the doppler radars, the Terminal Doppler Weather Radar and the NEXRAD, these radars, as we've heard several times today, take several minutes to do an entire volume scan, but one benefit of waiting all those minutes is that you get a three- dimensional picture of the storm structure, whereas the airport surveillance radar updates every 30 seconds. It's a very current picture of what's going on, but it's only a two-dimensional picture. So, with the three-dimensional reflectivity information from these doppler radars, we use that three-dimensional storm structure to determine whether the planes were flying above or below the storms. If you only did a study like this using ASR information, the data would be very current, but you wouldn't know if the planes were above or below the storms. Using the three doppler radars together, they each update on average every five or six minutes, but you have three of them. So, really the update rate is more like every two or three minutes, and the TDWRs update over the airports every one minute. So, it's very current three-dimensional information about what's going on. We computed a number of two-dimensional projections of that three-dimensional storm structure, some variables that have already been described today, vertically-integrated liquid, maximum reflectivity in a vertical column. What was the echo tops or how -- how high were the clouds? What was the echo bottom? Was the rain reaching the ground or not? What was the thickness of the clouds? What was the probability that there's severe hail in that location, and then from the Terminal Doppler Weather Radars, were there microbursts or gust fronts? We looked at the cloud-to-ground flash rate from the Lightening Detection Network, and then from the flight track data, we computed several interesting variables. We had the flight ID. So, we knew -- on most of the days, we had flight IDs. So, we knew which airlines the planes were flying from, what kind of aircraft it was, what fix they came in over, what runway they landed on. Where were they when they encountered the storm? How far from the airport were they? What was the total path link it took them from the time they entered the terminal air space to fly to the runways? Were they a leader or a follower, where that's something that's a variable that we made up, and the definition was that if another plane had flown the same route in the preceding 10 minutes, then you are a follower. If you are the first plane to fly this particular route within a 10-minute time span, which we arbitrarily made up, then we called them a leader. What was the altitude? What time did they arrive, and we also had the delay information. We knew what time -- we knew how long their flight had been. We knew what time they were supposed to take off, what time they did take off, etc. So, let's look at a couple of examples of -- I'm going to show you movies now of two of these stormy days. Is there any way we can focus that just a little bit better or is that as good as it gets? Okay. That's what we have. This is the Dallas-Fort Worth Terminal Area seen from above. The DFW runways are right in the center, and to give you an idea of scale, each of these little whisker lines right here is one nautical mile long. So, the runways are very small in here. It's about 40 nautical miles or 45 out to the edge of the TRACON. Arriving traffic typically enters the DFW air space over these four corner posts. Each of the blue numbers that you see here is the current location of an aircraft, and then I have 10-minute long contrails, if you will, in red. So, we can see when the flow is in a very steady state, all those red lines line up over each other, and you have a simple flow. We'll start the animation in just a minute, and you can see that when planes start deviating, the red lines diverge and go around the storms. The colors are a little hard to see in the projector, but greens again are Level 1 and 2, anything yellow or orange or red are Level 3 and above. Would you start the animation, please? Thank you. Here it goes. Every time the screen updates, that's 30 seconds of real time. So, this is much, much faster, you'll be happy to know, than the planes fly in real time. But here we have four arrival streams coming in, five arrival streams actually because there are two from the southwest. These are planes landing at DFW and at Love Field. Some storms are starting to build. Some near the nominal flight path. So, we'll start to see -- you can see the red line bowing. Now you see planes going on both sides. There's some deviation starting there. Now they're coming in. They're diverting around the weather. This storm also is moving through the nominal flight path. So, the arrivals have stopped for a moment, and you can see the planes flying to the north around the weather and to the south. These storms will continue up to the north and disrupt this flight path which at the moment is fairly nominal, though you see it's going to start to be affected. The planes are holding outside the TRACON here. Some are finding a path through the storms. Normal arrivals from the southwest, deviations coming in from the southeast. On this day, we'll see in a chart -- I believe on this day, there were an equal number of penetrations and deviations. You can see some -- a little bit of holding here in the terminal air space. This storm that just came up across DFW dropped a microburst on the air field, and the planes scattered and held, but the storm was fast-moving. It moved off the air field, and the planes came right back in. They didn't have to leave the TRACON. The ATC held the planes at the corner post for a few minutes. It was really beautifully choreographed compared to some instances where the wind shear sits on the air field, and everyone has to find somewhere else to go. Now we have planes here penetrating yellows and reds. It's hard to see with the way the projector's focused, but penetrating storms on final approach and near the airport, and I'll quantify that for you in a minute. Let's go to the next slide, which shows another, and this is -- I'm just going to show you two days from the study, and start the animation, please. The same air space, same parameters. Every time the screen updates, it's 30 seconds. Here we have a long line of storms across North Texas. There is a fortuitous gap at the moment in this line of storms over the air field. That gap is going to fill in, but in this movie, we're going to see pilots penetrating the storms Level 3 and higher for more than an hour on final approach, starting now. It's hard to see. There's yellow there, but we'll see it. No arrivals from the northwest because of that long line of storms that was there for awhile. Now, they're going to realize that there are some holes here that they can fly through, and we'll start to see arrivals coming in from the northwest. You can see the planes penetrating, and I'm sorry it's so hard to see on the projector that the storms there are yellow and brown, Level 3 and higher. They're going to continue to penetrate for a few more minutes. You see a little holding here at the corner posts, and now they're going to start penetrating again, and then, for some reason, I can't tell you why, they're going to stop penetrating, and it's going to be very chaotic for just a minute, and two planes that are just about to land are going to take very long pathways through very heavy precipitation, and they are going to leave the air space. You see some pilots here choosing to divert instead of penetrating, but mostly they're penetrating this Level 3 and 4 and 5. This chaos is about to begin, and the two planes that get the long pathways through this storm right here, there they are right there, you can see what they thought about the ride. They're leaving. Everyone else is landing safely, going around the ends of the storms and landing on the diagonal runway. If we have time at the end, we can show this again. You can watch it for quite a long time and always see something new. But we'll go on to the next slide. Thank you. As I mentioned, we looked at -- in the data set, we identified every penetration and every deviation. The penetrations were easy to find. The computer could do that and knows where the planes are, knows where the weather is, and, so, the surveillance radar that we use saw the plane every five seconds. For every aircraft location, we went in to each of those weather variable -- weather fields and pulled out the weather value for the location of the plane every five seconds. For deviations, we used human analysts who were familiar with the nominal routes in the TRACON from the corner posts to the runways to identify when aircraft were deviating around weather. They would use a mouse, draw a box around the storm that caused the deviation, and then the computer would take over again and go into each little grid point inside the box that the human drew. So, the human drew the large box here, and then we're illustrating each of the grid points that are inside that box, and the computer would pull out the value of every variable inside every grid point. This gave us a large database of penetrations and deviations, and here is the total data set. Nine days, again from late April of '97 through July 5th. The cases ranged from four hours up to -- we had the longest one was 12 hours. There were a total of 1,952 encounters with thunderstorms in this time period. These are arriving aircraft only. 2,000 times, the planes had a thunderstorm in their nominal flight path. One-third of the time, 642 times, they deviated around the storm and didn't penetrate. Two-thirds of the time, they penetrated the storms. Given that long list of variables from the earlier slide, we were able to break the planes -- we looked at the subset of planes that had encounters with Level 3 and higher weather, and we broke them into various subsets and did some statistical hypothesis tests to see if one group was more likely than the other to penetrate. So, the first test indicated on this chart is planes that were delayed somewhere upstream on the current leg of flight before they reached the Dallas air space. So, aircraft that had been delayed for 15 minutes or more somewhere upstream on this current leg of flight, after -- after leaving their departure airport. So, if they held on the ground, say, for two hours and were two hours late leaving, that doesn't count. It's a 15-minute delay in the air or longer versus the aircraft that had flown to the Dallas air space in their nominal flying time, and we found that to a very statistically significant extent, the aircraft that had been delayed upstream before they reached the Dallas air space were more likely to penetrate Level 3 and higher weather than the aircraft that arrived in the air space in the nominal flying time or less. This -- these hypothesis tests are usually characterized by a confidence level. The confidence level on this conclusion is that the one percent level. The chance that these numbers were so different by chance rather than by some sort of structure is less than one percent. The next set of subgroups that we split the data into were the leaders and the followers. Again, a follower is anyone who had another aircraft in front of him some time in the preceding 10 minutes, and as you could see in those movies, most of the folks were followers, but every time they break off and start a new path, that person is a leader. Again, the aircraft that had a plane flying the same route in the preceding 10 minutes, those aircraft that encountered Level 3 and higher were much more likely to go through than the aircraft that were the first one to drive up, pardon the term, to a Level 3 or higher storm. They were much more likely to deviate. Next, we split the encounters with the heavy weather, between those that occurred during daylight hours and those that occurred more than one hour after twilight. So, when it was clear that the sun was no longer illuminating the cloud and visual cues out the cockpit window would be -- the normal visual cues would be difficult, though, of course, even in the day time, it's difficult to see lightening strikes, and at night, it's -- you can detect the presence of lightening, if not its location perhaps, which found that storms that encountered these electrified -- pilots that encountered these electrified storms at night were more likely to penetrate the heavy weather than those that encountered them during the day time. So, we can't say conclusively that the visual cues play an important role, but that -- that result -- that's one possible conclusion you could draw from that result. The final test we did using the flight variables was to look -- to compare the performance of pilots or the behavior, I should say, of pilots at the different airlines. The five airlines with the most aircraft flying around in the Dallas air space in our data set are listed here, and there were small differences in the percentage of the pilots that penetrated both heavy weather and weak weather, but none of those differences were large enough to be statistically significant. So, the behavior of the aircraft -- of the -- of the pilots, there were no differences that we could discern across the airlines. None of those, however, were what NASA asked us to look at. They asked us to look at whether or not deviations could be predictable and particularly using weather variables, and what we found through our statistical analysis is that far from the airport, the answer is yes. More than kilometers from the airport, the plane -- the pilots' behavior was very well correlated with three types of variables: the intensity of the storm measured by almost any variable and almost any radar, the better with the 3-D pencil beam radars than the 2-D radars; the weather coverage, how much weather is in the area. When the pilots were encountering the only storm in the sky, they tended to go around it, but when there's widespread precipitation, and one would need to fly through some precipitation to get where one was going, they tended to penetrate, but then it curved again at the upper end of the scale. When there was widespread heavy precipitation, they tended to avoid the region. So, weather coverage was -- was correlated with their behavior, and, finally, the range from the airport, and we found that far from the airport, based on this data set of 2,000 encounters, if we -- if we trained the statistical classifier on some of those encounters and then showed -- tested it on the other encounters, where we were able to correctly predict what the pilots would do between 70 and 85 percent of the time. Now, NASA didn't ask us nor are we presumptuous enough to think that we would ever predict what a specific pilot is going to do, but the question is could you come up with a product that indicated the statistical likelihood that a storm would cause deviations. So, I don't want to imply that we were -- that we're predicting individual pilot behavior, but -- but we were able to predict the penetrations and deviations roughly three-quarters of the time far from the airport. Near the airport, however, it's much more complicated because the pilots almost never deviated. They almost always penetrated. This plot is showing as a function of range from the airport in kilometers. This is the entire data set. These are the deviations and the penetrations. The deviations are the dark bars, and the penetrations are the light bars, and I apologize if the people in the back can't see. I hope you can see the light bars. Far from the airport, we have a pretty homogenous mix of deviations and penetrations, but within 30 or within 25 or 20 kilometers of the airport, most of the encounters are taking place there, and as you can see, they're almost all penetrations. Very, very few deviations near the airport. We'll see on the next slide that those penetrations were of Level 1, 2, 3, 4 and 5 near the airport, and try as we might with our powerful statistical software, the behavior -- the ones that chose to deviate, it was not because there was anything different about the storms measured by the weather variables that we measure. Now, there may be some -- some variables that we didn't measure, but their behavior was not correlated with any of those storms -- those variables that we measured. This chart shows only the -- excuse me -- only the penetrations. Again, as a function of range from the airport, color- coded by the intensity level or the VIP level of the weather that they penetrated, and the yellow is showing up as sort of a sickly green here, but we've got Level 1s in the dark green. We mostly didn't pay much attention to Level 1. We've got Level 2s in the brighter green, and then this should be a yellow, and the yellow and magenta and red are Level 3, 4 and 5, and you can see that there are some penetrations of Level 3 and higher at all ranges, but within 25 kilometers of the airport, we have in this data set, if you count over to the left, more than 200 penetrations of Level 3 and higher, and you need to keep in mind that we had not only the two-dimensional weather information, but we had a 3-D picture of the storm. So, if I say it was a penetration, and I don't mean flying underneath it, the plane was flying through weather of that intensity level as measured by one of those radars. Some of the things that could be a factor near the airport that we've heard when presenting this work to various groups, and it just sort of makes sense, common sense, is that there is less lateral leeway as you get nearer the runway. You can deviate when you're far from the runway, maintain your place in line and stay on the nominal -- and -- and make your approach, but as you get nearer and nearer the runway, a deviation means necessarily that you're going to do a missed approach or an aborted approach. You're going to lose your place in line and fly back out into that storm field that you've just made your way through. There's a higher workload, higher pilot workload in the cockpit as you get closer and closer to the airport, and this is conjecture on my part, but it may be true that the pilots go from acquiring information more actively, where they have to devote some of their time to manipulating the radar and requesting -- requesting, for instance, a TWIP message, entering the request and waiting for it to come back, to a system of letting their eyes and ears give them the most information, to looking out the cockpit window, listening for wind shear alerts from the controllers and from PIREPs, from other pilots. This is the last slide before the summary. This shows -- it tries to answer the question of whether or not there's a, quote unquote, penalty for deviating at any particular range, and the lower line, the brighter purple line, shows as a function of range from the airport for the aircraft that penetrated storms at those ranges, what was their total path link, that it took some -- took them to fly from the edge of the TRACON to the runway, and you can see that the distance from the edge of the terminal air space to the runways is generally between 80 and 100 kilometers, depending on which runway you're landing on and which fix you're flying over, and that if you penetrate the storms and don't deviate, then the total path link is between 80 and 100 kilometers. For aircraft that deviate far from the airport, their total path link to get to the runways was very comparable to the planes that penetrate. So, there's very little cost in terms of an extra path link to get around the storms at those ranges, but as you get nearer and nearer the runway, in particular within 25 kilometers, then -- now if you deviate, this is just again validating the common sense. Now, if you deviate, you're committing to doing a missed approach, and you have to go out and -- and rejoin the que at some point, and, so, the total path link that those -- the average total path link that those planes flew was well over 50 percent more which just makes sense. So, we just wanted to check and make sure that it made sense, but it was interesting to see. We did not know what we would find out here far from the airport. Far from the airport, you can see really there's very little penalty in terms of a deviation. Your total -- the total path link it takes to get to the runway is comparable to if you had penetrated. So, in summary, far from the airport, we were able to fulfill NASA's requests quite well. The penetration and deviation behavior seems to be quite predictable if you have some measurement of the storm intensity and the coverage of the precipitation region, and there's some reason to hope that in the future, if automation systems or decision support systems like FAST are going to run on days when there are storms around as well, that we might be able to help them understand which pathways are likely to be closed and which pathways are likely to -- to not result in nominal flying times. Near the airport, however, we found that there were a great many penetrations of precipitation of all intensities, all of the levels, except for 6, no penetrations of Level 6, and that sometimes those streams of penetrations until something objectionable happen would then lead to the planes behind the one that had the objection having missed approaches and aborted approaches as well. We found that pilots were more likely to penetrate intense precipitation when they were following another aircraft, if they had been delayed upstream somewhere in this current leg of flight, and if they were flying after dark, and we did not find any discernible differences in the behavior of pilots from different airlines, and the full report, which has been entered in the record, is also available at our web site. That concludes my presentation. INTERVIEW BY TECHNICAL PANEL BY DR. BYRNE: Q Mr. Rhoda, just to re-emphasize or clarify one of your last statements here, was there any biases or trends in air carriers of which were more likely to penetrate than any other air carriers? A There were differences, but there were -- but those differences were not statistically significant. So, we had, you know, different numbers of encounters by different airlines just because no matter where you drew the line, if you take a larger data set, you would have different numbers of encounters, and there were differences in the percentages of pilots from the different airlines that -- that penetrated or deviated precipitation, but those differences, when you applied the statistical hypothesis test to see if those differences are statistically significant or not, none of the differences were significant. So, the numbers were different in the sense that the numerator and the denominator came up with a different ratio, but in the statistical sense, that we could say that this airline is more likely to penetrate or deviate, there were no differences. Q You also conducted similar research in Orlando, Florida. Did you find similar findings? A That study in Orlando was a preliminary study to this one, back in the Summer of 1993, and it wasn't as carefully controlled as this one in that the computer wasn't going in and pulling data out for every penetration. We did that small study to see what sort of software we would need to write to do a larger study. Generally speaking, what we found, broadly speaking, was similar. We saw penetrations of more intense precipitation near the airport, and farther from the airport, they tended to avoid things that were Level 3 and higher. But the encounters which we chose to examine were all subjectively selected. Myself and a couple of other analysts chose the encounters, quite frankly, that looked interesting, and then we -- rather than having the computer do the analysis, we sliced the radar data vertically by hand and went in and did a lot with pencil and paper. So, that small study paved the way for this one. This one, I would say, is more objective, but there were no -- broadly speaking, the results were the same. There were surprisingly high -- there were penetrations of surprisingly intense storms near the airport, even when viewed with the three-dimensional radar. So, the results were very similar. Q The research was also done at one of the world's busiest airports and home to a hub for two large major airlines. Would you expect to find anything different going to a little smaller airport, like Little Rock, with less traffic flow? A It's hard to say. Orlando's certainly smaller than Dallas, and the results were largely the same, and, so, I can only comment from that. I'm not sure how -- how the behavior would vary at the ends of the spokes, if you will, rather than at the hub. I'm not sure what you'd find. We'd be happy to take a look at some of that, but I don't know. Q What role do you think compliance and airborne weather radar systems and wind shear systems and the training that we've done in the past with wind shear recovery, what type of complacency do you think is a factor in this study? A We saw interesting behavior when there were microbursts at the airport, and in hindsight, we probably should have included not just was there a microburst there but was there a warning read to the pilot? Was there -- did the pilot receive a warning? As Wes indicated earlier, that's -- that's harder to measure. It would involve recording the voice transmissions between the towers and the pilots, but we did not see any penetrations of microbursts that were on the runways, and, so, it's possible that those deviations that we saw near the airport, which were not explained by any other -- any of the weather variables that we did include in our study, would be explained if you included the variables, saying pilot received a wind shear alert. So, perhaps that's when they decided to deviate, is when they received a wind shear alert. We did not see penetrations of microbursts in the regions where the TDWR system warns for microbursts. We did see one fairly disturbing incidence where pilots were penetrating Level 3, Level 4 and Level 5. All of a sudden, they stopped penetrating, presumably for some reason, possibly for some reason you'd hear about on the voice recording, but we didn't have it. Two planes broke out to the right, one plane broke off the approach to the left turning -- avoiding whatever was in front that presumably someone had objected to but turning into the rain shaft that was producing at the surface, this 60-knot microburst, a very strong microburst, but the microburst was beside the runway, not along the runway. It was far enough away from the runway that it was in a region where no warnings would be read to anyone. So, if the planes fly the nominal flight path, they have the benefit of those pilot reports. They have the benefit, particularly at a busy airport, of there's a high likelihood that someone has recently flown along there. Now, we know that the weather can change in those intervening minutes, but you have the benefit of the PIREPs, but as soon as you break off from the nominal approach route and fly to the left or to the right, presumably because of some hazard that's been identified in front of you, now you're flying in a region where there are no PIREPs because no one has flown there in the last few minutes, and the wind shear detection systems, even the most sophisticated ones, do not provide information to the controllers to read about any air space except the nominal flight path. So, the only really significant penetration of wind shear we saw was outside the warning area, but it was a very strong microburst. It was an ATR plane, didn't have any problem with that altitude and was not intending to land. So, it was not flying near the stall speed. In terms of complacency, because we did not record whether or not the pilots were receiving wind shear alerts in this study, I really can't comment about whether they complied with their rules, and also the rules vary slightly from airline to airline. It would be possible to do such a study, but it wasn't the focus of the study that we were doing. Q Your study indicates or demonstrates some very significant safety impacts. I'm sure you presented this to the FAA? A It's been presented, yes, at high levels at the FAA and at NASA, the folks at NASA who are doing the AWIN work. Q Has there been any request for further -- further research or further action regarding this matter? A To date, there have not. So, the study has been in circulation since June, and -- and we have put together a couple of proposals for some follow-on work, but nothing has come together yet. We're also testing the ITWS system. We -- we have the infrastructure in place to collect this type of data easily at several places around the country, and it wouldn't be hard for us to do this study, to extend it or to repeat it, to make sure the results hold either at other places or in more days at Dallas-Fort Worth, but we haven't yet been asked to do so. Q Mr. Rhoda, we'd like to thank you from the Tech Panel, and Mr. Feith has a question. MR. FEITH: Thank you. I'm going to jump in. BY MR. FEITH: Q This was a very interesting presentation. A couple of things. You didn't discern between airlines, but how about types of aircraft, jet versus turboprop? A We looked at that and didn't find a difference. Q Okay. A I should have -- I should have listed that. Turned out that there wasn't a difference. Q And you said that the report's been in circulation since June or thereabouts. Disseminated to airlines and -- and other facets of the aviation industry. Have you gotten any feedback from any of the air carriers that may have gotten your report, and what was -- what was their response to it? A We briefed the report at American a couple of times to different audiences. Once to one of the management pilot -- actually to a room full of several management pilots, and then later to the Allied Pilots Association and had very -- conversations that were very helpful to us in understanding some of the factors that come in to play. One of those viewgraphs reflected some of those conversations in terms of the restrictions in the lateral leeway. So, we gained some insight into what some of the factors are as the plane gets closer and closer to the runways. No one has expressed any particular surprise at what we've found. So, the reactions have been more along the lines of helping us understand the factors. You know, we are experts in weather sensing and the radar information and the interpretation of that information, and we've been testing our systems at FAA facilities for more than a decade now. So, we understand a little more perhaps from the FAA side or from the operational side inside the tower and the TRACON, and, so, those conversations have been illuminating to me in terms of what information is available to the pilots. No surprise particularly expressed when we have briefed this, but really comments to help us understand what's going on, and what factors are -- are being taken into account. Q Thank you very much. A Hm-hmm. CHAIRMAN HALL: Okay. We'll go to the tables. Boeing Commercial Airplane Group? MR. HINDERBERGER: Mr. Chairman, we have no questions. Thank you. CHAIRMAN HALL: American Airlines, Inc.? MR. BAKER: Thank you. We have no questions. CHAIRMAN HALL: The Allied Pilots Association? MR. ZWINGLE: Thank you, Mr. Chairman. INTERVIEW BY PARTIES TO THE HEARING BY MR. ZWINGLE: Q Mr. Rhoda, am I correct, did I hear you correctly, did you state that your work is not drawing any correlation between the behavior observed, the traffic at DFW Airport versus Little Rock? A No, we haven't examined any data from Little Rock. So, I wouldn't have any way to comment on what the behavioral -- Q So, this presentation does not draw a correlation between the two operations? A Nope. This -- this presentation is strictly behavior in the DFW air space. Q Okay. A The only other air space we've looked at is Orlando, and again that was a small exploratory study and not really large enough to draw statistically significant conclusions. Q Okay. You referenced a Level 3 rule of thumb. A Hm-hmm. Q Is this a -- you say this is a pilot rule of thumb or a controller rule of thumb? A Well, I don't know that we'll find it written -- well, we do find it written down in that IFR Magazine. It's something that we've heard first, something that personally I have heard first at FAA facilities. So, I've spent a great many hours in TRACONs and towers, sitting with controllers and traffic managers since they have helped us develop prototype weather systems. The Terminal Doppler Weather Radar and ITWS, and with their feedback, and, so, in talking to them about pilot behavior in the presence of thunderstorms, that's where I first heard what I'm referring to as this Level 3 rule of thumb, where they've said -- the only variable that those personnel have ever had to measure the intensity of storms is this VIP level, this six level scale that we've been talking about today, and, so, they don't have access to VIL or to anything that has any three-dimensional storm structure or to the echo top information or anything like that. So, they only have one variable. They only have one yardstick, and the rule of thumb that they have then on that one sort of dimensional scale is that Level 3 and higher, when the weather reaches Level 3 and higher, that -- that's when the pilots start to request deviations. Not a hard and fast rule that they always deviate, but that when the -- from a controller's viewpoint, as I understand it from these conversations, when the weather is reaching Level 3 intensity, you better start thinking about your routing options because they're going to start requesting deviations, and then, after having heard that in FAA facilities where we've been testing our products, so in Orlando and Memphis and Dallas, both in the en route centers and in the TRACONs and the towers, then I came across this reference in the print in a -- from the slide that I showed earlier in a magazine for pilots and to pilots about the weather information the controllers have and what to do in that area. Q Okay. So, in consideration of the fact that in my own informal survey here at the table with six pilots and probably 50 to 60,000 hours of experience who have never heard the rule of thumb, am I correct to assume this is not necessarily an air carrier -- A Certainly never anything I've ever seen in writing, except informally in this general interest magazine article in a pilot magazine. But something which is widely quoted in FAA facilities as their characterization of how pilots behave, if they have to express it as a function of the only variable they ever have to measure storm intensity. Q I understand. Thank you. Does your study indicate in consideration of the location at the Dallas-Fort Worth Airport and in consideration of the fact that this airport has -- is blessed with multiple sensors, as you referenced, how much of this information that was gathered from -- from those sensors was transmitted to the air crews during your study? A The air crews would not have received any graphical -- any color graphical depiction of the storms. The TWIP product has an alpha numeric -- Q Yes. A -- depiction that flight crews that have, if I understand correctly, a printer in the cockpit which would be a limited -- I'm not sure which air crews. There's some of the Delta planes, but I'm not sure which aircraft types that have a printer as well as an alpha numeric display for their ACARS. They can requested a map which would have little numbers, 1s and 2s and 3s and Ms for microbursts and lines of Gs for gust fronts. Those would have been the only air crews that could have -- would have received a graphical depiction from this plethora of sensors. Q So, -- go ahead. Continue. I'm sorry. A In terms of verbal description from a dispatcher or from a controller looking at these displays, it would be hard for me to say. No controller has one of these color displays in front of them. The supervisors in the towers and the TRACONs do. Some controllers, by virtue of where they're sitting, can see them, and sometimes on the frequency, we hear them giving descriptions to air crews that could only have come from our prototype systems and not from their PVD. But no air crews routinely receive graphic information, certainly color graphic information, from these modern ground-based sensors. Q I understand that. Thank you. Are you aware that not all air carrier aircraft are ACARS-equipped, not capable of receiving the TWIP? A I am aware of that. Q Okay. Do you know -- do you have data in your study to suggest that any aircraft were warned of -- of -- of hazardous weather? A I -- I know from having listened to the scanners, the ATC frequencies, on those days. During this time, I was the office manager of our Dallas -- it was a prototype system, and I worked on many of those days, and from listening to the scanners, I heard the wind shear alerts and the microburst alerts being read as the controllers are required to do. In terms of warnings about precipitation intensity, the controllers, as I hope we all know, are certainly not required to read - - to relay any information based on their weather radar information. They are very loathe to do so. They're often heard to say, as they are, I believe, in the cockpit voice recording in this incident, you have better information than I do because they are suspicious of the quality of the information that they have, as they should be, because of some data quality issues with the airport surveillance radar until it is cleaned up by systems like ITWS. So, I don't know. We didn't record the voice information. I don't know. Controllers do sometimes say I'm showing a Level 4, you know, what do you see or I'm showing this or what, what do you see? But they're not required to do so, and it's rare because they are concerned about liability issues. Q Charming. Does your data -- is your data broken down to reflect the percentage of flight time an aircraft is on a published route versus a radar vector? A No. No. That would be a useful thing to do, but it's not. Q Okay. Two more. Did you conduct any debriefs, flight crew debriefs after the penetrations were observed after the arrivals were completed? A I did not. Q Okay. And did you have any personal observation time on the flight decks of the air carrier aircraft? In other words, jumpseat observations? A In the study? Q Yes. A No. I have flown jumpseat, but not on any of these nine days. Q Thanks very much. A Hm-hmm. Q No further questions, sir. A All right. CHAIRMAN HALL: Association of Professional Flight Attendants? MS. LORD-JONES: We have no questions. Thank you. CHAIRMAN HALL: National Weather Service? MR. KUESSNER: National Weather Service has no questions. CHAIRMAN HALL: Little Rock National Airport? MS. SCHWARTZ: No questions, sir. CHAIRMAN HALL: Little Rock Fire Department? MR. CANTRELL: No questions, sir. Thank you. CHAIRMAN HALL: And the Federal Aviation Administration? MR. STREETER: No questions, Chairman. I do have your answer on the Little Rock Airport. CHAIRMAN HALL: All right. MR. STREETER: They are not scheduled to get TDWRS. CHAIRMAN HALL: So, -- and someone gave me a list. I guess there are 39 airports that presently have the Terminal Doppler Weather Radar. MR. STREETER: I don't know the exact numbers, but -- CHAIRMAN HALL: And four that are under construction, and then the one we got worked out up at JFK, I think. MR. STREETER: That's correct. CHAIRMAN HALL: So, as usual, you're getting about the top 40 airports in the country. What about the rest of them? Is there any other advance system planned at this point? MR. STREETER: Yeah. I have been advised there is another system planned. I also understand there are no more plans for additional TDWRs. So, it will go to the later system. CHAIRMAN HALL: Okay. Very well. Very well. THE WITNESS: That additional system was also developed in the Weather Sensing Group at Lincoln Laboratory. It's the ASR Weather Systems Processor, and I believe it's currently slated to go to 34 additional airports, 34 airports in addition to the ones that have received TDWR. CHAIRMAN HALL: Well, good. Okay. Well, thank you. I guess that's the tables. Mr. Sweedler? MR. SWEEDLER: No questions, Mr. Chairman. CHAIRMAN HALL: Mr. Berman? MR. BERMAN: No questions, sir. CHAIRMAN HALL: Mr. Haueter? MR. HAUETER: No questions. CHAIRMAN HALL: Mr. Clark? MR. CLARK: I have no questions. CHAIRMAN HALL: And the Chairman has no questions. Mr. Rhoda, we appreciate your presentation. It was certainly informative. Do you have any comments that -- additional comments you'd like to make? THE WITNESS: I don't, sir. Thank you very much for your time. CHAIRMAN HALL: Well, we appreciate you coming here, and I doubt if you're going to be leaving soon. But -- so, we hope you'll enjoy and follow the rest of the testimony. THE WITNESS: Thank you. (Whereupon, the witness was excused.) CHAIRMAN HALL: We are sitting at what, 5 after 6, and I'd like to get one more witness in tonight. I'm very aware that the President's giving the State of the Union here, and some may want to observe that, but let's take a short break, try to get back in about 10 minutes. The last witness is going to talk on the subject of the Automated Surface Observation System, the ASOS, which is yet another piece of equipment that is funded to -- to provide weather assistance. We'll be in recess for 10 minutes. (Whereupon, a recess was taken.) CHAIRMAN HALL: We will reconvene -- I guess i'm waiting. We will reconvene this hearing of the National Transportation Safety Board, and I ask Mr. Berman if he'll call our next witness. MR. BERMAN: Thank you. I call Mr. Rainer Dombrowsky. Whereupon, RAINER DOMBROWSKY having been first duly affirmed, was called as a witness herein and was examined and testified as follows: INTERVIEW BY BOARD OF INQUIRY BY MR. BERMAN: Q Sir, please state your full name and business address for the record. You need to flip the switch, sir. A For the record, my name is Rainer M. Dombrowsky. Q And by whom are you presently employed? A I am employed by the National Weather Service, located in Silver Spring, Maryland. Q What's your present position at the Weather Service? A At the present time, I am the Chief of the Surface Observations Branch of the National Weather Service. Q How long have you held that position, sir? A I've been in this position for one year. Prior to that, I held other Headquarters positions in the Office of Meteorology. Q Would you please briefly describe your duties and responsibilities in your current position? A In my current position, I oversee the policy, the implementation and the replacement of surface-observing systems which follow the range of our Cooperative Weather Observer Program and also the ASOS Program. Q And could you please describe your education and training and experience that brought you to your current qualifications? A I have a Bachelor's Degree from the University of Minnesota in Geography with a Meteorology Certification from the National Weather Service. I have been with the Air Weather Service of the Air Force, was with them for seven years and completed an additional 17 years with the Air National Guard, also operating a weather flight, and I've been with the National Weather Service since 1977, and I've worked just about every position in the Weather Service, from meteorological technician taking manual observations to my current position supervising the National Network. Q Okay. Thank you. And do you hold any current FAA airman certificates or current National Weather Service certifications? A None with the FAA. I do have an observer certificate with the National Weather Service which is still valid, both in observations and radar. Q Okay. Thank you very much. MR. BERMAN: Mr. Eick? INTERVIEW BY THE TECHNICAL PANEL BY MR. EICK: Q Good evening, Mr. Dombrowsky. A Good afternoon. Q I'd like to start our presentation this morning with giving you an eyewitness of the weather conditions at Little Rock on the evening of the accident. MR. EICK: The -- what is it? Six minutes? The clip is only 30 seconds, and it actually covers six minutes, but we'd like to give you a feeling of what was going on the evening of the accident. Will you please roll the video, please? MR. CLARK: Just a second. Do you want to describe where this video came from? MR. EICK: The video came from a security camera on the field at Little Rock and was -- well, four cameras, and provides us different viewpoints of the -- the airport area from the security cameras. The video runs from 0449 to 0454. So, I take that back on the time period. It's a little bit shorter, but again it's a short 30 seconds. (Video shown) MR. FEITH: What we are looking at is a ramp. The camera's showing the ramp, and it progressively gets worse as these clips play through. You're looking at four camera views per minute, five-second clips each, and as -- as we move into other camera views, and if you're watching the time, you see that it is raining. You can see objects on the ramp, but as the thunderstorm that has been talked about by various witnesses makes it progress towards the airport, the airplane is already down by our accounts as far as the landing time, but what this is showing is more or less a validation of the movement of the thunderstorm and the fact that the heavy rain does progress across the field as it's correlated to -- to all the time calculations from the witnesses talking about the wind shear and the LWAS system and the ASOS that we'll be talking about. What you're seeing is the rotating beacon on top of the tower. MR. EICK: That flash was a lightening strike nearby the airport. That's the end of the video. BY MR. EICK: Q Mr. Dombrowsky, can you provide us a quick summary of some of the advantages and limitations of the Automated Surface Observation System? A The latest generation of automated systems, the ASOS, was developed jointly with the FAA and the National Weather Service with some support from the Department of Defense in order to maximize the amount of observations that could be taken across the country. One of the limitations as has been heard earlier by the other federal agencies is that we're all operating under varying budgets, and as the budgets become tighter, we needed to look for other avenues because manual observations for one are expensive, and, secondly, the manual observations typically in the last decade or two have been taken in the basic weather watch mode rather than the continuous weather watch mode because of the need for the observers to be doing other duties while they were in the observation arena. So, the ASOS has allowed us to produce more timely observations, more frequent observations. Although the ASOS does not have the capability of detecting certain parameters, such as hail, currently most ASOSs do not have the thunderstorm capability but that will be coming, freezing precipitation was another issue, and we have freezing sensors on the systems now. But phenomena, such as hail, tornados, volcanic ash, things of that nature, are still limits to the system. Q Can you tell us, what is the official source of weather data or observations for the Little Rock Airport? A The official observation for Little Rock is the ASOS System, and the ASOS System is complemented by a contract observer augmentation under a contract with the FAA. Q Is the ASOS monitor available in the control tower for ATC use and reporting the most current conditions? A Not the monitor itself. The operator interface device is located one -- right across the hall and downstairs from the control tower, but there is an interface that allows the observation to be retransmitted to the tower. So, once the ASOS and the observer are in agreement, the observation is retransmitted to the control tower at the various positions within the tower. Q How often are observations made, and are there any special criteria used? A Observations are mandated to be taken once per hour with the stipulation that if certain special criteria are met during that hour, an observation will be taken to accommodate that particular special criteria. Now, those criteria range from cloud height to cloud visibility, beginning and ending of precipitation, and a number of other parameters, such as wind shifts and so on. Q I'd like to find out some more about some of the specific sensors. Can you tell us how the ASOS System gets its time stamps to verify the accuracy of the time of the observations? A Well, the ASOS -- the first part of your question, the ASOS has a number of various sensors attached to it. It's a full complete system which has a cloud height indicator, temperature and dew point sensor, and the standard make-up for observations that you would see at any manual site. The time stamping capability is done in a number of ways. The ASOS has three internal clocks. Those three clocks are compared against one another routinely during the day. If there's a clock drift of more than seven seconds, the system will automatically call our ASOS Operations Monitoring Center in Washington, actually in our Headquarters in Silver Spring, and if there is a problem, the folks at the AOMC, as we call it, will reset the system, but it has the intelligence to be able to compare itself, and if two of the three clocks are within time, the system's considered accurate. Q The next reporting element on the observation would be the wind. Can you tell us something about the wind anemometer, and how high is it located, and is this an international standard? A Well, the anemometer is located with the system. It's set at the standard 10 meters, which is the WMO or World Meteorological Organization standard. The National Weather Service and weather services throughout the world use the WMO guidelines. If they deviate from those, they ask for special permission to do so, and we have done that in a number of cases in our Federal Meteorological Handbook Number 1, but that does not apply to the wind. Q We just saw a presentation on LWAS. The LWAS sensors at the Little Rock Airport are located anywhere from 70 to a 150 feet above the ground. How would wind impact the height elevation differences between our 10 meter ASOS? A You -- you could definitely see differences, but again that's going to be dependent on the type of phenomena going through the area. But I would typically expect to see slightly higher winds the further up you would go. Q How does the ASOS wind algorithm work, and how is it with regards to its accuracy? A Well, in the simplest context, the algorithm has already been pretty much described by the gentleman from MIT in his description of the two-minute wind. As he mentioned there, it's a continuous averaging, and what the ASOS does is sample the wind data. It also has a quality data check internal to the algorithm, and if there are deviations from what the system is programmed to look for, then it will throw out those characters and again provide the folks out at our AOMC with data quality errors. So, if there is a major problem, then we can go back by way of notification to the local office having support responsibilities for it, but it has an internal quality control capability, and again the system will average out those 10-second readings that are taken over the minutes as they were described earlier. Q Would a wind report drive a special report in any cases? A In some instances, a wind report will drive a special. One of those is a wind shift. Any time the wind is 10 knots or greater, as detected by the instrument, and the wind direction changes by 45 degrees or greater, the system cues to begin sampling, and that sampling is predicated on the fact that that shift holds for approximately 15 minutes, and then it is verified as an official wind shift. Some of the other wind parameters that the ASOS looks for as far as specials include squalls, where there is a sudden doubling of the wind speed in excess of 22 knots, and that would also precipitate the issuance of a special. Q In reviewing some of the ASOS data, in Exhibit 5A on Page 19, we have some of the one-minute ASOS data. Looking at that, could you tell us where a wind shift report would be initially flagged? A Looking at the one-minute data that is presented here, I would say that the flagging of a special would occur some time around the minute 31 -- 0431. That would be the time that the system has begun to sense the 45-degree shift, and then once that shift has been initiated, it times out for 15 minutes just as it does with some of the lightening sensor systems that we do have at a few of our sites. It times as well. So, it has to have that 15 minutes before it initiates that special, and the reason that is done, and we use those parameters, is to ensure that we don't issue numerous specials because in a variable wind situation, the winds may be vacillating back and forth, and, you know, we would be inundating the tower and the folks who use the various products by generating specials more frequently. So, that's why we have that 15-minute time limit, and that 15-minute time limit is the official convention being used by the National Weather Service in its Federal Meteorological Handbook Number 1, and the FAA uses it also in their 7900 Series regulation, which also deals with the observations program. Q What was the National Weather Service official wind report at the time of the accident around 0450? A At 0450, if you were to use these observations here, at 0450, we had -- again depending on what you're looking for, the two- minute wind was reading at 285 degrees at 16 knots, and then the five-second gust was reading 302 degrees at 22 knots, and that was one minute prior to the accident time. As the hand-out shows, the accident was documented to have occurred on approximately 0451, and the winds were very similar to that previous minute. Q Moving on to the other sensors, how does the ASOS report its prevailing visibility? A The ASOS reports its prevailing visibility by examining a small segment of the atmosphere. The actual distance between the two sensors is only approximately two to three feet, but what it is sampling is the clarity of the atmosphere and applying that to a prevailing condition, and again it does a similar sampling process there as well. Q How does the ASOS System detect thunderstorms? A At the present time, the ASOS System does not detect thunderstorms. We are in the process of a joint venture with the FAA. We are -- currently, we have just completed the deployment or in the process of completing the deployment of the ASOS Software Version 2.60. That particular ASOS software supports the FAA's LDRS software, which relies on the National Lightening Detection Network. The data from those ground strokes go into the LDRS. The LDRS goes into -- feeds into the ASOS, and from that, the appropriate remarks are generated with respect to the thunderstorms' beginning and ending times, the location and whether it is on site or whether it's in the vicinity of the airport complex. Q In regards to the thunderstorm location, how many miles does the lightening strike have to be classify as a thunderstorm at the airport? A At the airport, it has to be within five statute miles. If it's outside that five-mile band, excuse me, it's considered in the vicinity, and then if it's beyond the 10-mile statute mile marker, it is considered at a distance, and there are appropriate remarks that account for that. Q How does the ASOS determine the weather type and intensity? A The weather type and intensity is done almost in the same fashion that the manual observer does. We have a sensor on the field which can detect only certain types of phenomena, such as fog and precipitation. When it comes to types of weather phenomena, such as hail, our system cannot detect that, and again what it does is it looks in reductions in clarity of the atmosphere based on the water droplet size and so on. In addition, we also use the rate-of-fall technique, which is also found in the Federal Meteorological Handbook Number 1, and using rate-of-fall techniques, we can calculate the intensity through the ranges of light to heavy. Q One of the critical factors in this accident is going to be precipitation amount. How does the ASOS measure rainfall? A The ASOS has a tipping bucket device. That device is an eight-inch diameter rain catch which has a pendulum with two 1/100th catch buckets on a pendulum, and as the rain drops into the bucket, it fills the first cup, gravity takes over, and it dumps that cup out, and the next cup fills. Each time a cup fills and empties, an electrical charge is sent through the system, and it sends a message to the system that 1/100th of rain has fallen, and it begins to count cumulatively during that hour. We can also generate data that give you one minute and 15 minute read-outs on the precipitation. Q How accurate is that in the high-wind situation of 20-30 knots? A In high-wind situations, you have a problem with overshooting the rain because of the angle at which the rain is coming down. You also have problems with heavy droplets causing a lot of splash. So that when you get into torrential rains, the tipping bucket is not one of the most accurate, but then again all of our systems suffer some degree in catch loss because of high winds and also because of the splashing effect. So, I would estimate in general we could lose as much as 10 percent, depending on the intensity of the rain and the direction of the rain that's coming. Q Are you familiar where the ASOS is located at the Little Rock Field? A Yes, I am. We visited that on our first day here. Q Can you describe where it is in relationship to Runway 4 Left and 4 Right? A It is at mid-field on 04 Left and approximately one-half to three-quarters of a mile further away from the 04 Right. Q Is it possible with the location of where the ASOS is that the rainfall rate would be different at the departure end of 4 Right? A It's definitely possible. If you're familiar with convective precipitation, it can have a very sharp demarcation line, and I've seen instances that I personally experienced where I might be standing on one side of the street, and it's pouring across the street, and the rain never gets to my side of the street. So, there -- there is a very fine demarcation that can occur. So, it is conceivable that the precipitation could have been heavier over the ASOS. Q Looking at the data in Exhibit 5A on Page 19, can you give us an idea of the intensity of the precipitation with the one- minute averages of .01, .02, .03? Give us an idea of the intensity of the rainfall at this time. A Actually, using these one-minute values, I would say that we are at a moderate to heavy rainfall rate, even with what looks like small numbers, at .01 and .02. See, by definition, in rainfall rates, any time that you have 6/100ths of an inch or greater during a six-minute interval, that constitutes heavy rain by the definition used in the FMH-1. Q In rainfall rates of .07 inch a minute? A .07, yeah. You're definitely in the heavy category. You're -- actually, you're on the verge of heavy even with some of those smaller amounts at 0436. Q In the course of the investigation, it was also discovered that the observer use was used to augment the system, could not issue a special report between 0 -- 47 and 20 seconds after the hour to 53 and 20 seconds after the hour because of what he referred to as a "lock-out". Can you describe what this lock-out is? A The lock-out was put into place to account for accomplishing the mandatory hourly observation during the hour. It was felt that the specials, unless they were of urgent specials, such as a tornado or a funnel cloud, something of that nature, would not warrant any override of that -- of that lock-out period, and again that was a decision jointly made by the FAA and the National Weather Service again with the -- some participation by the DoD through the ASOS Program Management Council, which meet quarterly. We -- since then, we have looked at that lock-out period more closely and feel that we -- we should be looking at allowing observers to put significant specials into that period of time, though the one limitation we discovered in our review was that the writing of the software was going to take much more time than we originally thought, and one of the other issues was that the program office for ASOS was being closed since the system was becoming operational, and our contract with our software folks expired, and during that period, we were in the transition of giving our Software Network Group within the National Weather Service working on this particular software fixes. We expect to have the lock-out period issue resolved some time later this year. We're estimating in the August-September time frame with the deployment of ASOS Version 2.80 which will give greater flexibility to the system. CHAIRMAN HALL: Mr. Eick, can I just interject and ask, sir, who is responsible for the maintenance of the system? THE WITNESS: At the present time, the National Weather Service is responsible for all maintenance, logistics and acquisition. There have been recent overtures by the FAA to consider taking the maintenance over, but the two agencies are still discussing that issue. CHAIRMAN HALL: So, is this -- is this piece of equipment co-located with the RVR out there? Is that where that is? That little box of equipment that's -- THE WITNESS: I think it's in the proximity, but the -- the system itself resides again at mid-field on 04 Left, and it also is maintained by the National Weather Service, although it is technically owned by the Federal Aviation Administration. CHAIRMAN HALL: Do your people actually maintain it or contract individuals? THE WITNESS: No. We -- we maintain it, sir. What we have is an agreement with the Federal Aviation Administration. They own about 75 percent of all of the ASOSs. We own the remainder. Ours are focused not only on aviation, but we also have ASOSs that are focused on climatological programs, and -- but the majority of the -- actually all of the FAA-owned ASOSs are in support of airport locations and the aviation program. CHAIRMAN HALL: And you all check -- do you all check to see how often they're working properly or -- THE WITNESS: Yes, we do. CHAIRMAN HALL: -- calibrated? THE WITNESS: Yes, sir. We have a system in place at our ASOS Operating Center in Washington that can monitor these systems, and whenever we get data quality errors, the system will alert us to those, and then we would then dispatch the responsible office to go out and deal with it. Now, depending on the type of sensor that's in question, that had -- they have different response times. The more critical sensors, such as visibility and wind, have a 12-hour response time. Some of the less critical ones, such as the precipitation, they might have as much as a 48-hour response time. CHAIRMAN HALL: And in the meantime, do the people in the tower know that it's out or -- THE WITNESS: Yes, they do. In most cases, the -- we can go as far as issuing a NOTAM for a particular sensor if it is out, but again in most instances like Little Rock, when we do have augmentation in place, if the automated system goes down, then the observer begins taking manual observations, and they have the equipment to do so. CHAIRMAN HALL: Okay. BY MR. EICK: Q Mr. Dombrowsky, you had a chance to review the ASOS observations during the period. In Exhibit 5B on Pages 1 through 7, we have the ASOS information. On Page 1 specifically, at Little Rock, we have an observation at 0423 Z, winds from the south at nine knots, seven miles, with thunderstorms, few clouds at 7,500 feet with cumulonimbus clouds, ceiling broken at 10,000 with remarks of frequent lightening in cloud, cloud-to-cloud, and the position of thunderstorms. Based on this observation, it appears that the observation, the ASOS was augmented by the observer. Looking beyond this point, do you see any time in between that the ASOS would have issued a special report without human involvement? A The ASOS would have been obligated or programmed to issue a special shortly before the onset of the hourly because of the wind shift that had occurred at 31 minutes past the hour. Q So, that would have been around 0446, prior to the lock- out period? A Yes, right. Just prior to the lock-out period. Q What about the reduction in visibility? We went from seven -- seven miles, and then at the special time at 0453 Z, we were at one mile and thunderstorm and heavy rain, mist. Would there have been any specials due to reductions in visibilities during this time period? A There could have been. Again, it depends on how -- how quickly you slip down through the criteria. In some instances, the observer had override control of the -- of the system, and the observer does have that capability because there are times when the ASOS may be showing a reduction in visibility which might be more in line with a sector visibility rather than a prevailing visibility, and in those situations, the augmenting observer has the option to go into the manual mode and keep the system there. The ASOS will continue to update itself because on what it is seeing, and it will continue to tell the observer, but when they're in that override mode or that edit mode, the observer has control. Now, there is a good chance that that particular observation that slipped down to a mile could have been generated by the ASOS or by the observer. It's hard to tell from the data sets that you have here. We'd have to go back and take a look at the one- minute archive data to confirm whether it was manually generated or whether it was generated by the -- by the system itself. Q So, it is possible the thunderstorm and rain would have been -- could have continued with the precipitation, but it did not hit a magic value of visibility of three miles and would not have provided a special? A Well, that's correct. See, what -- from what I see in the trend here, you went from a six mile down to a one mile. All right. That meets the criteria for a special because you have gone below that three-mile threshold. You would be obligated -- the system would be obligated to give you another special if you came back up through that special criteria. So, as the visibility -- as any of these parameters diminish, you have to go through that threshold. When you are on an improving trend, it's coming equal to or through that threshold that the system will generate observation -- generate specials. The system at Little Rock is programmed to follow the standard special criteria for -- as issued in the Federal Meteorological Handbook, and that would give you a special criteria requirement at three miles, at one mile, at one-half a mile, and special consideration given to a quarter mile, and in the situation of cloud heights, it would be less than 3,000, 2,000, 1,000, 500 and then every 100 feet below that. Q Based on these observations, what was the first time hail was reported at the airport? A That would be indicated by the augmentation remark issued in 05 -- excuse me -- 0509 observation, at least that's how it's listed here, where you have the remark -- well, let's see here. Where you have the GSB, that's where the hail began at 0458. That would be the first carry-on of when the hail began. Q And in closing, I guess, in your review of the data, you found the ASOS properly working during the period of the accident? A Following notification of the accident, I asked the AOMC to capture all of the data, and we have a staff in our Plan Product Improvement Group that reviewed this data for several days prior to talking to the folks here at Little Rock, and based on our analysis, the ASOS was operating as designed. Q Thank you. Mr. Dombrowsky, I appreciate you coming out in this nice snowy day, and that's all the questions I have. MR. EICK: Mr. Feith? MR. FEITH: I just got two questions real quick. BY MR. FEITH: Q The first one. You were talking about the configuration of where the system is in the current tower. Little Rock's building a new tower. Is this going to be the same configuration in the new tower? A Yes, it is, because the ASOSs per FAA agreement are to be placed at the mid-field touch down. Q Okay. A So, they're all placed that way across the country. Q Okay. And then -- CHAIRMAN HALL: How many fields get ASOSs? THE WITNESS: We have -- I couldn't tell you how many we are using purely for climatology, but we have roughly a thousand ASOSs around the country. One of the benefits of the ASOS system has been that we've been able to in some cases at what we call Level D sites -- I'm not sure if you're familiar with the airport level ratings of the FAA and the Airline Pilots Association have agreed to. But depending on the level designated to that particular airport, the office can -- the ASOS can either be a stand-alone at Level D where all you get is the automated observation. At those particular sites, we have adapted those systems to be able to detect lightening. We have 19 sites around there that have single site lightening detection capability. They also have a freezing rain sensor capability, and they have a precipitation catch capability. These were requirements placed on the National Weather Service by the Modernization Transition Committee, and until we met those requirements, we did not have stand-alone D level sites. If those sites do not have those capabilities, then those sites are required, both by the National Weather Service and the FAA, to retain contract augmentation, and at D level sites, we do have that. The service level also will dictate what additional parameters need to be reported, such as RVR and other enhancements that the control tower personnel can't handle, and under those situations, Level C, Level B and Level A-type support is provided through the augmentation process. CHAIRMAN HALL: Okay. BY MR. FEITH: Q How does the augmenter get the observation up to the tower? A The augmenter allows the ASOS to do what the ASOS is programmed to do. Once the observer concurs and does the transmission, and it becomes official, the augmenter then types the observation manually once again on the asynchronis system that links the augmenter to the tower and to the National Weather Service office because that system also provides a number of other meteorological products to the control tower. Q Okay. And last question. As far as the visibility and the documentation of the visibility, I believe that the sensors are at around six feet. A Yes, sir, hm-hmm. Q That -- that visibility at six feet, how was that number chosen versus 10 feet versus two feet versus a hundred feet? A The -- the convention that was used was adopted when flight was first really getting going. The average height of a pilot's cockpit was at approximately six feet, and that was the choice of height for both the temperature sensor and also for the -- for the visibility sensor as well initially, and we're looking at maybe modifying some of those as well. But that was the initial rationale behind the choice. Q So, the attempt to modify it is based on newer aircraft, being that we have cockpit heights higher than six feet? A Yes, sir, and, then again, what we're trying to replicate is the -- again, the prevailing visibility at what we deem to be a ground level height for the air crew when they're coming in, and then that's the rationale behind it. Q Will that take into consideration 747, MD-11s, type aircraft that are coming in at a high deck angle, where -- A Actually, it doesn't take that into consideration, plus you also have to deal with other issues that might impact air traffic coming in, is slant range visibility. We are trying to portray what the visibility is at surface, and that's what we're trying to do with that particular system, and, you know, it doesn't deal well or do slant range visibilities, and I know those can be critical because I've worked in support of CDSUs where slant range visibility of four miles was critical to incoming traffic, and it reduced the amount of traffic flow into a particular airport that I monitored. Q Very good. Thank you very much. CHAIRMAN HALL: You have a thousand of these items out there? THE WITNESS: Yes, sir. CHAIRMAN HALL: Do you monitor all thousand in Washington? THE WITNESS: Yes, sir. CHAIRMAN HALL: Hmm? THE WITNESS: National Weather Service monitors all of them through our system in Silver Spring, Maryland, and it's a continuous monitoring process, and if a system shows any flaw, giving us data quality errors, it automatically alerts us. We built that self-interrogation capability into the system so that when a system felt it was -- something was incorrect, it would flag us in Washington. We can do diagnostics from there, and, you know, again, if it is a major problem that requires replacement of a sensor, we can then go ahead and contact the maintenance support staff out here in the Little Rock area. CHAIRMAN HALL: Do you do the ones that are in Alaska, too, in these passes? THE WITNESS: Yes, sir. CHAIRMAN HALL: Okay. Let me see. American Airlines? MR. BAKER: Thank you. INTERVIEW BY THE PARTIES TO THE HEARING BY MR. BAKER: Q You mentioned that the -- the augmenter or observer takes the output from ASOS, checks it essentially, that it -- that it seems reasonable and makes sense, and then gets it to the tower. How long does that process typically take? A We have a five-minute edit window for all observations. So, -- and again, it takes approximately -- it would take a person approximately five minutes to do an accurate good manual observation, and that's the rationale behind using a five-minute edit window. Q On some other rain-measuring gear around the industry, units are outfitted with a wind shield to essentially help deal with that issue of very strong winds and angular water and inaccuracies. Has this ever been considered for ASOS? A We've not considered it for this particular system on ASOS. The wind shields typically give you the best results when you're dealing with frozen precipitation and not with liquid. So, the -- the adding of a wind shield wouldn't add any significant increase in the catch. So, we've opted to delay putting any type of equipment on the next generation system that we've got coming in. We are expecting to go to a next generation all purpose or all weather precipitation gauge, and we're in the process of reviewing the candidate systems right now, and I'm sure those candidate systems will have some additional skirting placed around them to -- to minimize that, but it becomes more of an issue with solid precipitation, days like today, rather than liquid precipitation, although there is some loss of catch, depending on the strength of the wind. Q I recall that when ASOS was being developed and initially rolled out, there was quite a debate between the air carriers, airports and the FAA, and presumably you folks were involved as well, on -- on the issue of RVR, and as I remember the -- the disagreement, the initial units did not have RVR measuring capability, and in essence, the -- the FAA's plan was to abandon the alternate or common RVR gear, leaving the airlines and the airports to essentially find some other way to -- to measure RVR. Has that debate been settled, and what's the status of RVR and ASOS? A The debate has not been settled. We are -- we have our software ready to report RVR once the FAA makes the determination as to what level of RVR capability they would like to have. The issue is scheduled for further debate, I believe next month, at the next ATA meeting in Washington, and because I know that the airlines are very interested in restoring the RVR capability that the National Weather Service had at one time. Once we eliminated the transmisometers, which are the systems that provide you the read-outs, once those were removed, there was no move to replace them with anything else, and that was the one -- that was one of the limitations that the ASOS had, and that was one of the reasons why we continue to maintain augmentation at the Level As and Bs and possibly some select Cs that require the RVR read-outs. Q You indicated that the majority of the ASOS units out there do not have currently the capability to detect thunderstorms. What equipment does the observer have to supplement ASOS or make up for -- for ASOS in regard to thunderstorm and lightening activity? A Here -- well, I'll just cite Little Rock because I just got to see their operation this week. What Little Rock has is basically just the -- the use of the human observer. The observer's ability to listen for thunder, which was a requirement or a cue, if you will, for the beginning of thunderstorms. They visually track. They receive the radar observations on their asynchronis system, so they can get a better feel for movement rather than trying to rely on their own eye contact, but that's about as much as the augmenter can do. Basically what the old manual observing system used to do, and that is use the radar, use whatever complementary sensing devices you have. If you've got a satellite image, that might help you a little bit, at least give you a first hint that's something coming or something's, you know, going to be developing. You can use the terminal forecast to judge the timing on when a storm might be coming in and to be more cognizant of the fact that that potential is there, but really it boils down to just the human approach to observing the weather, and that's really all they have right now. Q Was there a qualified observer on duty the evening of this accident? A Yes, there was. You happened -- actually, there were two on that evening because you were -- the event occurred right in the middle of a shift change. So, both observers were there for the better part of the hour. Q Are you aware of a discrepancy that apparently exists between the ASOS/observer data that suggest that the thunderstorm activity was moving to the northeast and the radar depictions that indicated it was moving to the southeast and east? A I was not aware that anyone thought that was a discrepancy. I've seen that phenomena happen. Individual cells might be moving in a northeasterly component while the main line is sliding southeast. So, depending on what the observer was trying to articulate, it's hard to say without being there. Q Is the tower controller ever designated as the observer? A The tower controllers are all certified under the limited observation or LORS Program because they -- they can provide tower visibility. As you know, there -- it can be critical at times for the air crews, when you have a shallow layer of fog, which might be above the head of the ground observer, but yet not above the control tower, and the prevailing visibility in the control tower might be unlimited, and you might have zero visibility on the surface. That is another additional bit of information that can be valuable to the pilot. Q Was the control tower -- were the tower controllers assigned that duty this -- on the evening of the accident? A They -- they had that ability at all times. The official observation will come from the observer, the augmenter that is under contract to the FAA, but the tower personnel can take observations. Typically, they will communicate any questions with the augmenter because we -- we asked the folks about that, and they do have a very good working relationship with the tower. Q Are there any special duties of the observer during this lock-out period that was referred to? A Well, during the lock-out period, all the -- what the observer is doing is taking his or her look at what is happening in the environment and then comparing what they are seeing visually to what the ASOS is reporting, and it's -- it's just that period of time where they begin to settle in on the most recent and accurate winds, the most recent pressure because, as you know in a convective situation, the winds can vary, and you don't want to jump out on the wind direction or speed too quickly. The pressure may vary, and it may require additional remarks, especially if you're in a situation where pressure might be on the verge of rising rapidly, and that requires a mandatory remark to be affixed to the observation. Q When ASOS was designed and planned and conceived, if you will, was it always in the plan to have an augmenter process along with the technology that is ASOS? A No, it was not. The plan was to eventually get to a level where we would have what we have now on a limited basis in the National Weather Service as Level D systems. Unfortunately, the state of the art in technology with regard to a number of capabilities involving present weather, the inability of the system to be able to accurately ascertain the intensity of drizzle from the perspective of rate-of-fall rather than from visibility. The problem there is the system can't differentiate between a mix of phenomena, such as fog and drizzle. So, we're trying to readapt our -- our sensor for present weather to account for drizzle situations and their intensity based on the rate-of-fall. That's just one example of one of the things that the system doesn't do well right now, and we do have a full-fledged funded program dealing with plan product improvement. We will be moving forward with a replacement dew point sensor this year. The first deployments will begin the latter part of this year. We've also completed our initial work on the ice-free wind sensor. Many parts of the country have problems with the wind anemometers freezing up. So, we are in the process of again trying to resolve what is most important to both the National Weather Service and the FAA. We periodically sit down and compare the priorities of what needs to be done in the ASOS arena to make it a true stand- alone system some time in the future. Q Thank you. That's very helpful. No further questions. CHAIRMAN HALL: Thank you. The Allied Pilots Association? MR. ZWINGLE: Yes, I do have some questions. BY MR. ZWINGLE: Q In consideration of the observer augmentation of the ASOS, is it true that there are at least four levels of service? A Yes, there are. Those four levels of service were agreed to in, I believe, 1996. It was put in the public record for review, and the four levels were agreed to by the various airline associations, the FAA and the National Weather Service. Q Okay. That noted, how -- how are the users of your product made aware of the real-time capability of the sensor or system? A You want -- are you talking about the -- how do the users know whether I'm on Level A, B or C? Q Yes, sir. A Okay. As of right now, that is still for me a point of contention with the FAA. I have asked the FAA to look at possibly putting that in the approach plates, so that folks would know exactly what level of service they are getting. That again has yet to be resolved between the two agencies. It's just a matter of notification. The -- at the present time, there is no way for anyone to know what level of service they're getting, other than the fact that if you do have a control tower, you are at least at a Level C. Level Ds do not have control towers. Level Cs, Bs and As have control towers, and depending on the complexity of the approach into the airport, the amount of traffic and the terrain, those are some of the parameters that were used to apply the various levels along with the types of weather phenomena that occur as well. So, if there was a high need for RVR, you would more -- you're going to more than likely see the majority of the RVR reporting capability go to the A and B level sites. Q Okay. Does the thunderstorm sensor on the ASOS System use cloud-to-ground or cloud-to-cloud lightening? A The -- there's a difference right now. There are 19 sites in the National Weather Service that are called single site lightening detection sensors. They have a maximum range of approximately 35 to 40 miles. They -- they can detect both cloud-to-ground and cloud- to-cloud. The problem is that the software cannot differentiate the direction of which the cloud-to-cloud is occurring. So, from the standpoint of having a complete system, it does not fully meet what we would like it to do. We are working with a number of vendors to try and resolve that particular issue. As far as the LDRS capability, which is the system of choice that will be deployed or actually is being deployed because it is part of the ASOS Version 2.60 currently in deployment, that capability is simply a turn-on-the-switch, and you'll be able to get the lightening detection information. The -- in that particular system, all it does detect is the cloud-to-ground lightening strikes. Q Thank you. Do you have Exhibit 5A available to you? A Yes, I do. Q Page 19, entitled ASOS Wind Information/Precipitation Amounts. A Yes, sir. Q Am I correct in observing that the total accumulated rainfall from the time precipitation began at, I believe, 0436 Z, until the annotated time of the accident is 41/100ths of an inch? A No. What you see -- actually, it's -- oh, let me -- let me double check here to make sure I've got this correct. If you look out in the -- in the -- in the table itself, there's one mistake in the table, I can tell you right now. The ASOS does not in its log annotate traces. Those were put in just so people knew that precipitation did occur, but it did not register 1/100th of an inch. What you see in that column that's marked one minute precip, those are the amounts that occurred each of those minutes. Q Hm-hmm. A Now, the total through the hour is the 1.09 you see in the bottom right. The 14/100ths is the 15-minute precipitation total from 4:30 to 4:45. Q I see. Okay. A As I mentioned, the system does monitor precipitation on a one-minute, a 15-minute and an hourly basis. Q Okay. How does the precipitation rate as recorded by the ASOS System compare or do you know how it compares to the precipitation intensity indicated by the WSR-88D? A We use the ASOS. We use eight-inch gauges as the ground truth for the 88D, and they are used to assist in the calibration of the 88D itself. So, we use the ASOS as a calibration tool for the 88D. Q Okay. Last question. There seems to be some -- several issues of reliability with the ASOS System. Did you have the same type of reliability with human observers? A I would say that based on the three studies that were done -- because that question comes up constantly, you know. How does the -- how can a system with limited optics do the same -- perform the same way as a human being? Well, you've got to remember the -- the human beings that you've been dealing with for the last 20 years have not been taking continuous observations, and we attempted in a number of studies, one done by the National Weather Service, one done by the FAA, and the most recent one done at the request of Senator Snowe in Maine, we went to the Air Force's Technological Labs and asked the Air Force to conduct a study on visibility and cloud height and compared the ability of the ASOS to the observer who was supplied by the Air Force and an augmenter that was working at that particular site, and from the five or six sites that we did in that study, it was proven that the ASOS is equivalent, if not slightly better, than what the human does with respect to clouds and visibility. Now, when it comes to other parameters that the system isn't designed for right now, like hail stones and things of that nature, yeah, that's where the human has the edge. Q Thank you very much. No further questions. CHAIRMAN HALL: The Association of Professional Flight Attendants? MS. LORD-JONES: We have no questions, sir. CHAIRMAN HALL: The National Weather Service? MR. KUESSNER: Am I the last one? CHAIRMAN HALL: Oh, no. Oh, no. You should be. Yes. I'm sorry. I apologize. It is late. The Little Rock National Airport? MS. SCHWARTZ: We have no questions, sir. CHAIRMAN HALL: The Little Rock Fire Department? MR. CANTRELL: No questions, sir. CHAIRMAN HALL: Federal Aviation Administration? MR. STREETER: No questions. CHAIRMAN HALL: The Boeing Commercial Airplane Group? MR. HINDERBERGER: No questions, sir. CHAIRMAN HALL: The National Weather Service? MR. KUESSNER: Thank you, sir. BY MR. KUESSNER: Q Just one thing, Mr. Dombrowsky. On Page 6 of -- I guess it's Exhibit 5B. CHAIRMAN HALL: I'm just glad to see there's not a machine there to answer the question -- ask the questions. MR. KUESSNER: Thank you. THE WITNESS: Page 6? MR. KUESSNER: Yeah. THE WITNESS: Okay. BY MR. KUESSNER: Q About one-third of the way down, where it says, "CMD Sys Log"? A Hm-hmm. Q Now, in -- in that area below that, is that where you expect any errors or problems that would have occurred during the times indicated would be logged there? A Right. What the system has done in that particular situation is has identified a loss of power, and in the system's log entry, it indicates that the uninterruptable power source did kick in appropriately, and then it also acknowledges the return to the standard power. So, you can see it does have an internal check capability, and in some instances, such as this, because the UPS is something that -- well, we just brought on board here in the last year or so. We're trying to resolve some of the issues that we might have with regard to local power sources. Q Now, this -- this part about the battery power on and restored, this occurred a bit before the accident in question, correct? A That's correct. Q Because this is in Greenwich Meridian Time? A Hm-hmm. Q Now, is there anything -- anything else on this page which would indicate there were any problems, I mean subsequent to that time? At least through the accident time, to make it easy. (Pause) THE WITNESS: About the only thing that I could say would be the -- let's see here the timing of that. I'll have to do something about getting these in proper time because they keep switching back and forth from the various times. Log-off, log-on entries, and then there is -- should be a -- if I remember correctly, there was a situation with a special -- BY MR. KUESSNER: Q I was referring under the -- under the Sys Log piece. A Under the Sys Log piece? Q Yeah. On Page 6. A Okay. Q I see at 0215, there was an AC power restored, and then the next -- A Right. Q -- entry is 0718. So, there's no indication that there were any problems? A My biggest problem right now is I can't focus on where you're looking at. Q Sorry. Do you have Page 6? A I've got Page 6. Q Exhibit 5B? A I've got the Sys Log entries, DCP UPS, power on, power restored. Q Okay. And then about -- a little bit further down and halfway down where it says, "6/2/99, 0214"? A 6/2/99. Other than the technician logging on. Q So, it was reliable? There was no problem? A Yeah. I -- I wasn't sure where you were going because, you know, the technicians always will, you know, -- if they do get a call to the AOMC in Washington, the technicians will call in and do a cursory review of what is happening internal to the system. Again, I don't even look at that as -- as a problem. Like I say, once the power is restored, I just ignore the rest of it because that's all routine. But, no, our technicians in Washington will call in to the system, and if they do see a problem, that's when they do their diagnostics, and they will make the appropriate calls. It was so obvious, I didn't see it. Q No further questions. Thanks. CHAIRMAN HALL: Mr. Sweedler? MR. SWEEDLER: No questions, Mr. Chairman. CHAIRMAN HALL: Mr. Berman? INTERVIEW BY THE BOARD OF INQUIRY BY MR. BERMAN: Q Sir, can you explain why the ASOS didn't show the 41- knot gust that was found on the LWAS? A It's -- as was explained earlier, the location can make a difference. You wouldn't necessarily see the 41-knot gust on the ASOS. You also -- I'm not sure what -- at what height that particular LWAS was. It could be -- could have been at 75 feet or whatever. So, there -- there are a number of reasons why 41 knots wouldn't show up, an LWAS wind would not show up on the ASOS. Q Thank you. A Hm-hmm. CHAIRMAN HALL: Mr. Haueter? MR. HAUETER: No questions. CHAIRMAN HALL: Mr. Clark? BY MR. CLARK: Q I've become confused on the lock-out, and I guess if we've covered this, I apologize. The -- during the period of the lock- out, were there any specials that -- let me rephrase that. If there were no lock-out present, would ASOS have sent any specials during that time period? A Yes, it would have, because the -- Q What sort of -- A -- visibilities dropped below three miles during that period. That would have constituted the system to have generated a special and sent it out. Because of the lock-out, the way it's set up, the observer cannot go in and make any changes in the system. Although it does document the actual visibility during that five-minute period, it makes that information available to the observer, so that the observer can annotate it. If we were not -- if we were using the old surface airways code, it would have probably generated what we call a record special, but in this particular instance, we're using the METAR code, and -- and there is no accommodation for that particular capability. That's one of the other reasons you don't see that generation of specials during an hourly period. Q Okay. Is the -- you mentioned the three-mile visibility. A Hm-hmm. Q Are there any others that would go with the gusting winds, the thunderstorm being present or nearby, or whatever rainfall was going on at that time? A Yeah. The -- from the standpoint of any additional specials, I -- I -- I didn't -- again, without going back and looking, I would say the visibility was probably the only one. Winds did not reach a level that could have done that. I know that the wind direction special did not engage for whatever reason, but that particular special -- that particular remark was carried forward into the -- into the hourly. So, that wind shift that would have occurred at 31 did get carried forward as part of the information in the hourly observation. Q Okay. You mentioned that there were archive data. Would any of that shed more light on my question to go back and look at that archive data rather than what's in the docket or what you have available today? A Well, we did look at all the archive data in Washington and -- and felt confident that the system was performing as designed. I think about the only -- well, as far as the lock-out goes, there's not much we can do about the lock-out. You can't override anything in that observation period time because the edit will not open up for the generation of specials. What it can do, though, is if there are certain code indicators that will allow you to do that, there are certain data fields that the observer can go in and edit, and the only ones that would supersede that and preempt the hourly from going out are observations that contain remarks pertinent to severe weather, such as tornados and things of that nature. Thunderstorms and -- and severe thunderstorms do not qualify under that particular lock-out. Q I guess that's the -- I understand that it worked the way it was designed. I guess my question goes more to if there are -- should we go back and reconsider that if there are thunderstorms or significant wind shifts, that we should be able to preempt that lock- out and get that information out in a timely manner? A We've already started that process, sir. In our Software Version Load 2.80, which is scheduled for release in the August- September time frame of this year, the lock-out issue has been resolved. We are testing that software load right now under a Demonstration Version of Load 2.75, and, you know, we're going to be testing that in the lab for the next several weeks. We began that testing process about a week or so ago, and once we prove that it works on our local sites -- as you know, everything always works well within a lab, but when you take it out in the field, it just, you know, -- that's the true test, and we'll be deploying the 2.75 Software Load to a number of sites around the country to validate and get, you know, a systems acceptance test completed, and then, once we have that, we'll be positioned to deploy the software load in that hopefully August-September time frame, barring any difficulties. Q Okay. But going back, the -- your review of the data, there were -- other than the three-mile visibility, going below three miles, there were no indicators that would have been a factor during this lock-out period based on your observations? A Based on what I've seen here, no. The only thing that could have been carried forward, and it was carried forward into the record hourly because that's the way the system is programmed, that if a phenomena occurs during the -- a special phenomena occurs during the lock-out period, it is still documented. It is still put into the observation. The only thing you don't see is a separate independent observation called a specia to acknowledge that, yes, I've had a reduction in visibility. It requires that special observation. What it does, it holds that observation, that special criteria that was met, into the -- into the hourly and then deploys the hourly with all of those elements updated to what is currently going on at the site. Q Okay. And I assume we have that data available? A Yes. Q All right. Thank you. A Yes, sir. BY MR. EICK: Q Mr. Dombrowsky, can you tell us what the peak wind observed on the evening of June 1st at Little Rock? A Based on my recollection, I believe it was 76 knots was the peak wind that was observed. I believe that was on the -- in one of the -- Q I think on Page 1 on the obs. A Either Page 1 of the obs or it was also, I think, in one of the tables that was showing the one-minute data. But 76 knots is what I -- what I recollect. Q And on the observations on the specias, too, or in the normal hourly obs, the observer could also -- the system could automatically also put in peak winds? A Yes. It generates peak wind remarks. It generates wind shift remarks. It generates squall remarks, if that happens to be the -- the criteria that is met during the observation time. Q And the definition of the -- of a gust is -- can you give us a definition of the gust, of this gust of 76 knots, versus why it wasn't reported as a squall? A Well, the -- the requirement of a squall is that you have to be at a -- a speed of 22 knots. That's the low end threshold. This is at -- and again it's -- by definition, a squall is a sudden doubling of the wind, and the wind has to be at least 22 knots or greater, and that criteria wasn't met. Q And the definition of a gust? A The definition of a gust is a difference between the prevailing wind speed, and the gust has to be greater than nine knots. Once that difference is greater than nine knots, it constitutes a -- a gust. Q Thank you very much. CHAIRMAN HALL: All right. Any other questions? (No response) CHAIRMAN HALL: Mr. Dombrowsky, the only inquiry I have is that this new system, you said there are some thought of turning the maintenance of this system over to the FAA? THE WITNESS: It's not our thought, sir. It's the thought of the FAA. There's a desire there to streamline the overall maintenance program, and the FAA has come to the National Weather Service on several occasions during the past year-year and a half in an effort to analyze and determine what is best for the Federal Government, and we each have our position. We would prefer to keep it, but the FAA would prefer to take it over, but again it would cost the FAA a significant amount of money up front to be able to do that, and right now, the budget the way it is, I -- I expect the FAA to probably back down on that. CHAIRMAN HALL: Is it your system? THE WITNESS: Actually it's a joint-owned system. CHAIRMAN HALL: I mean whose software and whose -- THE WITNESS: Oh, we -- we've done all of the -- CHAIRMAN HALL: Who's got the smarts that understands whether it's operating correctly? THE WITNESS: As of right now, we do, sir. We do all the plan product improvement. We do all the maintenance. The FAA has been our partner in this process, but most of the partnering that's been done by the FAA has been getting agreements on the various requirements and also getting additional funding to the National Weather Service to maintain the systems that the FAA has ownership over. CHAIRMAN HALL: And does it work now? THE WITNESS: It's working right now. Our position at the National Weather Service is, you know, why change? It's working, and it's the most efficient program right now. CHAIRMAN HALL: If I was the ATA, I'd be concerned about that. That's not my business, but I -- you know, if you all have the system, and I want to be sure the FAA's got all the things that it's responsible for working first. But, Mr. Streeter, are you part of that grab? MR. STREETER: No. That will continue -- CHAIRMAN HALL: Is that at your level? MR. STREETER: That will continue under discussion, sir. CHAIRMAN HALL: Okay. Well, I hope there's some -- some good reason for it. If there is, I'd like to -- like to know. Well, Mr. Dombrowsky, we would certainly welcome any closing thoughts that you would have. THE WITNESS: Well, from the standpoint of this accident, I really don't have any -- any more to add than what the preceding individuals have presented. Safety is paramount, and we will do everything we can in the National Weather Service to provide the best possible services and ensure that we do meet the requirements for flight safety. We're very, very much in favor of ensuring that we do that. CHAIRMAN HALL: Well, thank you very much. Now, that's going to conclude today. I want -- before the parties get antsy and move now, we've got six witnesses left. We have one gentleman, Dr. Dinges, who is going to come and talk to us about fatigue, but he is snowed in, and he's not going to be able to get here. Mr. Roman, our individual, is snowed in in St. Louis, but we can get his questions faxed here right, Mr. Feith, and ask those for him if he can't -- can't get here. I would just as soon start at 8:30 in the morning and attempt to finish tomorrow, unless there's a wave of desire to stay Saturday. Do I see a wave? (No response) CHAIRMAN HALL: I see -- I don't even see a wave in the audience. So, we will start at 8:30 in the morning. We will attempt to complete this hearing tomorrow, and I again thank the parties for their attention. I thank the audience and observers for your attendance, and if there's a Mr. Robert Gilford in the audience, I'd like to see him, if he'd come up to the podium. I'm sorry. Mr. Baker, yes, sir? MR. BAKER: Mr. Chairman, thank you. I have one final request on this snowy day. You know, as we've gone through the last two days, we've identified several programs and clocks that are tracking various things, the cockpit voice recorder, the FDR, the LWAS, and now ASOS, the WSR-88. I'm getting quite concerned and would ask that the staff, following these hearings, attempt to do a verification and synchronization of all the times in these systems, so that we don't make a mistake of when things happened. CHAIRMAN HALL: I -- I agree with you, and that's a good -- excellent thought. MR. BAKER: I have no further comments. Thank you. CHAIRMAN HALL: We're all challenged and continue to be challenged by technology. Any other comments from any of the other parties? (No response) CHAIRMAN HALL: We'd like to thank the National Weather Service for today's snow. Little Rock for the coffee. Is the Mayor here? The Mayor's left. So, there's no free dinner, I guess. So, we will all -- we will stand adjourned until 8:30 in the morning. (Whereupon, at 7:45 p.m., the hearing was adjourned, to reconvene tomorrow morning, Friday, January 28th, 2000, at 8:30 a.m.) 714 EXECUTIVE COURT REPORTERS, INC. (301) 565-0064 EXECUTIVE COURT REPORTERS, INC. (301) 565-0064