Solicitation Number:
W15P7T-09-RFI-ASPO-01

Notice Type:
Sources Sought

Synopsis:
The U.S. Army Space Programs Office, Program Manager Sequoyah, Alexandria, VA. is conducting market research to identify industry standards, metrics, best practices and lessons learned concerning commercial protocols for test and evaluation of automated foreign language translation products for speech and text translation. This request for information is for planning purposes only; this is not a request for Quotations or Proposals. No solicitation document exists and a formal solicitation may or may not be issued by the Government as a result of the responses to this survey. The Government will not be liable for payment of any response preparation expenses. Interested parties are responsible for adequately marking proprietary or competition sensitive information contained in their response. All interested firms that possess the knowledge addressed herein are encouraged to respond to this survey by providing the information specified below, on or before 2:00 pm EST on 06 FEB 09. Responses to this RFI should be unclassified. Sources responding should indicate their business size in relation to the following NAICS code: 334220. Direct all questions regarding this survey to the attention of Ronald Asan, Telephone number 732-427-2130, Fax number 732-427-4176, Email: Roanld.Asan@us.army.mil. All requests for further information must be via email, telephonic requests for additional information will not be honored.

The Program Manager Sequoyah, under the Product Director, Army Space Program Office (ASPO) of Program Executive Officer (PEO) Intelligence and Electronics Warfare and Sensors (IEW&S) has a mandate to develop and procure automated foreign language translation products for both speech and text translation in a variety of languages and on a variety of platforms. In partial response to this mandate, PM Sequoyah will plan and execute developmental testing and operational testing of such products over the life of the program, beginning in FY09. The purpose of this market survey is to identify industry best practices for the testing and evaluation of these products. In fulfilling the many requirements for material support tasked to the Sequoyah Program Office different solution sets (configurations, languages, and domains) will be considered.

Interested parties should consider the following information about the products to be evaluated in response to this RFI:
a) PM Sequoyah is interested in developing software products in the following configurations: Two-Way Speech-to-Speech (2WS2S), Two-Way Text-to-Text (2WT2T), One-Way Text-to-Speech (1WT2S), and One-Way Speech-to-Text (1WS2T) translations systems. Two-Way products would perform translation from English to target language and target language to English. 1WT2S products would perform translation from English to target language. 1WS2T products would perform translation from target language to English.
b) Hardware Platforms will include handheld (HH) portable devices and laptop/tablet (PC) mobile devices operating in a stand-alone mode, as well as server platforms operating in a web-enabled mode. These platforms could be operating with Windows or LINUX operating systems.
c) Target languages may include, but will not be limited to, Iraqi Arabic, Modern Standard Arabic, Pashto, and Dari.
d) Products will be targeted for use within specified domains. Examples of such domains include Checkpoint Operations, Base Security, Convoy Operations, Tactical Questioning, Internment and Detention, Combat Life Saving, and Combat Operations focused on Counterinsurgency.

PM Sequoyah will conduct test and evaluation at multiple points in the development and acquisition processes. During Developmental Testing, testing will be required at the component level of the products, including Automatic Speech Recognition (ASR), Optical Character Recognition (OCR), Machine Translation (MT), and Speech Synthesis (SS) as appropriate to the specific product. Tests will consider both accuracy/adequacy and speed of operation. During Operational Testing (and also during Developmental Testing), testing will be required to assess the overall system performance in performing tasks.

Respondents should specify for all of the various kinds of tests their experience and recommendations for procedures, methodologies, facilities, data requirements, and metrics for effective testing and evaluation. Maturity and demonstrated validity of test procedures should be described. Criticism of known test metrics, e.g., shortfalls, reliability/consistency of results, complexity/costliness of implementation, etc., may be included. Recognized industry practices for evaluating quality and accuracy should be noted. Metrics for system performance, especially, should be recommended that correlate well with human assessments of performance, e.g. Interagency Language Roundtable (ILR) ratings. A unique need for this program is to assess whether a non-linguist user assisted by a Sequoyah product is able to accomplish the same tasks (within a specified domain) as a linguist rated at a particular ILR level. If such a user is able to accomplish the same tasks as a linguist with an ILR rating of X, then we would say the Sequoyah product performs at ILR level X. PM Sequoyah is particularly interested in respondents recommendations for metrics and procedures for performing this assessment. Recommendations of new metrics/methodologies that demonstrate a high degree of correlation to the ILR linguist evaluation scheme are of great interest of the Program Office. Metrics/test methodologies that correlate well to a quantifiable scale based on a discernable language proficiency are also of interest.