Topic: Japanese Regulation of Insider Trading <desc> Description: Document will inform on Japan's regulation of insider trading. <narr> Narrative: A relevant document will provide data on Japanese laws, regulations, and/or practices which help the foreigner understand how Japan controls, or does not control, stock market practices which could be labeled as insider trading. <con> Concept(s): 1. insider trading 2. Japan 3. Ministry of Finance, Securities and Exchange Council, Osaka Securities Exchange, Tokyo Stock Exchange 4. Securities and Exchange Law, Article 58, law, legislation, guidelines, self-regulation 5. Nikko Securities, Yamaichi Securities, Nomura Securities, Daiwa Securities, Big Four brokerage firms <fac> Factor(s): <nat> Nationality: Japan <Ifac> <Aop> This topic actually consists of two different statements of the same query: the natural language specification con- sisting of <desc> and <narr> fields, and an expert- selected list of key terms which are often far more infor- mative than the narrative part (in some cases these terms were selected via feedback in actual retrieval attempts). The table below shows the seatch query obtained from fields <desc> and <nair> of topic 107, after an expansion with similar terms and deleting low-content terms. Ouery 107 standard+trade idf=16.81 weight""0.38 standard+trade idf=16.8l weight""0.38 regulate+japanese idf=15.40 weight=1.00 standard+japanese idf=14.08 weight"'0.38 regulate+trade idf=12.84 weight=l.00 regulate+trade idf=12.84 weight=l.00 controls idf"'9.97 weight=l.00 labele idf~.20 weight=l.00 trade+inside idf=8.62 weight=l.OO trade+inside idf=8.62 weight=l.00 inside idf"'iA9 weight=l.00 inside idf""iA9 weight=l.00 inside idf~A9 weight=l.00 regulate idf=5.66 weight=l.00 regulate idf=5.66 weight=l.00 131 regulate idf=5.66 weight=l.00 practice idf=5A6 weight=l.00 practice idf""5.46 weight=l.00 data idf=4.91 weight=l.00 data idf=4.91 weight"'0.51 data idf=4.91 weight=O.26 japanese idf=4.84 weight=l.00 japanese idf=4.84 weight=l.00 standard idf=4.81 weight=O.38 standard idf1"4.81 weight=O.38 standard idf=4.81 weight~.38 inform idf=4.71 weight=l.00 inform idf"'4.71 weight"'0.26 inform idf=4.71 weight"'0.51 protect idf=4.69 weight=O.41 Note that many `function' words have been removed from the query, e.g., provide, understand, as well as other `common words' such as document and relevant (this is in addition to our regular list of `stopwords'). Some still remain, however, e.g., data and inform, because these could not be uniformly considered as `common' across all queries. Results obtained for queries using text fields only and those based primarily on keyword fields are reported separately. The purpose of this distinction was to demon- strate that (or whether) an intensive natural language pro- cessing can make an imprecise and frequendy convo- luted narrative into a better query that an expert would create.14 The ad-hoc category runs were done as follows (these are the official IkEC-2 results): (1) nyuirl: An automatic run of topics 101-150 against the WSJ database with the following fields used: <tide>, <desc>, and <narr> only. Both syntactic phrases and term similarities were included. (2) nyuir2: An automatic run of topics 101-150 against the WSJ database with the following fields used: <tide>, <desc>, <con> and 4ac> only. Both syntactic phrases and term similarities were included. 14 Some results on the impact of different fields in ~~EC topics on the final recall/precision results were reported by Broglio and Croft (1993) at the ARPA HLT workshop, although text-only runs were not included. One of the most striking observations they have made is that the narrative field is entirely disposable, and moreover that its inclusion in the query actually hurts the system's performance. Croft (personal communication, 1992) has suggested that excluding all expert-made fields (i.e., <con> and <fac>) would make the queries quite ineffective. Broglio (personal communication, 1993) confirms this showing that text-oaly retrieval (i.e., with <desc> and <narr>) shows an average pre- cision at more than 30% below that of <con>based retrieval.

with a different combination of fields from the topics used to create search queries. A typical topic is shown below: 41ead> Tipster Topic Description Number: 107 Domain: International Economics Topic: Japanese Regulation of Insider Trading <desc> Description: Document will inform on Japan's regulation of insider trading. <narr> Narrative: A relevant document will provide data on Japanese laws, regulations, and/or practices which help the foreigner understand how Japan controls, or does not control, stock market practices which could be labeled as insider trading. <con> Concept(s): 1. insider trading 2. Japan 3. Ministry of Finance, Securities and Exchange Council, Osaka Securities Exchange, Tokyo Stock Exchange 4. Securities and Exchange Law, Article 58, law, legislation, guidelines, self-regulation 5. Nikko Securities, Yamaichi Securities, Nomura Securities, Daiwa Securities, Big Four brokerage firms <fac> Factor(s): <nat> Nationality: Japan <Ifac> <Aop> This topic actually consists of two different statements of the same query: the natural language specification con- sisting of <desc> and <narr> fields, and an expert- selected list of key terms which are often far more infor- mative than the narrative part (in some cases these terms were selected via feedback in actual retrieval attempts). The table below shows the seatch query obtained from fields <desc> and <nair> of topic 107, after an expansion with similar terms and deleting low-content terms. Ouery 107 standard+trade idf=16.81 weight""0.38 standard+trade idf=16.8l weight""0.38 regulate+japanese idf=15.40 weight=1.00 standard+japanese idf=14.08 weight"'0.38 regulate+trade idf=12.84 weight=l.00 regulate+trade idf=12.84 weight=l.00 controls idf"'9.97 weight=l.00 labele idf~.20 weight=l.00 trade+inside idf=8.62 weight=l.OO trade+inside idf=8.62 weight=l.00 inside idf"'iA9 weight=l.00 inside idf""iA9 weight=l.00 inside idf~A9 weight=l.00 regulate idf=5.66 weight=l.00 regulate idf=5.66 weight=l.00 131 regulate idf=5.66 weight=l.00 practice idf=5A6 weight=l.00 practice idf""5.46 weight=l.00 data idf=4.91 weight=l.00 data idf=4.91 weight"'0.51 data idf=4.91 weight=O.26 japanese idf=4.84 weight=l.00 japanese idf=4.84 weight=l.00 standard idf=4.81 weight=O.38 standard idf1"4.81 weight=O.38 standard idf=4.81 weight~.38 inform idf=4.71 weight=l.00 inform idf"'4.71 weight"'0.26 inform idf=4.71 weight"'0.51 protect idf=4.69 weight=O.41 Note that many `function' words have been removed from the query, e.g., provide, understand, as well as other `common words' such as document and relevant (this is in addition to our regular list of `stopwords'). Some still remain, however, e.g., data and inform, because these could not be uniformly considered as `common' across all queries. Results obtained for queries using text fields only and those based primarily on keyword fields are reported separately. The purpose of this distinction was to demon- strate that (or whether) an intensive natural language pro- cessing can make an imprecise and frequendy convo- luted narrative into a better query that an expert would create.14 The ad-hoc category runs were done as follows (these are the official IkEC-2 results): (1) nyuirl: An automatic run of topics 101-150 against the WSJ database with the following fields used: <tide>, <desc>, and <narr> only. Both syntactic phrases and term similarities were included. (2) nyuir2: An automatic run of topics 101-150 against the WSJ database with the following fields used: <tide>, <desc>, <con> and 4ac> only. Both syntactic phrases and term similarities were included. 14 Some results on the impact of different fields in ~~EC topics on the final recall/precision results were reported by Broglio and Croft (1993) at the ARPA HLT workshop, although text-only runs were not included. One of the most striking observations they have made is that the narrative field is entirely disposable, and moreover that its inclusion in the query actually hurts the system's performance. Croft (personal communication, 1992) has suggested that excluding all expert-made fields (i.e., <con> and <fac>) would make the queries quite ineffective. Broglio (personal communication, 1993) confirms this showing that text-oaly retrieval (i.e., with <desc> and <narr>) shows an average pre- cision at more than 30% below that of <con>based retrieval.