Text mining - Misplaced Pages

This is an old revision of this page, as edited by 67.87.16.42 (talk) at 17:54, 24 February 2007 (→Commercial Software and Applications). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

Revision as of 17:54, 24 February 2007 by 67.87.16.42 (talk) (→Commercial Software and Applications)(diff) ← Previous revision | Latest revision (diff) | Newer revision → (diff)

Text mining, sometimes alternately referred to as text data mining, refers generally to the process of deriving high quality information from text. High quality information is typically derived through the divining of patterns and trends through means such as statistical pattern learning. Text mining usually involves the process of structuring the input text (usually parsing, along with the addition of some derived linguistic features and the removal of others, and subsequent insertion into a database), deriving patterns within the structured data, and finally evaluation and interpretation of the output. 'High quality' in text mining usually refers to some combination of relevance, novelty, and interestingness. Typical text mining tasks include text categorization, text clustering, concept/entity extraction, sentiment analysis, document summarization, and entity relation modeling (i.e., learning relations between named entities).

History

Labour-intensive manual text-mining approaches first surfaced in the mid-1980s, but technological advances have enabled the field to advance swiftly during the past decade. Text mining is an interdisciplinary field which draws on information retrieval, data mining, machine learning, statistics, and computational linguistics. As most information (over 80%) is currently stored as text, text mining is believed to have a high commercial potential value.

Applications

Recently, text mining has been receiving attention in many areas, most notably in the security, commercial, and academic fields.

Security applications

One of the largest text mining applications that exists is probably the classified ECHELON surveillance system.

Software and Applications

Research and development departments of major companies, including IBM and Microsoft, are researching text mining techniques and developing programs to further automate the mining and analysis processes.

Academic applications

The issue of text mining is of importance to publishers who hold large databases of information requiring indexing for retrieval. This is particularly true in scientific disciplines, in which highly specific information is often contained within written text. Therefore, initiatives have been begun such as Nature's proposal for an open text mining interface (OTMI) and NIH's common Journal Publishing Document Type Definition (DTD) that would provide semantic cues to machines to answer specific queries contained within text without removing publisher barriers to public access.

Academic institutions have also become involved in the text mining initiative: The National Centre for Text Mining (NaCTeM), a collaborative effort between the Universities of Manchester, Liverpool and Salford, funded by the Joint Information Systems Committee (JISC) and two of the UK Research Councils aim to provide tools, carry out research and offer advice to the academic community, with an initial focus on text mining in the biological and biomedical sciences. In the United States, the School of Information at University of California, Berkeley is developing a program called BioText to assist bioscience researchers in text mining and analysis.

Commercial Software and Applications

Attensity - suite of text mining solutions for a variety of industries.
Clarabridge - text mining and categorization applications for customer, healthcare, and investigative analytics.
Clearforest - text mining software to extract meaning from various forms of textual information.
IBM Intelligent Miner for Text - commercial text mining software
Inxight - provider of text analytics, search, and unstructured visualization technologies.
Nstein Technologies - provider of text analytics, and asset/web content management technologies. (See Nstein)
PolyAnalyst - commercial text mining software
SAS Enterprise Miner - commercial text mining software
SPSS - provider of TextSmart, SPSS Text Analysis for Surveys, and Clementine, commercial text analysis software
TEMIS - TEMIS is a software editor providing innovative Information Discovery solutions to serve the Information Intelligence needs of business corporations.
TextAnalyst - commercial text mining software
Textalyser - an online text analysis tool for generating text analysis statistics of web pages and other texts.
Topicalizer - an online text analysis tool for generating text analysis statistics of web pages and other texts.
The "Ultimate Research Assistant" - a knowledge management tool that uses a combination of traditional search engine technology and text mining techniques to facilitate online research of complex topics.
Anderson Analytics - provider of text analytics and content analysis especially as it relates to consumer behavior.

Implications

Until recently websites most often used text-based lexical searches. Text mining may allow searches to be directly answered by the semantic web. Text mining is also used in some email spam filters.

References

Ronen Feldman and James Sanger, The Text Mining Handbook, Cambridge University Press, ISBN 9780521836579
Vitorino Ramos, Juan J. Merelo, Self-Organized Stigmergic Document Maps: Environment as a Mechanism for Context Learning, in AEB´2002 – 1st Spanish Conference on Evolutionary and Bio-Inspired Algorithms, E. Alba, F. Herrera, J.J. Merelo et al. (Eds.), pp. 284-293, Centro Univ. de Mérida, Mérida, Spain, 6-8 Feb. 2002.