tutorial

Natural Language Technology(*)Lillian LeeCornell Universityhttp://www.cs.cornell.edu/home/llee(*) Some of this material comes from a joint tutorial, co-organized with JohnLafferty, at the Sixteenth National Conference on Artificial Intelligence, 1999.OutlineI. Overview of the FieldII. The Statistical RevolutionIII. Language as a Statistical SourceIV. Tools of the TradeV. The Sparse Data ProblemVI. Conclusions and ReferencesI. Overview of the FieldNatural Language Processing (NLP)Goal: computers using natural language as input and/or outputlanguage languagecomputerunderstanding(NLU)generation (NLG)NLU example: convert an utterance into a sequence of computerinstructions.NLG example: produce a summary of a patient’s records.Why NLP?Lots of information is in natural language format.• Documents• News broadcasts• User utterancesLots of users want to communicate in natural language.• “Do what I mean!”NLP is UsefulTask Input Outputsummarization “document(s)” (CNN broadcasts) summarymachine translation signal in language 1 signal in language 2question answering query answer to query→ information retrieval query relevant documentsuser interfaces command in natural language computer instructions“Now we’re betting the company on these natural interface technologies”– Bill Gates, 1997NLP is Cross-DisciplinaryExcellent opportunities for interdisciplinary work.• Linguistics: models of language emphasizes 100% accuracy (competence)• ...