Dialog Modeling Dialog manager Manage flow of conversation Input: Semantic representation of the input Output: Semantic representation of the output Utilize additional knowledge User information Dialog History Task-specific information
2020-09-20
Motivation 🎯 Goal: generate natural language from semantic representation (or other data) Examples Pollen Forecast Pollen Forecast for Scotland Taking six numbers as input, a simple NLG system generates a short textual summary of pollen levels
2020-09-19
Definition Natural language understanding Representing the semantics of natural language Possible view: Translation from natural language to representation of meaning Difficulties Ambiguities Lexical Syntax Referential Vagueness E.g., “I had a late lunch.
2020-09-18
Definition Question Answering Automatically answer questions posed by humans in natural language Give user short answer to their question Gather and consult necessary information Related topics Information Retrieval Reading Comprehension
2020-09-18
TL;DR Text summarization Most important technique Extraction Tasks: Key word extraction Sentence extraction Algorithms: Supervised Unsupervised Abstract summarization still an open problem Introduction What is Summarization? Reduce natural language text document Goal: Compress text by extracting the most important/relevant parts 💪 Applications Articles, news: Outlines or abstracts
2020-09-16
TL;DR Representing and Analyze Sentence Structure Phrase structure grammar Context free grammar Problems: Ambiguities : PP Attachment Traditional Approaches Stochastically Parsing Probabilistic Context Free Grammar CYK Algorithm Transition-based parsing
2020-09-15
Introduction Definition Named Entity: some entity represented by a name Named Entity Recognition: Find and classify named entities in text Why useful? Create indices & hyperlinks Information extraction Establish relationships between named entities, build knowledge base Question answering: answers often NEs
2020-09-15
Part-of-Speech Tagging What is Part-of-Speech Tagging? Part-of-Speech tagging: Grammatical tagging Word-category disambiguation Task: Marking up a word in a text as corresponding to a particular part of speech based on definition and context Word level task: Assign one class to every word
2020-09-15
Introduction Definition Sentiment analysis / opinion mining Determine opinion, sentiment and subjectivity in text What is the authors opinion about something? What are the pros and cons? Important task in natural language processing Application Automatically maintain review and opinion-aggregation websites Web search target towards reviews generate results with variety of opinions Improve customer relationship management Automatically analyze customer feedback Predict public attitudes towards brand/politics Ad placement Advertise products near positive text Summarization Question-answering Challenges Deep undetstanding Co-reference resolution Negation handling Different hints in the text Tasks of SA Polarity classification binary classifier if text, sentence, document is positive or negative Agreement detection Do two text agree on their opinion?
2020-09-15
Introduction Definition Word Sense Disambiguation Determine which sense/meaning of a word is used in a particular context Classification problem Sense inventory considered senses of the words Word Sense Discrimination Divide usages of a word into different meanings Unsupervised Algorithms Task Determine which sense of a word is activated in a context
2020-09-14