Introduction

8/25/20

+ +

Natural Language Processing CS159 – Fall 2020 David Kauchak

+ + Administrivia Administrivia http://www.cs.pomona.edu/classes/cs159/ n Office hours, schedule, assigned readings, assignments Assignment 0 posted already n Everything will be posted there n Shouldn’t take too long n Due Thursday by 11:59pm (your time zone) Read the “administrive” handout! n ~7 assignments (in a variety of languages) n 4 quizzes Assignment 1 posted n final project for the last 3-4 weeks n Won’t cover all material until Thursday n teams of 2-3 people n Due Wednesday 9/2 (11:59 – your time zone) n class participation n Readings

Academic Honesty Collaboration

1 8/25/20

+ + What to expect… Requirements and goals

This course will be challenging for many of you Requirements

n assignments will be non-trivial n Competent programmer n content can be challenging n Some assignments in Java, but I will allow/encourage other languages after the first few assignments But it is a fun field! n Comfortable with mathematical thinking

n We’ll use a fair amount of probability, which I will review We’ll cover n Other basic concepts, like logs, summation, etc.

n basic linguistics n Data structures n probability n trees, hashtables, etc. n the common problems Goals n many techniques and algorithms n Learn the problems and techniques of NLP n common machine learning techniques n Build real NLP tools n some recent advances in neural networks for language n Understand what the current research problems are in the field processing n NLP applications

+ + Getting to know eachother Videos before class

2 8/25/20

+ + What is NLP? What is NLP?

Natural language processing (NLP) is a field of computer science The goal of this new field is to get computers to perform useful and linguistics concerned with the interactions between tasks involving human language… computers and human (natural) languages.

- Wikipedia - The book

+ + Key: Natural text Why do we need computers for dealing with natural text? “A growing number of businesses are making Facebook an indispensible part of hanging out their shingles. Small businesses are using …”

Natural text is written by people, generally for people

Why do we even care about natural text in computer science?

https://searchengineland.com/googles-search-indexes-hits-130-trillion-pages-documents-263378

3 8/25/20

+ Web is just the start… + Why is NLP hard?

Iraqi Head Seeks Arms e-mail Juvenile Court to Try Shooting Defendant

Stolen Painting Found by Tree

~500 million tweets a day Kids Make Nutritious Snacks ~200-300 billion e-mails a day Local HS Dropouts Cut in Half Blogs: ~200 million different blogs Obesity Study Looks for Larger Test Group corporate databases British Left Waffles on Falkland Islands Red Tape Holds Up New Bridges

Hospitals Are Sued by 7 Foot Doctors Facebook

+ + Why is NLP hard? Why is NLP hard?

Natural language: User: Where is Escape Room playing in the Claremont Area? n is highly ambiguous at many different levels System: Escape Room is playing at the Edwards in La Verne. n is complex and contains subtle use of context to convey meaning User:When is it playing there? n is probabilistic? System: It’s playing at 2pm, 5pm and 8pm n involves reasoning about the world n is highly social User: I’d like 1 adult and 2 children for the first show. How much would that cost? n is a key part in how people interact

However, some NLP problems can be surprisingly easy

4 8/25/20

+ + Different levels of NLP NLP problems and applications

pragmatics/discourse: how does the context affect the interpretation? What are some places where you have semantics: what does it mean? seen NLP used?

syntax: phrases, how do words interact What are NLP problems?

words: morphology, classes of words

+ + NLP problems and applications NLP problems and applications

Lots of problems of varying difficulty Lots of problems of varying difficulty Easier Easier Word segmentation: where are the words? Word segmentation: where are the words?

I would’ve liked Dr. Dave to finish early. But he didn’t.

5 8/25/20

+ + NLP problems and applications NLP problems and applications

Lots of problems of varying difficulty Easier continued n truecasing Easier i would’ve liked dr. dave to finish early. but he didn’t. n Speech segmentation n spell checking Identifying mispellings is challenging especially in the dessert. n Sentence splitting (aka sentence breaking, sentence boundary disambiguation) n OCR I would’ve liked Dr. Dave to finish early. But he didn’t.

n Language identification Soy un maestro con queso.

+ + NLP problems and applications NLP problems and applications Moderately difficult Moderately difficult continued n morphological analysis/stemming n text segmentation: break up text by topics smarter n part of speech tagging (and inducing word classes) smarter smartly smart n parsing smartest S smart VP n speech recognition NP

NP PP

n text classification J L PRP V N IN N SPAM sentiment analysis I eat sushi with tuna

6 8/25/20

+ + NLP problems and applications NLP problems and applications

Moderately difficult continued n word sense disambiguation Hard (many of these contain many smaller problems) As he walked along the side of the stream, he spotted some money by the bank. The money had gotten muddy from Machine translation being so close to the water.

n grammar correction 美国关岛国际机场及其办公室均接获一 The U.S. island of Guam is maintaining a high state of alert after the Guam airport and its 名自称沙地阿拉伯富商拉登等发出的电 offices both received an e-mail from someone 子邮件，威胁将会向机场等公众地方发 calling himself the Saudi Arabian Osama bin 动生化袭击後，关岛经保持高度戒备。 Laden and threatening a biological/chemical attack against public places such as the airport . n speech synthesis

+ + NLP problems and applications NLP problems and applications Summarization

Information extraction

IBM hired Fred Smith as president.

7 8/25/20

+ + NLP problems and applications NLP problems and applications

Natural language understanding n Text => semantic representation (e.g. logic, probabilistic relationships) Text simplification

Alfonso Perez Munoz, usually referred to as Information retrieval and question answering Alfonso, is a former Spanish footballer, in the striker position. "How many programmers in the child care department make over $50,000?”

“Who was the fourteenth president?”

“How did he die?” Alfonso Perez is a former Spanish football player.