You might use the technology every day without even knowing it.

For example,

Google Search knows what you’re looking for through either text or speech.

Gmail generates Smart Reply based on messages.

Google Translate translates in 133 languages, including 24 new languages, according to the Google I/O 2022 keynote.

Google Assistant interprets your intention and conducts interactive conversations.

Google is launching a new auto summary function for products in Workspace, including Docs, Google Chat, and Meet.

Behind all these applications, NLP technology plays a crucial role.

So what is NLP exactly?

Well, the name tells it all;

At first, that sounds impossible to achieve.

The nuance of languages is difficult to catch even for a human.

How is it possible for a machine to understand languages?

Well, equipped with

• nearly unlimited memory (such as cloud storage services),

• super computing power (such as TPU, the Tensor Processor Unit), and

• ever-evolving techniques (such as advanced ML models), computers might make the impossible possible.

Imagining NLP as an input-process-output system.

The input is natural human languages, which can be either text or speech.

The computer faces a couple of challenges to process natural language.

First, the human language has to be translated to a machine language that a computer understands.

This process is called text representation.

Second, how can language be processed?

There are two methods: Rule-based reasoning means that the computer is told what to do based on hand-coded rules such as dictionary lookup.

For example, the machine is told to match hello in English to hola in Spanish.

Machine learning means that the computer learns the rules on its own through a large amount of data and its corresponding results.

For example, fed with thousands and thousands of instances of customer feedback and ratings, the machine is able to predict a rating the next time it reads the comments.

Obviously the second method is much more scalable.

What is the output? Among the many tasks NLP can do, here a list of a few essentials:

• Text classification: predict correct labels that you want to assign to a document. For example,

  • sentiment analysis (deciding the rating based on the comments of a customer),

  • spam identification (determining whether an email is potentially a spam), and

  • topic classification (identifying the topic of the content).

Entity extraction (or entity recognition): identify entities, or the relevant specifics, within text. For example, Google Assistant understands that you’re asking about the weather (which is the topic) for today (which is the date) in Denver (which is the location).

Machine translation.

Interactive conversation: such as chatbot.

You may already discover that NLP is a marriage between linguistics and artificial intelligence.

It’s multidisciplinary …and involves

linguistics,

computer science,

AI and ML,

and statistics.

Let’s discuss natural language models in more detail.

Natural language is a sequence.

The order of the words matters.

Sequence models have been used to capture and model natural language.

Sequences can be the input and the output of a machine learning model.

Broadly, sequence models fit into three types:

one-to-sequence,

sequence-to-one,

and sequence-to-sequence.

In a one-to-sequence model, one input is passed in, and the model generates a sequence as the output.

A typical example is image captioning, where the description of an image, normally a few sentences, is generated based on this one image.

In a sequence-to-one model, a sequence is passed in and one output is generated.

For example, sentiment analysis is a typical sequence-to-one problem.

Based on the comments of a few sentences, the machine can generate one rating.

Finally, in a sequence-to-sequence model, sequences serve as both the input to and the output of the model.

Google Translate is a typical example in dealing with sequence-to-sequence problems.