Just another Network site

Deep Learning Research Review: Natural Language Processing

#DeepLearning Research Review: Natural Language Processing

  • Since deep learning loves math, we’re going to represent each word as a d-dimensional vector.
  • Extracting the rows from this matrix can give us a simple initialization of our word vectors.
  • The above cost function is basically saying that we’re going to add the log probabilities of ‘I’ and ‘love’ as well as ‘NLP’ and ‘love’ (where ‘love’ is the center word in both cases).
  • One Sentence Summary: Word2Vec seeks to find vector representations of different words by maximizing the log probability of context words given a center word and modifying the vectors through SGD.
  • Bonus: Another cool word vector initialization method: GloVe (Combines the ideas of coocurence matrices with Word2Vec)


This edition of Deep Learning Research Review explains recent research papers in Natural Language Processing (NLP). If you don’t have the time to read the top papers yourself, or need an overview of NLP with Deep Learning, this post is for you.

This edition of Deep Learning Research Review explains recent research papers in Natural Language Processing (NLP). If you don’t have the time to read the top papers yourself, or need an overview of NLP with Deep Learning, this post is for you.

Natural language processing (NLP) is all about creating systems that process or “understand” language in order to perform certain tasks. These tasks could include

The traditional approach to NLP involved a lot of domain knowledge of linguistics itself. Understanding terms such as phonemes and morphemes were pretty standard as there are whole linguistic classes dedicated to their study. Let’s look at how traditional NLP would try to understand the following word.

Let’s say our goal is to gather some information about this word (characterize its sentiment, find its definition, etc). Using our domain knowledge of language, we can break up this word into 3 parts.

We understand that the prefix “un” indicates an opposing or opposite idea and we know that “ed” can specify the time period (past tense) of the word. By recognizing the meaning of the stem word “interest”, we can easily deduce the definition and sentiment of the whole word. Seems pretty simple right? However, when you consider all the different prefixes and suffixes in the English language, it would take a very skilled linguist to understand all the possible combinations and meanings.

Deep Learning Research Review: Natural Language Processing

Comments are closed, but trackbacks and pingbacks are open.