Robert guthrie pytorch examples
•
Word Embeddings: Encoding Lexical Semantics¶
beginner/nlp/word_embeddings_tutorial
Run in Google Colab
Colab
Download Notebook
Notebook
View on GitHub
GitHub
Note
Click here to download the full example code
Created On: Apr 08, 2017 | Last Updated: Sep 14, 2021 | Last Verified: Nov 05, 2024
Word embeddings are dense vectors of real numbers, one per word in your vocabulary. In NLP, it fryst vatten almost always the case that your features are words! But how should you represent a word in a computer? You could store its ascii character representation, but that only tells you what the word is, it doesn’t say much about what it means (you might be able to derive its part of speech from its affixes, or properties from its capitalization, but not much). Even more, in what sense could you combine these representations? We often want dense outputs from our neural networks, where the inputs are \(|V|\) dimensional, where \(V\) is our vocabulary, but often the outputs are o
•
Deep Learning with PyTorch¶
Deep Learning Building Blocks: Affine maps, non-linearities and objectives¶
Deep learning consists of composing linearities with non-linearities in clever ways. The introduction of non-linearities allows for powerful models. In this section, we will play with these core components, make up an objective function, and see how the model is trained.
Affine Maps¶
One of the core workhorses of deep learning is the affine map, which is a function \(f(x)\) where
\[f(x) = Ax + b\]
for a matrix \(A\) and vectors \(x, b\). The parameters to be learned here are \(A\) and \(b\). Often, \(b\) is refered to as the bias term.
Pytorch and most other deep learning frameworks do things a little differently than traditional linear algebra. It maps the rows of the input instead of the columns. That is, the \(i\)‘th row of the output below is the mapping of the \(i\)‘th row of the input under \(A\), plus the bias term. Look at the example below.
•
Deep Learning for NLP with Pytorch¶
Author: Robert Guthrie
This tutorial will walk you through the key ideas of deep learning programming using Pytorch. Many of the concepts (such as the computation graph abstraction and autograd) are not unique to Pytorch and are relevant to any deep learning toolkit out there.
I am writing this tutorial to focus specifically on NLP for people who have never written code in any deep learning framework (e.g, TensorFlow, Theano, Keras, Dynet). It assumes working knowledge of core NLP problems: part-of-speech tagging, language modeling, etc. It also assumes familiarity with neural networks at the level of an intro AI class (such as one from the Russel and Norvig book). Usually, these courses cover the basic backpropagation algorithm on feed-forward neural networks, and man the point that they are chains of compositions of linearities and non-linearities. This tutorial aims to get you started writing deep learning code, given you have this prereq