Machine learning for NLP 2

SL5BE021

The course provides the fundamental concepts of supervised classification via deep learning methods, with typical examples in NLP.

1. General concepts for supervised classification

• methodology
• evaluation metrics

2. A first classifier : k-NN

3. Linear and log-linear models

• linear separability
• prediction with a (log-)linear classifier
• perceptron learning algorithm
• kernel methods
• logistic regression

4. Extension to Multi-layer perceptrons

• non linearity
• fully connected feed-forward neural network
• universal approximation theorem

5. Learning as loss minimization

• usual loss functions
• stochastic gradient descent
• backpropagation algorithm

6. vectorial representations

• word embeddings as dense features
• word embeddings learning

Lab sessions will illustrate the course, introducing in particular:

• tensor manipulation in numpy / pytorch
• sklearn and pytorch libraries

Bibliography

• Hal Daumé III : An introduction to Machine Learning, http://ciml.info/
• “Neural Network Methods in Natural Language Processing”, 2016, Morgan & Claypool
  • preliminary version available here : A primer on neural network models for natural language processing (http://u.cs.biu.ac.il/~yogo/nnlp.pdf)
• Goodfellow, Bengio & Courville “Deep Learning”, MIT Press, 2016 http://www.deeplearningbook.org/