Machine Learning Resources

Neural Networks

• Neural Networks and Deep Learning

Recurrent Neural Networks

• Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs
• Recurrent Neural Networks Tutorial, Part 2 – Implementing a RNN with Python, Numpy and Theano
• Recurrent Neural Networks Tutorial, Part 3 – Backpropagation Through Time and Vanishing Gradients
• Recurrent Neural Network Tutorial, Part 4 – Implementing a GRU/LSTM RNN with Python and Theano
• Understanding LSTM Networks

Deep Learning

Deep Learning Book (Ian Goodfellow, Yoshua Bengio and Aaron Courville) (link)

1. Introduction
2. Linear Algebra
3. Probability and Information Theory
4. Numerical Computation
5. Machine Learning Basics
6. Deep Feedforward Networks
7. Regularization
8. Optimization for Training Deep Models
9. Convolutional Networks
10. Sequence Modeling: Recurrent and Recursive Nets
11. Practical Methodology
12. Applications

NLP

Deep Learning for NLP (Stanford CS224)

• Lecture Notes 1
• Lecture Notes 2
• Lecture Notes 3
• Lecture Notes 4

Torch

Machine Learning (University of Oxford)

• Introduction to Lua and Torch
• Linear models
• Classifying digits and tuning optimizers
• Implementing your own layer
• nngraph
• LSTMs for language modelling

Theano

• theano-01-basics-key
• theano-02-advanced-key
• theano-03-internals

Mathematics Resources

Algebra

• Applied Numerical Linear Algebra (James W. Demmel)
• Numerical Linear Algebra (Lloyd N. Trefethen, and David Bau III)
• The Matrix Cookbook
• 高等代数学 (姚慕生)
• Linear Algebra Review and Reference (cs229)
• Machine Arithmetic: Fixed-Point and Floating-Point Numbers (MIT 18.330)

Optimization

• Numerical Optimization (Jorge Nocedal and Stephen J. Wright)
• Convex Optimization (slides, solutions)
• Proximal Algorithms
• Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers
• Convex Optimization Overview (cs229)
• Convex Optimization Overview 2 (cs229)
• An Introduction to the Conjugate Gradient Method Without the Agonizing Pain

Probability and Statistics

• The Elements of Statistical Learning
• An Introduction to Statistical Learning
• All of Statistics
• The Multivariate Gaussian Distribution (cs229)
• More on Multivariate Gaussians (cs229)
• Review of Probability Theory (cs229)

My Notes

Machine Learning

1. Linear Regression
2. Logistic Regression
3. Principal Components Analysis
4. K-Means
5. K-Nearest Neighbors
6. Decision Tree
7. Naive Bayes
8. Neural Networks
9. Recurrent Neural Network
10. LSTM Networks

Mathematics

1. Singular Value Decomposition
2. Lagrange Duality
3. Norms
4. Projectors

NLP

1. Word Embedding

Recommender Systems

1. Collaborative Filtering

Others

1. Electricity Data Analysis Based on Sparse Coding
2. Memory based Model