Deep-Learning

In this early morning of Super Bowl Day, I finally finished Deep Learning Specialization taught by Andrew Ng.

This specialization includes 5 modules:

Course 1: Neural Networks and Deep Learning

• Understand the major technology trends driving Deep Learning.

• Be able to build, train and apply fully connected deep neural networks.

• Know how to implement efficient (vectorized) neural networks.

• Understand the key parameters in a neural network's architecture.

  • Week1: Be able to explain the major trends driving the rise of deep learning, and understand where and how it is applied today.

    • Python basics with numpy

  • Week2: Learn to set up a machine learning problem with a neural network mindset. Learn to use vectorization to speed up your models.

    • Logistic Regression with a Neural Network mindset

  • Week3: Learn to build a neural network with one hidden layer, using forward propagation and backpropagation.

    • Planar data classification with a hidden layer

  • Week4: Understand the key computations underlying deep learning, use them to build and train deep neural networks, and apply it to computer vision.

    • Build your deep neural network: Step by Step
    • Deep Neural Network Application

Course 2: Improving Deeping Neural Networks

• Understand industry best-practices for building deep learning applications.

• Be able to effectively use the common neural network "tricks", including initialization, L2 and dropout regularization, Batch normalization, gradient checking.

• Be able to implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence.

• Understand new best-practices for the deep learning era of how to set up train/dev/test sets and analyze bias/variance.

• Be able to implement a neural network in TensorFlow.

  • Week 1: Practical aspects of Deep Learning

    • Initialization
    • Regularization
    • Gradient Chekcing

  • Week 2: Optimization algorithms

    • Optimization

  • Week 3: Hyperparameter tuning, Batch Normalization and Programming Frameworks

    • Tensorflow Tutorial

Course 3: Sturcturing Machine Learning Projects

• Understand how to diagnose errors in a machine learning system.

• Be able to prioritize the most promising directions for reducing error.

• Understand complex ML settings, such as mismatched training/test sets, and comparing to and/or surpassing human-level performance.

• Know how to apply end-to-end learning, transfer learning, and multi-task learning.

  • Week 1: ML Strategy (1)

    • Bird recognition

  • Week 2: ML Strategy (2)

    • Autonomous driving

Course 4: Convolutional Neural Networks

• Understand how to build a convolutional neural network, including recent variations such as residual networks.

• Know how to apply convolutional networks to visual detection and recognition tasks.

• Know to use neural style transfer to generate art.

• Be able to apply these algorithms to a variety of image, video, and other 2D or 3D data.

  • Week 1: Foundations of Convolutional Neural Networks

    • Convolutional Model: step by step
    • Convolutional model: application

  • Week 2: Deep convolutional models: case studies

    • Residual Networks

  • Week 3: Object detection

    • Car detection with YOLOv2

  • Week 4: Special applications: Face recognition & Neural style transfer

    • Art generation with Neural Stype Transfer
    • Face Recognition for the Happy House

Course 5: Sequence Models

• Understand how to build and train Recurrent Neural Networks (RNNs), and commonly-used variants such as GRUs and LSTMs.

• Be able to apply sequence models to natural language problems, including text synthesis.

• Be able to apply sequence models to audio applications, including speech recognition and music synthesis.

  • Week 1: Recurrent Neural Networks

    • Build RNN: step by step
    • Dinasaur Island: Character-level language modeling
    • Jazz improvisation with LSTM

  • Week 2: Natural Language Processing & Word Embeddings

    • Operation on word vectors - Debiasing
    • Emojify: Sentimental analysis

  • Week 3: Sequence models & Attention mechanism

    • Neural Machine Translation with Attention
    • Trigger word detection

Please see my GitHub for details.

I have also reviewed two amazing courses offered by Stanford University, which are

• CS231n: Convolutional Neural Networks for Visual Recognition

• CS224n: Natural Language Processing with Deep Learning

For the basic of machine learning, please refer to Andrew's Maching Learning on Coursera and CS229: Maching Learning. Here is my GitHub repo for Andrew's Machine Learning course as guidance if needed.

Other Resources

Deep Learning textbook: Ian Goodfellow and Yoshua Bengio and Aaron Courville

Cheat Sheets for Deep Learning

Deep Learning Projects

TensorFlow and Deep Learning without a PhD (LOL)

• CMU maching learning

Introduction to Machine Learning

Advanced Introduction to Machine Learning

• UBC maching learning

Machine Learning and Data Mining

Machine Learning

• Hunag-yi Lee videos

Maching Learning and Deep Learning resources