T81 558:Applications of Deep Neural Networks

Washington University in St. Louis

Instructor: Jeff Heaton

Spring 2017, Mondays, class room Lopata Hall #103

Please note, this semester is using TensorFlow 0.12.1.

Course Description

Deep learning is a group of exciting new technologies for neural networks. By using a combination of advanced training techniques and neural network architectural components, it is now possible to train neural networks of much greater complexity. This course will introduce the student to deep belief neural networks, regularization units (ReLU), convolution neural networks and recurrent neural networks. High performance computing (HPC) aspects will demonstrate how deep learning can be leveraged both on graphical processing units (GPUs), as well as grids. Deep learning allows a model to learn hierarchies of information in a way that is similar to the function of the human brain. Focus will be primarily upon the application of deep learning, with some introduction to the mathematical foundations of deep learning. Students will use the Python programming language to architect a deep learning model for several of real-world data sets and interpret the results of these networks.

Objectives

1. Explain how neural networks (deep and otherwise) compare to other machine learning models.
2. Determine when a deep neural network would be a good choice for a particular problem.
3. Demonstrate your understanding of the material through a final project uploaded to GitHub.

Syllabus

This syllabus presents the expected class schedule, due dates, and reading assignments. Download current syllabus.

Class	Content
MLK Day 01/16/2017	No class session
Class 1 01/23/2017	Python for Machine Learning
Class 2 01/30/2017	Neural Network & Machine Learning Basics Introduction to TensorFlow Assignment: Read Chapter 1
Class 3 02/06/2017	Training a Neural Network Assignment: Read Chapters 4 & 5
Class 4 02/13/2017	Classification & Regression Assignment: Program 1 Due
Class 5 02/20/2017	Backpropagation Assignment: Read Chapter 6
Class 6 02/27/2017	Preprocessing Assignment: Program 2 Due (Tuesday, 2-28 at midnight)
Class 7 03/06/2017	Kaggle Datasets Evaluating Neural Network Performance Assignment: Take Home Midterm Due (Tuesday, 3-7 at midnight)
Spring Break 03/13/2017	No class session
Class 8 03/20/2017	Convolutional Neural Networks
Class 9 03/27/2017	Regularization and Dropout Assignment: Read Chapter 12
Class 10 04/03/2017	Timeseries and Recurrent Neural Networks Assignment: Read Chapter 13, Program 3 Due (Tuesday, 4-4 at midnight) (submitted on Kaggle.com)
Class 11 04/10/2017	Natural Language Processing Assignment: Read Chapter 14
Class 12 04/17/2017	Applications of Neural Networks Assignment: Program 4 Due (Tuesday, 4-18 at midnight)
Class 13 04/24/2017	Advanced Deep Learning Topics
Class 14 05/01/2017	GPU, HPC and Cloud Assignment: Final Project Due (Tuesday, 5-2 at midnight)

Datasets

• Iris - Classify between 3 iris species.
• Auto MPG - Regression to determine MPG.
• WC Breast Cancer - Binary classification: malignant or benign.
• toy1 - The toy1 dataset, regression for weights of geometric solids.

*Note: Other datasets will be added as the class progresses.

Assignments

• Programming Assignment #1
• Programming Assignment #2
• Midterm
• Programming Assignment #3
• Programming Assignment #4
• Final Assignment

Other Information

• Helpful Functions - Helpful Python functions for this class.
• KDD99 Example
• Video Tutorial on Using Data Scientist Workbench - See how to add data to Data Scientist Workbench
• TensorFlow Versions - How to install the version of TensorFlow used in this class.
• Care and Feeding of Python - Some useful commands for a local Python install. Not needed if you are using Data Scientist Workbench.