Deep Learning Specialization on Coursera

Introduction

This repo contains all my work for this specialization. The code and images, are taken from Deep Learning Specialization on Coursera.

In five courses, you are going learn the foundations of Deep Learning, understand how to build neural networks, and learn how to lead successful machine learning projects. You will learn about Convolutional networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, and more. You will work on case studies from healthcare, autonomous driving, sign language reading, music generation, and natural language processing. You will master not only the theory, but also see how it is applied in industry. You will practice all these ideas in Python and in TensorFlow, which we will teach.

Notification

The Github Repository is designed to help viewers improving programming skills and revisiting basic Deep Learning knowledge.

Please follow and respect the Coursera Honor Code if you are enrolled with any Coursera Deep Learning courses. It is OK to use this repo as a reference to debug your program, but it is wrong to copy-paste codes from the repo just to getting by the Lab Assignments. Knowledge and practical experience are more important than certificates.

Github: https://github.com/cuicaihao/coursera-deeplearning-specialization

Personally speaking, even with a PhD in computer science, I still find few Lab Assignments are quite difficult to get all right at the first run , especially, the labs on the Deep ConvNets and B-LSTM models. One typo will cost you more time to debug, but it definitely worth it. You are improving your programming skills, learning new knowledges and knowing yourself at the same time.


Comments and Recommendation

There are also some disadvantage about this DL course. For example, the TensorFlow package used in the Lab is version 1.x. The most cutting edge package of TensorFlow is already 2.4.x (TensorFlow ), which means in real practice, all the code you learned in these courses have to updated, but the math are still the same. The Keras today is part of TensorFlow 2.0 instead of an indenpendent framework.

Beside TensorFlow, I find PyTorch and Paddle are also really good open source machine learning frameworks, accelerating the path from research prototyping to production deployment.

PyTorch
TensorFlow
PaddlePaddle Baidu 百度

Moreover, I highly recommend this Paper With Code website https://paperswithcode.com/, which create a free and open resource with Machine Learning papers, code and evaluation tables.

Paper with Code: Browse State-ofthe-Art

For example, we see the trends of paper implementations grouped of frameworks. It is clear to see all those authors prefer PyTorch, thus you know what to do.

Paper Implementations grouped by framework from 2010 to 2020

Programming Assignments

Course 1: Neural Networks and Deep Learning

Objectives:

  • Understand the major technology trends driving Deep Learning.
  • Be able to build, train and apply fully connected deep neural networks.
  • Know how to implement efficient (vectorized) neural networks.
  • Understand the key parameters in a neural network’s architecture.

Code:

Course 2: Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization

Objectives:

  • Understand industry best-practices for building deep learning applications.
  • Be able to effectively use the common neural network “tricks”, including initialization, L2 and dropout regularization, Batch normalization, gradient checking,
  • Be able to implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence.
  • Understand new best-practices for the deep learning era of how to set up train/dev/test sets and analyze bias/variance
  • Be able to implement a neural network in TensorFlow.

Code:

Course 3: Structuring Machine Learning Projects

Objectives:

  • Understand how to diagnose errors in a machine learning system, and
  • Be able to prioritize the most promising directions for reducing error
  • Understand complex ML settings, such as mismatched training/test sets, and comparing to and/or surpassing human-level performance
  • Know how to apply end-to-end learning, transfer learning, and multi-task learning

Code:

  • There is no Program Assignments for this course. But this course comes with very interesting case study quizzes.

Course 4: Convolutional Neural Networks

Objectives:

  • Understand how to build a convolutional neural network, including recent variations such as residual networks.
  • Know how to apply convolutional networks to visual detection and recognition tasks.
  • Know to use neural style transfer to generate art.
  • Be able to apply these algorithms to a variety of image, video, and other 2D or 3D data.

Code:

Course 5: Sequence Models

Objectives:

  • Understand how to build and train Recurrent Neural Networks (RNNs), and commonly-used variants such as GRUs and LSTMs.
  • Be able to apply sequence models to natural language problems, including text synthesis.
  • Be able to apply sequence models to audio applications, including speech recognition and music synthesis.

Code:

Research Papers List

This is a list of research papers referenced by the Deep Learning Specialization course.

Convolutional Neural Network

Classic Networks
Resnets
Networks in Networks and 1×1 Convolutions
Networks in Networks and 1×1 Convolutions

Min Lin, Qiang Chen, Shuicheng Yan – “Network In Network”

Inception Networks
Convolutional Implementation of Sliding Windows
Bounding Box Predictions
Region Proposals
Siamese Network
Triplet Loss
What are deep ConvNets learning?
Neural Style
Image Recognition

NLP Sequence Models

GRU
LSTM
Skip-Grams, Hierarchical Softmax
Word Embeddings
Negative Sampling
Glove
Debaising Word Embeddings
Sequence to Sequence Model
Image Captioning
Bleu Score
Attention based intuition
Speech Recognition
  • Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks
Further Reading List
Master Deep Learning, and Break into AI

Author: Caihao (Chris) Cui

Data Scientist and Machine Learning Technologist: Translating modern machine learning and computer vision techniques into engineering and bringing ideas to life to built a better future.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s