Open Courses, Free Softwares and Frameworks of Deep Learning

Just sharing my sources of Deep learning, if anyone finds this post helpful, please share it. The content will be updated with the new techniques and information.

The last update time is 17/12/2016 in Melbourne. (I plan to add some figures of the following links and include all the amazing  blogs of deep learning  in the future.)

1 Open Courses:

Machine Learning   -by  Andrew Ng

Description:  In this course, you’ll learn about some of the most widely used and successful machine learning techniques. You’ll have the opportunity to implement these algorithms yourself and gain practice with them. You will also learn some of practical hands-on tricks and techniques (rarely discussed in textbooks) that help get learning algorithms to work well. This is an “applied” machine learning class, and we emphasize the intuitions and know-how needed to get learning algorithms to work in practice, rather than the mathematical derivations.

Familiarity with programming, basic linear algebra (matrices, vectors, matrix-vector multiplication), and basic probability (random variables, basic properties of probability) is assumed. Basic calculus (derivatives and partial derivatives) would be helpful and would give you additional intuitions about the algorithms, but isn’t required to fully complete this course.

Machine Learning : This course (CS229) — taught by Professor Andrew Ng — provides a broad introduction to machine learning and statistical pattern recognition. Topics include supervised learning, unsupervised learning, learning theory, reinforcement learning and adaptive control. Recent applications of machine learning, such as to robotic control, data mining, autonomous navigation, bioinformatics, speech recognition, and text and web data processing are also discussed. Everyone could download all the learning materials as follows:

UFLDL Tutorial

Description: This tutorial will teach you the main ideas of Unsupervised Feature Learning and Deep Learning. By working through it, you will also get to implement several feature learning/deep learning algorithms, get to see them work for yourself, and learn how to apply/adapt these ideas to new problems.

This tutorial assumes a basic knowledge of machine learning (specifically, familiarity with the ideas of supervised learning, logistic regression, gradient descent). If you are not familiar with these ideas, we suggest you go to this Machine Learning course and complete sections II, III, IV (up to Logistic Regression) first.

Neural Networks for Machine Learning   -by Geoffrey Hinton (Starts Dec 12 on Coursera)

Description: Learn about artificial neural networks and how they’re being used for machine learning, as applied to speech and object recognition, image segmentation, modeling language and human motion, etc. We’ll emphasize both the basic algorithms and the practical tricks needed to get them to work well.

YouTube  Lecture : Lecture from the course Neural Networks for Machine Learning, as taught by Geoffrey Hinton (University of Toronto) on Coursera in 2012.

CS231n: Convolutional Neural Networks for Visual Recognition -by Fei Fei Li, Andrej Karpathy and  Justin Johnson

Description:  Computer Vision has become ubiquitous in our society, with applications in search, image understanding, apps, mapping, medicine, drones, and self-driving cars. Core to many of these applications is visual recognition tasks such as image classification, localization, and detection. Recent developments in the neural network (aka “deep learning”) approaches have greatly advanced the performance of these state-of-the-art visual recognition systems. This course is a deep dive into details of the deep learning architectures with a focus on learning end-to-end models for these tasks, particularly image classification. During the 10-week course, students will learn to implement, train and debug their own neural networks and gain a detailed understanding of cutting-edge research in computer vision. The final assignment will involve training a multi-million parameter convolutional neural network and applying it to the largest image classification dataset (ImageNet). We will focus on teaching how to set up the problem of image recognition, the learning algorithms (e.g. backpropagation), practical engineering tricks for training and fine-tuning the networks and guide the students through hands-on assignments and a final course project. Much of the background and materials of this course will be drawn from the ImageNet Challenge.

CS231n:  Notebooks and assignments ;  Lectures on YouTube ;

2 Softwares:

Matlab (Academic and Students) :

Description: Millions of engineers and scientists worldwide use MATLAB® to analyze and design the systems and products transforming our world. MATLAB is in automobile active safety systems, interplanetary spacecraft, health monitoring devices, smart power grids, and LTE cellular networks. It is used for machine learning, signal processing, image processing, computer vision, communications, computational finance, control design, robotics, and much more.

Deep learning with MATLAB : Deep learning is a branch of machine learning that uses multiple nonlinear processing layers to learn useful representations of features directly from data. Deep learning models can achieve state-of-the-art accuracy in object classification, sometimes exceeding human-level performance. Models are trained by using a large set of labeled data and neural network architectures that contain many layers.

 MATLAB Tutorials : Get Start with Matlab – In these interactive tutorials, you’ll learn the basics of MATLAB with lessons, demonstrations, quizzes, and hands-on practice sessions that reinforce learning.

MatConvnet : MatConvNet is a MATLAB toolbox implementing Convolutional Neural Networks (CNNs) for computer vision applications. It is simple, efficient, and can run and learn state-of-the-art CNNs. Many pre-trained CNNs for image classification, segmentation, face recognition, and text detection are available.

Python :

Description: Python is a programming language that lets you work quickly
and integrate systems more effectively.

Anaconda Python (Strongly suggested) :  Anaconda is the leading open data science platform powered by Python. The open source version of Anaconda is a high performance distribution of Python and R and includes over 100 of the most popular Python, R and Scala packages for data science.

Additionally, you’ll have access to over 720 packages that can easily be installed with conda, our renowned package, dependency, and environment manager, that is included in Anaconda. See the packages included with Anaconda and the Anaconda changelog.

Packages:

• Numpy: numPy is the fundamental package for scientific computing with Python.
• Matplotlib: matplotlib is a python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. matplotlib can be used in python scripts.
• Pandas: pandas is an open source, the BSD-licensed library providing high-performance, easy-to-use data structures, and data analysis tools for the Python programming language.
• SciKit : scikit-learn is Machine Learning toolbox built on NumPy, SciPy, and Matplotlib in Python. It is simple and efficient tools for data mining and data analysis.

Nvidia:

Description:  The NVIDIA Accelerated Computing Toolkit includes all the tools, libraries, training and information you need to accelerate your applications.

SDK provides high-performance tools and libraries to power innovative GPU-accelerated machine learning applications in the cloud, data centers, workstations, and embedded platforms

NVIDIA cuDNN : The NVIDIA CUDA® Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, pooling, normalization, and activation layers. cuDNN is part of the NVIDIA Deep Learning SDK.

3 Frameworks

Tensorflow :

Description: TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API.

Theano :

Description: Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently.

Keras :

Description: Keras is a high-level neural networks library, written in Python and capable of running on top of either TensorFlow or Theano. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research.

Torch :

Description: Torch is a scientific computing framework with wide support for machine learning algorithms that puts GPUs first. It is easy to use and efficient, thanks to an easy and fast scripting language, LuaJIT, and an underlying C/CUDA implementation.

MXNet :

Description: MXNet is an open-source deep learning framework that allows you to define, train, and deploy deep neural networks on a wide array of devices, from cloud infrastructure to mobile devices. It is highly scalable, allowing for fast model training, and supports a flexible programming model and multiple languages.

Caffe :

Description:  Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by the Berkeley Vision and Learning Center (BVLC) and by community contributors.