With this post I am presenting a short list of quality introductory material for anyone interested in entering the amazing world of neural networks.
Why would anyone want to do that? There are plenty of reasons. During the past decades, the biggest motivation would be academic and research interest. But today, neural networks popularity has grown to unprecedented levels because of recent scientific advances and new practical applications. They can be employed to a wide range of problems, including image recognition, speech recognition, text mining … self driving cars … etc, and with mind blowing accuracy!
So if you find yourself considering neural networks in order to solve problems in your projects, then perhaps you are just on the right track.
This has been the reason I started learning neural networks myself. An image processing problem, where all other statistical approaches failed miserably.
So, what is presented here, is the point of view of someone without any prior knowledge on the field, who at the same time is a computer engineer with pure practical and not academic interest in neural networks. At least that was the picture in the beginning, since after studying and working with neural nets for a while, I have to admit that I am absolutely captivated by the field and I seriously regret not entering earlier.
In any case, for anyone that has a problem to solve and is considering applying neural networks, the first thing I have to say is that a fair amount of neural net theory is necessary, even if a PhD is not their target. What I mean by this, is that although there are pretrained neural network models out there, I seriously doubt if anyone can go far into applying them towards solving a custom problem, without first knowing a few fundamental things about neural networks.
So this is where we start from. I found that the Stanford CS231n lecture notes and videos are one of the best starting points out there. Here are the links
Everything is very well explained, all the fundamental points are presented and by the end you will have a very good idea of what neural networks are all about. All it takes, is a little bit of patience since it takes some time and a lot of repetition in order to grasp the basic ideas. As a tip, try to follow the material while implementing what you read in python because something may look simple in theory, but once you try to implement it you get to deal with all the details you missed while reading it.
It would also be nice if you had a basic idea about linear algebra and probability theory. The more advanced you are in these fields, the faster you will be able to progress on the course.
Also, programming skills are necessary, but assuming that you have a developer background, this will not be an issue. It will also be a good chance to learn python and numpy, if you are not already familiar with them.
Another great university course, is the Caltech University machine learning course “Learning From Data”. Although covering a much broader range than just neural networks, it serves as an excellent course of placing the various aspects of machine learning (including neural networks) into one big picture.
Personally, I started with Lecture 10, but I am definitely following the lectures prior to that, with the first chance I get.
Try to follow both courses to completion, or at least to the point that you feel you have the knowledge you need in order to actually try and do something with neural networks.
Once you have reached this point you will be able to start building your own models or using the pretrained models that exist. You will have to choose and learn a suitable framework for that (Torch, Tensorflow, Caffe, Theano, etc) but I am sure this is a process that a modern software engineer is doing over and over again (I personally see this as one of the advantages of our profession). Here is a link to help you choose.
If you are on the other hand interested in building on the knowledge you have acquired and exploring deeper into the neural network research field, there are endless resources out there for you to follow. If you do so, please share with us whatever you find exciting.
Although the list I presented here, can hardly be described as a list, as it only contains a couple of items, I believe that, as in any other computer science field, what is most important for a beginner, is to become confident with the fundamental knowledge upon which everything else is based and not to be exposed to an overwhelming amount of different and advanced stuff. If in doubt try to read this first 🙂
Thanks for reading! Please feel free to share any thoughts in the comments section.
