artificial intelligence and related field

Details

Last Updated: Friday, 12 May 2023 23:24

Published: Friday, 23 August 2019 20:11

Hits: 1355

Artificial Intelligence (AI):

branch of computer science with goal of making computers or machines as intelligent as humans. John McCarthy is called the father of AI. He demonstarted 1st AI pgm in 1956 at CMU. AI is dominant in fields of gaming (chess, poker, go), natural language processing (NLP, which understands natural language spoken by humans), vision systems (which can comprehend visual images, video), speech recognition, handwriting recognition, robotics, etc.

AI can be classified in 2 - narrow AI, and general AI. narrow AI refers to use of AI for a narrow purpose as self driving car, smart speaker, etc, while general AI refers to doing anything that a human can do.

AI is basically supervised learning, where after the learning process, a system can be made smart enough where it can produce correct o/p for any given i/p. There is so much interest in AI today, is because traditional AI didn't perform better even when the amount of training data grew. But now with specific branches of AI, we can scale AI performance with the amount of data, i.e more the data, higher the accuracy of prediction. This is specifically true for neural nets, and more so for deep neural nets, which are just one of the approaches of AI.

Machine learning (ML):

AI started as wide field, with thousands of approaches to achieve this intelligence in m/c. All these approaches were being worked until 1980s. However, then a particular subset of AI called machine learning (ML) began to flourish, as it showed much greater promise. It was the practice of using algorithms to parse data, learn from it, and then make a determination or prediction about something in the world. It's algorithmic approaches included decision tree learning, inductive logic programming. clustering, reinforcement learning and Bayesian networks among others. One of the very best application areas for machine learning for many years was "computer vision", though it still required great deal of hand-coding. "computer vision or image detection" was the field of identifying images and videos to make sense of that image. However, it never achieved the goal of general AI, at best it was narrow AI, and even then it was very error prone.

Neural Network (NN) or artificial neural networks (ANN):

A particular algorithmic approach of machine learning, called artificial neural networks (ANN) (or simply NN) had been a topic of research for several decades. They were based on analogy of how human brain works. However, they were mostly shunned by AI research community as never showed much promise of intelligence. The problem was even the most basic neural networks were very computationally intensive, it just wasn’t a practical approach. However, in 2000 as super computers got more powerful, this started to change. ANN were able to learn from a large set of data, and get good results. Previous algo in ML saturated in how well they performed, even after more and more data was used to train them. But ANN scaled with data, and allowed the error rate to keep on going down, as more and more data was used to train them. In 2001, for the first time, human face detection was possible from images. This algo from research came into a product in 2006, when camera started having face detection technology in realtime. This was a major accomplishment. However, with ANN, even these powerful computers weren't able to unleash full power of ANN as they weren't able to handle massive computation. 

Deep learning or Deep Neural network:

Then in mid 2000, the field of ANN got major breakthrough. GPU started getting deployed for doing massive computation needed for ANNs. GPU do a lot of computation in parallel, as compared to CPU which do mostly serial computation. In 2012, Andrew Ng at google (previously a professor at Stanford) took these neural networks, and essentially made them huge, increased the layers and the neurons, and then ran massive amounts of data through the system to train it. In Ng’s case it was images from 10 million YouTube videos. Andrew Ng put the “deep” in deep learning, which describes all the layers in these neural networks. He ran a neural network trained using deep learning algo running on 16,000 CPU cores, that learned to recognize cats after watching only youtube videos, and w/o ever having been told what a "cat" is. For the first time, the promise of "intelligence" was realized. Deep learning is one of the methods applied to ANNs, and can be considered a subset of ANN. It just increases the depth of NN, and by doing that, it makes them deep. This makes NN perform exponentially better. Instead of deep NN, you can design shallow NN too if you have limited computed resource, but they do not perform that well (early ANN were shallow).

Convolution neural network (CNN):

There are various classes of ANNs. The class used in deep learning, that we saw above, is called deep neural network. There are various classes within deep neural network. One that is most commonly applied to analyze images and videos is called CNN. In 2010, ImageNet started in image classification challenge. The error rate in 1st 2 years hovered around 25%. In 2012,  an algorithm dropped error rate from 25% in previous years to 15%. This magical algorithm was CNN, and it started the runaway success of CNN. CNN have since dominated the field, and effort every year has been to improve CNN, rather than spending effort/dollar on alternative algo. We will only focus on CNN from now on. The term "convolution" indicates that the network employs a mathematical operation called convolution. Convolution is a specialized kind of linear operation. Convolutional networks are simply neural networks that use convolution in place of general matrix multiplication in at least one of their layer. Even though CNN is a class within deep learning which itself is a subset of ANN, they have all become synonymous with deep learning, and the 3 terms are used interchangeably (DL/ANN,NN/CNN).

One of the reasons CNN are so popular is that they use relatively little pre-processing compared to other image classification algo. This means that the network learns the filters that in traditional algorithms were hand engineered. This independence from prior knowledge and human effort in feature design is a major advantage.

One very good explanation of CNN is here: https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/

Other link here (read both parts 1 and 2): https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner's-Guide-To-Understanding-Convolutional-Neural-Networks/

3Blue1Brown video series on Youtube are absolutely amazing (4 chapters on NN series): https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi

This site is one recommeneded by 3B1B. It's a completely free online Book with 6 chapters: http://neuralnetworksanddeeplearning.com/index.html

You should look at Andrej Karpathy website: https://karpathy.ai. Lots of fun and life learning stuff. This course by Andrej Karpathy on coding your own GPT (videos build up from basics): https://karpathy.ai/zero-to-hero.html

There are video lectures by Prof Andrew Ng (from Stanford) on www.coursera.org

This is a very good starting point for AI, and I would base my next sections on his lectures on Coursera.