What is Machine Learning?

Even those who know little about Machine Learning are likely to use it every day. Here's a basic introduction to ML (no math required) to get you started on a topic that may soon literally drive the world.

One of the most challenging tasks we adults may have is to learn a new language.  Even after years of being taught, practicing and using our new language, we may still be far from fluency.  However, each of us have been able to learn our first language or even a second language, within the first few years of our lives, without ever being taught or even realizing we were trying to learn.

We begin by taking the many sounds we hear and then put them together to form a word. Next we join these words with one or two other words and then later put more words together into fully formed sentences.  The amazing part is when we start joining words together they aren't necessarily ones we've heard put together, and the same is true for sentences. Somehow we were able to recognize patterns of words and grammatical rules without being taught. We learned the rules from the data then applied the rules to output language in new ways  - this learning is very inductive, going from data to hypothesis rather than the other way around. We learn that more than one dog is "dogs". When we learn the word horse, we don't have to be taught that more than one horse is "horses".  Because language is invented primarily by adults, and they are not quite as logical as a child, they use words like held instead of the very logical "holded".  Based on feedback from the data, over time the child soon begins to use "held" instead of "holded."

Learning, applying, adjusting and repeating is perhaps what has allowed humans to change ourselves and the world around us, where as, all other animals, as far as we know, are programmed via DNA (evolution) and primarily mimic and copy their ancestors and remain just as they were hundreds of thousands or millions of years ago. Recently, computer science has been adapting a similar learning approach for software, where problems are too complex for traditional rules based programming. These problems or challenges include autonomous driving, recommendation systems where choices are near limitless, predictive systems that require massive data and variables, computer vision and voice recognition and response systems. It's becoming an underlying feature in many business and consumer applications and will most likely become ubiquitous in all parts of our lives in the not too distant future.


Machine Learning

The most basic definition of Machine Learning (ML) was given by Arthur Samuel, a pioneer in Artificial Intelligence, in 1959 as a "field of study that gives computers the ability to learn without being explicitly programmed." We don't exactly know how a child's brain and mind gives rise to learning a language but computer scientists model a similar process with machine learning. A more technical, yet logical, definition by computer scientist and professor, Tom Mitchell, is:

"A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E."

For example, task T, could be selecting a movie on Netflix, E is the selection history from all customers and P is the accuracy of suggested "What you may like" titles. 


ML Application

It's easy to think of machine learning as just beginning (it is), with so much potential still in the future (very true) and currently only used in the fringes of X labs (it's not). ML Applications have become a daily part of our lives to help solve some problems or answer questions. Like Google replying to a search request with exactly what we were looking for, Siri understanding what we are asking and responding accordingly, Amazon and Netflix properly suggesting what we would like, Pandora playing new songs that we would prefer over others, Zillow estimating the market value of any house we search, Uber using driverless cars to take us from point A to point B safely, and Facebook recognizing your face in a photo, we're interacting with this technology throughout the majority of our days. Of course, none of these are perfect at predicting, but the goal of ML is not perfect predictions. The goal is for less than perfect but useful predictions. Less than perfect but useful is far better than millions of potential possibilities for our human minds to select or for us to try to traditionally program to help select.


ML is not just helping us become better consumers but also improves our productivity on the job. Recently one of my colleagues based in Ecuador, Milton, and I tested Skype real time language translation between Spanish and English. I went into the test with very high and probably unfair expectations. The first step is for the application to recognize what Milton or I was saying. It had difficulty with certain names, especially if they were not common. In the second step, translation, there were times it was directly translating where direct translation doesn't necessarily equate to the same meaning in the other language. The third step was turning the translation into speech, which was pretty good. We just had to get used to the gaps between replies required for each of us to hear and understand the translated speech before replying. Initially, I was focusing on the flaws, as we all tend to do, and was disappointed. But after stepping back a bit and looking at the big picture of what we did, I realized that it was rather amazing. 80% to 90% of what we said was recognized and translated effectively, nearly in real-time, using a free application anyone can download. It's just a matter of time before the algorithms and hardware improvements get it to near 100%.

ML Basics (Very Basic)

ML applications can be split into two categories:

Supervised Machine Learning - this represents the majority of ML problems being tackled currently.  In this case you have inputs and known outputs and the goal is to learn and constantly improve the mapping function to where the predicted outputs of the mapping function are highly accurate. It's like a teacher telling you "right" or "wrong" and you tweaking your learning to get more "rights" over time. These problems can be further split into classification or regression problems.  Classification problems are where the output are attributes such as blue, red, cat, mug, sick, etc. Regression problems are where the output is numerical value such as price, quantity, temperature, etc. 

An example of this is feeding large number of pictures of coffee mugs (in essence labeled as mugs) where the algorithm begins to understand the visual features (via pixels and numbers) of coffee mugs to the point of being able to identify new and different pictures of coffee mugs as "coffee mugs".  Another, and much more basic example of supervised learning, is to feed a learning algorithm the many attributable features of sold houses such as location, square footage, number of bedrooms and the known selling price. Using that data (more data the better), the algorithm will approximate a function that can output expected sales price for any house based on the input attributes modeled.

Unsupervised Machine Learning - this represents ML problems that more closely model to a child learning language. In these problems you have a large amount of input data but no corresponding output data. The goal of the algorithm is understand the underlying model, structure, relationships and rules within the data. These problems can be split into clustering and association problems.  Some clustering examples would be to group customers according to purchases, or group friends based on social network data. An association example would be that a customer who buys x tend to buy y as well.

Google successfully used Unsupervised ML on YouTube videos, where the algorithm was able to determine that human faces, cat faces and human bodies were relevant features of the data. The machines were not asked to find any of these or given any labeled data. They had no way to know that it was a human face or a cat face. They just knew the set of pixels that make up those features in the video are distinct and relevant. 

Some problems can use a combination of supervised and unsupervised and is referred to as Semi-Supervised Machine Learning. For all three types, many different mathematical and statistical algorithms are used and have been around for some time. What's different is that more data is available and accessible and processing power is growing. Those two are the main ingredients for all ML problems and are catching up to the math part of the solution allowing us to take on greater challenges.


The Future of ML (A Possible Future)

Andrew Ng, a leading ML and AI researcher, tells a story in one of his educational talks about his goal 15 years ago to program a helicopter to fly on its own. It was a challenging problem, as flying a helicopter is very difficult - the pilot uses both hands and both feet to adjust controls to maneuver the helicopter as desired based on both flight plan, aerodynamics and environmental factors. His vision was to model the mathematics involved into a computer program but of course the mathematics of both aerodynamics and the control of the helicopter are extremely complex with large amounts of data and variables. He realized it was too difficult and perhaps impossible. 

Later he and students at Stanford approached the problem differently.  Remote controlled, small helicopters are also a challenge to fly - especially flying advanced aerobatics. Ng's team used machine learning to learn via watching (using the same data) advanced pilots of these remote controlled helicopters.  Using a reinforcement learning algorithm they were able to autonomously do similar aerobatics and even fly inverted for a long period of time which the human pilots could not do. This shows the power of machine learning to tackle problems that are too difficult for even the most advanced programmers.

A possible future, according to Andrew Ng, is one where there is one Master Algorithm rather than the many currently used for various learning problems. Why? That seems to be the way the best learning algorithm, the human brain, works. Neuroscientists have tried experiments where they rewire eyes away from the visual part of the brain to the audio part of the brain. Interestingly the brain can still process visual information.  The same result happened when eyes were rewired to the part of the brain used for physical touch. These results lead them to believe that the brain uses the same algorithm for all of this. 

A master algorithm that processes similar to the human brain is probably far into the future, but what is already the case is that the application and problems that ML and it's more advanced incarnation, Deep Learning, can help solve are infinite. More applications will be using ML requiring more of us with ML skills and more of us who will be able to recognize the problems we can use ML to solve. Advanced chips from companies like Nvidia and Intel, ML tools and platforms from Google and Microsoft, IoT sensors and the growth of public clouds to hold massive amounts of data are all leading to Machine Learning becoming ever more larger part of our lives.

It's a good time to learn more about Machine Learning!

Correcting The IT Productivity Problem