There are four widely recognized styles of machine learning: supervised, unsupervised, semi-supervised and reinforcement learning. These styles have been discussed in great depth in the literature and are included in most introductory lectures on machine learning algorithms. As a recap, the table below summarizes these styles. For a comprehensive mapping of machine learning algorithms to machine learning styles, check out this blog post.

Table 1: Summary of machine learning styles

What is lesser known is the concept of machine learning strategies which apply basic machine learning algorithms in creative ways to solve real-world problems. As a machine learning newbie, this is something that I’ve found to be quite interesting. There are four strategies that I will discuss in this post: online learning, transfer learning, ensemble learning and deep learning.  The good news is that the strategies can be applied with any of the four styles of learning described in the table above.

1. Online learning
   Online learning uses data in motion. These models are updated as data arrives, and they do not store previous data. Some online learning algorithms are also “adaptive,” meaning the models can change over time to follow or track changes in the data. They do so by gradually “forgetting” the outdated data and are well-suited for applications where the environment/model changes with time.Batch (or offline) learning, which uses data at rest, can serve as a warm-start for online learning. In batch learning, one model is trained on all the data at once. Many online learning algorithms use a batch/offline algorithm (on a small batch of data) to initialize (warm-start) the model. This practice can significantly speed up the convergence.
2. Transfer learning
   Transfer learning applies knowledge from one domain to another. It reuses old data, models and parameters for new problems, which is crucial to lifelong learning of machine learning models. Transfer learning comes naturally to people. For example, when learning a new language, we apply the knowledge (vocabulary, grammar, etc.) from languages we already know. The closer the two languages are, the easier it is to transfer the knowledge.
3. Ensemble learning
   While a single learner model uses only one learner (algorithm), ensemble learning uses multiple learner/experts. Common ensemble algorithms include gradient boosting, bootstrap aggregating (bagging), forest (decision forest), stacked ensembles and super learner. Ensemble learning algorithms can combine weaker learners (in many cases learners with prediction accuracy only slightly better than random guessing) to produce a strong, accurate model.
4. Deep learning
   Deep learning contains multiple layers and learns hierarchical or multi-scale representa­tions of the data. This is in contrast to “shallow learning” which is simply the application of standard machine learning modeling algorithms. Shallow learning typically requires feature engineering to ensure appropriate representation of inputs, while deep learning inherently learns the features during training.

Machine learning strategies are one more aspect to consider when applying machine learning algorithms to everyday business problems. To read more about machine learning and its application, download this white paper: The Machine Learning Landscape.