![The Kullback–Leibler divergence between discrete probability distributions](https://blogs.sas.com/content/iml/files/2020/05/KLDiv4.png)
If you have been learning about machine learning or mathematical statistics, you might have heard about the Kullback–Leibler divergence. The Kullback–Leibler divergence is a measure of dissimilarity between two probability distributions. It measures how much one distribution differs from a reference distribution. This article explains the Kullback–Leibler divergence and shows