Blog

The $ \phi$ entropy is defined as $ \text{Ent}_{\phi}[X]= \mathbb{E}[\phi (X)]-\phi(\mathbb{E}[ X])$ where $ X$ is a random variabel and $ \phi$ is a convex function ($ \text{Ent}_{\phi}[X] \geq 0$ ). By choosing $ \phi(x)=x^2$ we get $ \text{Ent}_{\phi}[X]=\text{Var}(X)$ . if $ \phi(x)=x\log x$ and $ X=\frac{dv}{d\mu} $ (radom nikodym derivative) we get $ \text{Ent}_{\phi}[X]=Read more