This technical guide explores the application of information theory in artificial intelligence, focusing on entropy and KL divergence. It specifically examines the mathematical reasoning behind why cross-entropy serves as the default loss function for many machine learning models.