Browse Teach Curate

Cross-Entropy as an Optimisation Objective

The information-theoretic quantity measuring the cost of describing one probability distribution using another, and its use as a loss function minimised to make a model distribution agree with a target distribution. Covers its relation to KL divergence, why it is minimised rather than maximised, and its role as a fitting criterion for probabilistic embeddings and classifiers.

jeremy Video

Using Cross Entropy Logits in Metric Learning Embeddings using Matplotlib and NumPy

The core principle is that Cross Entropy Logits Loss can be repurposed for metric learning by interpreting pairwise dot products between unit-length embeddings as logits within a softmax distribution…