A linguist trains a transformer model on 3 billion words and observes perplexity at 45. If perplexity is defined as 2^H where H is the causal entropy in bits per word, what is the average entropy H per word in decimal form?

Mon, 04 May 2026 11:13:35 +0000

Content under review