papersTODAY 04:00 UTC
Paper Proposes Method to Restore Zipfian Frequency Patterns in Unsupervised Term Discovery
A revised arXiv paper examines how unsupervised term discovery systems segment unlabelled speech and group the resulting units into candidate word or syllable types. The authors note that real lexicons follow a Zipfian frequency distribution, but the widely used centre-based clustering approach does not reproduce it. Their work introduces a method aimed at recovering that distribution in the discovered lexicon.