Publikation
BIRA: Improved Predictive Exchange Word Clustering
Jon Dehdari; Li Ling Tan; Josef van Genabith
In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL). Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL-2016), June 13-15, San Diego, California, USA, Association for Computational Linguistics, 2016.
Zusammenfassung
Word clusters are useful for many NLP tasks including training neural network language models, but current increases in datasets are outpacing the ability of word clusterers to handle them.
Little attention has been paid thus far on inducing high-quality word clusters at a large scale.
The predictive exchange algorithm is quite scalable, but sometimes does not provide as good perplexity as other slower clustering algorithms.
We introduce the bidirectional, interpolated, refining, and alternating (BIRA) predictive exchange algorithm.
It improves upon the predictive exchange algorithm's perplexity by up to 18%, giving it perplexities comparable to the slower two-sided exchange algorithm, and better perplexities than the slower Brown clustering algorithm.
Our BIRA implementation is fast, clustering a 2.5 billion token English News Crawl corpus in 3 hours.
It also reduces machine translation training time while preserving translation quality.
Our implementation is portable and freely available.