Shared-Parameter Alternating Tweedie Model for Co-occurrence Embeddings
A matrix-factorization model that learns dense word or item embeddings from a high-dimensional co-occurrence count matrix by treating the two low-rank factor matrices as shared parameters updated in alternation and modeling the weighted counts with a Tweedie distribution, which admits positive mass at zero and a mean-dependent variance. It is fit by Fisher scoring with a learning-rate adjustment (or Adam), which the authors show outperforms plain gradient descent and a pseudo-likelihood variant.
2501.00623
A statistical model, SA-Tweedie, for learning dense vector embeddings of words or items directly from a high-dimensional co-occurrence count matrix. Unlike ordinary regression there are no observed c…