Publication
Whittle Networks: A Deep Likelihood Model for Time Series
Zhongjie Yu; Fabrizio Ventola; Kristian Kersting
In: Marina Meila; Tong Zhang (Hrsg.). Proceedings of the 38th International Conference on Machine Learning. International Conference on Machine Learning (ICML-2021), July 18-24, Pages 12177-12186, Proceedings of Machine Learning Research, Vol. 139, PMLR, 2021.
Abstract
While probabilistic circuits have been extensively explored for tabular data, less attention has been paid to time series. Here, the goal is to estimate joint densities among the entire time series and, in turn, determining, for instance, conditional independence relations between them. To this end, we propose the first probabilistic circuits (PCs) approach for modeling the joint distribution of multivariate time series, called Whittle sum-product networks (WSPNs). WSPNs leverage the Whittle approximation, casting the likelihood in the frequency domain, and place a complex-valued sum-product network, the most prominent PC, over the frequencies. The conditional independence relations among the time series can then be determined efficiently in the spectral domain. Moreover, WSPNs can naturally be placed into the deep neural learning stack for time series, resulting in Whittle Networks, opening the likelihood toolbox for training deep neural models and inspecting their behaviour. Our experiments show that Whittle Networks can indeed capture complex dependencies between time series and provide a useful measure of uncertainty for neural networks.