Scaling of model approximation errors and expected entropy distances

Title:

Scaling of model approximation errors and expected entropy distances

Creator:

Montúfar, Guido F. and Rauh, Johannes

Identifier:

https://cdk.lib.cas.cz/client/handle/uuid:dc7cc742-2853-452a-8de3-3a0c59acd439
uuid:dc7cc742-2853-452a-8de3-3a0c59acd439

Subject:

exponential families, KL divergence, MLE, and Dirichlet prior

Type:

model:article and TEXT

Format:

bez média and svazek

Description:

We compute the expected value of the Kullback-Leibler divergence of various fundamental statistical models with respect to Dirichlet priors. For the uniform prior, the expected divergence of any model containing the uniform distribution is bounded by a constant 1−γ. For the models that we consider this bound is approached as the cardinality of the sample space tends to infinity, if the model dimension remains relatively small. For Dirichlet priors with reasonable concentration parameters the expected values of the divergence behave in a similar way. These results serve as a reference to rank the approximation capabilities of other statistical models.

Language:

English

Rights:

http://creativecommons.org/publicdomain/mark/1.0/
policy:public

Coverage:

[234]-245

Source:

Kybernetika | 2014 Volume:50 | Number:2

Harvested from:

CDK

Metadata only:

false