Mean and Variance in Models for Count Data
I collect here a few useful results on the mean and variance under various models for count data.
1. Poisson
In a Poisson distribution with parameter \(\mu\), the density is \[ \Pr\{ Y = y \} = \frac{ \mu^y e^{-\mu} }{ y! } \] and thus the probability of zero is \[ \Pr\{ Y = 0 \} = e^{-\mu} \] The expected value and variance are \[ E(Y) = \mu \quad\mbox{and}\quad \operatorname{var}(Y) = \mu \]
2. Negative Binomial
In a negative binomial distribution with parameters \(\mu\) and \(\alpha\), the density is \[ \newcommand{\k}{\alpha^{-1}} \Pr\{ Y = y \} = \frac{\Gamma(y + \k)}{y!\Gamma(\k)} \left( \frac{\k}{\mu + \k} \right) ^{\k} \left( \frac{\mu}{\mu + \k} \right)^y \] In particular, the probability of a zero count simplifies to \[ \Pr\{Y=0\} = (1 + \alpha\mu)^{-1/\alpha} \] The mean is simply \[ E(Y) = \mu \] and the variance is \[ \operatorname{var}(Y) = \mu(1 + \alpha\mu) \] a quadratic function of the mean for \(\alpha > 0\), equal to the Poisson variance if \(\alpha=0\).
The handout at https://grodri.github.io/glms/notes/c4a.pdf uses \(\sigma^2\) for the parameter called \(\alpha\) here, and shows how the negative binomial can be obtained as a gamma mixture of Poissons. It also fits overdispersed Poisson and negative binomial models to data from Long and Freese(2006) and compares the two variance functions.
3. Zero-Inflated Poisson
This is a finite mixture model where \(Y=0\) when \(Z=1\) (the so-called "always zero" condition) and \(Y\) has a Poisson distribution with mean \(\mu\) when \(Z=0\) (which of course includes the possibility of zero).
The probability of zero in this model is \[ \Pr\{ Y=0\} = \pi + (1-\pi) e^{-\mu} \] showing the two sources of zeroes.
The expected count is \[ E(Y) = (1-\pi)\mu \] and the variance is \[ \operatorname{var}(Y) = (1 - \pi) \mu (1 + \mu\pi) \] The simplest way to derive the last result is using the law of iterated expectations.
4. Zero-Inflated Negative Binomial
The same kind of model, but assuming the count in the not-always-zero group has a negative binomial distribution with mean \(\mu\) and overdispersion parameter \(\alpha\).
The probability of zero is then \[ \Pr\{ Y=0\} = \pi + (1-\pi) (1 + \alpha\mu)^{-1/\alpha} \] where the last term is based on the probability of zero in a negative binomial given earlier.
The expected count is \[ E(Y) = (1-\pi)\mu \] and the variance is \[ \operatorname{var}(Y) = (1-\pi)\mu(1 + \mu(\pi+\alpha)) \] You may verify that for \(\alpha=0\) we obtain the zero-inflated Poisson variance.
5. Truncated Poisson
A zero-truncated Poisson distribution is the distribution of a Poisson r.v. conditional on it taking positive values. The conditional density is \[ \Pr\{Y=y|Y>0\} = \frac{f(y)}{1-f(0)}, y=1,2,\dots \] where \(f(y)\) is the unconditional density given in Section 1.
The (conditional) mean is \[ E(Y|Y>0) = \frac{\mu}{1 - e^{-\mu}} \] not to be confused with \(\mu\), which is the mean of the entire Poisson distribution.
The (conditional) variance is best written as \[ \operatorname{var}(Y|Y>0) = \frac{\mu}{1-f(0)}- f(0)[E(Y|Y>0)]^2 \] where \(f(0)\) is the probability of zero as given in Section 1.
This can be a component of a hurdle model, as shown further below.
6. Truncated Negative Binomial
A zero-truncated negative binomial distribution is the distribution of a negative binomial r.v. conditional on it taking positive values. The density has the same form as the Poisson, with the complement of the probability of zero as a normalizing factor.
The (conditional) mean is \[ E(Y|Y>0) = \frac{\mu}{1 - (1+\alpha\mu)^{-1/\alpha}} \] The conditional variance is best written as \[ \operatorname{var}{Y|Y>0} = \frac{\mu(1 + \alpha\mu)}{1-f(0)} - f(0)[E(Y|Y>0)]^2 \] where \(f(0)\) is the probability of zero as given in Section 2.
This can also be used in a hurdle model.
7. A Poisson Hurdle Model
A hurdle model assumes that there is a Bernoulli r.v. that determines whether a count will be zero or positive. If positive, there is a separate truncated Poisson r.v. that determines the actual count. Thus \[ \Pr\{ Y = y \} = \begin{cases} \pi, y = 0, \\ (1-\pi) \mu^y e^{-\mu} /y!, y = 1, 2, \dots \end{cases} \]
Unlike the zero-inflated models discussed earlier, there is only one source of zeroes in this model, and the two equations can be fitted separately, for example using a logit model for zero or positive counts, combined with a truncated Poisson model for positive counts.
The expected count is \[ E(Y) = (1-\pi) E(Y|Y>0) = (1-\pi) \frac{\mu}{1-e^{-\mu}} \] The variance can be written as \[ \operatorname{var}(Y) = (1-\pi)\operatorname{var}(Y|Y>0)+\pi(1-\pi)[E(Y|Y>0)]^2 \] where the expectation and variance on the right-hand-side correspond to the truncated Poisson distribution as given in Section 5.
8. A Negative Binomial Hurdle Model
The final model changes the distribution of the positive counts to be a truncated negative binomial.
The expected count is then \[ E(Y) = (1 - \pi) \frac{\mu}{1 - (1 + \alpha\mu )^{-1/\alpha}} \] The variance can be written, as we did for the Poisson case, as
\[ \operatorname{var}(Y) = (1-\pi)\operatorname{var}(Y|Y>0)+\pi(1-\pi)[E(Y|Y>0)]^2 \] where the mean and variance on the right-hand-side correspond to the truncated negative binomial distribution as given in Section 6.
Appendix: A Note on Derivations
The results on the means follow from first principles and coincide with the results in Long and Freese (2006), see in particular equations (8.6) to (8.9) in pages 382-383. Getting the variances requires a bit more work.
For the zero-inflated models I used the law of iterated expectations, which takes advantage of the fact that the mean and variance in the always zero class are zero, to obtain \[ E(Y) = (1-\pi)E(Y|Z=0) \] and \[ \operatorname{var}(Y) = (1-\pi)\operatorname{var}(Y|Z=0) + \pi(1-pi)[E(Y|Z=0)]^2 \] Results in Sections 3 and 4 follow by substituting the Poisson and negative binomial mean and variance.
For truncation I worked from first principles, to obtain \[ E(Y|Y>0) = \frac{E(Y)}{1-f(0)} \] For the variance note that \(\operatorname{var}(Y|Y>0)=E(Y^2|Y>0)-[E(Y|Y>0)]^2\), and the terms on the right-hand side are easy to obtain. This leads directly to \[ \operatorname{var}(Y|Y>0) = \frac{ E(Y^2) }{ 1-f(0)} - \left[ \frac{ E(Y) }{ 1-f(0) } \right]^2 \] This expression can also be written in terms of the unconditional variance as \[ \operatorname{var}(Y|Y>0) = \frac{ \operatorname{var}(Y) }{ 1-f(0) } - f(0) \left[ \frac{ E(Y) }{ 1-f(0) }\right]^2 \] Plug in in the mean, variance and probability of zero in the Poisson and negative binomial to obtain the results in sections 5 and 6.
For hurdle models I used again the law of iterated expectations to write the moments in terms of the conditional moments when \(Z\) is one and zero. The mean is then \[ E(Y) = (1-\pi)E(Y|Y>0) \] and the variance is \[ \operatorname{var}(Y) = (1-\pi)\operatorname{var}(Y|Y>0) + \pi(1-\pi)[E(Y|Y>0)]^2 \] All we need to do is plug in the truncated mean and variance from above.
All results were verified using simulation, and agree with results in the references were available. Shinkwiler (2016) notes that the variance for the truncated negative binomial in Cameron and Trivedi (1998) was incorrect.
References
Cameron, A. C. and Trivedi, P. K. (1998) Regression Analysis of Count Data. Cambridge: Cambridge University Press.
Long, J.S. and Freese, J (2006) Regression Models for Categorical Dependent Variables Using Stata. College Station, Texas: Stata Press.
Shonkwiler, J. S. (2016) Variance of the Truncated Negative Binomial Distribution. Journal of Econometrics 195:209-210.
