# Survey Size with a Bayesian Prior

Posted

Putting a proposition before a large population of voters can be expensive, so an organization wishing to do so would like to have a reasonable assurance that a given proposition will pass. One approach is to take a survey of a randomly chosen subset of voters and use the results to estimate the proposition's chances amongst the general population. The larger the survey size and the larger the margin that the proposition passes in the survey, the larger the chances are that the proposition would pass for a general vote. The basic mathematics for this was discussed in Surveying for a Voter Proposition.

The previous discussion assumed that a reasonable choice for the Bayesian prior was the uniform distribution on p (the chance of a yes vote) because it is “unbiased,” in the sense that all possible outcomes were equally likely, including all yes votes to all no votes. Other common priors are commonly used that imply a different sense of “unbiased,” such as the Haldane prior p-1(1-p)-1 and the Jeffreys prior p(1-p). The Haldane prior assumes that, by and large, the entire population is in agreement, so p is most likely either 0 or 1. The Jeffreys prior assumes that each decade of p is equally likely, so a p between 0.001 and 0.01 is as likely as a p between 0.01 and 0.1. The Haldane and Jeffreys priors both imply that marginal survey results (close to 50%) are less likely than the extremes.

Secondly, since the Bayesian prior is intended to encompass prior knowledge of the distribution of p, a pessimist (optimist) has the right to incorporate pessimism (optimism) into the prior. A pessimistic prior would assume that the general population is more likely to vote no. A prior can also be used to assume a certain amount of indifference. For example, an election between identical twins might motivate a prior where the likelihood of strong sentiment toward either candidate is small.

The functional form of Bayesian prior generally used for these sorts of problems is the Beta distribution B(Y,N). The previously assumed uniform prior is then B(1,1), the Jeffreys prior is B(½,½), and the Haldane prior is B(0,0). In general, Y corresponds to increased likelihood of yes votes, while N corresponding to increased likelihood of no votes. The Beta distribution with strictly positive integer Y and N can be modeled using the Pólya urn model.

The distribution of yes votes in a survey is also a Beta distribution,

$P\left(y\text{|}s,p\right)=\left(\begin{array}{c}s\\ y\end{array}\right){p}^{y}{\left(1-p\right)}^{s-y}\text{.}$

(1)

Using Bayes' rule, the distribution of p is therefore

$P\left(p\text{|}s,y\right)=\frac{P\left(p\right)}{P\left(y\right)}\left(\begin{array}{c}s\\ y\end{array}\right){p}^{y}{\left(1-p\right)}^{s-y}\text{.}$

(2)

Here P(p) is the required Bayesian prior. Inserting in a Beta function for the prior gives

$P\left(p\text{|}s,y\right)=\frac{1}{P\left(y\right)}\left(\begin{array}{c}s\\ y\end{array}\right){p}^{y+Y-1}{\left(1-p\right)}^{s-y+N-1}\text{.}$

(3)

Since this is just another Beta distribution, the normalization (terms not depending on p) must be

$P\left(p\text{|}s,y\right)=\frac{{p}^{y+Y-1}{\left(1-p\right)}^{s-y+N-1}}{B\left(y+Y,s-y+N\right)}\text{.}$

(4)

As before, the probability that the vote fails by less than a majority is given by the cumulative distribution function, resulting in the regularized incomplete beta function,

$\mathrm{\delta }=\frac{B\left(½;y+Y,s-y+N\right)}{B\left(y+Y,s-y+N\right)}\text{.}$

(5)

The infinite series expansion (for integer Y and N, and perhaps in general) is

$\frac{B\left(x,\mathrm{\alpha },\mathrm{\beta }\right)}{B\left(\mathrm{\alpha },\mathrm{\beta }\right)}=\sum _{j=\mathrm{\alpha }}^{\mathrm{\alpha }+\mathrm{\beta }-1}\left(\begin{array}{c}\mathrm{\alpha }+\mathrm{\beta }-1\\ j\end{array}\right){x}^{j}{\left(1-x\right)}^{\mathrm{\alpha }+\mathrm{\beta }-1-j}\text{.}$

(6)

Applying this to δ gives

$\mathrm{\delta }=\frac{1}{{2}^{s+Y+N-1}}\sum _{j=y+1}^{s+Y+N-1}\left(\begin{array}{c}s+Y+N-1\\ j\end{array}\right)\text{.}$

(7)

The implications of this formula are interesting. Since the expression (s+N+Y–1) appears everywhere, both the pessimist and the optimist will change the size of a survey in exactly the same way; the direction of bias has no effect, only the magnitude.

For a fixed y (the number of required yes votes) and fixed δ, the Jeffreys and Haldane priors increase the size of the survey by 1 voter and 2 voters, respectively. These priors thus pose no difficulties (which is too bad, since both are somewhat inappropriate for a typical election situation).

Priors with Y+N>2 reduce the survey size relative to the uniform prior. Since the survey size is now smaller, but y has not changed, the required fraction of yes votes in the survey has increased. For a fixed estimate of pest=y/s, a larger survey is required to achieve the same y/s ratio. Luckily, incorporating a Bayesian prior into the computation of δ is a trivial change.

This computation was suggested to me by David Chaum, who wanted to know if a closed-form solution existed. See rsvoting.org for more information.

Categories