Sunday, November 30, 2014

DEFF is an ex-ante measure

The Design Effect is measure of efficiency defined as the ratio between two sampling variances: $$DEFF=\frac{Var1}{Var2}.$$
Before discussing the design effect properties, it is necessary to define what a sampling strategy is. This is a pair composed of an estimator (of a single parameter of interest) and a probability measure (or sample design). In the computation of the design effect, the sampling strategy is univariate in the sense that it is focused in the strategy used to estimate that single parameter of interest.

However, It is of common knowledge that in survey samples, some parameters may be estimated with a generic estimator (for example, the Horvitz-Thompson estimator) and some other parameters with some other estimators (for example, calibration estimators). That is, the estimation stage in a single survey may be given by the use of several estimators.

Now, the design effect is a broadly accepted measure used as a criterium for planning and designing probabilistic samples. Its definition, as the ratio of the variance of a complex sampling strategy Var1 and the variance of the simple random sampling strategy Var2 is the basis of the initial sample size calculation. The design effect does not induce neither the estimation strategy nor the computation of sampling errors. It is on the contrary, after defining the estimator for the main parameter of interest along with defining how to measure the uncertainty generated by the use of that very estimator, then DEFF is computed and reported.

In this regard, it is clear that DEFF is an ex-ante measure (used before collecting primary information) that depends on an estimator and a sampling design. Then, by its ex-ante nature: it must not be used for estimating the sampling errors in a survey, which by definition are ex-post (as they are computed after collection of primary information).

No comments:

Post a Comment