452
May 2001
Amer. J. Agr. Econ.
three response formats may each gener- recoding resulted in the largest welfare esti-
mates(table 5). For our data the actual
choose-one data resulted in the smallest wel-
fare estimate (excluding the negative esti-
mate for ratingsrecoded to rankswithout
ties). In addition, Roe, Boyle, and Teisl found
that ratingsrecoded to rankshad the smallest
confidence intervals, while here the ordered-
probit analysis of the rank data has the
tightest confidence interval around the mean
(−39%, +49%).
ate different statistical information regard-
ing preferences. This suggestive evidence is
confirmed when the likelihood-ratio tests are
conducted. The parametersof the ordered-
probit model using ratings recoded to ranks
(tiesallowed) isisgnificantly different from
the ordered-probit model of the original rank
data. Likewise, the vectors of parameters
from the probit modelsof ratingsrecoded to
choose one (tiesallowed) and ranksrecoded
to choose one are significantly different from
the probit model of the choose-one data.
The lack of statistical differences when ties
are not allowed isismilar to the finding of
Krosnick and Alwin, but the perverse welfare
estimate for ratingsrecoded to ranksand the
large reduction in the number of observations
suggests that the lack of significance may be
a statistical anomaly.
The fact that people use ties in the rat-
ings, but not the other responses formats,
leads one to question whether the absence
of tiesin the ranksand chooes-one data
isreflective of repsondentsmaking more
careful distinctions or making forced, arbi-
trary distinctions. The results of Ben-Akiva,
Morikawa, and Shiroish; and Feather sug-
gest that there may be problems with requir-
ing forced rank. sThisisan useis that
deserves further consideration, even in stud-
ies that present one choice versus the sta-
tus quo (Johnson and Desvousges; Johnston
and Swallow) or two alternativesplusa no
choice (Adamowicz, Louviere, and Williams
and Adamowicz et al.) if respondents are
asked to make multiple, repeated choices.
Our results are also qualitatively similar
to those of Elrod, Louviere, and Davey and
Kalish and Nelson. While the comparisons
with ties result in different structural models
asreflected by the tets of parameter vec-
tors, the confidence intervals on the wel-
fare estimates overlap. We are concerned
about the sizes of the confidence intervals,
but using alternative procedures to calculate
the confidence intervalsreusltsin ismilar or
even larger confidence intervals. Using the
Krinsky–Robb procedure resulted in a confi-
dence interval for the choose-one estimate of
(353, 1635) versus the interval of (451, 1562)
reported in table 3.
Finally, it isworth noting that the dif-
ferencesin the welfare etsimatesfrom the
rating, ranking and choose-one data, as
reported in table 3, are actually smaller
than Welsh and Poe found for contingent-
valuation response formats. Welsh and Poe
report estimated means of $37, $54, and $98,
respectively, for payment-card, open-ended,
and dichotomous-choice data. Thisisa range
of $61 that is 165% of the smallest esti-
mate. The comparable range for our data
is$646, which isonly 68% of the ms allets
estimate ($957 for choose one). It is also
interesting to note that while dichotomous
choice and choose one are the closest
response formats conceptually and analyti-
cally, dichotomouschoice providesthe high-
est contingent-valuation welfare estimate and
choose one provides the lowest conjoint wel-
fare estimate. However, the 95% confidence
interval on Welsh and Poe’s dichotomous-
choice estimate is (−16%, +26%), which
ismuch ms aller than the 95% confidence
interval on our choose-one estimate (+53%,
+63%).
Conclusions
Collectively, our results indicate that conver-
gent validity of ratings, ranks, and choose
one is not established. It is not advisable to
recode ratingsto ranksor chooes one. The
inconsistent use of ties (only in ratings) sug-
gests that the problem may be methodolog-
ical and not a violation of the transitivity
of preferences. The comparison of ranks and
choose one, however, is not complicated by
the issue of ties and recoding ranks to choose
one did not result in the same model as the
actual choose-one data. The proposed statis-
tical efficiency advocated by Mackenzie does
not hold for estimates of Hicksian surplus.
Rankingshave the tightets confidence inter-
val around the mean (−39%, +49%), while
the confidence intervalsfor ratings(−47%,
Roe, Boyle, and Teisl and Stevens, Barrett,
and Willisfound that recoding ratingsto
choose-one data resulted in the smallest wel-
fare estimates, where as in our study this