8(34)
Methodological Issues
The Problem of Evaluation
Ideally, an evaluation of a regulatory framework should be based on an
RCT. For several reasons, RCTs do not represent a viable alternative in
the area of public procurement legislation. Firstly, legislation is national,
which makes it difficult to apply different sets of rules in different parts of
a country. Secondly, those who are subject to a treatment to be evaluated
should preferably be ignorant of their status, which is of course impossible
in this area. Finally, strategic behavior should be expected among both
procuring entities and suppliers, which distorts the outcome of an
experiment.
When the first-best solution of an RCT is not feasible, natural experiments
are often the fallback alternative (Dunning 2012). A possibility that
naturally comes to mind is to use the variation among countries concerning
threshold values in order to estimate the effects on contract values for
similar projects. A moment of reflection shows that this is no more
attractive as a basis for analysis than the previous alternative. It is almost
impossible to define matching projects in different countries that would be
identical in all important dimensions. Further, countries differ in so many
other respects that it is difficult to imagine the elimination of the effect of
all relevant background variables. In summary, neither RCTs nor natural
experiments seem to be feasible as a basis for deciding on an appropriate
threshold value for public procurement.
The road followed in the present analysis is to derive supply curves from
the tenders presented in actual auctions and to estimate a generic supply
curve in order to define the stylized situation on which the choice of
threshold value should be based. Based on this stylized supply curve, a
reference value for the price in the absence of a regulatory framework is
derived. The gain is defined as the difference between this reference value
and the minimum price among the tenders. Two methodological problems
have to be solved in this context. The first is how to derive the underlying
supply curve from the tenders presented. From the suppliers’ point of view,
procurement is a game of competition in which strategic aspects might
influence the behavior of participants.
The second problem is to define a reference value assuming the supply
curve has been derived. Behavior will vary across procurement situations,
procuring officers, sectors, over time, and so on. These problems are
tackled in the following subsections.
