Received: February 13, 2025; Accepted: September 22, 2025; Published: August 2, 2026

Farmers, experts and students’ subjective probability distributions on methane emission reductions in livestock farming: An experimental comparison across elicitation methods

Claudia Magnapera^1,* , Austėja Kažemekaitytė² , Roberta Raffaelli³ , Simone Cerroni^3,4

¹ Institute of Natural Resource Sciences (IUNR), Zurich University of Applied Sciences (ZHAW), Wädenswil, Switzerland

² Faculty of Economics and Management, Free University of Bozen-Bolzano: Bolzano, Italy

³ Department of Economics and Management, University of Trento, Trento, Italy

⁴ Centre Agriculture Food Environment (C3A), University of Trento, Trento, Italy

*Corresponding author, e-mail address: claudiamagnapera@gmail.com

Abstract. Subjective probabilities are important determinants of economic choice behaviour, and their elicitation is not trivial. Different methods are available in the literature. This paper compares three of them using an economic experiment with farmers, other experts (animal nutrition and production scientists, vets, as well as animal feed, dairy, and meat company representatives), and agricultural students: the frequency method (FM), the interval method (IM), and the quadratic scoring rule method (QSR). These methods vary in the degree of complexity and saliency of the incentive scheme. Elicited subjective probability distributions refer to methane emissions reductions that can be achieved by changing animal diets in livestock farming. The study investigates whether these methods produce consistent results across methods and participant groups. Subjective probability distributions do not significantly differ across methods and participant groups. Overall, these results support that the use of less complex methods such as the FM and IM may be preferable.

Keywords: subjective probability distributions, quadratic scoring rule, frequency method, interval method, methane emission reductions.

Index

Introduction

1. Literature Review

1.1. Elicitation of farmers’ subjective probability distributions

1.2. Comparing methods for eliciting subjective probabilities distributions

1.3. Frequency method (FM), Interval method (IM), and Quadratic scoring rules (QSR)

2. Empirical application

3. Methods

3.1 Sample characteristics and data collection

3.2 Experimental design

4. Results and discussion

4.1 Comparisons across elicitation methods

4.2 Comparisons across groups

5. Conclusion

References

Introduction

Subjective beliefs help in understanding economic agents’ behaviour, particularly in contexts characterised by risk and uncertainty (Manski, 2004; 2018), and the agri-food sector is a fitting example (Cerroni and Rippo, 2023). Eliciting stakeholders’ beliefs along the agri-food supply chain is essential to disentangle the role these beliefs play in their decision-making processes (Hardaker and Lien, 2010). There is a vast literature showing that farmers’ beliefs are correlated with farmers’ decisions in different contexts: contract farming (e.g., Cerroni, 2020; Cerroni et al., 2023), insurance uptake (e.g., Arata et al., 2023; Čop et al., 2023), risk management (e.g., Meraner and Finger, 2014; van Winsen et al., 2014), technology adoption (e.g., Purvis et al., 1995; Tjernström et al., 2021), and adoption of sustainability practices (e.g., Trujillo-Barrera et al., 2016; Dessart et al., 2019). Therefore, knowing farmers’ beliefs can help policymakers designing information-based interventions that create awareness in the farming community and facilitate a shift towards production patterns that maximize societal welfare (Manski, 2018; Cerroni and Rippo, 2023).

However, eliciting beliefs is not a trivial task. A wide array of methods has been used in the literature, and there is no consensus on the methods that provide the most reliable estimates in mainstream economics (e.g., Trautmann and Van de Kuilen, 2015; Charness et al., 2021). The literature eliciting beliefs in agricultural economics, and more specifically farmers’ and experts’ beliefs, is limited and very few elicitation methods have been employed to retrieve expectations (Cerroni and Rippo, 2023).

This paper contributes to this literature by comparing farmers’ and experts’ beliefs elicited via three different methods, the frequency method (FM) (Hurley and Shogren, 2005), the interval method (IM) (Dufwenberg and Gneezy, 2000), and the quadratic scoring rule (QSR) (Brier, 1950) using a framed field experiment (Harrison and List, 2004). A similar exercise has been proposed on farmers’ risk preferences by Reynaud and Couture (2012).

Most attempts to elicit farmers’ beliefs use Likert scales that elicit qualitative judgments regarding the probability and/or the magnitude of an event (e.g., Meraner et al., 2019; Feyisa et al., 2023). Despite the Likert scales requiring a low cognitive effort from participants, qualitative judgments cannot be incorporated into economic models of decision-making under risk and uncertainty like the subjective expected utility model (SEUT, Savage, 1954). More sophisticated methods can be used to retrieve farmers’ beliefs expressed in a probabilistic fashion. These methods can be paired with the use of monetary incentives (see Cerroni et al., 2023 for a review), and, if properly designed, they allow eliciting subjective probability distributions that can be incorporated in standard models of decision making under uncertainty (e.g., Manski 2004; Cerroni et al., 2012). The literature on the elicitation of farmers’ subjective probability distributions is relatively limited, especially regarding direct comparison of different incentivised methods (Cerroni and Rippo, 2023).

Several incentive-compatible methods can elicit truthful subjective probability distributions. Some studies have explored whether different elicitation methods produce different subjective probability distributions (Trautmann and van de Kuilen, 2015), and we are not aware of any study that has compared farmers’ subjective probability distributions elicited via different incentivized and incentive-compatible mechanisms while linking potential differences to complexity and saliency of the incentive scheme. Complexity refers to the cognitive cost of the elicitation mechanism for participants (Charness et al., 2021). Saliency of the incentive scheme refer to the potential influence that experimental payoffs have on subjects’ actions in the experiment (Charness et al., 2016; Kleinlercher and Stöckl, 2017). This study aims to fill this methodological gap by specifically comparing the FM (Hurley and Shogren, 2005), IM (Dufwenberg and Gneezy, 2000), and QSR (Brier, 1950). QSR has been already used to elicit farmers’ subjective probabilities distributions in the literature (e.g., Cerroni, 2020; Cerroni et al., 2023; Čop et al., 2023). While the FM and IM were used as well (e.g., Menapace et al., 2013; Čop et al., 2023), these methods were never paired with a proper monetary incentive scheme. The FM asks participants to report how frequently different outcomes of a random variable occur. If the reported frequency matches exactly the empirical frequency (known a priori), participants receive a fixed prize x. The IM is equivalent to the FM, except that, in the IM participants are paid a fixed prize x if the reported frequency approximately matches the empirical frequency. This implies that the saliency of pay-off is higher in the IM than in the FM. According to Charness et al. (2021) FM and IM are less complex than QSR.

In particular, our study focuses on farmers and other experts’ beliefs about the potential impact of adopting a new agricultural practice to improve the sustainability of food production. Specifically, it elicits dairy farmers and other experts’ beliefs about the methane reductions that can be achieved when introducing specific feed additives (i.e., essential oils) to bovine diets. Other experts involved in our study are animal nutrition and production scientists, vets, as well as animal feed, dairy, and meat company representatives. We also included a group of postgraduate agricultural students (i.e. master-level), both as future sector participants and to contribute to the ongoing debate on the use of student proxies in experiments. Including agricultural students provides an additional test of whether farmer and agricultural students behaviour are consistent in economic experiments (e.g., Grüner et al., 2022; Höhler et al., 2024). Some evidence finds that students and farmers can behave similarly in generic risk tasks (e.g., Grüner et al., 2022), whereas other work warns that lack of domain expertise may lead to divergent responses (e.g., Höhler et al., 2024). Students offer practical advantages for experiments due to their accessibility and learning capacity (Grüner, 2022), but their differences from farmers in age, income, and sector-specific experience may limit how well their responses generalize to the farming population (e.g, Belot et al., 2015). Testing whether student-elicited distributions align with those of practitioners therefore provides valuable evidence on the external validity of experimental results in the agricultural context.

The emphasis on zootechnical feed additives, specifically essential oils, and their impact on reducing methane emissions from livestock production stems from the fact that livestock contributes 14.5% of the total greenhouse gas emissions (GHGEs) from anthropogenic activities (FAO, 2017). Most livestock-related methane emissions are due to enteric fermentation, and the use of essential oils in animal diets can help reduce the sector’s carbon footprint (Hristov et al., 2013; Honan et al., 2021). This aligns with EU policies, including the EU Green Deal, the Farm-to-Fork strategy, and the planned revision of the Industrial Emission Directive, which are pushing livestock farmers to lower their carbon footprint. Consequently, farmers are encouraged to adopt innovations that support this goal (Block et al., 2024). The introduction of feed additives such as essential oils is a readily available and cost-effective option to reduce GHGEs at the farm level (Belanche et al., 2020).

In our experiment, each participant provides a subjective probability distribution three times, each time facing a different elicitation mechanism. The order of exposure is randomized. Results show that farmers and other experts’ subjective probability distributions do not differ across methods, suggesting that it is preferable to implement simpler methods over complex ones. Farmers, other experts, and students’ subjective probability distributions are similar.

This paper is structured as follows: we start with a review of the elicitation of beliefs in the agricultural sector and a description of the three methods employed in this study. Next, we present the empirical application, describe our sample, the experimental design, and the main findings. Finally, we discuss the implications of these findings.

1. Literature Review

1.1. Elicitation of farmers’ subjective probability distributions

Farmers’ probabilistic beliefs can be elicited using many different approaches. Most studies elicit probabilistic qualitative judgments about the occurrence of given events using Likert scales. Among others, events refer to climatic adverse events, loss in production or income, and efficacy of an innovative risk-reducing technology or insurance product in reducing production or income losses (see Meraner et al., 2019; Feyisa et al., 2023; Villacis et al., 2023). A typical question would be asking how likely an x% loss in production due to climatic events is on a five-point scale from 1 (very unlikely) to 5 (very likely). Probabilistic qualitative judgments are relatively easy to elicit as they require a relatively low design effort from a researcher’s perspective and a relatively low cognitive effort from a participant’s perspective (e.g., Delavande et al., 2011). However, these strengths are counterbalanced by some limitations. Likert scales are not incentive-compatible, meaning that they do not induce respondents to truthfully reveal their beliefs. In addition, probabilistic qualitative judgments do not facilitate inter- and intra-personal comparisons (e.g. Manski, 2004). Finally, they cannot be incorporated into decision making models of decision making under risk and uncertainty (e.g. Cerroni et al., 2012).

A useful way to circumvent some of these disadvantages is the use of direct elicitation methods (or direct introspection). Participants are directly asked to state the probability of an event to occur. If several direct questions are asked about different states of the world, it is possible to map the entire probability distribution of an outcome variable. This approach has been widely used to elicit farmers’ subjective probability in the literature in both developing and developed countries (see Hardaker and Lien, 2010; Cerroni, 2020; Cerroni and Rippo, 2023 for reviews). Direct methods are generally paired with the use of visual aids that facilitate farmers’ understanding of the elicitation task. An example would be asking farmers to allocate a given number of tokens (generally 100) among different states of the world that may characterize a given outcome variable, suggesting that each token is equivalent to a probability point. Čop et al. (2023) and Fezzi et al. (2021) used this approach to elicit farmers’ subjective probability distributions regarding farm income losses in Croatia and regarding loss in production due to extreme climatic events in Italy. Direct methods are widely used to elicit subjective probability distributions in developing countries (Delavande et al., 2011; Cerroni and Rippo, 2023). While direct introspection is relatively straightforward to implement for researchers, it might be cognitively demanding and challenging for respondents who are not always able and willing to express the likelihood of events in a probabilistic way (Manski, 2004). New web interfaces were recently created to facilitate farmers’ understanding of direct elicitation methods (see Crosetto and de Haan, 2023). Direct methods are not generally incentive-compatible; hence, they do not overcome the issue of the potential of misreported probabilities (Trautmann and van de Kuilen, 2015; Cerroni, 2020; Charness et al., 2021).

To overcome the latter issue and elicit truthful beliefs, it is possible to use indirect methods. If associated with a proper incentive scheme, these methods are incentive compatible. These methods ask respondents to make choices between lotteries and allow retrieving subjective probabilities distributions based on respondents’ gambling behaviour. For a comprehensive review of these methods, we refer to Cerroni and Rippo (2023). To date, only a very few studies have used these methods to elicit farmer’s subjective probability distributions. All these studies have used proper scoring rules. Linear scoring rules have been used by Grisley and Kellong (1983) and Smith and Mandac (1995) in developing countries. Quadratic scoring rules have been used by Cerroni (2020) in Scotland, Cerroni et al. (2023) in Zimbabwe, and Čop et al. (2023) in Croatia. These methods are not free from criticism. A recent study by Danz et al. (2022) argued that individuals’ actual behaviour in response to incentives may lead respondents to deviate from truth-telling when eliciting beliefs.

1.2. Comparing methods for eliciting subjective probabilities distributions

Reliable subjective probability distributions are often needed in economic modelling to explain or predict human behaviour. Hence, it is essential to have reliable measures of individuals’ subjective beliefs. Several methods are available and choosing the right one for eliciting truthful beliefs is a relevant issue (Charness et al., 2021). The reliability of elicited subjective probability distributions is contingent upon the methodology employed and the participants’ comprehension of it.

Charness et al. (2021) ranked the methods available to elicit beliefs according to a scale based on complexity, which is the subjective burden a rule places on a decision-maker (Oprea, 2020). In our study, complexity pertains to the extent of the understanding required to generate utility-maximising belief reports. Some methods, such as the FM and the IM, have a low degree of complexity. In contrast, some others, such as QSR, entail a higher degree of complexity and may not be fully understood by participants at the cost of truthful belief reports.

To the best of our knowledge, there is limited research that compares complex methods to simpler ones, and this paper aims to fill the gap. Trautmann and Van de Kuilen (2015) compared subjective probabilities related to whether a participant accepted or rejected an allocation submitted by a proposer in a simple two-player ultimatum game using different elicitation techniques. Specifically, they tested non-incentivised direct method (i.e., direct introspection) against incentivised and incentive compatible methods such as outcome-matching, probability matching, and QSR with or without corrections for deviations from risk neutrality. Internal validity was investigated by testing for additivity and consistency of players’ beliefs with their allocation decisions during the game, while external validity was investigated by testing whether elicited beliefs matched objective probability measured at the end of the game (i.e. accuracy). Results suggest that incentivised and incentive-compatible methods predict better respondents’ behaviour during the game than non-incentivised introspection without necessarily having better performances in terms of additivity and accuracy. Schlag et al. (2015) and Charness et al. (2021) discuss the strengths and limitations of different elicitation mechanisms and conclude that more research in this direction is needed. For a comprehensive discussion of the most suitable methods to elicit subjective probability distributions for agricultural research, which particularly focuses on developing countries, we suggest reading Cerroni and Rippo (2023). We contribute to this literature by comparing three subjective probability distribution methods that differ in terms of complexity, and saliency of the incentive scheme.

1.3. Frequency method (FM), Interval method (IM), and Quadratic scoring rules (QSR)

The FM is among the least complex methods. Participants are asked to guess the empirical frequency of a random variable, either when the random variable has two possible outcomes (Hurley and Shogren, 2005) or multiple ones (Schlag and Tremewan, 2021). In both cases, participants are rewarded with a fixed prize x if and only if their reported frequencies match exactly the empirical frequencies. This rewarding mechanism can be applied either when empirical frequencies are observable or not yet observable. In the latter case, a few options to match unobservable frequencies would be by asking experts about it, or by using forecasts based on historical data. The main advantage of the method is that it is very easy to implement and easy to understand, in fact, it does not require any mathematical calculations, and natural frequencies are processed better by individuals than other formats (e.g., numbers, percentages) (Schlag and Tremewan, 2021). These characteristics (i.e., better understanding, fast responding, and less likelihood of choosing focal points) make the frequency method FM theoretically robust to deviations from risk neutrality and therefore more likely to be empirically incentive compatible (Schlag and Tremewan, 2021). Nevertheless, when faced with situations where there are many possible outcomes and unfamiliar topics, people may feel that their chances of achieving a reward are low because they must make correct predictions for each potential outcome. As a result, they may tend to misreport their beliefs, as they perceive the likelihood of being rewarded is tied to accurate predictions.

This major disadvantage of the FM can be alleviated by the IM, which is de facto a variation of the latter. It is still ranked as a simple method. Yet, it mitigates the low likelihood of being rewarded by allowing participants to guess correctly when their reported frequencies are approximately equal to the underlying ones. The primary challenge is in establishing an appropriate margin of error to not incentivize under- and over-reporting. While this approach significantly reduces complexity by retaining similar characteristics to the previous method, the literature reflects a scarcity of applications utilising the IM (e.g., Dufwenberg and Gneezy, 2000; Charness and Dufwenberg, 2006). A similar approach is eliciting beliefs using most-likely intervals (MLI) (Schlag and van der Weele, 2015). The interval method aims at providing a probability distribution within a certain percentage range of the underlying distribution, while MLI consists of specifying an interval that contains the most likely outcomes. This specification changes how the reward is calculated. While in the IM, participants are rewarded based on how well their entire probability distribution aligns with the underlying probability distribution within a certain margin of error, in the MLI participants are typically rewarded based on the width of the interval they provide, with narrower intervals being rewarded more favourably.

Moving to more complex methods, QSR is a frequently used incentive-compatible method to elicit beliefs, which first appeared in Brier (1950). It is a specification of the family of the proper scoring rules, which assign a numerical score to the individuals’ probabilistic predictions regarding future outcomes, rewarding them for the accuracy of their reports, while having severe punishments if they miss the true allocation in that interval (Harrison et al., 2017). The payoff score is defined as

$S=\left(2\times r_k\right)-{\sum }_{i=1,..,K}{\left(r_i\right)}^2$ (Eq. 1)

where $K$ corresponds to the number of intervals, $r_k$ are the reports of the likelihood that the event (i.e., methane reduction) falls in the interval $\mathrm{k}=1,\mathrm{}\dots ,\mathrm{K}$ . Each $r_k\ge 0,\forall k$ and ${\sum }_{i=1,..,K}\left(r_i\right)=1$ Matheson and Winkler, 1976). The method is incentive-compatible under the assumption of risk neutrality and expected utility maximisation (Kadane and Wrinkler, 1988; Schlag and van der Weele, 2015; Winkler and Murphy, 1970). Under these assumptions, an individual who is sufficiently risk averse would be drawn to report a probability distribution close to a uniform one, or report probabilities equal to 50% in the case of binary events (Andersen et al., 2014). It is good practice to accompany the task with measurements of individuals’ risk attitudes, to control for the initial assumption of risk neutrality (Andersen et al., 2014; Offerman et al., 2009). However, Harrison et al. (2017) demonstrated that having a large enough set of intervals (i.e. states of the world k) allows a reliable elicitation of subjective probability distributions without undertaking calibration for risk attitudes. QSR is classified as a complex elicitation method by Charness et al. (2021). Participants may have difficulties in understanding the multiple layering of the QSR, which, accompanied by a substantial aversion to complexity (Oprea, 2020), might impair truthful reports.

2. Empirical application

The Global Methane Pledge at a global level, the EU Green Deal, the Farm to Fork Strategy, and the EU Methane Strategy at a European level, are among the variety of initiatives currently underway to mitigate the significant climate impact of methane (CH₄) from one of its predominant sources, that is the agricultural sector. Mitigating methane emissions would allow for short-term effects on the climate, due to the evidence that CH₄ has about 82 times the climate-changing impact of carbon dioxide (CO₂) over 20 years (Smith et al., 2021). In the agricultural sector, animal production is a significant contributor, particularly through ruminants’ enteric fermentation, which alone accounted for 69% of CH₄ emissions within the sector at the European level (EEA, 2020).

There is a vast range of mitigation technologies available in agriculture, such as production intensification, dietary and rumen manipulation, and selection of low-CH₄-producing animals (Beauchemin et al., 2022). Some of these may be available at low cost and bring additional benefits for farms, but barriers such as insufficient knowledge and expertise and lack of financial incentives (Long et al., 2016) need to be addressed since their uptake is uneven across the European Union.

Different methane-reducing technologies have different potential for mitigation (Arndt et al., 2022). In our empirical application, we consider the wide class of CH₄-reducing essential oils that can be categorised as zootechnical feed additives for animal diets. Including essential oils in bovine feed is a practice that is effective in the short term, and studies suggest it is leading to a positive impact on feed efficiency and hence milk production (Wells, 2024). Essential oils, derived through a steam distillation process of various parts of plant species, act as rumen modifiers and can cause an inhibition of the methanogenesis process in ruminants (Glasson, 2022). Many studies show that, even though there is a substantial reduction in enteric CH₄, the results are variable (Honan et al., 2021; Hristov et al., 2013). Arndt et al. (2022) built a database reporting the effects of various mitigation strategies, including 115 studies on additives. Feed additives, such as essential oils, have a mean value of the reduction in daily CH₄ emissions of -8.3%, with a 95% confidence interval (-9.8%, -6.8%). The variation depends on several factors such as the environment, the animal’s diet and health, and others.

3. Methods

3.1 Sample characteristics and data collection

The sample consists of 60 individuals, evenly distributed between two groups: 20 dairy farmers from various Italian regions, 10 livestock experts, and 30 master’s students from various agri-food faculties.¹ For simplicity, the group of farmers and experts is going to be called the “dairy sector” throughout the paper. The sample was recruited using a snowball sampling method, a non-probabilistic approach where initial participants were asked to refer others who fit the study criteria. Data was gathered during the summer and the autumn of 2023 primarily through one-to-one with farmers and experts or group experimental sessions with students on the Zoom platform.² Participants were asked to report their probability on the web interface generated via the software o-Tree (Chen et al., 2016). Ethical clearance was granted by the Ethical Review Board at the University of Trento (Italy).³

3.2 Experimental design

In this study, farmers, experts, and students were asked to report their subjective probability distribution regarding the reduction (in percentage value) of enteric methane emissions which could be achieved using essential oils in bovine diets in a dairy farm across three consecutive tasks. Specifically, each participant completed the same task three times, each time under different elicitation methods: FM, IM, and QSR. The only difference among the three tasks was how the monetary reward was attributed. To control for order effects, the order of the task was randomized. Participants were divided into two groups. Specifically, one group was exposed to tasks in the following order: i) FM, ii) IM, and iii) QSR, and the other group as follows: i) QSR, ii) FM, and iii) IM. The FM was always presented before the IM to facilitate the understanding of the IM, which is a variation of the FM.

Before the elicitation of the subjective probability distributions, participants were provided with the following standardised information: i) a description of policy initiatives aiming at reducing methane emissions from the agricultural sector, ii) a description of strategies to reduce methane emissions in dairy farming with a focus on essential oils, and iii) a description of benefit and opportunities of using essential oils. Experimental instructions are available in the Supplementary materials.

Regardless of the elicitation mechanism, participants were informed that the reduction in methane emissions can fall in any percentage value between -6.6% and -10% given the information retrieved from the literature (Arndt et al., 2022). Using the mean and confidence interval reported in Arndt et al. (2022), we approximated an empirical benchmark distribution by fitting a normal distribution, against which participants’ subjective probabilities were compared. The normal distribution can be a good approximation because it provides a continuous, two‐parameter reference distribution for comparing participants’ subjective probabilities without further assumptions. Participants were instructed to allocate 70 tokens across seven predefined intervals each representing a range of methane emission reduction percentages using the interface depicted in Figure 1. These intervals included the following percentages of reduction: (-10%, -9.6%), (-9.5%, -9.1%), (-9%, -8.6%), (-8.5%, -8.1%), (-8%, -7.6%), (-7.5%, -7.1%), and (-7%, -6.6%). They were informed that the number of tokens allocated to each interval should reflect their perceived likelihood of the corresponding methane emission reduction occurring in the future. A higher allocation of tokens to a particular interval indicates a higher subjective likelihood of that reduction range according to the participant’s beliefs.

The experiment was incentivised. Participants did not receive any participation fee, but they could get a reward of up to €25, knowing that, at the end of the session, one of the three tasks would have been randomly extracted and used to determine their final payoff. Each elicitation method has a different incentivisation scheme. The FM pays €25 if and only if the probability distribution matches exactly the empirical distribution, otherwise it pays €0. Matching exactly meant that the number of tokens allocated by the participant perfectly corresponded to the probabilities defined by the empirical distribution. The IM pays €25 if and only if the subjective probability distribution matches the empirical distribution with an error margin equal to 10% in each interval, otherwise it pays €0. The payment scheme of the QSR is completely different as the QSR allows for a more continuous spectrum of payoffs as explained in Eq. 1.

4. Results and discussion

4.1 Comparisons across elicitation methods

Figure 2 shows the subjective probability distributions across elicitation methods when the sample is pooled (dairy sector plus students).

We test differences in subjective probability distributions across elicitation methods using the Wilcoxon signed-rank test for paired samples.⁴ Results suggest no statistically significant differences across methods when the sample is pooled (dairy sector plus students). Results hold when we estimate a generalized linear model with an underlying binomial distribution for our dependent variable and a logistic link function a la Papke and Woolridge (1996) (see Table 1). This modelling approach is particularly well suited when the state space of the dependent variable ranges from 0 to 1 as is the case in our study (e.g. Cerroni et al., 2012). Our dependent variable $p_i{,}_k$ is the probability associated with each participant $i$ to each interval $k$ (i.e. state of the world) shown in Figure 2.⁵ Our discrete independent variables are associated with the elicitation mechanism: $F{M}_k$ equals 1 if the probability was elicited using the FM, and $I{M}_k$ equals 1 if the probability was elicited using the IM. The variable $Q{SR}_k$ is used as a baseline and equals 1 if the probability was elicited using the QSR.⁶ Standard errors are clustered at the individual level. The model is specified as follows:

$P_{i,k}=\alpha +{\beta }_{FM}F{M}_i{,}_k+{\beta }_{IM}I{M}_i{,}_k+{\epsilon }_i{,}_k$ (Eq. 2)

These results suggest that different methods elicit equivalent subjective probability distributions. We can conclude that less complex (i.e. cognitively demanding) elicitation methods (FM and IM) are performing equivalently to a complex elicitation method such as the QSR. Subjective probability distributions are stable regardless of the use of completely different incentive schemes (FM and IM vs QSR). In addition, the difference in the saliency of payoffs between FM and IM does not produce any significant effect on the elicited subjective probability distributions. Therefore, we conclude that the use of less cognitively demanding techniques like IM and IF might be a viable approach for the elicitation of subjective probability distributions. We acknowledge that failure to detect differences across methods within subjects may be because subjects tend to confirm their beliefs over time, showing a sort of confirmatory bias (e.g., Rabin and Schrag, 1999; Zappalà, 2023). We evaluated the presence of confirmatory bias by taking advantage of the task randomization. More specifically, we are comparing subjective probability distributions elicited via the FM and the QSR when these were the first methods faced by our participants. This between-subject analysis rules out the presence of confirmatory bias. Results from the Kolmogorov-Smirnov test suggest that there is no difference in subjective probabilities across groups, indicating that subjective probability distributions are similar across methods even when confirmatory bias does not play any role.⁷ Results hold when estimating generalised linear models (Eq. 3) (see Table 2), as above. In this case $F{M\_first}_k$ equals 1 if the probability was elicited using the FM as a first task, and $QSR{\_first}_k$ equals 1 if the probability was elicited using the QSR as the first task. The latter is used as a baseline.

$P_{i,k}=\alpha +{\beta }_{F{M_{first}}_k}F{M_{first}}_k+{\epsilon }_i{,}_k$ (Eq. 3)

4.2 Comparisons across groups

Figure 3 and Figure 4 show subjective probability distributions across elicitation methods for the dairy sector (farmers plus experts) and students, respectively. These provide insights into probabilistic beliefs of dairy farmers, experts, and students across different outcomes. It is noticeable that, overall, the probability mass is concentrated in the three central intervals, suggesting that these groups generally expect moderate to average outcomes of future reduction in methane emissions in dairy farming. This is a typical stance when evidence or confidence in the topic is neither very strong nor very weak. However, it is interesting to note that students assign a moderate probability (around 10%) to the scenario with the lowest reduction in methane emissions. This probability rises to approximately 15% and 20% when assessed by farmers and experts, indicating a degree of scepticism about the efficacy of essential oils in reducing methane emissions, and signalling a need for further research on the topic.

Differences in subjective probability distributions across groups (dairy sector vs students) were assessed using the Kolmogorov-Smirnov test for the between-subjects comparison.⁸ Results indicate that there are no significant differences in token allocation across groups. Equivalent results are obtained when we estimate a generalized linear model (binomial distribution and logistic link function as above) where the dependent variable $p_{i,k}$ is the probability associated with each participant $i$ to each interval $k$ (i.e., state of the world) shown in Figure 1 (see Table 3). Our discrete independent variable is $Dairy\_Sec{tor}_k$ that equals 1 if the probability was elicited from a farmer or an expert (otherwise = 0), while students were used as a baseline. Standard errors are clustered at the individual level. The model is specified as follows:

$P_{i,k}=\alpha +{\beta }_{Dairy\_Sector}Dairy\_Sec{tor}_{i,k}+{\epsilon }_i{,}_k$ (Eq. 4)

These results may be driven by the fact that agricultural students are very aware of climate change mitigation strategies that can be implemented at the farm level and hence they are as knowledgeable and aware as farmers and experts regarding the specific topic under investigation. On the other hand, it might be possible that, given the novelty of the topic and the high degree of uncertainty related to the efficacy of introducing feed additives to reduce methane emissions from livestock production, both students, farmers and experts have little knowledge and awareness of the problem.

5. Conclusion

This study compares three indirect methods – FM, IM, and QSR – to elicit farmers’, other experts’ and students’ subjective probability distributions about a specific agricultural outcome using a framed economic experiment. The outcome variable considered is the methane emission reduction which can be achieved by the introduction of zootechnical feed additives to bovine diets. Feed additives, more specifically essential oils, are readily available and cost-effective solutions to reduce the carbon footprint of animal production (Belanche et al., 2020). As the EU is designing policies that require farmers to reduce their GHGEs (e.g., the Green Deal, the Farm-to-fork Strategy, and the Industrial Emission Directive), understanding farmers’ and other stakeholders’ beliefs regarding the effectiveness of GHGE-reducing innovations can help anticipate adoption behaviour.

The three elicitation methods differ in two main features: the level of complexity (i.e., the cognitive cost of the elicitation mechanism for participants) and the saliency of the incentive scheme (i.e., the ability of the incentive scheme to alter behavior). Following Charness et al. (2021), we classify the QSR as complex, and the FM and IM as simple. Similarly, the IM and the QSR have more salient incentive schemes than the FM. The QSR has been recently used to elicit farmers’ subjective probability distributions using proper incentive schemes (Cerroni et al., 2020; Cerroni et al., 2023; Čop et al., 2023), while the FM and IM have been applied without incentivization in earlier studies (e.g., Menapace et al., 2013; Čop et al., 2023).

In our study, each participant was asked to provide a subjective probability distribution when exposed to these three different methods. The order of the tasks was randomized to mitigate order effects. Our sample consists of farmers, other experts (i.e., animal nutrition and production scientists, veterinaries as well as feed, dairy, and meat company representatives), and postgraduate agricultural students. Our results indicate that, on average, the subjective probability distributions elicited using the three methods do not differ significantly, implying a consistent elicitation through tasks. This leads us to suggest that less complex elicitation methods (FM and IM) may be preferable when feasible in experimental studies. No difference has been highlighted between groups. In fact, subjective probability distributions were elicited consistently across the dairy sector (i.e., farmers and experts), and agricultural students. Dairy farmers, experts, and students generally expect moderate to average reductions in methane emissions from essential oils. However, while students see a moderate chance of achieving the lowest methane reduction possible, dairy farmers and experts are more sceptical, highlighting the need for further research on the efficacy of essential oils.

Little variation across groups may suggest that agricultural students are as knowledgeable as experts and farmers about reductions in methane emission that can be achieved using essential oils. Nevertheless, an alternative explanation is that, given the high degree of uncertainty surrounding the outcome variable, both the dairy sector (farmers and experts) and students form similar uncertain beliefs.

In this regard, as the exercise focused on a highly uncertain outcome, that is methane reductions achievable through the use of essential oils, about which even experts hold wide-ranging opinions, it remains an open question whether method performance would remain equivalent when eliciting beliefs about more familiar or less ambiguous outcomes (for example, crop yields or weather events).

Although our primary objective was methodological (comparing FM, IM, and QSR) rather than producing broadly representative estimates, the modest sample size may still cast doubt on external validity. To reinforce and extend these findings, future research should replicate the experiment with larger, stratified samples spanning different regions, crop systems, and farm structures.

In practical terms, our results offer clear guidance for researchers in agricultural and environmental economics: simpler, low-burden elicitation methods can be as effective as complex scoring rules, easing implementation without compromising data quality. To build on this work, future research should (1) replicate the comparison across different domains of uncertainty, (2) employ larger, stratified samples to bolster statistical power, and (3) explore alternative interfaces or incentive schemes. Such extensions will help sharpen our understanding of how methodological choices shape the elicitation of subjective probabilities in decision-making under uncertainty.

While this study is primarily methodological, understanding how farmers and other stakeholders form beliefs may indirectly support the design of information-based interventions. For instance, clearer communication (e.g., informational workshops, webinars or farmer-friendly explanatory materials) or demonstration activities (such as on-farm trials where farmers can directly observe outcomes) could help reduce scepticism toward innovations such as feed additives. Nevertheless, such applications fall beyond the scope of this study and would require further targeted research.

References

Andersen, S., Fountain, J., Harrison, G. W., et al. (2014). Estimating subjective probabilities. Journal of Risk and Uncertainty, 48: 207–229. https://doi.org/10.1007/s11166-014-9194-z

Arata, L., Cerroni, S., Santeramo, F. G., Trestini, S., Severini, S. (2023). Towards a holistic approach to sustainable risk management in agriculture in the EU: A literature review. Bio-Based and Applied Economics, 12(3): 165-182. https://doi.org/10.36253/bae-14492

Arndt, C., Hristov, A. N., Price, W. J., McClelland, S. C., Pelaez, A. M., Cueva, S. F., … & Yu, Z. (2022). Full adoption of the most effective strategies to mitigate methane emissions by ruminants can help meet the 1.5°C target by 2030 but not 2050. Proceedings of the National Academy of Sciences, 119(20). https://doi.org/10.1073/pnas.2111294119

Beauchemin, K. A., Ungerfeld, E. M., Abdalla, A. L., Alvarez, C., Arndt, C., Becquet, P., … & Kebreab, E. (2022). Invited review: Current enteric methane mitigation options. Journal of Dairy Science, 105(12). https://doi.org/10.3168/jds.2022-22091

Belanche, A., Newbold, C. J., Morgavi, D. P., Bach, A., Zweifel, B., & Yáñez-Ruiz, D. R. (2020). A meta-analysis describing the effects of the essential oils blend Agolin Ruminant on performance, rumen fermentation and methane emissions in dairy cows. Animals, 10(4): 620. https://doi.org/10.3390/ani10040620

Belot, M., Duch, R. and Miller, L. (2015). A comprehensive comparison of students and non-students in classic experimental games, Journal of Economic Behavior & Organization 113, 26–33. https://doi.org/10.1016/j.jebo.2015.02.007

Block, J. B., Michels, M., Mußhoff, O., & Hermann, D. (2024). How to reduce the carbon footprint of the agricultural sector? Factors influencing farmers’ decision to participate in carbon sequestration programs. Journal of Environmental Management, 359, https://doi.org/10.1016/j.jenvman.2024.121019

Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1): 1-3. https://doi.org/10.1175/1520-0493(1950)078%3C0001:vofeit%3E2.0.co;2

Cerroni, S., Bozzola, M., & Rippo, R. (2023). Eliciting farmers’ risk and uncertainty preferences using contextualized field experiments in developing countries: A pilot study. In Managing Risk in Agriculture: A Development Perspective, 85-100. GB: CABI. https://doi.org/10.1079/9781800622289.0006

Cerroni, S. (2020). Eliciting farmers’ subjective probabilities, risk, and uncertainty preferences using contextualized field experiments. Agricultural Economics, 51(5): 707-724. https://doi.org/10.1111/agec.12587

Cerroni, S., Notaro, S., & Shaw, W. D. (2012). Eliciting and estimating valid subjective probabilities: An experimental investigation of the exchangeability method. Journal of Economic Behavior & Organization, 84(1): 201-215. https://doi.org/10.1016/j.jebo.2012.08.001

Cerroni, S., & Rippo, R. (2023). Subjective probabilities and farmers’ decision-making in developing countries. In Managing Risk in Agriculture: A Development Perspective : 35-49. GB: CABI. https://doi.org/10.1079/9781800622289.0003

Charness, G., & Dufwenberg, M. (2006). Promises and partnership. Econometrica, 74(6): 1579-1601. https://doi.org/10.1111/j.1468-0262.2006.00719.x

Charness, G., Gneezy, U., & Halladay, B. (2016). Experimental methods: Pay one or pay all. Journal of Economic Behavior & Organization, 131: 141-150. https://doi.org/10.1016/j.jebo.2016.08.010

Charness, G., Gneezy, U., & Rasocha, V. (2021). Experimental methods: Eliciting beliefs. Journal of Economic Behavior & Organization, 189: 234-256. https://doi.org/10.1016/j.jebo.2021.06.032

Chen, D. L., Schonger, M., & Wickens, C. (2016). oTree—An open-source platform for laboratory, online, and field experiments. Journal of Behavioral and Experimental Finance, 9: 88-97. https://doi.org/10.1016/j.jbef.2015.12.001

Čop, T., Cerroni, S., & Njavro, M. (2023). Farmers’ acceptance of the income stabilisation tool: a discrete choice experiment application. European review of agricultural economics, 50(4): 1520-1546. https://doi.org/10.1093/erae/jbad025

Crosetto, P., & De Haan, T. (2023). Comparing input interfaces to elicit belief distributions. Judgment and Decision Making, 18: e27. https://doi.org/10.1017/jdm.2023.21

Danz, D., Vesterlund, L., & Wilson, A. J. (2022). Belief elicitation and behavioral incentive compatibility. American Economic Review, 112(9): 2851-2883. https://doi.org/10.1257/aer.20201248

Delavande, A., Giné, X., & McKenzie, D. (2011). Measuring subjective expectations in developing countries: A critical review and new evidence. Journal of Development Economics, 94(2): 151-163. https://doi.org/10.1016/j.jdeveco.2010.01.008

Dessart, F. J., Barreiro-Hurlé, J., & Van Bavel, R. (2019). Behavioural factors affecting the adoption of sustainable farming practices: a policy-oriented review. European Review of Agricultural Economics, 46(3): 417-471. https://doi.org/10.1093/erae/jbz019

Dufwenberg, M., & Gneezy, U. (2000). Measuring beliefs in an experimental lost wallet game. Games and Economic Behavior, 30(2): 163-182. https://doi.org/10.1006/game.1999.0715

Feyisa, A. D., Maertens, M., & de Mey, Y. (2023). Relating risk preferences and risk perceptions over different agricultural risk domains: Insights from Ethiopia. World Development, 162. https://doi.org/10.1016/j.worlddev.2022.106137

Fezzi, C., Menapace, L., & Raffaelli, R. (2021). Estimating risk preferences integrating insurance choices with subjective beliefs. European Economic Review, 135. https://doi.org/10.1016/j.euroecorev.2021.103717

Glasson, C. R., Kinley, R. D., de Nys, R., King, N., Adams, S. L., Packer, M. A., … & Magnusson, M. (2022). Benefits and risks of including the bromoform containing seaweed Asparagopsis in feed for the reduction of methane production from ruminants. Algal Research, 64. https://doi.org/10.1016/j.algal.2022.102673

Grisley, W., & Kellogg, E. D. (1983). Farmers’ subjective probabilities in Northern Thailand: an elicitation analysis. American Journal of Agricultural Economics, 65(1): 74-82. https://doi.org/10.2307/1240339

Grüner, S., Lehberger, M., Hirschauer, N., & Mußhoff, O. (2022). How (un)informative are experiments with students for other social groups? A study of agricultural students and farmers. Australian Journal of Agricultural and Resource Economics, 66(3): 471-504. https://doi.org/10.1111/1467-8489.12485

Hardaker, J. B., & Lien, G. (2010). Probabilities for decision analysis in agriculture and rural resource economics: the need for a paradigm change. Agricultural Systems, 103, 345–350. https://doi.org/10.1016/j.agsy.2010.01.001

Harrison, G. W., & List, J. A. (2004). Field experiments. Journal of Economic Literature, 42(4): 1009–1055. https://doi.org/10.1257/0022051043004577

Harrison, G. W., Martínez-Correa, J., Swarthout, J. T., & Ulm, E. R. (2017). Scoring rules for subjective probability distributions. Journal of Economic Behavior & Organization, 134: 430-448. https://doi.org/10.1016/j.jebo.2016.12.001

Honan, M., Feng, X., Tricarico, J. M., & Kebreab, E. (2021). Feed additives as a strategic approach to reduce enteric methane production in cattle: Modes of action, effectiveness, and safety. Animal Production Science, 62: 1303-1317. https://doi.org/10.1071/an20295

Höhler, J., Barreiro-Hurlé, J., Czajkowski, M., Dessart, F. J., Ferraro, P. J., Li, T., … & Rommel, J. (2024). Perspectives on stakeholder participation in the design of economic experiments for agricultural policymaking: Pros, cons, and twelve recommendations for researchers. Applied Economic Perspectives and Policy, 46(1): 338-359. https://doi.org/10.1002/aepp.13385

Hristov, A. N., Oh, J., Firkins, J. L., Dijkstra, J., Kebreab, E., Waghorn, G., … & Tricarico, J. M. (2013). Special topics—Mitigation of methane and nitrous oxide emissions from animal operations: I. A review of enteric methane mitigation options. Journal of Animal Science, 91(11): 5045-5069. https://doi.org/10.2527/jas.2013-6583

Hurley, T. M., & Shogren, J. F. (2005). An experimental comparison of induced and elicited beliefs. Journal of Risk and Uncertainty, 30: 169-188. https://doi.org/10.1007/s11166-005-6565-5

Kadane, J. B., & Winkler, R. L. (1988). Separating probability elicitation from utilities. Journal of the American Statistical Association, 83(402): 357-363. https://doi.org/10.1080/01621459.1988.10478605

Kleinlercher, D., & Stöckl, T. (2018). On the provision of incentives in finance experiments. Experimental Economics, 21: 154-179. https://doi.org/10.1007/s10683-017-9530-7

Manski, C. F. (2004). Measuring expectations. Econometrica, 72: 1329–1376. https://doi.org/10.1111/j.1468-0262.2004.00537.x.

Manski, C. F. (2018). Survey measurement of probabilistic macroeconomic expectations: progress and promise. NBER Macroeconomics Annual, 32(1): 411-471. https://doi.org/10.1086/696061

Matheson, J. E., & Winkler, R. L. (1976). Scoring rules for continuous probability distributions. Management Science, 22(10): 1087-1096. https://doi.org/10.1287/mnsc.22.10.1087

Menapace, L., Colson, G., & Raffaelli, R. (2013). Risk aversion, subjective beliefs, and farmer risk management strategies. American Journal of Agricultural Economics, 95(2): 384-389. https://doi.org/10.1093/ajae/aas107

Meraner, M., & Finger, R. (2019). Risk perceptions, preferences and management strategies: evidence from a case study using German livestock farmers. Journal of Risk Research, 22(1): 110-135. https://doi.org/10.1080/13669877.2017.1351476

Meraner, M., Musshoff, O., & Finger, R. (2018). Using involvement to reduce inconsistencies in risk preference elicitation. Journal of Behavioral and Experimental Economics, 73: 22-33. https://doi.org/10.1016/j.socec.2018.01.001

Offerman, T., Sonnemans, J., Van De Kuilen, G., & Wakker, P. P. (2009). A truth serum for non-Bayesians: correcting proper scoring rules for risk attitudes. Review of Economic Studies, 76: 1461–1489. https://doi.org/10.1111/j.1467-937X.2009.00557.x.

Oprea, R. (2020). What makes a rule complex? American Economic Review, 110(12): 3913-3951. https://doi.org/10.1257/aer.20191717

Papke, L. E., & Wooldridge, J. M. (1996). Econometric methods for fractional response variables with an application to 401(k) plan participation rates. Journal of Applied Econometrics, 11(6): 619–632. https://doi.org/10.1002/(sici)1099-1255(199611)11:6<619::aid-jae418>3.0.co;2-1

Purvis, A., Boggess, W. G., Moss, C. B., & Holt, J. (1995). Technology adoption decisions under irreversibility and uncertainty: an ex ante approach. American Journal of Agricultural Economics, 77: 541-551. https://doi.org/10.2307/1243223.

Rabin, M., & Schrag, J. L. (1999). First impressions matter: a model of confirmatory bias. The Quarterly Journal of Economics, 114(1): 37-82. https://doi.org/10.1162/003355399555945

Reynaud, A., & Couture, S. (2012). Stability of risk preference measures: results from a field experiment on French farmers. Theory and Decision, 73: 203-221. https://doi.org/10.1007/s11238-012-9296-5

Savage, L. J. (1954). The foundations of statistics. Naval Research Logistics Quarterly, 1: 236–236. https://doi.org/10.1002/nav.3800010316.

Schlag, K., & Tremewan, J. (2021). Simple belief elicitation: An experimental evaluation. Journal of Risk and Uncertainty, 62: 137-155. https://doi.org/10.1007/s11166-021-09349-6

Schlag, K. H., Tremewan, J., & van der Weele, J. J. (2015). A penny for your thoughts: a survey of methods for eliciting beliefs. Experimental Economics, 18: 457–490. https://doi.org/10.1007/s10683-014-9416-x.

Schlag, K. H., & van der Weele, J. J. (2015). A method to elicit beliefs as most likely intervals. Judgment and Decision Making, 10(5): 456–468. https://doi.org/10.1017/S1930297500005593.

Smith, J., & Mandac, A. M. (1995). Subjective versus objective yield distributions as measures of production risk. American Journal of Agricultural Economics, 77(1): 152-161. https://doi.org/10.2307/1243897

Smith, C., Nicholls, Z. R. J., Armour, K., Collins, W., Forster, P., Meinshausen, M., … & Watanabe, M. (2021). The Earth’s energy budget, climate feedbacks, and climate sensitivity supplementary material. In Climate Change 2021: The Physical Science Basis. Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change [Masson-Delmotte, V., Zhai, P., Pirani, A., Connors, S. L., Péan, C., Berger, S., … Zhou, B. (eds.)]. Retrieved from https://www.ipcc.ch/.

Trautmann, S. T., & van de Kuilen, G. (2015). Belief elicitation: A horse race among truth serums. The Economic Journal, 125(589): 2116-2135. https://doi.org/10.1111/ecoj.12160

Trujillo-Barrera, A., Pennings, J. M., & Hofenk, D. (2016). Understanding producers’ motives for adopting sustainable practices: the role of expected rewards, risk perception and risk tolerance. European Review of Agricultural Economics, 43(3): 359-382. https://doi.org/10.1093/erae/jbv038

Tjernström, E., Lybbert, T. J., Hernández, R. F., & Correa, J. S. (2021). Learning by (virtually) doing: experimentation and belief updating in smallholder agriculture. Journal of Economic Behavior & Organization, 189: 28-50. https://doi.org/10.1016/j.jebo.2021.03.001

van Winsen, F., de Mey, Y., Lauwers, L., Van Passel, S., Vancauteren, M., & Wauters, E. (2016). Determinants of risk behaviour: effects of perceived risks and risk attitude on farmer’s adoption of risk management strategies. Journal of Risk Research, 19(1): 56-78. https://doi.org/10.1080/13669877.2014.940597

Villacis, A. H., Mishra, A. K., Barrera, V., & Dominguez, J. (2023). Risk perceptions of climate change, risk preferences and cacao farming. In Managing Risk in Agriculture: A Development Perspective, 225-237. GB: CABI. https://doi.org/10.1079/9781800622289.0016

Zappalà, G. (2023). Drought exposure and accuracy: motivated reasoning in climate change beliefs. Environmental Resource Economics, 85: 649–672. https://doi.org/10.1007/s10640-023-00779-1.

Wells, C. W. (2023). Effects of essential oils on economically important characteristics of ruminant species: a comprehensive review. Animal Nutrition, 16: 1-10. https://doi.org/10.1016/j.aninu.2023.05.017

Winkler, R. L., & Murphy, A. H. (1970). Nonlinear utility and the probability score. Journal of Applied Meteorology and Climatology, 9(1): 143-148. https://doi.org/10.1175/1520-0450(1970)009<0143:nuatps>2.0.co;2

---