A recent post by Dr Becky Batagol had me thinking about the importance of good data collection in the field of ADR. ADR proponents make many claims about the benefits of ADR. We also know that most dispute resolution processes occur behind closed doors, which challenges our ability to make sweeping statements about “what goes on” in ADR processes. Good data, collected from third party facilitators, disputing parties and non participant observers, helps us to make strong claims about the efficacy of ADR forums.
Such data may help us to more clearly consider questions that we believe we already know the answer to. Do ADR processes really preserve relationships when compared to litigation? Are the values of self-determination and autonomy actively promoted in ADR? Do parties focus more on their interests, rather than their positions in ADR? Are remedies achieved in ADR more flexible and varied, compared to court adjudication?
ADR researchers also need to consider methodological approaches that allow us to say things like:
- ADR is cheaper than litigation.
ADR is quicker than litigation.
Settlement rates in ADR are high (compared to what?)
Party satisfaction with ADR processes is high.
When I think about these claims, I think about the (initial) words of Thomas Jefferson and the American Declaration of Independence… “We hold these truths to be sacred and undeniable”. I think good data allows us to cast sunlight on many of ADR’s claims. It is important that we do shine light on these claims, so that users and providers of ADR services understand what ADR can and cannot sensibly lay claim to. Becky made reference in her earlier post to data that would be useful to have:
- settlement rates
factors that may influence settlement rates, such as referral stage
what happens when disputes are not settled at ADR
participant satisfaction with ADR and perceptions of fairness, the time and costs expended by participants and service providers.
For members of the ADR research network and readers of this blog – what other data do you think we should be trying to collect??
James
The Australian Dispute Resolution Research Network 
If what we are taking about is the integrity of the ADR process, just as we consider the institutional integrity of the courts, then there is more data on ADR processes I would like to see publicly available.
The measures listed relating to outcomes are all subjective measures of the outcome of an ADR process. There is no test of the ‘fairness’ of the process beyond participant satisfaction. Participant satisfaction is very important in a process such as mediation which is premised upon the idea of party empowerment and self-determination. But when we consider the integrity of the process we need to look at it from all angles, and one perspective upon mediation is that as a private process, it allows for unfair outcomes when a stronger party negotiates an outcome that a weaker party doesn’t even realise is unfair. Can we measure how the outcome from mediation compares to what would have been received in the same case as ordered by a court? That’s a hypothetical and is hard to answer. Can we use legal advice as a proxy for each party receiving information about legal entitlements, and so bringing an element of objective fairness into the negotiations? The problem with this measure is we know nothing about the extent and quality of the legal advice. One semi-objective measure I have used in the past is to document whether an agreement made in mediation is altered subsequently following legal advice or a court process. This measure can then be compared to participants’ satisfaction with the outcome to explore the gaps between the parties own views of their agreement and their legal entitlements, providing a more balanced view of the fairness of the process. . .
LikeLike
This is such an important conversation – research quality is so fundamentally affected by the quality of data and as James noted, gathering data is one of the great challenges for DR researchers, as most processes that we are interested in are private. I agree with Becky that party satisfaction is subjective and doesn’t tell us enough about the fairness of or bases for decision making. But if we are serious about self determination and responsiveness, it is false to compare what people agree to privately against what an objective third party, with imperfect information, deems would have been the “likely legal outcome”. Perhaps something that could be done is to ask participants for more information about what they used as reference points when making their decision. Some possible questions might include the following (I anticipate that these would benefit from much refining and critique):
1. Did you receive legal advice prior to making your final decision about the outcome? (follow up questions about details – prior to the process, on the phone during the process, after the process, with or without new information being provided to the lawyer, written or verbal advice etc)
2. Did you receive advice from any other people before making your final decision? (follow up questions about from who, about what, whether or not the advice determined whether or not to reach the agreement/walk away)
3. What were the alternatives that you considered when making your final decision? (wise and well prepared parties won’t agree to something that isn’t better than their BATNA, but if they haven’t considered alternatives in decision making then they won’t have conducted that analysis)
4. Did you feel pressured to agree to the ultimate outcome? (follow up by who, how?)
5. How did the final outcome compare to your main aim in dealing with your dispute?
All of these questions relate to the outcome. There is also the question of process, with procedural justice being a powerful indicator of satisfaction. Questions about perceptions of fairness are important and many researchers have asked questions about perceptions of fairness in the past. These precedents can provide a useful starting point for developing our own empirical research questions. Of course, if we ask the same questions we can also compare our own results with the results of previous studies.
Other questions about empirical research data gathering include how we access participants, when we approach them, how we approach them, how we ask our questions etc. Whose cooperation do we need to capture our target research group?
I think the more we talk about these challenges and mine previous research for ideas, the more robust and successful our own research is likely to be.
LikeLike
This is a great discussion and very important. When reading James’ post I found myself reacting to the fact that – as James accurately points out – many in the DR community in Australia – and also internationally – are stuck in a bit of a comparative loop with litigation. It seems we can’t get away from comparing and perhaps engaging in occasional hyperbole – ADR is cheaper than litigation, ADR is quicker than litigation. Yes sometimes it is – but sometimes it might not be. Surely we are now more sophisticated? These comparisons were necessary in the 70s and 80s as DR was trying to gain traction in a system that was very litigation-centric. But DR is now much more widely acknowledged as a positive component of the lawyer’s (and many other professional’s) toolbox. What we should be doing is espousing the benefits and legitimacy of DR in its own right – and collecting data that helps us establish that.
LikeLike
Rachael makes an important point about gathering data about the legitimacy of DR of itself rather than more narrowly about its usefulness in comparison with litigation. Data which measures effectiveness, whether settlement rates, compliance with agreements post process and measures of party satisfaction, are of a kind persuasive to policy makers and funders. Of these, settlement rates are the easiest to measure and arguably the least representative of actual effectiveness. Still this measure as compared with litigation can present a strong argument to government as it has to date.
I am an advocate for data that helps us build theory about the nature, principles and process of ADR and mediation in particular. Research methods that open conversations with practitioners and elicit data about how they understand and use the process produce a wealth of conceptual understanding and practical application that can inform mediation’s further development.
We are seeing mediation emerge as a profession in its own right and as one that draws upon its legal and social and behavioural roots. I am particularly interested in exploring the socio-legal constructions of mediation’s principles and practice and advocate an explicit acknowledgement and exploration of this blend of principle and practice. In my research in Family Dispute Resolution Practice (FDRP), for example, I have found, of the samples of practitioners I have interviewed, that all have social science backgrounds while few have legal backgrounds. In my view it is the blend of socio-legal principles and practice which give mediation its particular character and strength. Research that elicits and acknowledges this blend asserts mediation as a stand alone DR process quite distinct from litigation.
LikeLike
Pingback: Mapping the new world of accredited ADR schemes | UK Administrative Justice Institute