Measuring Access to Justice in 7 Simple Steps

Step 1: Define the Path to Justice

What is a Path to Justice?

In order to meet their existing justice needs the users of justice bring their problems to an existing structured mechanism for resolution of justice needs. These mechanisms or paths to justice are what the methodology measures. In a more abstract way a Path to Justice is defined as ‘commonly applied process that people address in order to cope with their Legal Problems’. A court procedure is an obvious example of a path to justice. However, the definition includes both formal and informal procedures. This means that a mediation procedure, or a procedure before an informal Commission or other neutral also qualify as a path to justice.

Who is the user?

In every dispute there are at least two disputing parties. The methodology measures the experiences of the active party – the one that starts the process as response to her legal needs. Depending on the type of dispute resolution process this party could be called differently – a claimant, petitioner, victim etc.

When a Path to Justice begins?

A Path to Justice begins when a user first takes action to solve the problem with the means of formal or informal justice. This can be the moment when the user seeks information about her possibilities, seeks legal advice (internet, legal adviser, police, mediator, etc.) or files a petition. There is an important period between the moment of the first action and the moment when the official or unofficial dispute resolution procedure commences. In this period the user can endure transaction costs in terms of stress, negative emotions, loss opportunities or money. It is important to measure these costs as part of the overall costs of the Path to Justice.

When a Path to Justice ends

The end of a Path to Justice is the moment when a neutral takes a decision, the parties come to an agreement, or one of the parties leaves the procedure, which ends the procedure.

What is an outcome of a Path to Justice?

An outcome of a Path to Justice is the situation at the moment a path to justice ends. It can be an award for damages, a agreement about future conduct, or an apology, or combinations of these.

What is the measurement timeframe?

One of the important questions at the design phase is about the scope of the measurement. The particular circumstances could require that we measure all users who obtained an outcome in the last calendar year or in the last several years. It is recommendable that the perceptions of the users are studied as soon as the outcome is delivered. On the other hand, on many paths to justice larger periods of time have to be covered in order to ensure sufficient sample size.

Step 2: Identify the users of justice

How to find the users of justice

Study of the costs and quality of paths to justice requires access to the users who used the path. Depending on the circumstances one can study all users or just a fraction – a sample. Identifying the users is key to the success to the measurement. Here are several tips for locating users of justice:

  • Look in the records of the neutral who presides over the dispute;

  • Access records of service providers such as attorneys, victim support organizations, para-legals, legal aid providers etc.;

  • Trace users and their contact details in public registers;

  • Identify users of justice on the spot where justice is delivered – court rooms, public service counters, web sites etc.

 

How many users of justice have to be interviewed

In order to have good (valid and reliable) results at least 30-50 users have to be interviewed. Larger samples provide richer data which could answer more questions.

Step 3: Adapt the methodology

Three versions of the questionnaire can be downloaded. All three have the same core of questions and the difference is in the breadth of the information that they collect.

In some instances the methodology has to be adapted to the specifics of the measured path to justice. Adaptation can tailor the measurement to the specific environment but can also lead to incompatible results. Therefore the process of adaptation should be conducted carefully and with concern for the comparability of the data with other studies. After all one of the most important purposes of measuring paths to justice is to compare them to other paths.

Step 4: Contact the users and collect data

After the Path to Justice and its users are identified and the measurement tool is adapted, it is time to start the measurement. Many different modes of collecting data are possible. In the context of measuring the costs and quality of paths to justice we recommend two quantitative and one qualitative methods. Cross-sectional survey and a diary method belong to the family of the quantitative research methods, whereas the focus group interview collects data which is qualitative.

Survey

Survey is the most frequently used form for gathering perceptioinal data. The important question with the surveys is how to deliver the questionnaire to the respondents. Here are the most widely used strategies:

  • Self-administered questionnaires sent to the users’ address or distributed at places where the users of justice appear (court rooms, offices of attorneys, mediators or arbitrators, police and other public institutions etc.)

  • Telephone interviews;

  • Internet based questionnaire;

  • Face-to-face interview.

Diary study

The diary study is a form of longitudinal panel study. It is a data collection method which requests participants to keep a diary or journal (can be outlined by the researchers) of their experiences with the subject under study. Information may include date and time, emotions encountered, issues with the subject (the process), etc. The study can also consist of questionnaire distributed at several points in time.

Focus group interview

Focus group interview is qualitative method where a group of people are asked information regarding their perceptions and attitudes towards a given subject – in this case their experiences with the path to justice. In an interactive group setting, participants are invited to discuss their issues with other group members. Focus groups are useful for bringing attention to the unexpected and offering further exploration.

Step 5: Analyse the data

In general, the collected data could be analyzed in two analytical frameworks – descriptive and explanatory. A descriptive perspective means that the aim of the data collection is to depict a path to justice – its costs and quality. It just tells us how the path looks like in the eyes of its users, without getting deeper to explain why. On the other hand, the explanatory framework aims to address a wide array of “why” and “how” questions – why the outcome scores high or low, why some users pay more, how the perceptions on the procedural quality impact the outcome etc.

The questionnaire collects predominantly quantitative data in four dimensions – characteristics of the path and the its users, costs, quality of the procedure and quality of the outcome. At the core of the data analysis is the description of the indicators. All of the indicators are composed of numerous items (questions). The analysis could look at the descriptive statistics at the item level or comment the value of several items which make a unified concept – i.e. opportunity costs, procedural justice or transparency of the outcome.

Step 6: Compute an index of the path

An index is a composite measure which rank orders the paths to justice under study and is based on more than one data item. The goal is to summarize several indicators into one final score. An index of the path to justice is a single value which reflects the different indicators of a path to justice.

Briefly, the construction of an index of a path to justice is about aggregating the three indicators into one composite measure. This means that first aggregate values of the costs, the quality of the procedure and the quality of the outcome have to be computed. Two models are possible: simple (the aggregation is a plain calculation of the mean of the related indicators) and complex (weights are applied before the means are computed). The latter option returns better results but has one stringent requirement – knowledge on the comparative preferences towards the three indicators. Thus the former option of simple aggregation is more feasible.

Step 7: Improve the Paths to Justice

Knowledge on what do the users think of the Path to Justice is valuable tool for policy makers, service providers or civil society groups who want to improve justice and access to justice. Here are just a few dimensions of improvement that could be based on the evidence that the measurement exercise provide:

  • Benchmarking the Path to Justice to other paths in the same or other jurisdiction;

  • Benchmarking the development of the Path to Justice across time;

  • Analysing the effect of policy interventions or management approaches;

  • Improved access to the path after potential barriers are identified;

  • Increased transparency on the path to justice;

  • Provide the current and potential users with more information for better decisions.