Empirical research with judicial professionals and courts: Methods and practices
Coordinators: Sharyn Roach Anleu (Matthew Flinders Distinguished Professor, Flinders University), João Paulo Dias (Centre for Social Studies, University of Coimbra), Paula Casaleiro (Centre for Social Studies, University of Coimbra)
Description of the meeting
Many studies of judicial professionals and courts rely on secondary data such as court statistics and administrative data (Opeskin, 2013), and other documents such as cases, biographies, and other publicly available material (Moran, 2006; Roberts, 2014). There are also important ethnographic studies of courtrooms (e.g. Carlen, 1976). However, studies that obtain data directly from judicial professionals are scarce but increasingly being undertaken, using multiple research methods, such as interviews, focus groups, surveys, and observations, besides the conventional legal research (e.g. The Judicial Research Project at Flinders University https://sites.flinders.edu.au/judicialresearchproject/; Centre for Social Studies of the University of Coimbra: Who are they? Insights into professional characterisation of judges and public prosecutors in Portugal; QUALIS - Working conditions of judicial professions in Portugal).
Topics such as judicial stress, working conditions and/or intersections between work and family are currently of interest and concern to judicial professionals and their courts administrations. This is also a time in which judicial reforms, both in the global north and south, focus on the performance of the judicial systems and their capacity to provide an adequate response, within a reasonable time, to the demand for justice expressed by citizens. The shift from the classic ‘rule of law’ paradigm to the ‘new public management’ paradigm contributed to the emergence of empirical research concerning new models of judicial management, judicial organisation and judicial professional’s performance and working contexts (Dias, 2004; Bastard and Mouhanna, 2010; Verzelloni, 2016; Dias and Almeida, 2010; Roach Anleu and Mack, 2014).
In the past decades, socio-legal scholars have described judicial professionals at courts as difficult populations for research. Some socio-legal researchers describe the judiciary as “a ‘hard-to-reach’ group” due to obstacles presented by gatekeepers (Cowan, Blandy, Hitchings, Hunter & Nixon, 2006: 548), the judiciary’s “high status and professional remoteness” (Dobbin et al., 2001: 287) and their concerns about confidentiality of responses (Hunter, Nixon & Blandy, 2008: 87). Thus, problems of access, time constraints and perceived utility of socio-legal and social science research in general have proved to be significant challenges for empirical research that obtains data directly from judicial personnel.
This workshop will address methodological and practical issues that can arise when researching judicial administration/organization and judicial professionals using a range of socio-legal (social science) research methods and data (such as interviews, surveys, observation, secondary data) and conventional legal analysis. It will draw on the experiences of socio-legal researchers within the field and discuss a range of socio-legal insights, methodological approaches and methods from disciplines such as anthropology, law, political science, psychology and sociology.
The workshop aims to identify and evaluate the different research methods and approaches to researching the everyday work of judicial professionals and their courts. It will challenge the traditional construction of judicial professionals and courts as a difficult population to study and propose strategies to promote and facilitate the access to these research sites.
This workshop will address four main themes:
- Empirical strategies to access and conduct research with judicial officers. Members of any population of study should be motivated to participate in the study. The fact that a few researchers have been able to gain the participation of judges or public prosecutors suggests that there may be particular issues that these professionals find important enough to respond to, methods of conducting research that they may find more congenial, or classes of judges or public prosecutors who may view social and behavioural science research more favourably than do others. In the workshop we will discuss how to obtain access, what mechanisms/strategies work and which have been unsuccessful. For instance, the relevance of the research topic to the judicial professions and the level of understanding of the topic showed by researchers are factors that contribute to motivate participation in studies;
- Use of different methodologies, especially surveys, focus groups and interviews, and mixed methods, combining court observations with interviews/surveys, legal analysis and statistical data. Socio-legal researchers increasingly recognise the need to employ a wide variety of methods in studying judicial and legal phenomena, and the need to be informed by an understanding of debates about theory and method in mainstream social science (Banakar & Travers, 2005). Some studies suggest that the instruments used may also be a factor to the low participation of the judicial professionals. Darbyshire (2011) pointed out that judges welcomed the opportunity to open up the judiciary to outside scrutiny and to the work-shadowing method used by the study. Thus, extensive surveys may not be the best approach to this population.
- Partnerships with courts and judicial organisations – the support of governmental and judicial institutions, associations or academies in the preparation, dissemination and call for participation in the studies facilitates access to a ´difficult population´; it helps surpass the assumed resentment or unwillingness to be tested, and concerns by judicial professionals about the anonymity and confidentiality of survey responses.
- Diversity of judicial management models, judicial organisation and judicial professionals. Despite the international diffusion of managerial models and judicial reforms to improve the capacity/quality of courts, there remains a diversity of judicial management models. Moreover, there is a wide range of judicial professions such as magistrates, judges, public prosecutors, law clerks and judicial officers, whose role and status can vary considerably. The workshop will consider the challenges posed by the specificities of the national judicial contexts and judicial careers to the empirical research.
Banakar, Reza & Travers, Max. 2005, Theory and Method in Socio-Legal Research. London: Hart Publishing. Series: Oñati International Series in Law and Society.
Bastard, Benoit & Mouhanna, Christian. 2010, ‘Procureurs et substituts: l’ évolution du système de production des décisions pénales’, Droit et Société. Nº 74: 33-53.
Carlen, Pat. 1976, Magistrates' Justice. London: Martin Robertson.
Cowan, Dave, Sarah Blandy, Emma Hitchings, Caroline Hunter and Judy Nixon 2006, 'District Judges and Possession Proceedings', Journal of Law and Society 33: 547-71.
Darbyshire, Penny. 2011, Sitting in Judgment: The Working Lives of Judges. Oxford and Portland, Oregon: Hart Publishing.
Dias, João Paulo. 2004, O mundo dos magistrados: a evolução da organização e do auto-governo judiciário [The world of magistrates: evolution of its judicial organisation and self-governance]. Coimbra: Almedina.
Dias, João Paulo & Jorge Almeida. 2010, ‘The external and internal conditions for the independence of the judiciary in Portugal’, in De Groot, Leny; Rombouts, Wannes (eds) Separation of powers in theory and practice: an international perspective. Nijmegen: Wolf Legal Publishers, 225-252.
Dobbin, Shirley A., Sophia I. Gatowski, Gerald P. Ginsburg, Mara l. Merlino, Veronica Dahir and James T. Richardson. 2001, 'Surveying Difficult Populations: Lessons Learned from a National Survey of State Trial Court Judges', Justice System Journal 22: 287-307.
Hunter, Caroline, Judy Nixon and Sarah Blandy. 2008, 'Researching the Judiciary: Exploring the Invisible in Judicial Decision Making ', Journal of Law and Society 35: 76-90.
Moran, Leslie J. 2006, 'Judging Pictures: A Case Study of Portraits of the Chief Justices, Supreme Court of New South Wales', International Journal of Law in Context 5: 295-314.
Opeskin, Brian. 2013, 'The State of the Judicature: A Statistical Profile of Australian Courts and Judges', Sydney Law Review 35: 489-517.
Roach Anleu, S. & Mack, K. 2014, ‘Judicial Performance and experiences of judicial work: findings from socio-legal research’, Oñati Socio-Legal Series, 4(5): 1015-1040.
Roberts, Heather. 2014, Telling a History of Australian Women Judges Through Courts' Ceremonial Archives' Australian Feminist Law Journal, 40:1, 147-162.
Verzelloni, Luca. 2016, A Gestão dos Sistemas de Justiça: Governo dos Tribunais numa Análise Comparada [The management of judicial systems: governance of courts in a comparative analysis], in Maria de Lurdes Rodrigues, Nuno Garoupa, Pedro Magalhães, Alexandra Leitão, Conceição Gomes (org.), 40 Anos de Politicas de Justiça em Portugal, 709-736.
Our experience in Oñati
On Thursday 23 and Friday 24 June 2022 a very successful workshop was held on the challenges of undertaking empirical research that involves courts and judicial professionals. There were 16 presentations over the two days, with most participants able to be present in Oñati in person and enjoy the surroundings of the Institute and local cuisine, and others were able to join by zoom.
The workshop provided an opportunity for participants to take time out of our embeddedness in research and to be reflexive about our projects that involve studying judges, courts, judicial professionals, and others in and around courts. The format of workshop allowed for extended discussions of common themes, sharing experiences and networking rather than an occasion to deliver a finalised paper with fully fledged conclusions.
The focus of the workshop was on research design and methods, yet the discussions teased out the difficulties in completely separating questions of method from findings, and the converse. We stopped to consider big picture questions such as ‘what is data’ and the strengths and limitations of certain kinds of collected or produced data, statistics assembled by agencies for reporting purposes, rather than research, and the role of gatekeepers and other constraints on access to the judiciary and its work.
Participants described their research projects which involved an array of socio-legal/social science data collection/construction techniques — ethnographic methods, interviews, language processing techniques, online and paper surveys, judgments, photography, and field diaries — undertaken either alone or in some combination. The projects involved interdisciplinary teams of researchers and different kinds of collaboration with courts and other judicial institutions. There were several examples of high levels of enthusiasm among judicial professionals to be involved in research projects that can lead to better court outcomes and experiences for some of the most marginalised court users.
The discussions canvassed issues of trust, rapport, buy in, judicial values, using emotion in research, and reflexivity regarding the relationship of the researcher to the research setting. They also reconsidered the public impact or seduction of policy demands on research design and findings.
We appreciate the support of the Institute in enabling this workshop to occur, and sincerely thank the dedicated staff who made sure every detail was checked, kept everyone up to date, facilitated the audio-visual technology and arranged the catering, restaurants, and airport transfers.