Data mapping project: Apples, Oranges and Lemons
Examining how administrative data is collected and used in the Victorian legal assistance sector.
Background
Reliable data is powerful. It has potential to illuminate the forces shaping our society. Data is increasingly used to help us understand complex issues and one important source is administrative data.
Administrative data is information collected and stored as part of the everyday function of organisations. It can be used to generate insights into the operation of the legal system.
In the legal context, service providers are demanding more from their data to improve and design effective services. Used well, administrative data can measure not only the volume and type of work done, but the success of services and programs.
To unlock the potential of this data, we first need to know what data exists, its form and quality, how it is currently used, and what it can be used for. Hence this project to map data collected by the Victorian civil justice system.
This is key to more fully exploiting its potential as a tool for answering important access to justice questions.
Mapping Justice
The Data Mapping Project mapped use and utility of administrative data across a broad range of civil justice bodies, including courts and tribunals, legal assistance organisations and dispute resolution bodies.
This work identified opportunities and challenges in using existing data to improve access to justice through the operations of large players in the sector, and set out some possible new directions.
Stage one: Apples oranges and lemons
For Apples, Oranges and Lemons: the use and utility of administrative data in the Victorian legal assistance sector we interviewed 29 public legal services. These included a cross-section of legal assistance providers in metropolitan and regional Victoria, covering both generalist and specialist services. We also spoke to representatives from peak bodies and funders.
Key findings
Data practice was variable
Data collection practices across the sector varied considerably. Between organisations there was inconsistency in how data was collected, counted and measured. These variations reflected pragmatic choices of individual organisations, with data collection tailored to the needs of services and their clients.
Data quality was variable
Despite efforts to improve legal assistance sector data practices, there was evidence of significant variation in data quality and consistency.
Variation in quality and practice has implications
Variable data quality and data practice created difficulty interpreting data and means that Victoria's legal assistance administrative data cannot be meaningfully aggregated at a sector level or easily used to compare organisations.
Innovative data practice was constrained by database limitations
There was widespread evidence of administrative data being used in innovative ways by individual organisations to learn more about clients and services, and to better respond to legal need. Some participants reported, however, that inflexible data systems constrained their efforts and were frequently difficult to modify to extract the information they wanted.
Data capability varied but was universally constrained by resourcing
Data capability also varied between organisations. While some participants had the skills and capacity to work confidently with their data, others reported challenges and barriers. The single greatest barrier was a lack of funding dedicated to improving data systems and practices. Finite funding meant that frontline legal assistance was understandably prioritised. This polarisation of data capability could exacerbate the variation in data quality and practice, further inhibiting sector-level analysis.
Capturing complex clients and services requires complex data
Participating organisations reported that administrative data did not adequately capture the complexity of clients’ circumstances and needs, nor the extent of the problems they faced. Even with improvement, administrative data remains one of a range of data sources, and complementary methods are needed to fully capture the impact and value of legal assistance services.
Capturing outcomes and quantifying impacts presents additional challenges
There was widespread interest in measuring service outcomes, learning about impact and the value of services provided. This was seen as important for service design and sector funding. Measuring service outcomes, however, raises additional data collection and resource challenges.
Implications and way forward
Administrative data is imperfect and carries risks
In its current form, the administrative data collected by the legal assistance sector is not sufficiently reliable for sector-wide analyses, to compare organisations or meaningfully monitor performance.
A data quality framework could help improve legal assistance data
Victoria’s legal assistance sector data is currently not filtered through a data quality framework. This is a common tool used to interrogate administrative data before using it for analysis and decision making. It would be a valuable addition.
Administrative data is only one tool in understanding access to justice
Even if the quality of legal assistance sector administrative data is improved, its limitations must be acknowledged. Administrative data is only one of a range of information and data sources required for a comprehensive picture of access to justice in Victoria. Collection of other sources and forms of data is also needed to build the evidence base to inform service design and policy.
Strategies to unlock the utility of administrative data
Strategy 1: Improved data quality and standardised data practices
Modern public legal assistance services require data management systems that meet organisational needs.
Strategy 2: Quality data requires leadership, collaboration, and coordination
Building a quality evidence base needs strategic thinking and commitment to drive improved data culture and practice.
Strategy 3: Quality data requires investment in people and time
There needs to be investment in the data capability of individual organisations.
Strategy 4: Quality data requires funding
Increasing the data capability across the legal assistance sector to a consistent standard requires dedicated funding from all governments.
Examining how administrative data is collected and used in the Victorian legal assistance sector.
Publication resources
Related publications
You may be interested in these other research publications.
Preparing lawyers for work in the community legal sector.
Building and sustaining the community legal centre workforce.
Understand better justice
Data and empirical evidence to help understand access to justice and build a better justice system.