Public interest documents are records that shine light on the workings of government. Examples of these documents include proceedings of Federal, State and Territory parliaments (Hansards), submissions to public inquiries, transcripts of public hearings, Ministerial diaries, conflict of interest and lobbying registers. Significant efforts have been made to make these documents more openly available for interested parties over the past few decades. 

However, a ‘last-mile’ problem remains where most datasets of public interest documents are difficult to systematically consume and analyse because they are primarily shared as individual files intended for human rather than machine reading. This limits the ability of researchers across the HASS and Indigenous research community to effectively use these valuable documents and tackle key research questions in the public interest. This is partly because of gaps in the datasets and partly due to the wide variety of formats and structures used. 

Significant data transformation and processing is required to make them usable for anything beyond reading individual files. Only people with specific technical skills and expertise can do this. This leads to rework, repetition, and wasted effort on the part of researchers. Many valuable projects are abandoned or limited in scope before they even start due to the high costs for researchers to access and work with these documents. This limitation affects not just computational uses of these datasets, but also limits qualitative and interpretive analysis. The volume of documents available typically requires careful filtering and selection of examples to be analysed in more detail and these documents are rarely published in ways that support this filtering. 

This project aims to: 

• Deliver a platform that makes existing public interest documents (starting with Federal Hansard and public inquiries and reviews) more easily accessible to a wider audience of researchers, civil society organisations and the broader public. 

• Provide a standard for Public Interest (PI) document metadata that supports FAIR (Findable, Accessible, Interoperable, Reusable) principles. This will include compatibility with existing metadata standards such as the Australian Government Record Keeping Standard. 

• Support researchers by creating and documenting research workflows and use cases, including both computational and qualitative workflows. 

The RAPID project builds on, and will contribute to the Language Data Commons of Australia (LDaCA) technology architecture. Specifically, the RAPID project will apply the LDACA RO-Crate tools and the Oni data portal to collections of public interest documents consistent with the PILARS protocols. 

This application will develop the underlying technologies by applying them to new use cases and developing new functionality that could potentially be integrated into the LDaCA project architecture.