Trove Query Builder

Trove Query Builder is a user interface to assist researchers query the National Library of Australia's Trove API. It's designed to be integrated with other web based software, and provide both a convenient user interface and also a convenient programming interface for developers who are integrating Trove with some other web-based application. In that sense, it's designed to be customised by developers to suit the needs of their individual applications.

The software takes the form of a Javascript-based Single-Page Application which displays an HTML form which a user can fill out, generate a query URL, and download the result using their browser or a harvester application. Researchers can use the Query Builder to create and execute Trove API queries without doing any programming.

The Query Builder software is designed to be able to communicate with Trove's API either directly, or via a proxy server. However, direct communication Trove's API is not possible at the moment because of a known bug in Trove's API.

The Query Builder form includes access to certain query capabilities additional to those provided by Trove's API, by communicating with Trove via an intermediate layer provided by the TroveProxy web application. The TroveProxy web application forwards the user's query to the Trove API, retrieves the data from Trove, and then performs enhancement and conversions before returning it to the user. See the TroveProxy repository for more information on the proxy application.

Most of the items on the form correspond to query parameters in Trove's API, and the TroveProxy server will pass these parameters through to Trove, but a few of them correspond to features of the TroveProxy application itself. Most importantly, the Query Builder form includes a list of output formats, allowing users to direct the Proxy to convert the Trove data into a desired format before returning it. The Query Builder form includes this format option in the URL it sends to the TroveProxy, in the form of a proxy-format parameter which instructs the TroveProxy application to transform the results of a Trove API query into one of three distinct data formats, suitable for use with three different types of software application. NB all the proxy-specific parameters in the form, and in the URLs generated by the form, start with the prefix proxy-.

How the Query Builder software is organised

The software is entirely client-side, browser-based code, consisting of HTML and JavaScript ciles in the /web directory. In principle it can be integrated easily with other web based applications by including HTML form markup, and links to the JavaScript files into the HTML layer of those other applications.

However, in this repository it's also packaged in the form of a Docker container which includes a web server to serve up the HTML and JavaScript files.

JavaScript code

The repository contains three JavaScript libraries which each define a custom HTML form element:

• The trove-facet element provides a drop-down list to represent one of Trove's "facet" parameters. The facet values in the list are automatically loaded from Trove's API, and updated when necessary.
• The trove-query element provides an embedded form in which users can build a query by specifying values for different Trove fields, and combining them using boolean logic.
• The trove-dataset-metadata element provides an embedded form in which users can add metadata about their query, such as a name and description.

The search form

The repository also contains a web page containing an HTML form containing input fields for all the documented parameters of the Trove API.

This very generic form can be used to explore Trove's search API in full, but the intention is that when the Query Builder software is integrated into a particular application, a new version of this form will be created by cutting the generic form down to fit a more particular need, removing unnecessary fields, and restricting options. For instance, an application which always wanted to retrieve data in CSV format could replace the proxy-format drop-down list of formats with a hidden field of the same name, with the fixed value csv.

Developing

To build the docker application, naming the image trove-query-builder:

docker build -t trove-query-builder .

To launch the trove-query-builder container:

docker run --publish 80:80 trove-query-builder

For convenience while developing the application, you can use the --mount command to mount the web folder into the container, so that you can edit the HTML and JS code and have the changes reflected in the running container immediately, without having to rebuild the docker image.

docker run --publish 80:80 --mount type=bind,src=`pwd`/web,dst=/usr/local/apache2/htdocs trove-query-builder