What is the SRB?
SRB handles data through a client-server architecture and allows users to access location-independent data: the user does not need to know where files are located and how they are stored. The SRB organises data logically into a single virtual file system in a similar way to a filesystem found on a stand-alone computer rather than physically. The use of stored metadata about each file facilitates the querying of these distributed data.
Metadata – data about the data – allow users to quickly find a dataset, what it contains, its format, when or where the data were collected or created, etc. Metadata querying and browsing enables user communities to have transparent access to each other’s data collections. Users can store or replicate their data collections across several servers to facilitate data preservation, while not losing local access control. Access permissions on individual files can be set so that other users can access the files. Users wishing to access these collections will view a single collection through the SRB middleware and do not need to know the physical location of the data.
The SRB servers that provide access to the archival resources, the Metadata Catalogue (MCAT) and the SRB clients are the main elements of an SRB domain. Files uploaded into the SRB are referenced by logical file handles chosen by the user – a name that is meaningful to the user. The MCAT maps these logical handles to the physical file locations on individual resources, and stores the metadata associated with the files, as well as information about the users and the physical resources managed by the SRB. The MCAT is usually associated to one SRB server (MCAT-SRB server).
How does it work?
Figure 1 illustrates how the SRB works in a typical scenario (the arrows show the typical flow of data). A user needs a file stored in ‘Data Resource 2’ which could be a tape archive or a hard disk etc. However the user does not have this information. The only thing that the user has to do is login and request the file by making a query via an SRB client. What follows happens behind the scene. The client contacts the local SRB Server A and requests the file (1). Server A contacts the MCAT-enabled SRB Server B in order to find the physical location of the file (2). Server B looks up the information in the MCAT and passes this information back to Server A (3). As the file is located on a resource maintained by SRB Server C, Server A redirects the request to SRB Server C (4). Server C retrieves the file from Data Resource 2 (5 and 6) and services the client directly (7). The files held on these devices appear to the user as part of a single file system. The full process is transparent to the user who does not need to know in which data resource the requested file is located.
 |
| Figure 1: SRB data pathway |
Usage and deployment of SRB
QCIF is currently supporting the deployment and support of SRB for a number of different scientific communities. In 2004 the UQ HPC unit upgraded its data storage capacity to 120TB of tape storage. The storage capacity was further increased in 2006, with an upgrade of another 100TB of tape. SRB is currently being deployed on the data storage infrastructure.
A number of projects are using the SRB, including the SensorNet and e-Archaeology projects.
For more information visit the VisLab Storage Resource Broker project page.
Contacts
David Gwynne, Terry Simmich, Nicole BordesVislab, University of Queensland
Publications
Bordes, N., Ulm, S., Pettersen, O. et al., 2006. "Data Grid for the Management, Reconstruction, Analysis and Visualisation of Archaeological Data". Accepted for publication: In print. (2006)Bordes, N., Ulm, S. et al., 2006. "Towards an Australian Archaeological Data Grid". Submitted to Computer Applications and Quantitative Methods in Archaeology (2006)