Technical Obstacles in Implementing the (Interception and Access) Amendment ( Retention) Bill 2014

Summary:

In my professional opinion, the data handling defined by the Bill is unworkable and cannot be implemented at a reasonable cost. Besides this, other countries have shown that data retention schemes do not provide value, and there are major issues in identifying perpetrators of major terrorist acts to even know who to monitor. “ Notices” already in use in Australia are more than sufficient for the required tasks. This Bill should be withdrawn.

My Background.

I am a private individual, with a background of nearly 30 years experience in the ICT industry, developing complex software and information systems. I can neither confirm nor deny that I was involved in IT analysis at the Defence Signals Directorate, which may or may not have given me an understanding of intelligence matters.

My submission.

The Bill proposes the implementation of a data collection and delivery system at an industry wide level, but with data collection by individual service providers. It appears to have been developed without taking technical and other obstacles into account, which would prevent it being effective in its current form.

A data collection process has several stages:

1) Capture the data

2) Load the data (incorporating extraction and transformation)

3) Store the data

4) Retrieve the data

5) Deliver the data While there is a widespread belief that computers are powerful machines capable of anything at minimal cost, this typically only applies to the storage and transport of data. The collection and processing of data is highly complicated, and can present unsolvable issues to design and other constraints.

A data sharing system needs a common format to store the data in and to transport it around. Otherwise, it will need significant processing whenever it is retrieved and delivered with the possibility of errors in that processing. Many industries have created umbrella organisations to develop industry standard data formats (aka XML) that are then used to store and transport information. They include attributes such as the type, size and structure of the various sets of data. Without an agreed and accepted format for the retention data, this Act could not be implemented without significant overhead and increased chance of error.

There seems to be a large amount of spurious data included in the Bill, irrelevant to the intelligence gathering. While this is evidently seen as useful, it has major implications for the processing of the data. Typically, the retention data would be processed and loaded overnight in a short window. The length of this time frame is dictated by various factors such as the need for system availability, the time needed to the system, etc. This time window is currently used by large companies to extract, transform and load their own data into their systems. The time needed to load retention data needs to be as short as possible, and there may not be time to load it after the existing load takes place. Because the Bill requires specific data to be stored, this creates an additional processing requirement. In order to comply with the Act, companies may to need to completely redesign their data warehousing systems at a major cost, and if this is prohibitive, they may have no other choice but to pay non-compliance fines. The requirement to store this data may jeopardise the viability of smaller service providers (who may not currently have the IT capability to perform it), and would be a significant deterrent for new market entrants (decreasing industry competition).

The amount of data that needs to be stored, processed & retrieved is phenomenal. It appears that a 2 year retention period was randomly selected, but this is unworkable in terms of organising and indexing the data in a way that is sensible and easily retrieved. The data is stored by individual service providers, but will likely need collation across multiple service providers which adds significant complexity. Even a 3 month window is highly challenging with consumers changing locations, technologies and types of interaction (particularly as the intelligence targets may be actively seeking to obscure their activities). As and access is increasingly from mobile devices, this also adds significant complexity to location and timing capture (e.g short mobile phone call starts in WA, then ends in SA several hours earlier)

This leads into another major challenge. All that is being stored is raw data. In current intelligence work, a dossier of information is collated and compiled based on specific data accessed via the authority of a warrant, e.g. the “Data Preservation Notices” that are active for 3 months for specific persons of interest. The raw data is collated as it is received and summarized into relevant information. This information can then be leveraged as part of a larger investigation.

The Bill is dictating the storage of raw data, across multiple organisations, across consumers with multiple devices and multiple accounts or aliases, moving across cities or across the country. The ability to reconstruct the data into any form of meaningful information is a complex undertaking, and likely to be ambiguous or incomplete, and possibly open to legal challenge as to its legitimacy. This data reconstruction is also jeopardised since some data cannot be captured, e.g. Internet traffic that can be hidden (e.g. in public Internet facilities), encrypted or masked, or simply occur out of Australia's legislative jurisdiction.

There are many other technical challenges involved in storing and retrieving the data (listed in the data collection process above) that are too technical to go into. In my professional opinion, I would recommend that this project does not proceed because of the high risk of failure or cost overrun.

Given all the complexity and cost of this project, there must be a significant benefit realised. Have other nations been successful in leveraging this data to prevent illegal activity? Both and America have shown that the resulting data retention schemes have negligible value.

Even with these systems providing this information, major acts of terrorism (e.g. Sydney, London, Boston, Paris) have occurred where the perpetrators were known to authorities but not seen as being a threat, and were apparently not being monitored.

For all the reasons outlined above, this Bill should be withdrawn.