AI and Big Data require oversight to be implemented ethically
Broeders et al. 17 [Dennis Broeders is Associate Porfessor of Security and Technology and Senior Fellow of the Hague Program for Cyber Norms and the Institute of Security and Global Affairs, Leiden University https://www.sciencedirect.com/science/article/pii/S0267364917300675#s0110]//LP
The use of Big Data in the security domain requires intensified oversight. An effective and confidence-inspiring oversight regime, in its turn, requires a higher degree of data processing transparency. In this, transparency is not an aim in itself but serves the interests of accountability. Citizens and organisations must also have opportunities to discuss the accuracy and proportionality of decisions based on data analyses and made by government institutions and, if necessary, to have them assessed by the courts. 13.1. Big Data, big oversight? Current oversight of data processing leaves a lot to be desired, even more so in view of the ongoing rapid digitisation of government and use of data analysis. DPAs and the various forms of oversight on security and intelligence agencies do not appear to be properly equipped to face the challenges of the Big Data era in terms of powers, expertise and financial resources.74 In the Netherlands, many parties, including the oversight committee CTIVD itself75, believe that the planned expansion of the powers of the Dutch general and military intelligence agencies requires oversight capacity and technical expertise to be significantly expanded at all levels.76 Although the powers and resources of national DPAs will in principle be increased through the entry into force of the GDPR, it is up to the national legislatures to allocate the corresponding financial resources, powers and capacity. As indicated above, oversight of the analysis phase will be of paramount importance. Therefore, if possibilities for collecting and analysing data increase, independent oversight should be strengthened. For regulating the intelligence services, it would be appropriate, in view of the need to protect fundamental rights, to introduce an overriding power/the possibility of passing judgments regarding lawfulness.77 13.2. Increasing (layered) transparency Big Data also requires greater transparency in the government's data-processing activities. There is still a lot to be gained on this point, as data processing is a ‘black box’ in many cases. In addition, data subjects are not so quick to invoke their right to information because they often simply do not know that their data is being collected. Given the sensitive nature of the work of law enforcement and national security agencies, there cannot be full transparency, which is precluded by the danger of ‘gaming the system’ and the need to protect sensitive and classified information. Nevertheless, it is possible to work with a layered system of transparency, as was suggested above in relation to reports by the police and the intelligence services to the competent oversight authorities, which, in their turn, report to parliament. It would also be desirable to give citizens more insight into the frequency of data collection, the reasons why it is done, and, if possible, what effect complex data analyses have. In the field of security, some organisations have the legal right to keep all or parts of their operations secret from data subjects and the general public. However, the growing amount of data that the government can obtain under existing secrecy provisions is out of step with the data processing transparency that is required. There are indications, moreover, that the agencies involved in national security are prone to a culture of ‘overclassification’.78 Furthermore, better information meets a democratic need; a well-informed discussion of the use of Big Data solutions requires a better understanding of the use of data by government organisations in the area of security policy. Therefore, data processing transparency should be enhanced, and a better balance should be achieved between the secrecy requirement and the interests of openness as regards Big Data applications that affect fundamental freedoms. Greater transparency is needed in at least two areas. A growing number of organisations in the field of security is involved in Big Data applications, above all in the area of fighting fraud. There is a lack of good regulation in this area. Although a lot of relevant information about data processing in partnerships is contained in publically available covenants and decisions, this is not very accessible. Citizens should not have to be detectives to find the relevant information. There should be greater openness when organisations intend to work with Big Data applications, for example, by requiring them to draw up a policy plan detailing what Big Data applications they use, what public policy they pursue, at what cost and what results they expect from the application. At the accountability level, too, more is possible than is currently being done, for example in annual reporting. Compared to the Netherlands, some European countries are practising a significantly higher degree of openness about their intelligence techniques and operations, without this hindering the work of their intelligence services in any noteworthy way. This is being done in Belgium, for example, with the aim of being able to conduct an informed discussion about how the intelligence and security services work, what their powers are and how they are monitored and regulated.79