Rahul Matthan

Abstract

Data protection and copyright law are two of the principal legal frameworks invoked in the governance of artificial intelligence. This paper argues that applying these frameworks to AI without reconsidering their underlying premises risks constraining the technology and limiting its potential benefits. Data protection law rests on the premise that privacy is best protected by limiting the collection and processing of personal data, while copyright law seeks to protect creators from unauthorised reproduction and use of their works. Both frameworks were developed in response to earlier technological conditions that differ from those of AI and abundant data. The paper proposes a re-evaluation of these frameworks: replacing prescriptive, process-based data protection with a harms-based approach focused on outcomes, and shifting copyright enforcement from the ingestion of protected material during training towards outputs generated by AI models.

Introduction

The governance of transformative technologies poses a recurring structural challenge for legal systems. Technologies of genuinely novel character arrive in advance of the regulatory categories adequate to their governance, and the institutional pressures that demand some form of regulatory response do not ordinarily permit the deliberation that principled category development would require. The characteristic response – the application of existing legal frameworks through analogical extension – is institutionally understandable but analytically hazardous. The danger is not just that existing legal frameworks are applied before new ones can be developed, but that these analogical extensions proceed without a systematic examination of whether the foundational premises of those frameworks are fully appropriate to the new technology. Where those premises do not survive, the result is the entrenchment of regulatory structures conceptually mismatched to their objects, compliance obligations that track the wrong variables, and enforcement activities that address the wrong harms.

This article examines two domains in which this structural problem is most acute in relation to the governance of artificial intelligence. The first is data protection law, which operates in most jurisdictions through a set of principles – notice and consent, data minimisation, purpose and use limitation, and retention restrictions. These were first developed in response to the emergence of large-scale data processing in the early 1970s and have since been codified in a succession of legal instruments and national legislation, most influentially the European Union’s General Data Protection Regulation (GDPR) of 2016. The second is copyright law, which protects creative works from reproduction without their authors' permission, and is being invoked to challenge the practice of training large language models and other generative AI systems on corpora of copyrighted text and other works. Although the analysis draws principally on the EU and U.S. frameworks that have been most influential globally, the structural argument is intended to travel across jurisdictions.

The central contention of this article is that both legal frameworks, when applied to artificial intelligence, fail at both practical and conceptual levels. Data protection law presumes that privacy is most effectively protected by, among other things, constraining the volume of data available for processing – an assumption systematically undermined by a technology whose performance is directly a function of the volume of data on which it is trained. Copyright law, in its AI-era applications, assumes that the ingestion of copyrighted works into an AI training pipeline constitutes a form of reproduction within the meaning of copyright doctrine – an assumption that must be questioned given how AI models are trained. Neither of these is an instance of doctrinal difficulty but rather a foundational mismatch between the conceptual architecture of the legal frameworks and the technology to which they are being applied.

The analysis proceeds in four parts. First, the author examines the data protection framework, tracing its historical assumptions, the failure of its enforcement mechanism, and its structural incompatibility with AI. Next, the author examines copyright law, its historical design, the foundational significance of the idea/expression dichotomy, and the challenge of applying it to the process of AI training. Lastly, the author provides suggestions for regulating these sectors in ways that are better aligned with the objectives that need to be achieved.

Data Protection Law

The origin of contemporary data protection doctrine lies in the work of a United States government advisory committee whose 1973 report, Records, Computers and the Rights of Citizens, identified the proliferation of automated personal data systems as a potential threat to individual privacy and proposed procedural safeguards in response. These safeguards operated on the basis that one of the best ways to guard against the misuse of individuals’ personal data would be to ensure data scarcity (the less personal data an organisation collects, the lower the risk of a privacy violation). To that end, entities were required not to collect personal data without a legitimate purpose, retain it longer than was necessary to fulfil that purpose, or use it in ways that exceed the scope of the consent obtained. These principles were subsequently adopted and elaborated upon by the OECD in its 1980 Guidelines on the Protection of Privacy and Transborder Flows of Personal Data and the Council of Europe’s Convention 108 of the same year and were ultimately codified into law by the European Union through a series of data protection directives that culminated in the General Data Protection Regulation of 2016.

In the early 1970s, large-scale personal data processing was confined to mainframe systems operated by governments and major corporations. Storage was costly, the computational resources required to process large datasets were scarce, and the barriers to collection and processing were high enough that this regulatory approach was considered appropriate. But over time, as the cost of storage and compute fell, cloud-based alternatives began to dramatically expand the quantum of data that could be processed. Despite this, data protection instruments such as the GDPR, which were enacted long after these trends were evident, persisted with the regulatory logic of data scarcity.

Modern AI systems improve as they consume more training data. Any regulation designed to promote data scarcity will limit the ability of these systems to improve. Purpose and use limitations, which require that data be used only for purposes specified in advance at the time of collection, are incompatible with AI systems whose emergent capabilities mean that the ultimate use to which they will be put cannot be described in advance. Retention restrictions, which require that personal data be deleted once it has served the purpose for which it was collected, are difficult to give effect to in digital systems where the data in question is embedded in the model’s parameters in a form that bears no straightforward relationship to its original form. 

Data protection regulations stipulate procedures that must be complied with to achieve their objectives. The trouble is that hard-coding procedures as a means of compliance ignored the technological realities of a fast-changing world, where data became easy to collect, process, and store. As a result, the behaviours these laws were supposed to prevent occurred anyway, and wherever possible, corporations used consent to achieve their ends. While these principles might have been effective when first conceptualised, they failed to address the trade-offs implicit in a world where data can offer benefits, particularly when used to train AI systems that are capable of unlocking development where it is most needed. Even though consent is only one of several lawful bases for processing, corporations seeking to do more with data often use it to frame the permissions they need in terms so broad that, through the expansiveness of their construction, they can achieve all their intended outcomes.

One challenge in developing a harm-based framework is identifying the catalogue of prohibited harms. While the mere fact that information about a person is capable of being inferred without being explicitly disclosed is often viewed as an excessive breach of personal privacy, unless this knowledge is put to use in ways that result in actual harm, this new framing will not be found unlawful. What sorts of harms would fall within such a framework? Some examples include financial harms (the use of personal information to extract or divert money), reputational harms (the use of personal information to affect how that person is perceived in public) and physical harms (the disclosure of a person’s physical location with a view to inflicting bodily harm). This list is merely indicative and not exhaustive. Since this is true of any harm-based framing, we can, at best, use an illustrative definition to arrive at an answer. 

A harm-based framework must also confront an objection of timing: a purely ex post regime that responds only after harm has occurred may be inadequate for harms that are serious and irreversible. The proposal is therefore not to abandon ex ante safeguards altogether but to shift the centre of gravity from process compliance to accountability for outcomes, retaining anticipatory obligations (such as risk assessment) where the gravity and irreversibility of potential harm justify them. 

There are also questions of liability that need to be addressed, particularly regarding their allocation between the developer of an AI system and its deployer. Particularly in complex agentic systems, the data protection harms may not be neatly attributable to a single entity when they arise from complex interactions among multiple systems. Many privacy and safety harms originate in choices made during foundation-model training, while others can be traced to the choices that deployers make when building AI applications. Any complete framework needs to allocate responsibility all along the value chain according to which actor is best placed to foresee and prevent a given harm. These are additional issues that will need to be addressed in the context of AI to ensure that the liability for privacy harms is appropriately applied. 

It is clear that regulations prescribing processes to be followed are an inadequate approach to privacy protection – particularly in a world where AI offers so much. At the same time, there is no gainsaying the fact that personal privacy needs to be protected – perhaps more so when the very same intelligence systems are capable of causing far greater harm than ever before. In this context, the appropriate response would be to reconceptualise data protection in terms of the desired outcomes rather than through process prescription. Rather than stating which processes organisations need to put in place to comply, we should simply specify the harms that AI systems must not produce and hold organisations accountable for ensuring those harms do not occur, with liability consequences when they do. This reorientation creates incentives for investment in AI safety and fairness in outcomes rather than limiting data ingestion. What is more, because this approach is technologically agnostic, the obligation to avoid specific harms remains operative regardless of the direction AI technology takes.

III. Copyright Law

Copyright was designed to give creators a bundle of entitlements over their work – control, recognition, and economic reward. Of these, control over reproduction is the one most directly engaged by AI training. 

Unlike a patent, copyright is not an exclusionary monopoly over an idea; it protects expression while leaving ideas free, and in many traditions, it also protects the author’s moral rights of attribution and integrity. The concern it was built to address was straightforward: technology made it possible for anyone to copy a book, a painting, or a piece of music and sell that copy, depriving the creator of the benefits of their labour. By stipulating that no one other than the author of a given work has the right to make copies of the work, creators can control how their creation can be perceived, experienced, and consumed, and by whom. By exercising this control, they can earn income from their creations. In essence, control over the reproduction of the work lay at the very core of the rights made available under copyright law.

Fundamental to the rights granted under copyright law is the principle that protection is available in respect of the specific form and expression of an idea but not for the idea itself. It is the specific words an author chooses, the arrangement of sentences and paragraphs, and their particular stylistic choices that give a literary work its identifiable character that is entitled to protection, not the ideas that they express. This limitation was introduced to ensure that the first person to articulate a creative concept cannot prevent anyone else from building upon it. It recognises that all creative endeavours, across literature, art and culture, are, in essence, collaborative and cumulative journeys in which works develop through a process of layering, with each new artist building upon the work of those who came before. This distinction has been maintained across jurisdictions and across centuries of technological change, from the earliest copyright cases through to the international frameworks that govern the field today.

In this context, we need to examine whether the process of training AI models constitutes an infringement of copyright.

The process of training a large language model involves ingesting vast amounts of content to build neural networks capable of performing next-token prediction, which can present the knowledge distilled from this content in various useful ways. It is important to note that while the process of training the model does require content (some of which might be protected by copyright) to be processed, that content is not reproduced and stored within the model. Instead, the model analyses the new content it is presented with in the context of all the other material it has already processed and adjusts its “weights” accordingly. 

In other words, even though it may appear that the AI model is “copying” the work during the training process, it is, to the contrary, merely processing the work without reproducing it in the way that has long been associated with copyright infringement. The argument that it does constitute copying merely because the process of training a model involves the ingestion of content conflates two things that copyright doctrine has always treated as distinct: the processing of a work and its reproduction. 

To be clear, nowhere in the letter of the law (neither under the Berne Convention nor its implementing statutes in the U.S., U.K., EU and India) has the reproduction right been confined only to copies a human can perceive. Nevertheless, the argument being advanced here is that when a model is trained, what is extracted is not protected expression, but ideas, patterns, and statistical relationships of the kind typically excluded from copyright protection. Reference to human perceptibility is thus offered not as a doctrinal test but as a way to demonstrate that the harm copyright exists to prevent simply is not manifest at the training stage.

When a large language model trains on a corpus of text, the entire corpus is first chunked and then represented as a vector that encodes statistical relationships between words, phrases, and concepts. These representations are then used to adjust the weights of the model’s neural network through an iterative optimisation process that is designed to improve the model’s ability to predict the next token in a sequence. At the end of a training run, the text itself is nowhere to be seen. What remains are billions of numerical parameters that do not have any recoverable relationship to a given sentence, paragraph, or passage in the original corpus. No human who is given the weights of a trained large language model will be able to discern from them the works on which it was trained. While in certain instances it has been demonstrated that specific prompting can elicit verbatim passages from a large language model, this is more of an output-side phenomenon rather than evidence that training results in reproduction.

The process of training strips away exactly what copyright law was created to protect – the specific expressions, word selection, arrangement, and rhythm of the original author’s prose. It only retains that which copyright explicitly does not protect – the underlying concepts, informational content, and patterns of the relationships between ideas. The extraction of unprotected ideas from a work was never meant to be prohibited by copyright. That sort of interpretation can only be upheld if we are willing to abandon the idea/expression distinction.

A person who reads widely – across novels, scholarship, journalism, science – absorbs the vocabulary, argumentative structures, ways of framing problems, and the conceptual frameworks articulated in these different works, in the process learning from all they have read. Everything they write thereafter will, at least in some small way, be influenced by all they have read. This is not considered to be an infringement of copyright because the reader does not reproduce expression but instead internalises the ideas contained in the works – ideas that copyright has always left free to build upon. AI models work in much the same way. One way to understand the process of training AI models is to think about it as more akin to learning than to copying verbatim. 

This is not to say that copyright has no role to play in relation to AI. To the contrary, all this discussion makes clear is that we have been looking at the wrong end of the AI workflow. AI systems are designed to generate content in response to a prompt. When that content substantially reproduces copyright-protected expression, it does so in a form that humans can clearly identify. This suggests that, in relation to AI systems, rights-holders’ legitimate interests are best protected if enforcement efforts are focused on the output generated, not on the process of training the model.

The concerns expressed by creators about training AI systems often centre on the threat they perceive in the economic competition AI systems introduce. Creators worry that if an AI system can generate content at scale and at comparatively low cost, it will eventually drive down demand for their work. This is a legitimate concern that deserves a considered policy response through the appropriate legal framework. 

However, that framework cannot be copyright law, which was specifically designed to address unauthorised reproduction. What we need is a completely different framework that is capable of addressing these economic concerns. Using copyright for this purpose risks mischaracterising what training does and distorting the intellectual property doctrine that has served us well across many technological transitions.

IV. Finding Solutions

Artificial intelligence will force a re-evaluation of existing regulatory frameworks. Our current data protection regulations were designed to address data scarcity because it was believed necessary to safeguard personal privacy. This approach, however, runs counter to how AI systems function and fails to account for the fact that they perform better the more data they are allowed to process. The way our current copyright laws are framed explicitly prohibits the copying of protected works. This articulation does not explain what needs to be done when such works are ingested, but no copy is released as output. 

At the same time, neither of these frameworks addresses the ancillary harms that AI systems might be uniquely capable of causing. Since data protection laws allow data fiduciaries to process data with the consent of data principals, an end-user who has consented to the use of AI to process her data may not be able to argue that her privacy has been violated as a result. Since these systems are probabilistic and non-deterministic, they are expected to produce unexpected outcomes – a feature that does not align with the purpose and use limitations that define our current approach to data protection. 

Even though copyright laws can protect only the form and expression of a work, not its underlying ideas, there is no escaping the fact that these laws were designed to offer creators some degree of protection over their intellectual property. Now that AI systems can produce creative outputs of comparable quality to those generated by real people, they risk reducing demand for work by human creators. We need to find a way to offer the creative community some form of protection from the economic challenges they are about to face. 

If these concerns are to be addressed, it is clear that existing legal frameworks are not adequate. New frameworks will need to be designed, and in this final section, we will examine alternative approaches.

A. Data Protection

Redesigning data protection for the age of AI requires three structural reorientations. First, the organising principle of optimising for data scarcity (the belief that the best way to ensure privacy is to constrain the volume of data processed) must give way to a philosophy of responsible stewardship. Organisations should be held accountable for the harms that may result from their data processing activities, regardless of the volume of data they process. This reorientation ensures that we do not abandon the core principle of protecting individual privacy but does so through the instrument of accountability, which is better calibrated to the character of AI-era privacy harm.

Second, the mechanism of prior consent should make way for a framework that focuses on the outcomes of processing. To be clear, the purpose of this is not to eliminate transparency obligations that current data protection regulations impose on the entity processing information, or the rights individuals have to be informed about how it is processed and for what purpose. Those obligations will continue to serve legitimate purposes independent of the consent mechanism. What it does instead is relocate the focus of protection from prior permission to liability for the harms caused. Since it is the organisation that deploys the AI system that is best placed to understand the consequences that could unfold when it is used, it is they who should have a duty of care to ensure that harmful outcomes do not occur.

Third, the regulatory apparatus should be reoriented from ex ante process documentation to continuous outcome monitoring. Regulatory bodies need to upskill themselves with the technical competence that will enable them to assess the performance of AI systems based on the analysis of outputs rather than the organisation’s compliance with process checklists. They need to use their investigative and enforcement powers to focus on identifying and remediating the harms arising from deployed systems, responding dynamically to situations across different use cases and circumstances. 

B. Copyright

The AI developer ecosystem needs clarity on whether the AI training process is legal. This would require a clear articulation of which types of AI training are lawful under copyright law. Various countries have introduced fair use exceptions in their laws to allow text and data mining. Japan’s law probably goes the furthest in this regard, allowing a copyrighted work to be used for purposes that do not relate to the enjoyment of the thoughts or sentiments expressed in the copyrighted work (such as AI development or other forms of data analysis), if that use would not unreasonably prejudice the interests of the copyright holder, allowed without the permission of the copyright holder. While this position does remain the subject of active debate, language to this effect, if included in the copyright statute, will provide the AI ecosystem the confidence to proceed with training their models.

It is just as important to ensure that, in the AI context, enforcement is focused on the outputs generated by AI models. Rights-holders who find that their copyright-protected expression has been substantially reproduced in AI-generated outputs should be assured that they have enforceable claims under the existing law. Where procedural or evidentiary obstacles impede the practical exercise of these claims, targeted procedural reforms should be implemented to ensure that copyright protection extends to these instances of harm in the context of AI.

Third, care should be taken to ensure that policy efforts aimed at ensuring that economic benefit accrues to creators whose work contributed to AI development are designed in a manner that aligns with the conceptual design of the AI ecosystem. Levies, remuneration, or other benefit-sharing frameworks that are supposed to operate as mechanisms for the redistribution of value should achieve these objectives without doctrinal distortion and in a manner that verifiably achieves their stated objectives. The design of these instruments must also take into account questions of scope, effectiveness of administration, and equitable distribution and may be beyond the scope of this article to delineate.

 The regulatory challenges that AI presents are not problems of doctrinal application but of a structural mismatch between legal frameworks designed for one technological era and technologies that operate on fundamentally different principles. Just as the industrial revolution demanded the creation of labour law, product liability, and the modern law of corporations, AI will require its own legal innovations. The proposals outlined above illustrate the direction that such regulatory innovation could take. Until we are willing to build from first principles rather than extend by analogy, the governance of AI will remain an exercise of forcing a new technology into old categories. The costs of such an approach will be borne not by those who frame the regulations but by those whom AI has the greatest potential to benefit.