Governing AI Starts with Giving Users Control Over Their Data

Sept. 15, 2025 — Technology news at present is an endless firehose of artificial intelligence (AI) coverage, with a dissonant blend of optimistic hype and acute fear. While every innovation wave prompts these mixed reactions, emotional reactions to AI are staggering. Some people breathlessly assert that Artificial General Intelligence or AGI, is coming in a matter of months; others point out that the newest models continue to struggle with basic hallucinations, such as miscounting the number of letter “b”s in the word “blueberry,” not to mention more serious concerns about their safety. AI’s technical performance on some metrics continues to improve, yet notable critics such as Gary Marcus say the current generation of large language models have plateaued technically.

Credit: NicoElNino/Shutterstock

The interest and attention paid to AI mean there are many efforts to regulate it. Building on older concerns about AI injecting discrimination and bias into key life-shaping decisions such as whether to approve a mortgage or grant parole, the explosion of generative AI today leads to more nuanced and complex questions related to speech and creativity. If AI mimics people, should it be regulated as such? The makers of AI systems, civil society, and regulators around the world are seeing what can happen without effective safeguards, notably including instances of “AI-induced psychosis.”

There isn’t any single governance idea or framework that can address this complexity with simple interventions. But if we’ve learned anything from the (short) history of digital technologies, the heart of governance of AI will and must be grounded in the governance of the data powering it.

Data is valuable, as a thorough report from the Bennett Institute for Public Policy and Open Data Institute (ODI) showed years ago. That value has only become clearer since then. The UK government fully recognises this, and in the digital sector is presently looking at how to extend its existing Smart Data frameworks.

At the same time, the role of data in AI is complex. Another report from ODI presents a taxonomy of data in AI, ranging from model and training data in various forms to personal interactions with systems. Access to, and interoperability within, these types of data is key for the future of the AI ecosystem, to be sure. The issues are different for training data, so I will not consider them here; Croissant at MLCommons, for example, looks at this question. Here, I consider the data involved in AI use.

Today, the AI universe is fairly open, looked at from a personal data lens. It is customary for users to be able to download their conversation histories, and mainstream services are building with MCP and A2A and other shared, interoperable protocols.

Will it stay that way? Maybe. But some are concerned that future commercial incentives will push companies to lock users into their services by asserting control over personal data. Personal data is extremely valuable in this context, and despite a foundation of data protection law in much of the world granting user access and transfer rights, some attempts at control seem inevitable.

User data portability creates clear and sustainable benefits for privacy and competition alike in the digital sector. To protect these benefits, my organization, the Data Transfer Initiative (DTI), has proposed five principles for the portability of personal data in the context of AI:

• Users should be able, at their request, to download personal data from AI services, and to request the direct transfer of personal data between AI services. This data should be in a structured, machine-readable, and well-documented format.
• User-directed portability should focus on personal data, and should not extend to training data, model weights, or other elements of AI services not specifically related to the user initiating data transfer; however, personal data used to customise the actions of the AI service should be included as within scope.
• Data portability tools and interfaces should adhere to an open, interoperable technical specification to allow users to easily transfer personal data directly between AI services on a reciprocal basis.
• AI services, including generative services, AI agents, and tools, should communicate with other services through open protocols and should not impose unduly restrictive terms on data interfaces to ensure that users have their choice of products or services.
• Where data is transferred directly between service providers at a user’s request, all parties should employ reasonable, well-documented frameworks and practices for security vetting of the other party to the transfer, including organizational policies regarding data privacy, data security, end user transparency, and authentication of transfer parties and end users.

How do these principles fit with the ongoing regulatory efforts related to AI? Presently, these efforts, such as the European Union’s (EU) landmark AI Act, are not focused on personal data per se. Nor do they need to be, explicitly, given the existing landscape of portability obligations in both data protection and competition law, particularly in the EU. Yet there are some gaps in existing frameworks.

It is important that the market for AI services remains open, so innovation and competition will continue to flourish – at least with respect to the potential for gatekeeping behaviour over access to personal user data.

However, should any services begin to lock this data up, regulators should and will get involved. But the need for further complicated and burdensome regulation can be avoided with the widespread development of portability tools, policies, practices, and protocols so that the DTI principles set out here are realised in practice.

The views and opinions expressed in this post are those of the author(s) and not necessarily those of the Bennett Institute for Public Policy.

Source: Chris Riley, Data Transfer Initiative; Bennett Institute for Public Policy