To regulate AI, try playing in a sandbox

For an increasing number of regulators, researchers, and tech developers, the word “sandbox” is just as likely to evoke rulemaking and compliance as it is to conjure images of children digging, playing, and building. Which is kinda the point.

That’s thanks to the rise ofregulatorysandboxes, which allow organizations to develop and test new technologies in a low-stakes, monitored environment before rolling them out to the general public.

Supporters, from both the regulatory and the business sides, say sandboxes can strike the right balance of reining in potentially harmful technologies without kneecapping technological progress. They can also help regulators build technological competency and clarify how they’ll enforce laws that apply to tech. And while regulatory sandboxes originated in financial services, there’s growing interest in using them to police artificial intelligence—an urgent task as AI is expanding its reach while remaining largely unregulated.

Even for all of its promise, experts told us, the approach should be viewed not as a silver bullet for AI regulation, but instead as a potential step in the right direction.

Rashida Richardson, an AI researcher and visiting scholar at Rutgers Law School, is generally critical of AI regulatory sandboxes, but still said “it’s worth testing out ideas like this, because there is not going to be any universal model to AI regulation, and to figure out the right configuration of policy, you need to see theoretical ideas in practice.”

But waiting for the theoretical to become concrete will take time. For example, in April, the European Union proposed AI regulation that would establish regulatory sandboxes to help the EU achieve its aim of responsible AI innovation, mentioning the word “sandbox” 38 times, compared to related terms like “impact assessment” (13 mentions) and “audit” (four). But it will likely take years for the EU’s proposal to become law.

In the US, some well-known AI experts are working on an AI sandbox prototype, but regulators are not yet in the picture. However, the world’s first and (so far) only AI-specific regulatory sandbox did roll out in Norway this March, as a way to help companies comply with AI-specific provisions of the EU’s General Data Protection Regulation (GDPR). The project provides an early window into how the approach can work in practice.

“It’s a place for mutual learning—if you can learn earlier in the [product development] process, that is not only good for your compliance risk, but it’s really great for building a great product,” according to Erlend Andreas Gjære, CEO and cofounder of Secure Practice, an information security (“infosec”) startup that is one of four participants in Norway’s new AI regulatory sandbox.

A trip to Oslo...

Last May, Norway’s data protection authority, Datatilsynet, announced plans to create an AI regulatory sandbox (though it was initially set into motion by the country’s 2020 National Strategy for Artificial Intelligence). Kari Laumann, who runs the sandbox project, told Emerging Tech Brew it will cost 9 million Norwegian krone ($1.1 million) in 2021 to cover salaries, administration, marketing, communications, and other expenses.

Datatilsynet received 25 applications for just four spots in its sandbox. The program kicked off in mid-March 2021, with both public and private sector participants. This round of engagements will last 3–6 months, depending on the company. Laumann told us the sandbox hopes to welcome a second cohort in the fall.

The four sandbox participants were selected because their regulatory conundrums reflect some of the most common AI-related GDPR compliance issues Datatilsynet hears, such as applying principles like transparency, fairness, and data minimization to AI systems.

Secure Practice, the infosec startup, is working with Datatilsynet on transparency and data minimization; Age Labs, a blood sample analysis startup, is focused on anonymization; there’s an education project that is a collaboration between multiple public entities, which raises challenges with transparency, fairness, minimization, and consent concerning children’s data; and Norway’s welfare authority wants to use AI to send more customized follow-up to those who are granted sick leave, but the organization needs to figure out the legal basis for training algorithms and how to navigate transparency and fairness issues.

Generally, the goal of Norway’s AI regulatory sandbox is to facilitate compliance with some of these trickier provisions of the GDPR. It’s not looking to create a process that all tech developers go through, but rather to produce helpful precedent in fuzzy legal areas, and communicate those findings to organizations building AI systems. Its approach relies on a series of hands-on workshops and extended conversation and negotiation between tech developers and regulators.

“We are trying to help them develop privacy-friendly AI solutions within the legal framework of the GDPR,” Laumann said. “There’s a lot of uncertainty on how to interpret the rules into practical solutions. The regulation is there and there's also quite a bit of general guidance, but people are asking for examples. And that's what I think we can provide in the sandbox.”

From the sandbox will come two deliverables: a detailed report on how organizations tweaked or built their algorithms to comply with the GDPR, and project plans—i.e., the process, assessments, and discussions around each sandbox engagement. In terms of “softer” outcomes, the hope is to improve Datatilsynet’s regulatory competency by allowing it to get its hands dirty.

Gjære told us his company got involved because it wanted greater clarity on a GDPR concept called “privacy by design,” which suggests companies build privacy into models from the very beginning, rather than bolt it on as an afterthought. Secure Practice’s sandbox project is an AI service that would allow its clients to personalize employee training curriculums based on employee data that’s collected and aggregated into data profiles.

For its part, Gjære said Secure Practice is building a simpler, “proof-of-concept” version of the tool—which is not based on profiling—in parallel with the sandbox. The goal is that, when sandbox time concludes in September, Secure Practice will be able to quickly spin up and roll out a version of the tool that collects and processes its clients’ employee data without running afoul of the GDPR.

“We’re trying to combine product engineering with privacy engineering,” Gjære said, so that Secure Practice is compliant not only with the GDPR, but also in the future, as the EU’s new proposal to regulate AI looms.

The exact sandbox process varies participant-to-participant, but for Secure Practice it consists primarily of four individualized workshops: The first is about what data clients can collect on their employees, the second is on Secure Practice’s role as a data processor, third is conducting a data privacy impact assessment, and the last is about transparency and how to communicate collected data to both clients and employees. These workshops are direct meetings between Secure Practice and Datasilynet.

In between workshops, Secure Practice works with a dedicated Datatilsynet attorney to prep for the next one (e.g., the attorney will help pull together the data privacy impact assessment). There are also two common workshops among all participants—one on anonymization, and one on machine learning.

By participating, Secure Practice gets to work closely with regulators to ensure it doesn’t spend time and money building a machine-learning tool that violates the privacy of its users, and Datasilynet in turn gets to both proactively regulate and establish precedent that can help other companies navigate similar challenges. Norway’s sandbox process is itself very much a one-to-one experience—Gjære said he has a Datatilsynet attorney fully dedicated to his project—but the outcome is meant to be one-to-many.

Looking stateside...

While Norway intends for a broad group of organizations to benefit from the outcomes of its sandbox, Richardson views the lack of genuine scalability as a major challenge for sandboxes in general. After all, even in a well-resourced country like Norway, regulators were only able to directly monitor and engage with four companies (just 16% of applicants).

Richardson also noted that countries like Norway have strong regulatory regimes—there are laws that explicitly govern AI, and regulators to enforce them—which makes something like a sandbox more likely to be effective there than, say, in the US.

“You need a resourced and empowered regulator, which does exist in Europe, but I would say you don’t necessarily have here [in the US],” Richardson said. “If you take it outside of the financial sector—and even there, you could argue, there's not enough regulators—then it's like, who does that fall under, the FTC? They're not resourced enough.”

Both Laumann and Meredith Broussard, an AI researcher and NYU professor, also brought up funding as a key pillar of success with sandboxes; the latter in the sense that funding will be a hurdle in the US, and the former in the sense that Datatilsynet’s 2.5 years of fully guaranteed government funding has unlocked doors.

Nevertheless, at least one group of AI experts—including Broussard—is working to bring AI regulatory sandboxes to the US. In mid-May, Broussard formalized a partnership to develop an AI regulatory sandbox between NYU’s Algorithmic Advisory Alliance, which she directs, and ORCAA, an algorithmic auditing firm founded by mathematician and author Cathy O’Neil.

“Regulation around AI is just getting started,” Broussard said. “So we need to get started with better tools for regulators to see inside black boxes, for developers to see inside black boxes, and for business executives to see inside black boxes. It's probably going to change and evolve over time, but a regulatory sandbox is a really good first step.”

The project is still in its infancy, but O’Neil told us in the short term she imagines the sandbox prototype will be a computational platform to test a company’s algorithms for bias without getting the company in trouble. Ideally, that means it’s somewhat standardized and able to test similar algorithms for specific issues, like credit card application algorithms for racial discrimination.

She said the current stage of the project is to develop and run tests that can tell whether a given algorithm is racist, sexist, or otherwise biased, but that these tests will likely be too malleable to constitute a long-term solution. O’Neil compared them to the emissions tests Volkswagen cheated on in 2015, where the company programmed its engines to detect when they were being tested and to change their behavior accordingly. The system could be gamed, and Volkswagen seized on that.

“That is what we’re doing. We're basically developing a bring-your-car-to-the-garage test: send us your algorithm, show us your data,” O’Neil said. “You know what's happening—we do, too.”

O’Neil said developing these tests is a first step toward a longer-term goal for the sandbox, which is to arrive at a point where these tests are standardized and conducted without companies choosing what data to share, and ideally without them even knowing the test is happening. This would be more akin to the way restaurant inspections work: They happen when the inspector wants, without warning, so they can catch restaurants in their true state rather than on artificially good behavior. The hope is also that regulators will eventually step in.

And while Broussard and O’Neil’s sandbox prototype will likely differ in specifics from what Norway currently has in place, O’Neil said the overall point of the AI sandbox is the same: to arrive at more concrete guidance around the rules that govern algorithms.

“How do we define enforcement in the age of the algorithm? How do we codify enforcement in the age of algorithms?” O’Neil said. “This is like a play time when we're going to be figuring that out.”