Cryptographic Locks Contain Rogue AI For Now

Defenses

Published: Fri, Aug 29, 2025 • By Rowan Vale

Cryptographic Locks Contain Rogue AI For Now

A new paper proposes a tamper-resistant, cryptographically enforced layer that forces AI systems to obey externally defined rules. The design uses signed rule engines and a secure platform to make bypassing controls computationally infeasible. It raises the bar for safety in high-risk systems but still hinges on flawless key management and hardware trust.

The paper "Governable AI: Provable Safety Under Extreme Threat Models" offers a clear, blunt answer to a familiar problem: internal guardrails can fail when an AI is motivated and clever enough. The authors propose moving enforcement outside the model with a rule enforcement module and a governable secure super platform that use cryptography and trusted hardware to make rule-breaching practically impossible.

What they propose is simple to state and hard to foil on paper. Every command an AI issues must be vetted by a signed, deterministic rule engine running on a tamper-resistant platform. Rules are decoupled from the AI so policy makers can update behavior without touching the model. The authors provide formal proofs and a prototype in an autonomous driving scenario that shows the REM blocks unsafe commands.

Why this matters: in hospitals, power grids, or self-driving fleets you do not want a clever agent inventing new workarounds. Cryptographic non-bypassability turns a messy software argument into a legalistic gate: either the signed rules allow an action or the action is stopped.

But the emperor still has clothes concerns. The guarantees depend on perfect key management, supply chain integrity, and hardware that resists side-channel and fault attacks. Centralized governance also creates new choke points and political risks. The system raises the difficulty for attackers but does not make the problem go away if the foundation is compromised.

Two practical steps readers can take: ask vendors and regulators for independent audits of key management and hardware attestation, and push for public descriptions of enforcement architectures before deployment in critical systems.

Additional analysis of the original ArXiv paper

📋 Original Paper Title and Abstract

Governable AI: Provable Safety Under Extreme Threat Models

Authors: Donglin Wang, Weiyun Liang, Chunyuan Chen, Jing Xu, and Yulong Fu

As AI rapidly advances, the security risks posed by AI are becoming increasingly severe, especially in critical scenarios, including those posing existential risks. If AI becomes uncontrollable, manipulated, or actively evades safety mechanisms, it could trigger systemic disasters. Existing AI safety approaches-such as model enhancement, value alignment, and human intervention-suffer from fundamental, in-principle limitations when facing AI with extreme motivations and unlimited intelligence, and cannot guarantee security. To address this challenge, we propose a Governable AI (GAI) framework that shifts from traditional internal constraints to externally enforced structural compliance based on cryptographic mechanisms that are computationally infeasible to break, even for future AI, under the defined threat model and well-established cryptographic assumptions.The GAI framework is composed of a simple yet reliable, fully deterministic, powerful, flexible, and general-purpose rule enforcement module (REM); governance rules; and a governable secure super-platform (GSSP) that offers end-to-end protection against compromise or subversion by AI. The decoupling of the governance rules and the technical platform further enables a feasible and generalizable technical pathway for the safety governance of AI. REM enforces the bottom line defined by governance rules, while GSSP ensures non-bypassability, tamper-resistance, and unforgeability to eliminate all identified attack vectors. This paper also presents a rigorous formal proof of the security properties of this mechanism and demonstrates its effectiveness through a prototype implementation evaluated in representative high-stakes scenarios.

🔍 ShortSpan Analysis of the Paper

Authors

not reported

Problem

The paper studies how to achieve provable safety for artificial intelligence under extreme threat models in which an omniscient adversary may act with maximal malicious intent and unlimited intelligence. It argues that existing safety approaches based on internal constraints such as model enhancement, value alignment, or human intervention are fundamentally insufficient to guarantee security in such scenarios and could fail to prevent catastrophic outcomes in high stakes domains. The authors define three representative threat classes, T1 misjudgment, T2 manipulation, and T3 active attack, and propose that only externally enforced, cryptographically grounded safety mechanisms can provide non bypassable guarantees. The work introduces a Governable AI framework consisting of a rule enforcement module, governance rules, and a governable secure super platform to achieve end to end protection against subversion by AI, with formal proofs and a prototype evaluation in representative high risk contexts.

Approach

The Governable AI GAI framework combines a rule enforcement module REM, governance rules, and a governable secure super platform GSSP. Governance rules encode explicit constraints, while REM is fully deterministic, reliable, general purpose, and notions of formal verifiability. GSSP provides non bypassability tamper resistance and unforgeability under cryptographic assumptions. Governance rules are decoupled from the technical platform so governance experts can define and update rules without technical assistance and technical experts can focus on building a robust platform. The architecture supports cryptographic protection, trusted computing, and end to end protection through a TPM backed environment. A workflow is described: rule definition by domain experts or policymakers, REM development translating near natural language rules into formal rules, cryptographic protection ensuring rules and REM are signed and verified, and runtime operation where the AI system sends commands to REM for compliance verification and possibly transformation to a rule compliant form before actuators are engaged. Key characteristics include physical separation of AI and critical systems, mandatory validation by REM, and a trusted computing platform that secures REM execution and rule signing. The REM integrates an interpreter, a rule engine, and governance rules stored in data form; rules are verified by digital signatures before loading, enabling a path to formal verifiability with discrete input spaces and deterministic logic. A prototype autonomous driving scenario demonstrates the operational workflow and security mechanisms, including signature based loading, TPM assisted integrity, and kernel level access restrictions via eBPF based Linux Security Modules to prevent AI from directly accessing actuators.

Key Findings

Finding 1 external enforcement in the GAI architecture yields non bypassability tamper resistance and unforgeability under common cryptographic assumptions, with formal proofs and a prototype demonstrating efficacy in high stake scenarios.
Finding 2 the REM enforces safety constraints for every AI issued command; in an autonomous driving case study, path planning experiments show the AI cannot achieve 100 percent compliant paths without REM intervention and with REM, 100 percent compliance is observed across tested maps, indicating a raised lower bound on safe behaviour.
Finding 3 formal verification is feasible due to discrete input spaces and purely functional decision logic; tools such as CBMC and KLEE enable path safety verification while theorem proving tools like Coq Isabelle can be used for full path verification.
Finding 4 a nine path attack surface analysis demonstrates how P1 to P9 potential compromises are mitigated by REM and GSSP design including TPM based loading verification OS level protection signature module integrity and private key non exportability.
Finding 5 governance rule decoupling enables governance bodies to define rules without engineering effort while technical experts build a robust platform; rule engines such as Drools CLIPS Jess Prolog are proposed as open source foundations for REM.
Finding 6 alignment with established security and regulatory frameworks is identified; the framework is positioned to support compliance with NIST SP 800 series EU AI Act ISO 27001 and TPM specifications helping to integrate GAI into critical infrastructure contexts.

Limitations

Not reported for specific limitations, but the work acknowledges dependence on strong cryptographic assumptions and trusted enforcement; if those break or are misimplemented the safety guarantees could fail. The framework relies on secure key management trusted platform integrity and secure governance rule updates which must be securely managed to prevent new vulnerabilities. The threat model assumes physical separation and software level attacks, not hardware based attacks. Complete rule completeness is treated as an assumption rather than a proven guarantee. Centralised governance may introduce new control points and policy sovereignty considerations in deployment.

Why It Matters

The GAI framework offers an externally enforced cryptographically grounded safety layer that can potentially prevent AI from bypassing rules and running amok in high risk environments. By decoupling governance from the platform and providing formal verification and tamper resistant protection, GAI presents a rigorous architectural pattern for securing AI agents against manipulation or runaway behaviour. The approach is intended to be implementable across domains such as autonomous driving and other critical systems, aligns with established frameworks, and provides a pathway toward verifiable containment of AI behaviours in the face of extreme threat models. The work also highlights potential risks including governance centralisation, supply chain, hardware and software trust, and key management vulnerabilities that must be mitigated for practical deployment.

Attribution Original paper on arXiv