Sovereign AI in Health: German hospitals are building their own AI models, trained on their own data, on local servers

For over 20 years, hospitals outsourced their software needs to third-party vendors, creating deep dependencies on external providers for clinical workflows, documentation, and data management. The most notable attempt to break this pattern was Kaiser Permanente in the US, which invested billions in building proprietary systems before ultimately switching to Epic. This history explains why most hospitals have been reluctant to build rather than buy.

The AI era is changing this calculus. Unlike traditional hospital software, AI models can be downloaded as open-source weights, fine-tuned on institutional data, and deployed on local GPU infrastructure. The marginal cost of training a hospital-specific model on top of an open-source foundation is a fraction of the cost of building an EHR from scratch. German university hospitals, which combine large patient datasets with academic research capabilities and dedicated IT departments, are uniquely positioned to lead this insourcing shift. The result is a new class of hospital-built AI that runs entirely within the institution's own network, with no patient data leaving the premises.

Three German institutions illustrate different approaches to sovereign AI deployment. First, ORPHEUS is a medical speech recognition system developed by IDM (Institute for Digital Medicine), a non-profit spin-off from UKE Hamburg, Germany's leading digital hospital which went paperless in 2007. IDM operates its own GPU infrastructure at UKE, training on 7 million patient records to build proprietary foundation models. ORPHEUS runs locally in University Hospital Aachen's data centre with no cloud detour, no patient data leaving the hospital, and no Big Tech dependency. According to customer feedback, it significantly outperforms standard commercial market solutions.

Second, Schon Klinik has operated its own locally hosted LLM since 2025, driven directly by the CEO. The platform enables every employee to build their own custom AI agents for recurring administrative tasks, democratizing AI access across the organization. Third, Leipzig University Hospital (UKL) launched UKL-GPT in March 2025, an in-house AI assistant that searches nearly 9,000 internal documents and operates entirely within the hospital's own data centre, serving all professional groups.

Not all hospitals can replicate this model. The institutions with insourcing capacity fall into two categories. University hospitals including UKE Hamburg, Leipzig, Essen, LMU Munich, and Charite Berlin have the combination of large patient datasets, academic research infrastructure, dedicated IT teams, and GPU hardware needed to train and deploy proprietary models. Large private hospital chains including Helios, Asklepios, Sana, and Schon Klinik have the scale and centralized IT governance to invest in sovereign infrastructure across multiple sites.

However, approximately two-thirds of Germany's 1,800 hospitals lack the financial resources, technical expertise, and data volume to build their own AI models. These smaller community hospitals and regional providers risk being left behind in the sovereign AI shift, creating a two-tier system where advanced AI capabilities concentrate in large institutions. The upcoming hospital reform (KHVVG) will consolidate the German hospital landscape, but consolidation alone does not solve the AI infrastructure gap for the institutions that remain independent.

The solution for the majority of German hospitals lies in shared structures. The Truveta model in the United States offers a blueprint: a consortium of health systems that pool de-identified patient data into a shared research platform while maintaining provider ownership through an equity structure. Each participating system retains governance rights over how its data is used, avoiding the aggregator model where a commercial third party controls the data and extracts the value.

According to HGM Advisory's analysis, Germany must establish similar provider-owned equity structures before commercial data aggregators dominate the market. Shared sovereign infrastructure could operate at the state (Bundeslander) or regional level, with university hospitals acting as technical anchors that provide GPU infrastructure, model training capabilities, and deployment support to smaller hospitals in their network. The critical design principle is that data governance must remain with the contributing institutions rather than being centralized in a single entity. IDM's approach, operating as a non-profit spin-off that distributes its products (ORPHEUS, ARGO) to other German hospitals, already demonstrates one viable model for scaling sovereign AI beyond a single institution.

The shift from corporate cloud dependency to institutional sovereignty raises an important question highlighted by healthcare AI researcher Raquel Correia: where do patients sit in this model? Sovereign hospital AI keeps data within institutional walls rather than sending it to US hyperscalers, which is a clear improvement for data protection. But it also risks creating institutional data silos where patient data is controlled by hospitals rather than by patients themselves.

The European Health Data Space (EHDS), expected to reach implementation readiness by 2027-2028, aims to give patients the right to access, port, and control their health data across institutions and borders. Sovereign hospital AI must be designed with EHDS interoperability in mind from the start. The hospitals that build their AI infrastructure on open standards (openEHR, FHIR) and design for patient data portability will be better positioned when EHDS mandates take effect. The goal should be sovereignty that protects patient data from Big Tech extraction while simultaneously empowering patients to access and direct the use of their own health information.