Securing the Most Valuable Room in the Company

A conventional server room held a lot of replaceable hardware and data that was, usually, also somewhere else. A GPU hall is different on both counts: a single rack is a seven-figure asset with months of lead time, and the model weights and training data on the storage next to it may be the most valuable intellectual property the company owns. Security programs designed for the old room do not cover the new one.

Five Layers

We think about AI facility security as five layers: the site (fence, gates, cameras, lighting, loading dock), the building (reception, mantraps, badge-plus-biometric access, visitor and vendor escort), the hall (cage access, per-rack electronic locks, CCTV with retention), the rack (firmware attestation, encrypted media, tamper evidence), and the fabric (segmentation of GPU, storage, management and out-of-band networks, controlled access to BMCs, switches and the subnet manager). A gap at any layer undermines the others; a program that is strong at the building and silent at the fabric is the common pattern.

The Fabric Is Where General IT Security Is Weakest

InfiniBand and RoCE bypass the operating-system kernel for performance, which also means they bypass the host firewalls, agents and inspection most security teams rely on. Isolation has to be done in the fabric itself — partition keys on InfiniBand, VXLAN and EVPN segmentation on Ethernet fabrics, strict control of who can reach the subnet manager and switch operating systems. Management and out-of-band networks for BMCs deserve zero-trust access with per-session authentication, because a compromised BMC is a compromised node regardless of what runs on it.

Supply Chain Is a Security Control

Optics, cables, drives and even whole systems are counterfeited or tampered with in transit. Sourcing through authorised distribution, chain-of-custody records from distributor to rack, tamper-evident packaging and firmware verification on arrival are inexpensive controls that auditors increasingly expect and that most facilities skip.

Certification Is the Output, Not the Goal

Customers ask for SOC 2; international partners ask for ISO 27001; US government work follows NIST 800-171 toward CMMC or FedRAMP; facility ratings follow TIA-942 or Uptime. The certificate is evidence that the controls above exist and operate. Designing to the control set first and mapping to the frameworks second produces a program that passes several audits with one body of evidence, instead of a different scramble for each.

Where to Start

With an assessment: a threat model for the facility and workload, a gap analysis against the frameworks that are actually being asked for, a penetration test of the management networks and a physical walk-through. That produces a risk register and a phased roadmap that can be implemented while the hall stays in production — which is how most of these programs have to be delivered.