NVIDIA Photonics CPO Switches Deployed: Lambda Benchmarks 10,000 Blackwell Ultra GPUs at Scale

Lambda has deployed liquid-cooled NVIDIA Photonics CPO switches across its Quebec supercomputing campus, connecting over 10,000 NVIDIA Blackwell Ultra GPUs to achieve 10.1% lower pod-scale latency and 5x power efficiency.

Photo of author
NVIDIA Photonics CPO switches deployed in Lambda AI factory
Lambda deployed liquid-cooled optical switches to interconnect over 10,000 Blackwell Ultra GPUs at rack and pod scale. (Photo: NVIDIA / Lambda)

Artificial intelligence cloud hyperscaler Lambda has achieved a major milestone by deploying NVIDIA Photonics CPO switches across its primary datacenter campus. The breakthrough installation directly connects over 10,000 Blackwell Ultra GPUs to accelerate next-generation foundation model workloads.

By replacing fragile pluggable optical transceivers with co-packaged silicon engines, these high-bandwidth networking switches eliminate substantial signal attenuation while delivering unprecedented thermal efficiency and extended system reliability across large-scale distributed artificial intelligence clusters.

Architectural Specification NVIDIA Photonics CPO switches Conventional Pluggable Optical Transceivers
Switch Port Density & Bandwidth 144 Ports @ 800 Gb/s (115.2 Tbps aggregate bandwidth) 64–72 Ports @ 800 Gb/s (51.2–57.6 Tbps aggregate bandwidth)
Electro-Optical Engine Placement Co-packaged optics integrated directly on switch silicon substrate Front-panel pluggable cages (OSFP / QSFP-DD form factors)
Electrical Trace Length to Optics Millimeters (short package substrate traces) Inches (long PCB motherboard traces with signal loss)
Retimer / DSP Signal Requirements Eliminated entirely (no standalone retimer chips required) Mandatory power-hungry retimer DSP chips in every transceiver
Laser Source Configuration 18 Field-Replaceable External Light Sources (4x fewer lasers) Individual internal lasers embedded in every transceiver plug
Median Operating Temperature 36°C (actively liquid-cooled optical engines) 51°C (passive or forced-air heatsink cooling)
Interconnect Latency Improvement Up to 10.1% latency reduction across full pod-scale fabrics Baseline standard network latency
Energy Efficiency & Reliability 5x higher optical power efficiency, 15x higher MTBI reliability Baseline high electrical draw and frequent laser wear

Co-Packaged Optics Architecture: Eliminating Front-Panel Optical Transceivers

Modern frontier model training clusters require tens of thousands of compute dies to exchange matrix gradients simultaneously. Traditional networking fabrics suffer from electrical resistance at copper traces, prompting engineers to implement NVIDIA Photonics CPO switches across modern pods.

Conventional network switches route high-speed electrical signals several inches across printed circuit boards to reach front-panel transceivers. This legacy architecture demands power-hungry digital signal processing retimers that generate massive heat and increase overall cluster latency.

In stark contrast, modern NVIDIA Photonics CPO switches position miniature optical engines directly alongside the central switching silicon substrate. Shortening the electrical path from several inches to mere millimeters preserves pristine signal integrity and eradicates standalone retimer requirements.

By moving electro-optical conversion onto the package, each switch chassis drastically reduces signal degradation and parasitic capacitance. This unified packaging approach enables higher transmission frequencies without degrading high-speed serial links during intensive distributed training runs.

Co-packaged optics switch chassis and fiber harness
Internal fiber harness and co-packaged optics switch architecture deployed at Lambda’s Quebec campus. (Photo: Lambda Labs)

Enterprise datacenter operators adopting NVIDIA Photonics CPO switches achieve five times greater optical power efficiency than legacy solutions. Saved electrical wattage is directly returned to high-density compute nodes, mirroring architectural power-saving strategies highlighted in Apple’s M4 desktop silicon architecture.

Thermal Efficiency and Active Liquid Cooling: Operating Optics at 36°C

Thermal stress represents the leading cause of field failures for high-speed optical transceivers deployed inside dense modern datacenters. Integrating liquid cooling across NVIDIA Photonics CPO switches completely resolves persistent thermal dissipation bottlenecks in modern high-density racks.

Conventional pluggable optics operate at elevated median temperatures exceeding 51 degrees Celsius under sustained network traffic. Benchmark data confirms that NVIDIA Photonics CPO switches maintain a cool median operating temperature of just 36 degrees Celsius.

Silicon photonics optical engine co-packaged on ASIC
Co-packaged optical engines integrated alongside central switch silicon to eliminate electrical retimers. (Photo: NVIDIA)

This dramatic 15-degree thermal reduction prevents laser wavelength drift and preserves strict bit error rate tolerances under peak workloads. Direct-to-chip liquid cooling loops efficiently channel heat away from both switching silicon and co-packaged optical engines concurrently.

By stabilizing thermal envelopes across high-bandwidth backplanes, NVIDIA Photonics CPO switches dramatically curb thermal degradation over time. Maintaining stable optical operating conditions protects delicate semiconductor junctions, preventing premature component wear during prolonged deep learning training jobs.

Furthermore, eliminating air cooling requirements inside switch chassis significantly reduces acoustic noise and parasitic parasitic fan power draw. Facilities can redirect auxiliary airflow to secondary cooling duties, lowering overall campus power usage effectiveness across operational lifespans.

Interconnect Latency Benchmarks: 10% Latency Reduction Across Pod Scale

Latency penalties compound exponentially when training mixture-of-experts architectures and running autonomous agent inference chains across distributed clusters. Rigorous benchmarking reveals that NVIDIA Photonics CPO switches reduce fabric latency while preserving full deterministic network throughput.

Testing across isolated single-rack deployments demonstrates an immediate 3.8 percent network latency improvement compared to traditional optical transceivers. Eliminating retimer buffering stages shaves crucial nanoseconds off inter-GPU collective communications during synchronized backward gradient propagation steps.

NVIDIA Quantum-X Photonics 144-port 800G switch
The 144-port 800 Gbps NVIDIA Photonics switch chassis featuring liquid-cooled optical engines. (Photo: NVIDIA)

When scaling up to extensive pod configurations, NVIDIA Photonics CPO switches widen their performance lead to 10.1 percent lower latency. Maintaining low transit times across thousands of GPU endpoints accelerates collective operations like all-reduce and reduces barrier synchronization idling.

Crucially, this latency reduction occurs without compromising bit error rates or dropping line-rate throughput across all active ports. Engineers confirm that co-packaged optical architecture sustains identical bandwidth saturation as traditional optics while moving packets noticeably faster.

For multi-agent systems and real-time reasoning workloads, shaving fabric latency directly translates into responsive customer-facing inference pipelines. Minimizing tail latency ensures that complex agentic reasoning loops finish execution without hitting unexpected communication timeout thresholds.

Resiliency Engineering: Field-Replaceable Light Sources and MTBI Scaling

In massive distributed clusters, a single optical transceiver failure can halt thousands of synchronized processors during multi-week training checkpoints. To prevent costly job interruptions, NVIDIA’s engineering team designed NVIDIA Photonics CPO switches with a modular external light source architecture.

Each 144-port switch utilizes 18 modular external light sources rather than embedding fragile lasers directly onto inaccessible optical engines. This modular topology requires four times fewer lasers than traditional transceivers, reducing the statistical surface area for hardware malfunctions.

Hyperscale AI factory cluster scaling across datacenter racks
Hyperscale AI factory pod architecture interconnecting thousands of compute accelerators across modular rows. (Photo: NVIDIA)

Whenever a laser diode experiences degradation, technicians service NVIDIA Photonics CPO switches by simply hot-swapping the external laser module. Replacing the external cartridge takes seconds without disturbing the underlying liquid-cooled switch package or disconnecting intricate optical fibers.

As a direct consequence of this resilient modular design, hardware clusters achieve up to fifteen times greater operational reliability. Extending mean time between interruptions ensures multi-month model training runs proceed smoothly without costly cluster restarts or checkpoint rollbacks.

Data center reliability teams report that deploying NVIDIA Photonics CPO switches drastically simplifies routine preventative maintenance protocols. Standardizing on field-replaceable optical sources lowers spare parts inventory costs while guaranteeing high continuous cluster availability for tenants.

Cluster Scaling Tier Total Compute Accelerators Network Interconnect Fabric Measured Latency Delta with CPO
Single Rack Scale 72 NVIDIA GB300 NVL72 GPUs NVLink 5 Copper Backplane + Liquid-Cooled CPO 3.8% Latency Reduction
Scalable Unit (SU) Scale 576 Interconnected GPUs Two-Tier Leaf-Spine with Co-Packaged Switches 6.4% Latency Reduction
Full Pod Scale 1,152 to 2,304 GPUs Full Non-Blocking Fabric with Co-Packaged Optics 10.1% Latency Reduction
Hyperscale AI Factory 10,000+ Blackwell Ultra GPUs Multi-Pod Quantum-X / Spectrum-X Optical Backbone 10.1% Sustained Low Latency

Hyperscale AI Factory Economics: Deploying GB300 NVL72 Racks in Hydro-Powered Campuses

Lambda deployed its flagship photonics cluster at its state-of-the-art supercomputing campus located in Quebec to maximize environmental efficiency. Pairing NVIDIA Photonics CPO switches with clean hydroelectric power yields an exceptional power usage effectiveness rating of just 1.2.

The site commands 142 megawatts of secured renewable energy capacity, leveraging cold northern ambient air for free cooling throughout the year. Zero process water consumption for server cooling preserves regional aquatic ecosystems while delivering uninterrupted compute to global enterprise customers.

Racks of Blackwell Ultra GB300 NVL72 hardware are interconnected through high-density 800 Gbps optical links across the campus floor. By outfitting the facility with NVIDIA Photonics CPO switches, Lambda enables tenants to lease massive GPU capacity with enterprise-grade resilience.

The deployment proves that commercial cloud providers can operate cutting-edge optical fabrics at scale without incurring unsustainable operational overhead. Much like NVIDIA shifted enterprise ecosystems in its Windows 10 driver transition roadmap, NVIDIA Photonics CPO switches establish a new operating foundation.

As AI cluster sizes expand toward hundreds of thousands of accelerators, optical integration will become mandatory across all modern datacenters. Lambda’s successful operational deployment establishes a practical blueprint for hyperscale infrastructure engineering over the coming decade.

Frequently Asked Questions

What are NVIDIA Photonics CPO switches and how do they differ from pluggable optics?

Unlike conventional transceivers attached to front panels, NVIDIA Photonics CPO switches integrate optical engines directly onto the processor substrate. This design eliminates inches of copper trace resistance, removes retimer chips, and drastically improves overall electrical efficiency.

What latency improvements do co-packaged optical switches deliver in AI clusters?

In empirical benchmark testing, NVIDIA Photonics CPO switches reduced network latency by 3.8 percent at rack scale and 10.1 percent at pod scale. These latency reductions accelerate collective operations and minimize idle synchronization delays across thousands of interconnected GPUs.

Are lasers field-replaceable if a light source fails on co-packaged switches?

Yes, NVIDIA Photonics CPO switches feature 18 field-replaceable external light sources supplying optical power to engines via fiber. Technicians can replace failing laser cartridges in minutes without disrupting switch processors or complex internal liquid cooling loops.

How does active liquid cooling benefit co-packaged optics compared to air cooling?

Operating with integrated liquid cooling, NVIDIA Photonics CPO switches maintain a median temperature of 36°C, compared to 51°C for pluggable optics. This 15-degree reduction prevents laser wavelength drift, reduces bit errors, and extends hardware lifespan substantially.

Do co-packaged optics switches sacrifice network throughput to achieve lower power?

Yes, NVIDIA Photonics CPO switches maintain identical 800 Gbps per-port line-rate throughput across 144 ports, delivering 115.2 Tbps aggregate bandwidth. They achieve lower latency and 5x power efficiency without sacrificing raw interconnect bandwidth.

Enterprise Infrastructure Verdict: The Co-Packaged Networking Era Begins

The transition from pluggable transceivers to NVIDIA Photonics CPO switches marks a decisive turning point in supercomputing interconnect technology. Lambda has proven that co-packaged optics deliver measurable latency reductions, dramatic thermal advantages, and superior cluster resiliency at scale.

With 10,000 Blackwell Ultra GPUs humming smoothly across liquid-cooled fabrics, optical networking is no longer an experimental laboratory curiosity. High-density cluster deployments echo the architectural refinements seen when scaling hardware in our Ryzen 7 9800X3D architectural analysis.

Datacenter architects seeking to eliminate communication bottlenecks will inevitably look to NVIDIA Photonics CPO switches as the gold standard. As bandwidth demands skyrocket, co-packaged photonics will ensure global AI infrastructure scales sustainably to meet future computational demands.

Get the latest tech news & gaming updates.

SUBSCRIBE TO OUR NEWSLETTER

We don’t spam! Read our privacy policy for more info.