Photonic computing achieves practical inference speeds for transformer models at 10x energy reduction
A semiconductor startup demonstrated photonic matrix multiplication for transformer attention mechanisms with practical throughput at a measured 90% energy reduction versus equivalent GPU inference. Limitations in memory bandwidth and reconfiguration speed remain, but the proof-of-concept changes the economics of inference at scale.
The transformer architecture that underlies modern AI systems — the attention mechanism, in particular — is computationally dominated by large matrix multiplications. This is what GPUs are good at: performing thousands of multiply-accumulate operations in parallel using electrical signals. What photonic computing offers is a physically different substrate for the same mathematical operation: using light instead of electrons to perform the multiplications, with the potential for substantially lower energy consumption and higher throughput.
The physics is compelling. Multiplying two numbers by modulating the amplitude of a light signal and then detecting the result requires no current flow through a resistive element, which is where most of the energy in electronic computing is dissipated as heat. Optical matrix-vector products can, in principle, be performed at the speed of light with energy consumption proportional to the optical power rather than to the computational complexity.
The engineering challenges are what has kept photonic computing from displacing electronics for decades: converting between the digital electronic domain and the analog optical domain requires high-quality digital-to-analog and analog-to-digital converters, which are expensive, slow, and noisy. Storing intermediate results requires either converting back to electronics or developing optical memory, which remains a hard problem. Programming photonic circuits requires physical reconfiguration, which is much slower than loading new weights into GPU memory.
What the startup demonstration showed is a hybrid approach: the matrix multiplications in the attention mechanism run photonically, while memory access, control logic, and the non-linear activations run electronically. In this configuration, the system achieved the reported 90% energy reduction for the computational core, while accepting the overhead of domain conversion at the boundaries. For inference workloads — where the weights are fixed and the same matrix multiplications run millions of times — this overhead is amortizable.
The remaining limitation is reconfigurability: switching to a different model requires physically reconfiguring the photonic circuit, which currently takes seconds to minutes rather than the microseconds a GPU needs to load new weights. For large-scale inference serving where the same model runs continuously, this is manageable. For development environments where researchers switch models frequently, it is not.