Blog › Why FPGAs missed the AI boom
Why FPGAs missed the AI boom: a post-mortem
The FPGA was more programmable. The GPU was more programmable.
Published 2026-08-21. AI assisted in writing this post.
There is a curious historical puzzle in the field-programmable gate array (FPGA) industry.
By the early 2010s, many of the ideas that now seem central to modern computing were already familiar to FPGA engineers. General-purpose processor performance could not continue scaling indefinitely. Important workloads would require specialized acceleration. Data centers would become heterogeneous. Algorithms would keep changing, making fixed-function application-specific integrated circuits (ASICs) undesirable for some applications. It therefore seemed reasonable that a reprogrammable accelerator sitting beside the central processing unit (CPU) might become an important part of the future data center.
This was not merely a theoretical argument.
Microsoft’s Project Catapult began investigating FPGA acceleration for Bing around 2010. A 1,632-server scale pilot followed in 2012, and the 2014 paper describing it reported that the reconfigurable fabric improved per-server ranking throughput by 95% at a fixed latency distribution. Bing put the technology into production in early 2015, FPGA-enabled servers were subsequently deployed at scale across Bing and Azure, and the later Brainwave project demonstrated FPGA-based real-time neural-network inference.
So the basic hypothesis —
CPU + reconfigurable accelerator
— was not unreasonable. In fact, it worked.
Intel apparently saw a similar opportunity. In 2015 it agreed to acquire Altera for approximately $16.7 billion, explicitly planning to offer Altera FPGAs alongside Xeon processors as integrated data-center products. When the acquisition closed, Intel CEO Brian Krzanich named machine learning among the future applications the combination could enable.
Five years later, AMD announced its $35 billion acquisition of Xilinx. AMD’s argument was broader, but remarkably similar in spirit: future computing would be heterogeneous, algorithms would continually evolve, and adaptive computing would complement CPUs and graphics processing units (GPUs) in accelerating those workloads.
Then the artificial intelligence (AI) computing boom actually arrived.
And the FPGA was not its principal beneficiary.
The GPU was.
This raises a useful question for FPGA engineers: how did an industry built around programmable acceleration anticipate so much of the future correctly, while capturing relatively little of the enormous AI accelerator market that followed?
Part of the answer comes from an ambiguity in the word programmable.
Two notions of programmability
Consider two machines.
The first machine has a fixed architecture but can execute a large class of programs:
P → I(P) → H
where P is the programmer’s description, I(P) is a sequence of instructions or kernels, and H is a largely fixed hardware architecture.
A GPU is approximately such a machine.
The second machine allows the program to alter the architecture itself:
P → A(P) → C(P)
where A(P) is a hardware architecture constructed for the problem and C(P) is its physical configuration.
An FPGA is approximately this second machine.
From one perspective, the second machine is obviously more programmable. We can create arbitrary pipelines, memories, state machines, arithmetic structures, and interfaces.
But consider what happens when the user changes P.
For the first machine, we may need to compile another program.
For the second, we may need to construct another machine.
This suggests two different quantities that we might both informally call programmability: architectural flexibility, and the cost of changing the computation.
FPGAs are exceptional at the first.
GPUs turned out to be exceptional at the second.
For AI, the second quantity became enormously important.
The abstraction problem
Suppose we write
for (int i = 0; i < N; i++)
c[i] = a[i] * b[i];On a conventional processor, this program leaves most architectural questions unanswered because the architecture has already been selected.
On an FPGA, these unanswered questions are the architecture.
Should there be one multiplier or 256? Should the loop be fully unrolled? What is the initiation interval? Where should the arrays reside? How should the memories be partitioned? How deep should the pipeline be? What clock frequency should we target? Will the resulting design route? Will it meet timing?
The difficulty encountered by FPGA OpenCL and high-level synthesis (HLS) systems was therefore somewhat deeper than translating C into register-transfer level (RTL) code.
They were attempting to map a software description onto a space of possible machines.
One can certainly automate parts of this search. Modern HLS tools do so.
But there is a tension.
If the compiler selects a sufficiently generic architecture so that software behaves predictably, much of the advantage of constructing custom hardware disappears.
If the compiler aggressively specializes the hardware, physical architecture once again matters.
Put somewhat provocatively:
An FPGA made sufficiently easy to program risks becoming an inefficient processor implemented in programmable logic. An FPGA used to its full potential starts requiring hardware engineering again.
This seems to have been a difficult circle to square.
CUDA solved an easier problem — and perhaps the more important one
There is another historical point worth emphasizing.
FPGA vendors did not actually possess the software-platform head start that one might initially assume.
NVIDIA introduced CUDA in 2006, presenting the GPU as a general-purpose parallel computing platform programmable from familiar languages, initially C. Altera and Xilinx later developed increasingly sophisticated higher-level programming environments for programmable logic, but they faced a fundamentally different compilation problem.
A CUDA compiler knows the broad shape of the machine on which its program will execute.
An FPGA compiler is partly responsible for creating that machine.
The distinction is easy to underestimate because both processes are called compilation.
But they are not computationally or conceptually equivalent.
In one case we approximately have
program → instructions.
In the other,
program → microarchitecture → logic → placement → routing.
This additional freedom is precisely why FPGAs are useful.
It is also precisely why they are difficult.
AI then became unusually regular
There is another part of the story.
Suppose future computing had become dominated by rapidly changing collections of irregular bit-level algorithms. The FPGA might have been exceptionally well positioned.
Instead, an enormous fraction of AI computation converged toward variations of a relatively regular primitive:
C = AB.
There are, of course, many complications surrounding this operation — attention, normalization, activation functions, sparsity, communication, quantization, memory movement, mixture-of-experts routing, and so forth. Nevertheless, very large quantities of the computational work eventually reduced to dense or structured matrix operations.
This creates another curious question.
If we know that thousands of multipliers will spend most of their useful lives performing matrix operations, why should we repeatedly construct the interconnect between those multipliers out of programmable routing?
At some point it becomes attractive to harden the structure.
This is approximately what happened.
NVIDIA added Tensor Cores.
Google constructed tensor processing units (TPUs) around systolic matrix engines.
AMD GPUs acquired increasingly capable matrix hardware.
And, interestingly, Xilinx itself moved in this direction.
Versal is an interesting admission
The architecture of Versal is revealing.
A traditional FPGA provides programmable logic, digital signal processing (DSP) blocks, memories, and routing from which one can construct an accelerator. Versal added AI Engines: arrays of specialized vector/very-long-instruction-word (VLIW) processors with local memories and dedicated interconnect.
Thus the architecture moved from something resembling
programmable fabric
toward
programmable fabric + specialized processors + network-on-chip + CPU subsystem.
This should not be viewed as abandoning the FPGA idea. Quite the opposite: it is a sensible response to the observation that different types of programmability are appropriate at different levels.
But it does suggest something important.
For sufficiently regular computation, configuring individual lookup tables (LUTs) and programmable routes is often not the most economical level of abstraction.
The irony is that the FPGA industry’s own architectural evolution eventually rediscovered the same compromise that made GPUs effective:
Retain programmability, but harden the structures whose regularity has become predictable.
The memory problem
There is a similar argument for memory.
An FPGA is particularly attractive when a computation can be organized as a stream:
x → f1 → f2 → f3 → y.
A carefully designed FPGA can keep data moving through a custom pipeline with very little unnecessary storage or instruction overhead. Many signal-processing applications have exactly this form.
Large AI models increasingly do not.
A large model may require repeatedly moving billions of parameters between high-bandwidth memory (HBM) and arithmetic units. Once the problem becomes dominated by enormous matrix operations and enormous memory movement, the optimal machine begins looking rather different.
The flexibility of an FPGA’s programmable routing has real silicon cost. Configuration memory has cost. General LUT structures have cost.
Those resources are extremely valuable when the computation requires them.
They are less obviously valuable when the dominant task is moving tensors from HBM into a large collection of matrix multipliers.
Thus the question is not whether an FPGA can perform the computation. Clearly it can.
The question is what fraction of the silicon budget must be devoted to flexibility that the workload does not presently require.
For large language models (LLMs), that question had a decisive answer.
Every accelerator divides its die between the core compute element and everything that feeds, controls, and configures it. A GPU built for AI spends the dominant share of its silicon on hardened matrix arithmetic and the memory system that feeds it. An FPGA cannot make that allocation: a large fraction of its die is committed, before any user design exists, to configuration memory, programmable routing, and general-purpose LUT fabric — the machinery of reconfigurability itself.
Against a workload that had collapsed into one enormous regular primitive, no compiler could close that gap.
The LLM boom was never available to the FPGA.
The iteration-time problem
There is also a human timescale to consider.
Suppose a machine-learning researcher modifies an algorithm.
On a GPU the experimental loop might look approximately like
modify → compile → run → measure.
An FPGA implementation can introduce another loop:
modify → generate architecture → synthesize → place → route → check timing → run → measure.
For a production hardware design, this is entirely reasonable.
For a researcher performing dozens of experiments, it is a severe disadvantage.
And AI happened to develop through unusually rapid empirical iteration. Architectures changed. Numerical formats changed. Training methods changed. Attention mechanisms changed. Model sizes changed by orders of magnitude.
In this environment, the ability to reconfigure hardware was useful, but the ability not to have to reconfigure the hardware at all was often more useful.
This is the paradox.
The FPGA was more flexible in hardware.
The GPU was more flexible in practice.
Microsoft provides an important counterexample
One should be careful not to conclude from this that FPGA acceleration failed technically.
Microsoft demonstrated almost the opposite.
Project Catapult put FPGAs into production for Bing and Azure, and Brainwave demonstrated impressive low-latency neural-network inference. Microsoft describes FPGA-enabled servers deployed at data-center scale, and later used FPGAs for networking as well as computation.
But notice the economic setting.
Microsoft possessed enormous workloads, specialized hardware teams, control of its own data centers, millions of servers over which engineering costs could be amortized, and workloads valuable enough to justify specialized acceleration.
This is closer to ASIC economics with reconfigurable hardware than to general-purpose software development.
Microsoft’s success therefore establishes an important result:
FPGAs can be excellent data-center accelerators when the value of specialization exceeds the engineering cost of specialization.
It does not establish the stronger result:
FPGAs are the best general-purpose platform on which arbitrary AI developers should run arbitrary models.
These are quite different propositions.
Training may have determined inference
There is also a network effect.
Modern AI models were overwhelmingly developed and trained on GPUs.
Consequently the researcher already had
PyTorch → GPU libraries → CUDA → GPU.
The model was tested there. The kernels were optimized there. The profiling tools worked there. The distributed-computing infrastructure existed there. The engineers understood the system.
Once this ecosystem existed, moving inference to another architecture had to produce enough benefit to compensate for leaving it.
Thus training may have selected the inference platform indirectly.
This is a useful reminder that hardware competitions are not always decided by hardware.
Intel may therefore have been more right than wrong
This also reframes Intel’s acquisition of Altera.
Intel predicted that CPU-only computing would become insufficient. Correct.
It expected data centers to become heterogeneous. Correct.
It expected machine learning to become an important workload. Correct.
It believed specialized accelerators would become increasingly important as conventional processor scaling slowed. Again, correct.
The questionable assumption was more specific:
changing workload ⇒ value of hardware reconfiguration.
What actually happened was closer to
changing workload ⇒ value of software programmability on top of specialized hardware.
That is a much subtler forecasting error.
Intel correctly anticipated the problem and even the broad class of solution. It appears to have selected the wrong level of programmability at which to attack it.
AMD’s later Xilinx acquisition was more diversified. AMD explicitly described the combined portfolio as CPUs, GPUs, FPGAs, and adaptive SoCs rather than betting on programmable logic alone.
In retrospect, that distinction seems rather important.
A simple model
We might summarize the issue with an intentionally crude objective function.
Suppose the practical value of an accelerator is approximately
V = (P × E × D × S) / C
where P is computational performance, E is energy efficiency, D is developer productivity, S is ecosystem and deployment scale, and C is total cost.
FPGA discussions have traditionally emphasized P and E. And for many workloads they can be excellent.
But NVIDIA spent more than a decade increasing D and S. Eventually those terms became enormous.
CUDA’s greatest achievement may therefore not be that programmers can write CUDA kernels.
It may be that most AI programmers do not have to.
They write PyTorch.
The specialized hardware disappears beneath layers of software.
This is a considerably stronger abstraction than hiding RTL.
There may now be a new variable
There is, however, an interesting reason to revisit this history.
The cost of hardware engineering may itself be changing.
Historically, exploiting an FPGA efficiently required a human to manage much of the mapping
intent → architecture → RTL/HLS → verification → implementation → timing closure.
HLS attempted to automate one portion of this transformation. It never completely removed the architectural problem.
Modern AI agents potentially attack the problem differently.
Rather than demanding that a single compiler infer the optimal hardware architecture from a C program, an agent can in principle participate in an iterative engineering loop:
specification → candidate architecture → simulation → synthesis → timing report → modified architecture → …
The distinction is important.
An HLS compiler largely performs a transformation.
An engineering agent can perform a search.
It can generate an implementation, observe the result, modify the architecture, run the tools again, inspect failures, create tests, and repeat.
This begins to resemble what FPGA engineers already do.
The interesting question is therefore not necessarily whether AI can write RTL. That is probably the least interesting part.
The question is whether AI can substantially reduce the engineering cost of obtaining specialized hardware.
If that quantity falls by one or two orders of magnitude, some old economic assumptions about programmable logic deserve to be reconsidered.
Perhaps the FPGA was waiting for the wrong abstraction
For roughly fifteen years the FPGA industry attempted to move upward toward software developers:
RTL → HLS → OpenCL → software developer.
Perhaps this was the wrong direction in which to remove complexity.
The goal was to make hardware design resemble software development. But hardware does not behave like software. Placement matters. Routing matters. Memory architecture matters. Clocking matters. Timing matters. Physical resources matter.
Pretending otherwise tends to sacrifice the properties that make an FPGA useful.
There is another possibility:
human specification → AI engineering agent → the actual hardware workflow.
Here the hardware complexity is not removed.
It is absorbed by another engineering layer.
The agent can still reason about pipelines, block RAM (BRAM) banking, DSP utilization, clock-domain crossings, timing constraints, and placement.
The human does not necessarily need those details to disappear from the design process.
They merely need to disappear from the human’s critical path.
This is a substantially different proposition from HLS.
A possible second chance for programmable logic
None of this implies that FPGAs are about to replace GPUs.
The die-area argument makes the first boom permanently out of reach, not provisionally so: a machine that spends silicon on reconfigurability will not out-multiply a machine that spends the same silicon on multipliers, and for large transformer models the economics of dense matrix hardware and HBM remain formidable.
But the first boom is producing a second one.
Deployed AI systems generate demand for everything around the tensor computation:
network → FPGA → GPU/NPU → network.
The FPGA can perform operations for which the GPU is structurally less well suited: protocol termination, deterministic data movement, packet steering, compression, custom security functions, low-latency filtering, sensor interfaces, preprocessing, specialized numerical formats, and real-time control.
The GPU or neural processing unit (NPU) can then do what it does exceptionally well: large-scale tensor computation.
This second boom has the shape programmable logic was built for: diverse, irregular, latency-bound, and different at every deployment. No single hardened architecture will serve all of it.
What disqualifies the FPGA from workloads of this shape was never silicon. It is the second quantity of programmability — the cost of changing the computation.
Remove that quantity, and the second boom can go to the FPGA.
That is precisely the quantity an AI engineering agent attacks.
This is less exciting than declaring that the FPGA will replace the GPU.
It may also be much more useful.
The broader lesson
There is a tendency in engineering to identify flexibility with the number of things a device can be configured to do.
AI computing suggests another definition.
A system is flexible when the cost of changing what it does is small.
Under the first definition, an FPGA is extraordinarily flexible. Under the second, a GPU running a mature software stack can be even more flexible.
That distinction helps explain an otherwise puzzling period in semiconductor history.
Intel and AMD saw heterogeneous computing coming. Microsoft demonstrated that reconfigurable data-center acceleration actually worked. FPGA vendors built increasingly sophisticated high-level tools. Xilinx developed adaptive computing devices specifically intended to span conventional processing, programmable logic, and specialized AI engines.
And nevertheless, when perhaps the largest new accelerator market in computing history arrived, most of its value flowed elsewhere.
The industry did not misunderstand the need for specialization.
It misunderstood the economics of specialization.
For FPGA engineers, that is perhaps the more useful conclusion.
And if AI agents substantially reduce the cost of creating, verifying, and optimizing specialized hardware, then the experiment may not be finished.
The FPGA may not have needed a better C compiler.
It may have needed a machine that could become the FPGA engineer.
Sources
- Putnam et al., “A Reconfigurable Fabric for Accelerating Large-Scale Datacenter Services,” ISCA 2014. Describes the 1,632-server pilot and reports the 95% per-server ranking throughput improvement at fixed latency. Paper (PDF).
- Microsoft Research, Project Catapult project page. Timeline covering the 2010 origin, the 2012 scale pilot, at-scale deployment in Bing and Azure, and Project Brainwave. Project page.
- Intel, “Intel to Acquire Altera,” June 1, 2015. States the approximately $16.7 billion valuation and the plan to offer Altera FPGAs with Xeon processors as integrated products. Press release.
- Intel, “Intel Completes Acquisition of Altera,” December 28, 2015. Krzanich statement naming autonomous driving and machine learning. Press release.
- AMD, “AMD to Acquire Xilinx, Creating the Industry’s High Performance Computing Leader,” October 27, 2020. States the $35 billion all-stock valuation. Press release.
- AMD, “AMD Completes Acquisition of Xilinx,” February 14, 2022. Describes the combined portfolio as CPUs, GPUs, FPGAs, and Adaptive SoCs. Press release.
- NVIDIA introduced CUDA in November 2006 as a general-purpose parallel computing platform for its GPUs.