Resources › Numeric formats in the Altera FPGA AI Suite IP

Numeric formats in the Altera FPGA AI Suite IP

The FPGA AI Suite is Altera’s artificial intelligence (AI) inference block for its field-programmable gate array (FPGA) families. It is configured through an architecture file whose global parameters size the processing element (PE) array — k_vector (KVEC) PEs, each computing a c_vector (CVEC)-wide dot product, across num_lanes lanes — the on-chip feature cache, and the graph limits. One parameter, arch_precision, sets the number format for every multiply in the array and dominates both area and accuracy. This page reads section 2.5.2.1 of the reference manual: the four formats at the bit level, what blocking does to real numbers, where the intellectual property (IP) block uses fp32 and fp16 instead, what each format costs in digital signal processing (DSP) blocks, and the remaining global parameters.

Published 2026-08-22. Source: Altera FPGA AI Suite IP Reference Manual, version 2025.1. Every parameter range, bit layout, and DSP count on this page comes from that manual. Where the page adds an explanation the manual does not give, the text says so and cites the DSP block documentation it rests on.

The four formats at the bit level

Section 2.5 of the manual defines the legal values of arch_precision as FP11 = INT7-BFP (1s.6m.5e), FP12AGX = INT8-BFP (8m.5e, two’s complement), FP13AGX = INT9-BFP (9m.5e, two’s complement), and FP16 = INT12-BFP (1s.11m.5e). All four are block floating point (BFP). Each element stores a sign and a mantissa. A block of CVEC elements shares one 5-bit exponent.

Dashed cell: the most significant bit of a two’s-complement mantissa. It carries the weight −2^N−1, so it acts as the sign without being a separate field. In all four formats the usable magnitude is N − 1 bits: 6, 7, 8, and 11. The sign-magnitude formats have a redundant negative-zero code. The two’s-complement formats have the asymmetric range −2^N−1 to 2^N−1 − 1 and no negative zero.

Where the names come from

The manual does not explain the names. They are consistent with the scalar minifloat each format derives from: 1 sign bit, 5 exponent bits, and n − 6 fraction bits, so FP11 has 5 fraction bits and FP16 has 10. Converting to block floating point makes the hidden leading 1 explicit and moves the exponent out of the element. What remains is an (n − 4)-bit signed integer: INT7, INT8, INT9, INT12. The white paper the manual cites shows the same construction for a 9-bit minifloat blocked to int5bfp (source 3). The AGX suffix marks the two formats that map onto native signed multiplier modes of Agilex DSP blocks, covered below.

What the shared exponent costs

The exponent costs 5 bits per block at every block size. Element width, not exponent overhead, is what grows with precision. The table gives stored bits per element, including the amortized exponent, for each legal c_vector value: (N × CVEC + 5) ÷ CVEC.

At CVEC = 64 the exponent is 1.1 % of an FP11 block (5 of 453 bits) and 0.6 % of an FP16 block (5 of 773 bits). An FP16 block is 1.6 to 1.7 times the size of an FP11 block across the legal CVEC values: 53 versus 33 bits at CVEC = 4, 773 versus 453 at CVEC = 64. This matches the manual: arch_precision has a relatively large effect on accuracy, FP16 significantly increases the size of the IP, and c_vector has a generally small effect on accuracy.

What blocking does to real numbers

The block exponent E comes from the element with the largest magnitude: E = ⌊log₂ max |v|⌋. Every element is then divided by 2^E−P+1, where P = N − 1 is the magnitude width, and rounded to an integer. An element e binades below the block maximum loses e of its mantissa bits. An element N or more binades below it rounds to zero. The two tables apply this to the same four values in FP11 and FP16. Rounding is to nearest. The manual does not document the exponent field’s bias or range, and the example does not model them.

FP11: N = 7, P = 6. E = ⌊log₂ 1.9⌋ = 0. Least significant bit weight 2⁰⁻⁵ = 1/32 = 0.03125. Mantissa range −63 to 63.

FP16: N = 12, P = 11. E = 0. Least significant bit weight 2⁻¹⁰ = 0.0009765625. Mantissa range −2047 to 2047.

The pattern holds at any width: a large element in a block costs its small neighbors their low bits. A wider c_vector puts more elements under one exponent, which is the mechanism behind its small but nonzero effect on accuracy.

INT8 graphs

For a graph with 8-bit integer (INT8) symmetric quantization, the compiler sets the block exponent so that the INT8 weights are used directly as the mantissas. The manual limits this to architectures whose mantissa is 8 bits or wider, which excludes FP11. INT8 graphs do not change inference speed or FPGA resource use; the same hardware runs every graph. pe_array/enable_scale must be true for INT8-quantized graphs.

Where each numeric type is used

The manual fixes the precision of every stage. Features and weights use arch_precision. The PE accumulators are fp32. The accumulator bias value, the scale multiplier, the drain from the accumulators to the crossbar (XBAR), and the auxiliary modules are all fp16. The diagram reads those statements as a data path. The manual does not specify the order of the scale and bias operations inside the PE.

Shaded boxes hold block floating point. The dark box is fp32. Blue outlines are fp16.

Block floating point for the dot products

The manual states the consequences: lower precision raises frames per second and reduces area at the cost of accuracy; FP11 needs the fewest random-access memory (RAM) blocks and slightly reduces external memory traffic; FP16 significantly increases the size of the IP. The mechanism is implicit in the manual and explicit here. Within a block, a dot product of two BFP vectors is an integer dot product. The mantissa products are summed in integer arithmetic, and one exponent addition covers the whole block. That is the shape of a DSP block: a narrow integer multiplier feeding a wide adder. Per-element floating point would need an alignment shifter and a normalizer on every product. Blocking amortizes that to one per CVEC products. The shared 5-bit exponent, the same width as IEEE half precision, keeps floating-point dynamic range across blocks and layers while the in-block arithmetic stays fixed point. Multiplier width, block RAM width, and memory traffic therefore scale with N rather than with a full floating-point word.

fp32 for the accumulators

A convolution output sums products from many blocks: the input channels divided by CVEC, times the filter taps, each block with its own exponent. Adding those partial sums is again a floating-point operation, and the running sum is far larger than any single product. A 24-bit significand keeps late small products from being swamped and avoids overflow over long accumulations. A 7- to 12-bit mantissa cannot. The manual budgets one accumulator per PE per lane: KVEC × num_lanes DSP blocks in addition to the multipliers.

fp16 after the accumulator

Once a result leaves the accumulator it is a single value, not a member of a dot-product block. Scale, bias, the activation functions, pooling, elementwise multiplication, and softmax act on values individually. A shared exponent gains nothing there, and a per-element exponent is required. fp16 carries the value with its own exponent at half the width of fp32. That halves the drain and crossbar data paths (the manual requires the PE drain width to equal the XBAR KVEC), and fp16 is a native mode of Agilex DSP blocks (source 4). Results are re-blocked into arch_precision when written back to the stream buffer for the next layer.

Multiplications per DSP block

Table 4 of the manual gives multiplications per DSP block by format and device family. INT8 symmetric quantization is available with FP12AGX and higher.

Why two formats are two’s complement

The manual gives the counts. The reasons below come from the DSP block documentation (sources 4 and 5) and from the multiplier widths.

• FP13AGX matches the Agilex 7 DSP block. The Agilex 7 variable-precision DSP block has a fixed-point mode that computes four signed 9 × 9 products per block. A 9-bit two’s-complement mantissa is that operand exactly, so the sign needs no handling outside the DSP. The Arria 10 and Stratix 10 variable-precision DSP blocks offer 18 × 19 and 27 × 27 multipliers and no 9 × 9 mode, which is why the manual lists no FP13AGX support for them.
• FP12AGX matches the Agilex 5 AI tensor block. The enhanced DSP block on Agilex 5 performs twenty INT8 multiplications per block in tensor mode, with support for a block-floating exponent. An 8-bit two’s-complement mantissa is its native operand. The manual’s 20 multiplications per DSP, the 12 × 1 interleave requirement, and its statement that FP12AGX cuts the DSP count significantly all follow from that mode.
• FP11 and FP16 are sign-magnitude because they predate Agilex. Two unsigned 6-bit magnitudes fit in one 18-bit operand with their product fields 12 bits apart, which gives two products per 18 × 19 multiplier and four per block, with the sign resolved by an exclusive-OR outside the DSP. A two’s-complement operand cannot be packed that way. An 11-bit magnitude needs a whole multiplier, which gives FP16 two per block on every family. The packing is inference from the multiplier widths; the manual states only the counts.
• Agilex 5 is a special case. Tensor mode lifts FP11 and FP12AGX to 20 multiplications per DSP only with interleave 12 × 1. FP13AGX on Agilex 5 uses the INT9 vector mode, of which the suite currently uses four of the six multipliers. The manual warns that the multiplier count under-predicts DSP usage in tensor mode.

Estimating the DSP budget

From the manual: the PE array needs CVEC × KVEC × num_lanes multipliers, plus KVEC × num_lanes DSP blocks for the fp32 accumulators. Dividing the multiplier count by the multiplications per DSP gives the multiplier DSPs. Two examples with CVEC = 16, KVEC = 64, and one lane:

• Agilex 7, FP11: 16 × 64 × 1 = 1,024 multipliers, ÷ 4 = 256 DSP blocks, plus 64 accumulators: 320.
• Agilex 5, FP12AGX with 12 × 1 interleave: 1,024 ÷ 20 = 51.2, rounded up to 52, plus 64 accumulators: 116. Treat this as a floor because of the tensor-mode caveat above.

The manual’s constraints: KVEC must be a multiple of CVEC and divisible by the crossbar and auxiliary k_vector values. With the depthwise module, KVEC must equal CVEC.

Choosing a format

The example architectures shipped with the suite already use the recommended arch_precision for each family. The manual gives these reasons to change it.

Both knobs are fixed at IP generation. They are architecture parameters, not per-graph settings.

The rest of the global parameter group

Ranges and constraints are the manual’s. Parameters marked “optimizer” are normally set by the architecture optimizer rather than by hand.

family — A10, AGX5, AGX7, C10, S10

Target device family. AGX5 covers Agilex 5 and Agilex 3. C10 is Cyclone 10 GX. The family decides which arch_precision values and interleave factors are legal.

k_vector (KVEC) — 4 to 128, optimizer

Number of PEs in the array, equal to the number of filters processed at once. Must be a multiple of c_vector, divisible by the crossbar and auxiliary k_vector values, and equal to c_vector when the depthwise module is used.

c_vector (CVEC) — 4, 8, 16, 32, 64, optimizer

Width of the dot product in each PE, and therefore the block size of the number format. Must equal k_vector with the depthwise module.

num_lanes — 1, 2, 4

Output-height slices computed in parallel. Scales the PE array, its DSP count, and the total stream buffer. A 4-lane architecture with stream_buffer_depth 10k has a 40k total stream buffer, so set the depth to the per-lane share.

stream_buffer_depth — 2048 to 262144, optimizer

Depth of the on-chip cache for feature (image) data. Larger values cost logic and block RAM and raise performance. Features are held in arch_precision, so the format width multiplies this cost.

enable_eltwise_mult — true or false

Adds the elementwise multiplication layer. Required for MobileNetV3.

filter_size_width_max, filter_size_height_max — 14 or 28

Largest convolution filter the IP can run. Also bounds the Average Pool window, which the filter_scratchpad and filter_depth parameters may limit further.

output_image_height_max, output_image_width_max, output_channels_max — default 128 × 128, 8192 channels

Maximum output tensor size.

enable_debug — true or false

Turns on the debug network, which forwards control and status register (CSR) read requests to externally attached debug-capable modules. Generally not needed for production architectures.

enable_layout_transform — true or false

Dedicated hardware that transforms the input tensor layout and folds inputs into channels, for hostless systems or a slow or heavily loaded hard processor system (HPS). Configured through layout_transform_params. Area scales with input bus width, maximum tensor and stride dimensions, and CVEC; when max_stride_width × max_stride_height × max_channels exceeds CVEC it consumes significant memory for the overflow buffer. It cannot help when the first convolution’s stride is 1, because nothing can be folded; the manual suggests a lighter transform outside the IP in that case. In system-on-chip streaming-to-memory (S2M) mode the hardware transform is supported on Agilex 7 only; other devices use a lightweight external module.

Sources

1. FPGA AI Suite IP Reference Manual 2025.1, section 2.5.2.1, Parameter Group: Global Parameters. Parameter definitions, the arch_precision discussion, Table 4 (multiplications per DSP), INT8 compatibility, and the DSP count formula.
2. FPGA AI Suite IP Reference Manual 2025.1, section 2.5, IP Block Configuration. Bit definitions of the four formats, fp32 accumulator, fp16 bias, scale, drain, and auxiliary precisions, interleave factors, and the drain-width constraint.
3. Low Precision Networks for Efficient Inference on FPGAs, Intel Programmable Solutions Group, IEEE. Block floating point construction with the implicit 1 made explicit, the FP9 to int5bfp example, and the DSP packing rationale.
4. Agilex 7 Variable Precision DSP Blocks User Guide. The 9 × 9 signed sum-of-4 fixed-point mode and the FP16 floating-point mode.
5. Agilex 5 FPGAs: Enhanced DSP with AI Tensor Block, architecture brief. Twenty INT8 multiplications per block in tensor mode with a block-floating exponent, and the INT9 vector mode lineage.