Vibecoding an inference chip: Comments news update
Ankit Gupta used cloud coding agents to program an FPGA on an AWS F2 instance, lifting ternary model decode speeds by 33× to 213 tokens per second.
By Dillip Chowdary • Nov 21, 2026 • Source: ankitg.me

At a glance
- Product
- Ternary FPGA inference engine
- Hardware
- AWS F2 instance
- Author
- Ankit Gupta
- Date
- November 20, 2026
- Throughput
- 213.13 tokens per second
- Compute budget
- $10,000 in tokens
In this post
Ankit Gupta demonstrated an automated hardware synthesis workflow on November 20, 2026, compiling a ternary-weight large language model onto a field-programmable gate array using autonomous AI coding agents. As detailed in ankitg.me's report, cloud-hosted coding agents wrote the low-level C++ and Verilog required to reprogram physical hardware circuits for AI inference. The experiment targeted an Amazon Web Services F2 instance equipped with 16 GB of high-bandwidth memory, converting general-purpose logic gates into an inference accelerator without manual waveform analysis.
This report covers the implementation steps, automated optimization loops, memory architectures, ternary compute engine mechanics, and measured throughput scaling. It is intended for machine learning engineers, systems architects, and hardware designers evaluating whether generative models can author verifiable hardware description code. The narrative examines how an agent-led research workflow escalated initial bit-exact baselines to production-grade decoding speeds.
01Cloud agents and hardware instance setup
The development process began on an AWS F2 virtual machine reservation provisioned with assistance from Codex. The target hardware configuration featured 16 GB of high-bandwidth memory directly connected to FPGA fabric. Gupta paired this deployment with DeepGrove ternary weight architectures designed specifically for memory-constrained environments, which had previously demonstrated state-of-the-art results on Mac hardware with agentic tool calling.
Autonomous execution ran through a cloud agent managed in Conductor over several consecutive days. The system operated on a $10,000 token budget funded through Y Combinator, alternating between Astra and Opus models to direct both high-level design and lower-level code generation. The human operator contributed high-level directional prompts, such as porting CPU-bound operations and evaluating speculative decoding, while the models authored hardware description routines.
The automated research harness completed hundreds of distinct experiments across the run. During this execution lifecycle, the coding agents generated approximately 3 million lines of code and processed 30 million lines of simulation output files. The underlying codebase consisted primarily of C++ driver logic alongside Verilog circuit descriptions, removing the traditional requirement for manual hardware verification.
02Optimization sequence across fifteen stages
Throughput started at a bit-exact decode baseline of 6.43 tokens per second on the ternary model. Across fifteen distinct architectural revisions, the agent systematically removed host communication bottlenecks, restructured memory transfers, and overlapping arithmetic pipelines to lift performance to 213.13 tokens per second.
| Stage | Benchmark Tokens Per Second | Architectural Adjustment |
|---|---|---|
| Baseline | 6.43 | Initial bit-exact full-model decode |
| Host overhead | 8.07 | Reduction of host-to-device communication |
| Burst attention | 14.02 | Batched attention memory access |
| Parallel router | 20.53 | Concurrent expert routing evaluation |
| Glue engine | 30.24 | Hardware intermediate logic consolidation |
| New pipeline | 49.21 | Restructured execution pipeline stages |
| Prompt speculation | 51.25 | Speculative verification on prompt tokens |
| Router relay | 58.43 | Staged routing relay signals |
| Head shadow | 94.44 | Shadowing attention head computations |
| Draft model | 105.63 | Speculative draft decoding integration |
| Memory ports | 108.81 | Dedicated memory port allocations |
| Attention overlap | 134.99 | Overlapping attention with projection work |
| Weight packing | 171.30 | 2-bit weight streaming adjustments |
| Re-speculation | 184.19 | Revised speculative decode cadence |
| Parallel head loader | 213.13 | Dual memory bank reads and expert prefetch |
Note: Hardware timing adjustments required clocking the design down to 225/425 MHz after extended soak testing uncovered intermittent memory faults when operating at 231.25 MHz.
03Ternary engine architecture without multipliers
Advertisement
Tech Pulse Daily
Get tomorrow's pulse first
Join engineers who read Tech Pulse before stand-up. Free, weekday mornings.
Guidance provided by the DeepGrove founders informed the low-level compute pipeline, which implemented a dense batch-size-one decode engine tailored for Qwen-style models. The ternary math engine eliminated dedicated DSP multipliers entirely, replacing arithmetic units with 3:1 multiplexers driven by 2-bit weights.
- Multiplexer logic selects between x, inverted x, or zero based on the 2-bit weight encoding.
- Error trapping triggers an invalid state flag whenever the unused fourth 2-bit code appears in memory.
- Negation handling applies two's complement inversion while row reducers accumulate the required offset once per row.
- Activation grouping processes activations as INT12 or INT16 integers before applying FP32 group scales during accumulation.
- Deferred normalization quantizes activations ahead of pre-projection RMSNorm, applying normalization constants after accumulation.
The physical processing core distributed work across eight individual cores clustered into two distinct groups. Compute projections that execute independently, including gate and up projections, run concurrently on separated core blocks. This structural arrangement reduced DSP usage by half compared to INT8 and INT16 designs at equivalent rates, trading dedicated DSP slices for general lookup tables.
04High bandwidth memory layout and buffer contracts
Memory efficiency governed batch-one decode performance because streaming weights requires substantially more energy and time than computing operations. The design packed weights into 2-bit representations, allowing eight times as many weights per data transfer compared to 16-bit parameters. Parameters streamed directly from HBM into arithmetic lanes, entirely bypassing on-chip buffers because each weight is accessed only once per generated token.
High-bandwidth memory routing dedicated a single point-to-point chain to each memory channel rather than deploying a monolithic crossbar, organizing tensor data across all 32 channels to maintain continuous bandwidth saturation. Activations remained cached directly beside each core for reuse across rows. For mixture-of-experts workloads, expert weights were striped evenly across all 32 channels, and runtime address translation mapped expert identifiers directly to physical storage.
On-chip memory allocations enforced strict hardware lifecycle contracts to prevent race conditions during parallel execution. Buffers functioned as typed storage slots that progressed through explicit states: open, write, commit, read, and release. Dedicated sideband memories tracked exponents and scale factors alongside primary data, rejecting improper access ranges and unverified producer IDs at the circuit level.
05Attention pipelining and schedule control
The attention engine parsed key-value states in 64-token tiles, updating an online softmax running maximum and sum in FP32 without writing the complete attention score matrix to memory. The system compressed key-value caches using MXINT8 representation, assigning an 8-bit precision to cache values paired with one shared exponent for each 32-channel block. Attention probabilities were stored as UINT16 integers, while the final context vectors accumulated in FP32 precision.
Hardware pipeline latency was mitigated through C-slow interleaving, which multiplexed two independent compute jobs onto a single attention datapath without duplicating physical execution units. Normalization and rotary position embeddings for query and key heads ran on a separate, dedicated vector unit, freeing the main vector path to overlap with value projection calculations.
Overall execution was orchestrated by a central hardware controller that dispatched operations in strict program order using tracking tags and synchronization barriers. For mixture-of-experts layers, the router remained on the critical scheduling path because expert weights could not be fetched before routing decisions finalized. To prevent routing divergence, the router executed exclusively in FP32 precision while surrounding hardware pipelines overlapped around it.
Frequently asked questions
What hardware was used to run the vibe-coded inference engine?
The project ran on an AWS F2 cloud instance equipped with FPGA hardware and 16 GB of high-bandwidth memory.
How did the inference speed change throughout the experiments?
Execution throughput advanced from a baseline of 6.43 tokens per second to 213.13 tokens per second, representing a 33.13-fold speedup.
Why did the ternary arithmetic engine avoid hardware multipliers?
Weights were packed at 2-bit precision, enabling 3:1 multiplexers to route x, inverted x, or zero instead of relying on dedicated digital signal processing multipliers.
How was the key-value cache formatted to conserve bandwidth?
The cache utilized an MXINT8 format featuring 8-bit values accompanied by one shared exponent across each 32-channel block.
Sources
Author
Dillip Chowdary
Writes Tech Bytes coverage of AI, engineering, and the tools that actually ship. Editor of Tech Pulse Daily.
Related on Tech Bytes
Advertisement