The bottleneck
Two Python cores were pinned, and asyncio could not invent another one.
The problem was not that Python was running a little hot. One core was pinned at 100% ingesting market data, maintaining order books, and calculating implied volatility; a second was pinned at 100% fitting the surface. There was nowhere useful left for the rest of the application to go, and latency depended rather too heavily on whether the machine happened to be having a good day.
asyncio had already done its job. It managed several WebSocket connections without wasting a thread waiting for each exchange, but it could not make the Black formula, root finder, and surface fitter execute simultaneously. Python threads remained constrained by the GIL, while a process pool would have serialised rapidly changing books into duplicated state on cores which were already occupied.
We had also explored the other extreme. Kafka plus Kubernetes created good service boundaries and an AWS bill of roughly $1,000 in the first serious week. Uncle Jeff needs paying. The latest complete seven-day Cost Explorer window was $65.86 for the whole account, including $33.09 of EC2 and $16.80 for the remaining EKS control plane.
Native hot path
Rust connects to the exchanges and owns the volatility state.
The Rust engine is not a helper which receives an option price from Python and returns a volatility. It connects directly to Deribit, OKX, Bybit, and Binance, discovers and expires instruments, maintains exchange books, tracks index and futures references, interpolates the forward, calculates bid and ask implied volatilities, and constructs the expiry and strike groups required downstream.
Rust being genuinely multithreaded matters here. Network I/O runs asynchronously under Tokio while CPU work can execute on real worker threads across available cores, without a global interpreter lock turning concurrency back into a queue. Bounded non-blocking ingress preserves lifecycle messages while supersedable quote updates can coalesce under load.
Python still performs SVI calibration, downstream analytics, persistence, and product delivery. Rust owns the hot, stateful market-data path; Python receives coherent snapshots and ordered patches instead of issuing thousands of tiny calculation requests.
Process boundary
A Unix socket was simpler than pipes, shared memory, or an in-process extension.
Ordinary pipes work when one process starts another and data flows predictably in one direction. Python instead needed to connect and reconnect independently, request a fresh snapshot after a restart or sequence gap, exchange heartbeats, and continue without taking the Rust engine down with it.
Shared memory would avoid copies, but the bytes were the easy part. We would still need ownership rules, locking, versioning, and a consistency boundary which prevented Python reading a book while Rust was halfway through changing it. Embedding Rust through PyO3 would remove the socket copy but couple both runtimes, releases, ABI packaging, and failure domains again.
The Unix socket uses a versioned MessagePack protocol with a four-byte length prefix. Rust publishes an authoritative snapshot followed by monotonically sequenced patches; a sequence gap or new engine identity causes Python to request another snapshot. A slow consumer never gets to backpressure exchange ingestion.
Implied volatility
The previous accepted volatility is usually the best Newton starting point.
For each option side the engine solves the Black forward-price equation using ACT/365 time to expiry. Premiums are normalized to the exchange contract, inverse premiums are converted through the forward while linear premiums remain linear, fees are included in the effective bid and ask, and prices are checked against intrinsic and maximum bounds before iteration begins.
The last accepted volatility for the same instrument, book side, and effective input becomes the next Newton starting point. Implied volatility usually moves continuously, so the answer from a few milliseconds ago is an excellent initial guess. A newly observed option uses Corrado–Miller as its cold start.
Newton is guarded by a valid volatility bracket from effectively zero to 500%. If vega becomes too small or a proposed step leaves the bracket, the solver moves to its midpoint and can fall back to bisection. The normal path remains fast while the unpleasant cases remain bounded and deliberately boring.
Selective calculation
The fastest volatility calculation is still the one we do not repeat.
Metadata-only updates no longer trigger another solve, forward movements below 2.5 basis points reuse existing results, top-of-book vols are cached, deeper levels are keyed by effective inputs, and dirty expiry and strike groups are built once at the publication boundary rather than after every exchange message.
The initial live profile performed roughly 57 implied-volatility calculations per incoming message. After caching, gating, and building groups once per flush, the production-shaped replay averaged 6.59. That replay contained 1,798 instruments, including 1,784 options across twelve expiries, two exchanges, and five book levels, with 9,000 messages paced at 3,000 per second.
LLM-assisted rewrite
Python became the executable specification for the Rust implementation.
We built the Rust implementation with substantial help from LLM-assisted coding, using the working Python system as the test case. Python already contained the accumulated decisions about exchange messages, contract conventions, forwards, fees, book state, and implied-volatility behavior; describing all of that perfectly in a prompt would have been less reliable than replaying it.
Captured market data ran through both implementations and the resulting instruments, books, forwards, vols, and groups were compared. A disagreement produced a concrete case to investigate rather than an argument about which implementation looked plausible. Boundary prices, missing references, crossed books, reconnects, and deliberately unpleasant solver inputs became regression tests.
The useful combination was not asking an LLM to write a trading system. It was a mature Python reference, captured production inputs, measurable invariants, and an LLM able to accelerate translation and test generation. That changed the economics of a rewrite which would otherwise have been difficult to justify merely to save CPU.
Language choice
Why not C++?
C++ was the obvious alternative. It is mature, extremely fast, and embedded throughout trading; if a bank has priced options since before JavaScript existed, C++ is probably involved somewhere important.
It could build the same engine and might extract the last few percent in a carefully chosen benchmark, but we also needed exchange connectivity, asynchronous I/O, shared state, numerical code, and enough concurrency to make failures interesting. Rust occupies essentially the same native-performance class while enforcing ownership and shared-state rules at compile time.
Rust is also more fashionable, which is not a sound engineering reason but is still a reason people occasionally use while pretending not to. C++ developers can build the same system, probably in fewer nanoseconds, and are paid the big bucks partly because they can understand their code after the third template error has filled six terminal screens.
Measured result
The system moved from two saturated cores into a predictable operating range.
A live process sample on 25 August put the complete Rust exchange, book, and IV engine at 22.5% of one core; its longer-running process average was similarly around 21.8%. The combined Rust and remaining Python system averaged 28.1% CPU on a two-vCPU c7i.large over 24 hours, equivalent to about 0.56 of one core in total. The 95th percentile was roughly 0.65 core and the maximum five-minute sample remained below one full core.
The one-hour warm live soak completed without a dropped update or sequence gap. Exchange state reached Python with median latency of 173 ms, a 95th percentile of 263 ms, and a maximum of 495 ms. Applying the patch in Python took 0.60 ms at the median and 0.98 ms at the 95th percentile, leaving the deliberate publication cadence as most of the remaining latency.
Genuine reconnects recovered in 1.22 seconds for Deribit and 2.15 seconds for OKX. Python briefly marked the feed unready, received an authoritative snapshot, and resumed from a known sequence instead of guessing what it had missed. Memory remained stable during the soak, although the engine deliberately retains books and caches in memory; this is lower and more predictable CPU, not the magical disappearance of every resource.
BTC to ETH and SOL
BTC proved the engine; ETH became another partition.
The practical payoff is not only lower CPU. BTC established the architecture; adding ETH largely meant creating another currency partition through the same exchange, forward, book, and volatility machinery. The product now carries both currencies' surfaces, smiles, risk reversals, and flies through one engine and protocol instead of starting another bespoke pipeline.
SOL is next, although SOL brings its own issues. Contract availability, venue coverage, liquidity distribution, and the stability of the short-dated surface are not automatically identical merely because the currency field changed. That will probably deserve its own article once we have finished discovering precisely which assumptions it objects to.
The roadmap remains
Docker keeps Kafka and Kubernetes available without requiring them today.
Moving away from Kafka and Kubernetes for the present workload does not mean the earlier architecture was wasted. The Rust engine already runs as a Docker container behind an explicit versioned boundary, so a future distributed deployment can run the same engine as currency-partitioned Rust workers instead of rewriting it again.
The Unix socket can be replaced or supplemented by a Kafka transport adapter while preserving exchange connectivity, state, solver behavior, and the message model. Kafka still requires consumer offsets, partition ownership, and delivery semantics; it is not summoned merely by writing FROM rust. Thanks to the magic of Docker, however, the tested worker is already a portable deployment unit.
Python supplied the working implementation and test oracle, LLM-assisted coding made the translation practical, Rust supplied genuine multithreading and predictable performance, and Docker leaves a route into Kafka and Kubernetes when the workload merits it. Each component is doing the job it is good at, and the platform can add currencies and workers without being rewritten for a third time.
Prerequisites and product context
Place this step in the surface pipeline.
The deployed Rust engine owns live BTC and ETH exchange state and implied-volatility calculation before Python fits and publishes the surfaces shown in the dashboard.
Links