Original build
The project started as a live crypto options surface.
The proof of concept was simple on paper: ingest live options market data, compute implied volatility, fit an SVI surface, and stream smiles, skews, risk reversals, butterflies, and diagnostics to a frontend.
That description hides most of the real work. The surface fit mattered, but the surrounding system determined whether the output stayed usable while markets were moving.
- Consume websocket market data.
- Track live order-book state.
- Recalculate implied volatility.
- Maintain smile state by expiry.
- Calibrate an arbitrage-aware SVI surface.
- Stream surface updates to the dashboard.
State contract
The first durable boundary was the normalized surface state.
The system became easier to reason about once raw exchange messages stopped leaking into the surface layer. Venue payloads were decoded into an internal state model: option book state, expiry forward state, implied-volatility state, fitted smile state, and dashboard patch state.
That boundary mattered more than the language or framework. Once a quote carried provenance, timestamp, forward, strike, option type, and quality flags, every downstream decision could be audited: why IV was recalculated, why an SVI fit was accepted, why a risk node moved, or why the dashboard reused a previous value.
- Keep raw exchange payloads available for replay, but do not fit directly from them.
- Version quote state and forward state together so moneyness is reproducible.
- Publish fit timestamps, reject reasons, and residual summaries with every surface update.
- Treat dashboard patches as derived state, not as the source of truth.
Exchange payload -> normalized book state -> expiry forward state -> implied-volatility points -> SVI smile state -> surface snapshot -> dashboard patch
WebSocket boundary
The connector contract kept exchange traffic from owning the surface.
The early system became more stable when the exchange connector stopped being allowed to trigger arbitrary downstream work. Its responsibility was narrow: decode venue payloads, maintain connection health, normalize instrument identity, and publish quote-state changes with timestamps and sequence context.
That boundary made the real-time crypto volatility surface architecture easier to debug. A WebSocket reconnect, a sequence gap, and a stale wing quote could be observed before they became an SVI residual or a dashboard patch.
- Keep connection heartbeats separate from market-data freshness.
- Attach venue timestamp, local receive time, instrument key, and sequence state to every normalized update.
- Mark sequence gaps and resubscribe events as data-quality events, not only transport logs.
- Let the volatility layer decide whether a quote is analytically useful instead of making the connector fit-aware.
venue_ws_message -> connector_health -> normalize_instrument -> update_quote_state -> publish QuoteState -> volatility_worker decides IV / skip / reuse
Snapshot contract
The dashboard needed a surface snapshot contract, not live model internals.
The browser should not need to know whether the backend is single-process Python, a later Kafka worker graph, or a mixed native worker. It needs a coherent surface snapshot: fitted smiles, fixed-tenor rows, risk reversals, flies, timestamps, freshness state, and holdback reasons.
That product contract forced the backend to become explicit about what had changed. The useful output was not 'a quote updated'; it was a dashboard patch that said which expiry, smile, risk node, or diagnostic state had changed and which previous values were reused.
- Publish accepted smile versions rather than mutable in-memory curve objects.
- Separate full surface snapshots from small dashboard patches.
- Include source timestamps, fit timestamps, and stale-node flags in the user-facing payload.
- Keep API consumers and the dashboard on the same accepted surface state.
SurfaceSnapshot {
currency,
smiles,
fixed_tenors,
risk_nodes,
diagnostics,
source_timestamps,
accepted_at
}
DashboardPatch {
snapshot_id,
changed_expiries,
changed_nodes,
reused_nodes,
holdback_reasons
}First version
The single-exchange version held together until the workload changed.
The first version connected to one exchange, ran on a medium EC2 instance, and maintained a live surface well enough to prove the architecture was directionally sound.
Then the second exchange was added. The visible symptoms looked like websocket reliability problems: delayed heartbeats, messier reconnects, stale UI updates, and queues backing up. Profiling showed that the transport was not the real issue.
Latency budget
The useful latency metric was surface age, not message age.
A websocket message can arrive on time while the analytical state is late. The dashboard only becomes useful when the book update has passed through IV inversion, smile preparation, SVI calibration, risk-node extraction, persistence, and browser fanout.
The early monitoring therefore measured the age of each user-facing surface object. A fresh socket with stale fixed-tenor rows was still a stale product. A delayed message that did not change the visible smile was less important than a repricing that left a risk reversal or fly behind.
- Track source exchange timestamp, local receive time, IV calculation time, fit time, and browser delivery time separately.
- Measure staleness by expiry because front-end and long-dated BTC books fail differently.
- Expose dashboard snapshot age so operators can distinguish UI lag from market-data lag.
- Alert on stale risk nodes when the underlying smile has moved but derived panels have not.
Root cause
The system had quietly moved from I/O-bound to CPU-bound.
Adding a venue did not just add messages. It multiplied downstream work. Every update could trigger order-book aggregation, implied-volatility recalculation, ATM refreshes, Greeks, smile state, fit preparation, arbitrage validation, surface patch generation, frontend broadcasts, persistence writes, and lifecycle logging.
Once CPU became saturated, the connection layer started missing its own timing obligations. Heartbeats looked unreliable because downstream computation was stealing enough time that the event loop could no longer behave reliably.
- Market data became stale.
- Websocket queues backed up.
- Frontend latency increased.
- Reconnect handling degraded.
- The dashboard inherited uneven state.
Failure pattern
CPU pressure surfaced as market-data instability.
The misleading part was that the first visible failures were not stack traces from the volatility code. They were delayed websocket callbacks, missed heartbeat windows, uneven UI updates, and quote snapshots that arrived in bursts instead of a smooth stream.
That is why the fix could not be only a larger instance. The system needed to reduce unnecessary recomputation and separate the work that kept market data current from the work that produced expensive analytical updates.
- A spot move could invalidate many delta buckets even when most displayed vols barely changed.
- A wide or stale wing quote could trigger expensive recalculation that should have been rejected earlier.
- Multiple updates for the same expiry could queue repeated SVI preparation work.
- Persistence and frontend fanout could compete with the same event loop used for ingestion.
Optimisation phase
The useful fixes reduced work per update.
The obvious answer would have been to rewrite hot paths in Rust or C++. Lower-level code would help in places, but the bigger issue was architectural: the system was doing too much unnecessary work.
The next stage became less about making individual functions faster and more about deciding when full precision was actually needed.
- Ignore tiny spot moves that would not move displayed volatility by even 1bp.
- Use approximation paths for small moves instead of full implied-volatility recalculation.
- Batch incoming updates so duplicate recomputation is collapsed.
- Separate ingestion, fitting, persistence, and frontend broadcasting.
Update gating
The fast path decided whether a tick could change the displayed surface.
The important distinction was not quote update versus no quote update. It was whether the update could change a displayed volatility state enough to justify the expensive path. A one-lot mark change in an illiquid far wing does not deserve the same treatment as a broad BTC repricing across the front expiries.
Derivasys started gating work by market relevance: quote freshness, spread quality, forward movement, moneyness shift, expiry importance, and whether the previous accepted smile already explained the new point inside tolerance.
- Reject bad books before they reach the IV solver.
- Reuse previous IV when mark and forward movement are below the display threshold.
- Batch expiry-level recalculation so a burst creates one fit request, not one request per instrument.
- Promote full recalculation when risk reversals, flies, ATM term structure, or fixed-tenor rows would visibly change.
if quote_quality == "bad": reject_before_iv(reason="book_quality") elif surface_move_below_threshold(quote, forward, previous_smile): reuse_previous_state(reason="display_invariant") elif expiry_in_pending_batch(expiry): merge_update(expiry, quote) else: queue_surface_recalc(expiry, priority="visible_surface_change")
Replay harness
Replay was the only way to prove an optimization preserved the surface.
A faster path is not useful if it changes the surface in a way that cannot be explained. The first build therefore needed enough retained state to replay a market window through old and new logic: quote states, forward states, IV outputs, SVI candidates, accepted smiles, and dashboard patches.
This replay harness made performance tuning safer. A proposed shortcut could be checked against the same window that previously stressed the system, and the comparison could focus on market-facing outputs rather than only CPU time.
- Compare accepted SVI parameters, residuals, ATM IV, risk reversals, flies, and fixed-tenor rows.
- Track whether differences came from quote rejection, IV reuse, solver fallback, or fit acceptance.
- Keep the replay input contract close to the future Kafka topic contract.
- Use dashboard-patch equality as the final check before promoting an optimization.
Migration handoff
The MVP created the boundaries for the later Kafka design.
The first system did not need Kafka to prove the product, but it did need clean internal boundaries. Exchange ingestion, quote state, IV calculation, SVI fitting, surface snapshots, API publication, and dashboard fanout had to become separable concepts before they could become independent services.
That is the practical lesson from the MVP. The first build should not prematurely distribute every component, but it should make the state transitions explicit enough that the next architecture can split them without changing what users see.
- Exchange connectors should publish normalized events rather than mutate hidden global state.
- Volatility workers should consume quote-state records and produce auditable IV records.
- Surface workers should publish compact snapshots and patches that dashboard and API consumers can share.
- A Rust or C++ worker migration is safer when output equality can be checked against replayed quote states.
Outcome
The bottleneck became systems engineering.
After the optimisation work, the system stabilised around 5,000 market-data messages per second while still maintaining live smile state, updating the surface, broadcasting frontend updates, and persisting state.
That was enough to make the MVP useful, but not enough to scale cleanly forever. More exchanges, more currencies, and more downstream analytics all pointed toward a distributed ingestion and processing model with independent workers, distributed state, asynchronous fit jobs, decoupled persistence, and scalable broadcast infrastructure.
The quant model matters. The systems engineering around the model matters just as much.
Prerequisites and product context
Place this step in the surface pipeline.
This is the starting architecture behind the current dashboard gateway and SVI surface stream.
Links