Neural SDE Yield Curve — Nelson-Siegel Dynamics as Structural Constraints:

The yield curve encodes the market’s full term structure of interest-rate expectations. This project asks whether a Neural stochastic differential equation that embeds the Nelson-Siegel factor dynamics — level, slope, curvature — as structural constraints can learn a parsimonious yet flexible continuous-time model that forecasts the entire curve shape, while staying interpretable in economic factors. The state is the 3-D NS factor vector \(x_t = (\text{level}, \text{slope}, \text{curvature})\); a shared MLP parameterises both the drift \(f(x_t)\) and a lower-triangular diffusion factor \(L(x_t)\) (so the diffusion covariance \(LL^\top\) is PSD by construction), and the network is trained in plain torch by maximising the log-likelihood of the Euler-Maruyama discretisation — torchsde is never imported, the SDE step is written by hand. A Nelson-Siegel penalty anchors the drift to mean-reverting, economically coherent dynamics. The model is then benchmarked honestly against the Dynamic Nelson-Siegel VAR, an LSTM, and the stubborn random walk, with Diebold-Mariano tests. The candid result: on out-of-sample point RMSE the random walk on yields remains champion (the classic Duffee finding) — DNS-VAR ties it, the Neural SDE ties it (not significantly worse), and the flexible LSTM significantly loses by overfitting the factor dynamics. The SDE’s genuine value is therefore reframed precisely: not a sharper point forecaster but a coherent continuous-time scenario engine with calibrated multi-horizon forward-curve uncertainty fans, interpretable factor dynamics, and a convex VaR / stress framework. Built on Python 3.12 with torch, numpy, scipy, pandas, plotly, streamlit, pydantic v2 and typer; packaged with hatchling and tested with pytest against deterministic seed-42 fixtures.

I. Interactive Dashboard:

The dashboard below runs entirely in the browser via stlite (Streamlit on WebAssembly — no server). PyTorch cannot run in Pyodide, so the in-browser demo trains a pure-NumPy Euler-Maruyama Neural-SDE stand-in (per-factor OU drift + a diffusion factor by gradient descent) alongside the DNS-VAR and random-walk baselines, reproducing the identical pipeline: the animated forward-curve fan chart with forecast uncertainty bands, the NS factor decomposition time series, and an interest-rate scenario stress table, all on seed-42 synthetic presets with interactive sliders. The full project trains the torch Neural SDE and the LSTM. First load downloads Pyodide and may take 20–40 seconds.

II. Project Layout:

neural_sde_curve/
├── pyproject.toml                              # Build config, deps, ruff + pytest settings
├── .env.example                                # Optional FRED_API_KEY (real-data only)
├── dashboard.html                              # Self-contained stlite demo (numpy Euler-Maruyama)
├── scripts/
│   ├── make_thumbnail.py                       # Real matplotlib thumbnail (forward-curve fan)
│   └── download_data.py                        # Optional FRED par-yield + macro fetch
├── src/neural_sde_curve/
│   ├── data/
│   │   ├── synthetic.py                        # Seed-42 NS-factor + curve generator
│   │   ├── schemas.py                          # Pydantic v2 curve / factor / forecast records
│   │   ├── fetchers.py                         # FRED DGS* par yields + macro (optional)
│   │   └── store.py                            # DuckDB cache (explicit INSERT column lists)
│   ├── models/
│   │   ├── dns.py                              # Dynamic Nelson-Siegel factor extraction + VAR
│   │   ├── neural_sde.py                       # Plain-torch drift/diffusion MLP + EM log-likelihood
│   │   ├── lstm.py                             # LSTM factor-path benchmark
│   │   └── benchmarks.py                       # Random-walk + DNS-VAR baselines
│   ├── eval/
│   │   ├── train.py                            # Walk-forward, no-leak origins
│   │   └── metrics.py                          # Per-maturity RMSE / MAE + Diebold-Mariano (HLN)
│   ├── scenario/
│   │   └── var.py                              # 1000-path forward-curve sampling + bond-VaR / stress
│   ├── report/plots.py                         # Plotly: fan chart, factor series, stress table
│   ├── cli.py                                  # Typer CLI: compare | dmtest | scenario | isolation | stats
│   └── app.py                                  # Streamlit server-side dashboard
└── tests/                                      # 35 seed-42 offline tests
  

III. Dynamic Nelson-Siegel — The Interpretable Baseline (models/dns.py):

At each date the curve is collapsed onto three factors by OLS against the fixed Nelson-Siegel loadings on the eleven tenors (1m, 3m, 6m, 1y, 2y, 3y, 5y, 7y, 10y, 20y, 30y). The factors are the economically named level \(\beta_1\), slope \(\beta_2\), and curvature \(\beta_3\):

\[ y(\tau) = \beta_1 + \beta_2\,\frac{1 - e^{-\lambda\tau}}{\lambda\tau} + \beta_3\!\left(\frac{1 - e^{-\lambda\tau}}{\lambda\tau} - e^{-\lambda\tau}\right). \]

Stacking the per-date factors and fitting a joint VAR(1) on \((\beta_1,\beta_2,\beta_3)\) gives the Dynamic Nelson-Siegel forecaster — a strong, fully interpretable classical benchmark. Crucially, the factor representation is exact enough that the NS round-trip (curve → factors → curve) is verified to machine tolerance in the test suite; everything downstream operates on the 3-D factor state rather than the 11-D curve.

IV. The Neural SDE & Its NS Structural Constraint (models/neural_sde.py):

The factor state follows a stochastic differential equation \(dx_t = f(x_t)\,dt + g(x_t)\,dW_t\). A shared MLP trunk emits the drift \(f \in \mathbb R^3\) and a lower-triangular \(L \in \mathbb R^{3\times3}\), so the diffusion covariance \(\Sigma = LL^\top\,dt\) is positive semi-definite by construction. Training maximises the Gaussian log-likelihood of the hand-written Euler-Maruyama transition (monthly step \(dt = 1\)):

\[ x_{t+1}\,|\,x_t \sim \mathcal N\!\big(\,x_t + f(x_t)\,dt,\; (LL^\top)(x_t)\,dt\,\big), \qquad \text{NLL} = \tfrac12\big[(x_{t+1}-\mu)^\top \Sigma^{-1}(x_{t+1}-\mu) + \log\det\Sigma\big]. \]

To stop the network learning economically incoherent curve shapes, a Nelson-Siegel structural penalty is added to the loss. The first term pulls the drift toward an Ornstein-Uhlenbeck mean-reversion around the empirical factor mean \(\hat\mu\); the second caps the magnitude of the level-factor drift at \(c_{\text{level}}\), encoding the stylised fact that the curve level is highly persistent:

\[ P(\theta) = w_{\text{mr}}\,\overline{\big\|\,f(x_t) + \operatorname{diag}(\kappa)(x_t - \hat\mu)\,\big\|^2} \;+\; w_{\text{lvl}}\,\overline{\big[\operatorname{relu}\!\big(|f_{\text{level}}(x_t)| - c_{\text{level}}\big)\big]^2}. \]

def em_nll(self, x_t, x_next, dt=1.0):
    f, L = self.drift_and_chol(x_t)            # shared MLP -> drift, lower-tri L
    mu = x_t + f * dt
    Sigma = (L @ L.transpose(-1, -2)) * dt     # = g g^T dt, PSD by construction
    err = (x_next - mu).unsqueeze(-1)
    quad = err.transpose(-1, -2) @ torch.linalg.solve(Sigma, err)
    logdet = torch.logdet(Sigma)
    return 0.5 * (quad.squeeze() + logdet)     # Euler-Maruyama Gaussian NLL

def ns_penalty(self, x):                       # structural mean-reversion + level cap
    f, _ = self.drift_and_chol(x)
    ou = f + self.kappa * (x - self.mu_hat)
    cap = torch.relu(f[:, 0].abs() - self.c_level)
    return self.w_mr * (ou ** 2).mean() + self.w_lvl * (cap ** 2).mean()

An isolation test confirms the penalty bites: the constrained level-drift mean magnitude is 0.089 versus 0.154 unconstrained, with its maximum pinned at \(c_{\text{level}}\). Training is mini-batched over enough epochs that the loss genuinely decreases (no full-batch underfit), and the result is reproducible from the seed.

V. Forecast Leaderboard (cli.py compare):

Walk-forward, seed-42, 300 months, with every origin held out, predicted once, and scored against the realised curve. Errors are full-curve RMSE / MAE in basis points at the 1-month and 3-month horizons:

Honestly reported: on point RMSE the random walk on yields is essentially unbeatable — DNS-VAR edges it by a basis point of noise, the Neural SDE ties it (slightly worse but within sampling error), and the flexible LSTM significantly loses, overfitting the factor dynamics. This is the expected, deliberately un-oversold result; a yield curve is so close to a martingale that added machinery struggles to add point accuracy.

VI. Is Any Difference Significant? Diebold-Mariano (eval/metrics.py):

Raw RMSE gaps this small can be pure noise, so each model is tested against the random walk with the Diebold-Mariano statistic (Harvey-Leybourne-Newbold small-sample correction, Newey-West lag \(h-1\)):

The significance test sharpens the story: DNS-VAR and the Neural SDE are statistical ties with the random walk (\(p > 0.2\)), so neither the classical nor the neural model genuinely beats it on point forecasts — but neither is significantly worse. The LSTM is the only model the test can distinguish, and it distinguishes it as a loser (\(p < 0.05\) at both horizons). The Neural SDE’s structural constraint is precisely what keeps it pinned to the random walk instead of drifting off like the unconstrained LSTM.

VII. Where the SDE Actually Wins: Scenario Generation & VaR (scenario/var.py):

A random walk gives a point and nothing else; the fitted Neural SDE is a full generative model of the curve. Sampling 1000 forward-curve paths at the 3-month horizon and pricing a duration-matched bond portfolio (duration ~8.4) under each gives a coherent, calibrated risk distribution:

The parallel rate-shock stress is convex, as a bond book should be: a −200bp rally adds +19.6%, a +200bp sell-off costs −13.8%, and +300bp costs −18.6% — gains outrun losses for symmetric moves. VaR also rises monotonically with horizon as the forward-curve fan widens. This is the SDE’s real contribution: not a sharper number, but calibrated multi-horizon uncertainty and a convex VaR / stress engine the random walk simply cannot provide.

VIII. CLI — cli.py:

# Install
pip install -e ".[dev]"

# Forecast leaderboard: DNS-VAR vs Random Walk vs Neural SDE vs LSTM
nsde compare

# Diebold-Mariano significance tests vs the random walk
nsde dmtest

# Sample 1000 forward-curve paths -> bond-portfolio VaR + rate-shock stress
nsde scenario

# NS structural-constraint isolation test (constrained vs unconstrained drift)
nsde isolation

# Launch the server-side Streamlit dashboard
streamlit run src/neural_sde_curve/app.py
  

IX. Test Suite:

Thirty-five tests, fully offline, seed-42. Data tests verify the NS curve→factor→curve round-trip is exact and the causal baselines are leak-free — every walk-forward origin is held out, predicted exactly once, and scored against curves[origin + h]. Model tests confirm the Euler-Maruyama loss decreases under training (no full-batch underfit), that the diffusion covariance is PSD, and that two runs from the same seed are identical. The NS-penalty isolation test proves the constraint caps the level-drift magnitude. Scenario tests check VaR ordering across horizons and the convexity of the rate-shock stress; a DuckDB round-trip test guards the storage layer.

def test_ns_penalty_caps_level_drift(factors):
    free = train_sde(factors, w_lvl=0.0)
    cons = train_sde(factors, w_lvl=2.0, c_level=0.15)
    assert level_drift_mag(cons) < level_drift_mag(free)     # the constraint bites
    assert level_drift_mag(cons, reduce="max") <= 0.15 + 1e-3

def test_walk_forward_has_no_leakage(curves):
    preds = walk_forward(curves, horizon=1)
    for origin, pred in preds:                                # each origin OOS, predicted once
        assert target_of(pred) is curves[origin + 1]

X. Configuration & Setup:

cd assets/projects/neural_sde_curve
python -m venv .venv && .venv\Scripts\Activate.ps1        # Windows
pip install -e ".[dev]"
nsde compare                                              # reproduce the leaderboard
nsde dmtest                                               # Diebold-Mariano significance
PYTHONPATH=src pytest -q                                  # 35 tests, offline
streamlit run src/neural_sde_curve/app.py
  

No data download is required: the models, tests and dashboard all run on the seed-42 synthetic NS-factor generator with no API keys. The optional scripts/download_data.py and data/fetchers.py fetch real US Treasury par yields (FRED DGS1MO–DGS30) plus macro covariates (Fed Funds, CPI, unemployment), gated behind a FRED_API_KEY in .env — never touched by CI. An honest caveat: the synthetic generator is itself NS-mean-reverting, which makes DNS-VAR a near-oracle baseline and the random walk even harder to beat; on real FRED data the relative ordering would shift but the “random walk is hard to beat” punchline is robust.

Team:

Theodosios Dimitrasopoulos, personal project.

Tools & methods:

Python 3.12, PyTorch (in-house Neural SDE via hand-written Euler-Maruyama, no torchsde), NumPy, SciPy, pandas, Pydantic v2, Typer, rich, Plotly, Streamlit, pytest, ruff, hatchling. Methods: the Nelson-Siegel curve factorisation and Dynamic Nelson-Siegel VAR (Diebold-Li 2006); Neural stochastic differential equations with a shared drift/diffusion MLP and a Cholesky-parameterised PSD diffusion; maximum-likelihood training on the Euler-Maruyama Gaussian transition with a structural mean-reversion / level-persistence penalty; an LSTM factor-path benchmark; the random-walk martingale baseline (Duffee); walk-forward forecast evaluation with per-maturity RMSE / MAE and the Harvey-Leybourne-Newbold-corrected Diebold-Mariano test; and Monte-Carlo forward-curve scenario generation for duration-matched bond-portfolio VaR, CVaR and convex rate-shock stress testing.