Test Data · 2024-02-29 · 4 min read
Synthetic Data Generation – Part 1: Declarative YAML Data with Plaitpy
Introduction
It is common to be asked to test the performance of a system based on realistic future volumes—say, 1-year and 5-year business projections. When the system or business line is new, you face the Day-Zero data problem: there is no historical data to extrapolate from.
For our benchmark scenario, we model an FX futures broker expecting specific transaction volume splits across maturities (6M, 1Y, 5Y) and notional trade sizes (200,000, 1,000,000, 10,000,000).
Plaitpy Python Library
Plaitpy is a compact Python library that lets you define statistical data models in declarative YAML. It supports mixtures, conditional switches, CSV lookups, and Python lambda expressions.
Scenario 1: Equal Distribution Across Maturities
When the trade-size proportions are uniform across maturities, the YAML configuration is remarkably concise:
fields:
currency:
value: EUR/USD
maturity:
mixture:
- value: 1Y
weight: 28
- value: 5Y
weight: 14
- value: 6M
weight: 58
_base:
mixture:
- value: 200000
weight: 77
- value: 1000000
weight: 19
- value: 10000000
weight: 4
amount:
lambda: random.gauss(this._base * 1.1, this._base * 0.1)
Scenario 2: Maturity-Dependent Volume Splits (5-Year Projection)
When the trade-size distribution varies depending on the maturity (6M vs 1Y vs 5Y), we use Plaitpy's switch and onlyif directives:
fields:
currency:
value: EUR/USD
maturity:
mixture:
- value: 1Y
weight: 23
- value: 5Y
weight: 11
- value: 6M
weight: 66
_base6M:
mixture:
- value: 200000
weight: 96
- value: 1000000
weight: 3
- value: 10000000
weight: 1
_base1Y:
mixture:
- value: 200000
weight: 95
- value: 1000000
weight: 4
- value: 10000000
weight: 1
_base5Y:
mixture:
- value: 200000
weight: 48
- value: 1000000
weight: 38
- value: 10000000
weight: 14
_base:
switch:
- onlyif: this.maturity == "6M"
lambda: this._base6M
- onlyif: this.maturity == "1Y"
lambda: this._base1Y
- onlyif: this.maturity == "5Y"
lambda: this._base5Y
- default:
value: 0
amount:
lambda: random.gauss(this._base * 1.1, this._base * 0.1)
Conclusion on Plaitpy
Pros:
- Lightweight YAML files that live cleanly in Git
- Deterministic, reproducible, and easy to run in CI/CD
Cons:
- Not visual; nested conditional YAML structures take a little getting used to
Enjoyed this technical breakdown?
Subscribe for occasional engineering field notes on load testing, synthetic data, and observability—or talk to us about your system.