The hard parts of market and reference data. Handled.
Market and reference data come with hard backend work: identifiers
that disagree, feeds that drop out and leave gaps, and access rights
for every user. OpenMDTA takes care of it, built on more than twenty
years of experience founding and running a fintech, so your team can
focus on what makes your product unique.
01 / Data inYour reference and market feeds, through existing importers or
your own mapping plugins.
02 / Process & remixDurable ingestion and recovery, identity resolution, and remixing
into the Datasets your products need.
03 / Developer experienceOne Gateway and typed, schema-generated SDKs for records, live
values, tick history, and calculated figures.
04 / Your application outQuote pages, watchlists, instrument search, screeners, and
customer APIs. You add the users.
Search, live prices, tick history, key figures, and corporate actions
come built in. Adding a provider means adding its parser and mapping;
the rest of the platform is already in place.
01 / DATA IN
Your feeds. On a shared foundation.
You should choose feeds for your product—not to fit our connector
list. We implement or adapt the parsers, protocol handling, and
mappings for the providers you need, using the shared ingestion,
recovery, storage, and delivery infrastructure already in place. Your
pilot defines the exact interfaces and the integration work still
required.
Choosing a feed doesn’t grant rights to its data. Provider access,
use, and redistribution still depend on the applicable data
agreements.
02 / THE DATA MODEL
A Dataset defines what you cover.
Which entities do you have information about, and what can your
product do with it? A Dataset brings together a Catalog and,
optionally, one canonical Stream for the entities it covers. The
Catalog describes those entries; the Stream carries their time-ordered
updates.
Each Dataset has its own coverage: a broad instrument universe, a
venue’s listings, or a specialist set of structured-product terms.
Catalog — Model the things your product needs to understand.
A Catalog holds structured records: instruments, listings,
issuers, financial observations, classifications, and
relationships. Define typed fields, identifiers, and dimensions
around your data. Combine provider information with your own
fields while retaining provenance and access requirements.
Instruments and listings: identifiers, names,
currencies, venues, and product terms.
Companies and issuers: legal entities,
classifications, and issuer relationships.
Fundamentals and estimates: observations
organized by subject, metric, period, and provider.
Analyst data: structured ratings, price
targets, and recommendations.
Funds: holdings and constituent records linking
funds to instruments.
Corporate actions: dated events, split ratios,
statuses, and distributions.
A Catalog is not a fixed instrument schema or a static file. Its
records change as information arrives; your schema defines what
those records mean.
Stream — Model what happens over time.
A Stream carries a source-ordered sequence of typed updates:
quotes, trades, or other changing values. Your schema defines the
payload. OpenMDTA supplies live delivery, retained history,
explicit gaps, and recovery around it.
Source positions make continuity explicit. Live processing
continues alongside background recovery; recovered history does
not overwrite newer live state. Anything still missing remains
visible.
For example, a Catalog describes an instrument and its listings. A
Stream carries quote and trade updates. Services expose the
current values, individual historical ticks, and aggregated
candles.
The distinction is how you model and consume information—not
simply “slow data” versus “fast data.” A Catalog-only Dataset can
provide specialist information without a live Stream.
03 / CONNECT THE IDENTIFIERS
Different identifiers. The same entity.
One provider uses an ISIN, another a WKN, a third its own identifier.
Connect their information through shared identity resolution instead
of rebuilding cross-provider mapping tables in every application.
Typed identifiers. Traceable connections.
Identifiers carry their type: ISIN(…),
WKN(…), CUSIP(…), or a configured
vendor-specific type—not an unexplained string. Source records
assert which identifiers refer to the same entity. Identity
resolution connects those claims within the appropriate entity
type and selects a leading identifier using configured priorities.
Alternative identifiers and contributing source records remain
available. Your product gets a common reference without discarding
provider IDs or rewriting authoritative source data. An issuer’s
LEI identifies a legal entity, not the instrument it issued.
Same instrument does not mean same listing.
An instrument can have several trading lines. Keep its identity
separate from venue, currency, and other source-defined
dimensions. Resolve the instrument, then filter the selected
Catalog’s records for the trading context your product needs. A
lookup can return several records; even ISIN, MIC, and currency
are not universally sufficient to identify one listing.
When you need one exact source record, address it by
{catalog, record_key}. Its key stays source-owned—not
a universal instrument ID or a vendor-independent listing ID.
Changing claims, priorities, and exact references
Resolution uses source-supplied claims and configured policies,
not name matching or an assumed global identifier table. Source
updates and withdrawals recompute affected mappings; downstream
remixers follow the published Identity Catalog. Conflicts and
rebalances remain inspectable.
Different identifiers. One resolved entity.
SAP's German ISIN (DE0007164600), WKN, and FIGI share-class
identifier, connected through four source records. Actual screenshot
from our local evaluation deployment, cropped to the resolver's
selected outcome. Click to view full size.
04 / REMIX YOUR UNIVERSE
Remix your data. Your rules.
Remixing turns your inputs into a maintained, product-specific
Catalog—sometimes called a golden copy—of the entities you cover and
the fields you choose from each source. You can build several, and one
remixed Catalog can feed another. Market Streams remain separate from
these structured records; derived values can combine selected Catalog
and price inputs.
Exchange data
Regulatory data
Your own data
Remixed CatalogIdentities resolved, your rules applied
Your applicationsThrough the Gateway and typed SDKs
Specialist CatalogRemixed again, e.g. structured products
One remixed Catalog serves your applications and can feed further
specialist Catalogs.
Combine your sources into one view.
Choose which input Datasets determine membership in a broad master
Dataset. Resolve their records to shared entities, combine fields
with your own source priorities and rules, and enrich them from
other inputs. Remixing materializes its Catalog and updates
affected entities as inputs change, retaining provenance and
access requirements.
Remix again for specialist products.
Use a broad master as an input to a more focused Catalog, such as
one for structured products. Add specialist sources and fields
where that product needs depth, while the general master stays
lean. Each output has its own membership and model.
Discover depth without copying everything.
Publish a per-entity overview of membership in configured input
Catalogs, including record keys and lifecycle state. Your general
master can link to in-depth information in a specialist Dataset
without copying all its fields.
For example, show a “Structured-product details” link in your
instrument navigation header only when that Dataset covers the
instrument and the user has access—not simply because the
instrument has a structured-product category label.
05 / MAKE IT YOURS
Your fields. First-class everywhere.
Your product does not have to stop at our base data model. Add the
information that makes it yours—from the source to your developers’
SDKs, without changing OpenMDTA’s core.
Configuration first. Code when you need it.
Most source mappings and remixing rules fit our DSL: YAML or JSON
that the platform validates and runs, with no compiler involved.
When you need full flexibility, write the same logic in Rust,
compile it to WebAssembly, and deploy it as a plugin. Either way,
your logic ships separately from OpenMDTA’s service images, which
stay unchanged.
Add your own fields when remixing.
Define custom fields in your schema and populate them in your
remixing rules or plugin. Combine source values, derive
product-specific classifications, or add the business rules that
make your master Dataset useful to your team.
Extend the importer. Keep the platform.
Reuse an existing importer and extend its mapping to capture more
from the exchange feed. When the base model does not cover your
needs, derive custom fields directly from the decoded feed
messages, in the DSL or in code. Build and deploy your mapping
plugin independently—without recompiling the importer or
OpenMDTA’s service binaries.
First-class fields. All the way to your SDK.
Custom fields travel through the same schema-driven infrastructure
as standard fields. The Gateway exposes them through your
configured data model, and your auto-generated SDKs include their
types and accessors automatically. No separate API or hand-written
client wrapper for every field you add.
Lists and index families, ready for selectors.
Select instruments by list instead of maintaining member tables.
Index families such as GER40, US500, TECH100, and EU50 are Catalog
lists with metadata and identifier-only members. Use the base
lists, publish your own, or remix them. A venue selection resolves
each member to matching records, subject to coverage and access
rights.
Composition is fixed for a request. A new request or reconnect
reads it again. Weights, index calculation, and index values are
separate concerns.
Evolve your schema. Keep your clients working.
Every schema change is checked at build time and classified as
compatible or breaking.
Automatic when possible. Optional fields can be
added under the same name. Retired optional fields keep their
wire slots, so historical values stay readable and generated
SDKs keep their deprecated accessors.
Explicit when breaking. Removals and changes of
type, unit, or meaning are rejected under the existing name.
Introduce a new field version with a typed migration in the
schema or in Rust; lossy conversions require consumer opt-in.
Clients read the version they know. Gateway
reads and the generated SDKs request a supported version and
receive converted records.
06 / THE SERVICES
One Gateway. Purpose-built services.
Your team builds against one interface.
The Gateway brings the services together behind one customer-facing
interface. It authenticates requests, applies access rules, and
routes them to the right service. Your developers use a consistent,
schema-generated SDK for structured records, current values, tick
history, and calculated figures—not a separate integration for every
service.
READER / CATALOG ACCESS
Find and retrieve your structured data.
Turn a Catalog into a usable application interface. Retrieve
selected fields, look up records by keys or identifying
dimensions, and expose configured search and discovery
capabilities.
Record and field retrieval; identity-aware lookup through the
platform.
Autocomplete, text search, facets, and numeric filters.
Configured search projections and access rules.
Build an instrument finder, an issuer profile, or a searchable
fundamentals dataset.
LAST / LATEST VALUES
The current picture, continuously updated.
Read the latest available values, then follow updates through a
subscription. Shared upstream processing serves multiple consumers
without duplicating the entire ingestion path.
Current snapshots followed by typed updates—not just the last
traded price.
Applicable real-time, delayed, and end-of-day delivery policies.
Continuity and gap information alongside the data.
Build a watchlist, a quote page, or a customer-facing live API.
TIMESERIES / HISTORY & AGGREGATION
Tick history and OHLC. Built to scale.
Query individual historical observations and open/high/low/close
candles on the same platform. Partitioned storage, compressed
pages, and indexed range reads keep queries focused on the data
they need.
Tick-level history, OHLC aggregation, historical queries, and
live-following interfaces.
Explicit coverage and gaps; supported intervals follow the
configured model.
Separate query and materialization workers: add serving capacity
without repeating historical processing.
Build charts, historical analysis, or a vendor’s historical-data
API.
KEYFIGURES / CALCULATIONS & SCREENING
Turn data into useful calculations.
Combine Catalog information with selected price inputs. Define
calculation logic and expose results for individual instruments or
searchable result sets.
Configurable calculations, input selection, and derived numeric
values.
Filtering, sorting, and screening with price-freshness,
availability, and access policies.
Midpoint and spread are implemented examples; additional
financial formulas and their inputs are scoped separately.
Build a product screener or a customer-specific calculated
dataset.
OBSERVATION / WATCHES & TRIGGERS
Turn changing data into useful notifications.
Build persistent watches around the conditions your users care
about.
Durable task state, trigger transitions, and protection against
rapidly oscillating conditions.
Task state and pending notifications committed together;
delivery retries use a durable outbox.
Historical recovery produces separate summaries, not misleading
live alerts, and does not overwrite the current live state.
Your application supplies instrument resolution, typed rules, and
notification delivery. The reusable core is implemented; a
ready-made Gateway observation API or email/push integration is
not implied.
Build price alerts, threshold watches, or customer-specific
notifications that bring users back at the right moment.
Delivery semantics and integration boundaries
Catalog import cursors advance after writes complete; replayed
writes are idempotent. Immutable payloads precede manifest
publication. These are scoped semantics, not a blanket exactly-once
promise. Ordering is source-local; SDK reconnect delivery is at
least once. Clients handle gaps and replacement values according to
the selected API contract.
Catalog search exposes a configured projection to holders of its
search grant; per-record detail permissions do not filter that
projection. Review its fields before enabling it. Keyfigures search
checks the caller’s permitted inputs before returning results.
Search freshness and ranking follow the selected service’s contract;
there is no universal guarantee of exact global live ranking.
07 / BUILD
Typed data. Familiar code.
OpenMDTA generates type-safe SDKs in multiple languages from your
schema, so your fields reach your developers’ editors with their
types. Prefer plain HTTP? Every Gateway also publishes an OpenAPI
document for the same data, with a built-in explorer to try it.
This example from our generated Globex SDK streams MSFT and AAPL
quotes with the default blocks. Each typed message keeps the selector
that produced it, and prices stay exact Decimal values.
import { connect, selector, type ConnectOptions } from "@openmdta/globex";
export async function readQuoteSample(options: ConnectOptions) {
const connection = await connect(options);
try {
const requested = [selector.us("MSFT"), selector.us("AAPL")] as const;
// No options: the stream carries every block the Dataset publishes.
const quotes = connection.select(requested).latestStream();
let received = 0;
for await (const msg of quotes) {
const quote = msg.fields.bidAsk;
console.log(
msg.selector.expression,
quote?.bid?.price.toString(),
quote?.ask?.price.toString(),
);
if (++received === 10) break; // Leaving the loop cancels the stream.
}
} finally {
await connection.close();
}
}
The whole market, not just one quote.
Beyond single-instrument snapshots, a stream feed delivers the
market data of every instrument in a Dataset: in source order,
with gaps made explicit, in real-time, delayed, or end-of-day
quality.
All your reference data, in one feed.
A Catalog feed delivers every reference-data record with the
fields you choose and marks progress with cursors. Use it to fill
your own database, cache, or search index.
08 / ENGINEERING TARGET
Two million instruments. One fast CPU core.
Our goal: handle a feed of 2 million instruments at 200,000
updates per second, including ingestion, latest-state maintenance,
and time-series processing, on one fast CPU core.
Only the separate S3 storage server’s resources are excluded.
OpenMDTA’s encoding, compression, and client-side storage work
remain in scope.
09 / OPERATE
Durable storage. Your operations.
STORAGE / DURABLE STATE
Durable storage. Replaceable services.
OpenMDTA’s durable state lives in S3-compatible storage, not in an
irreplaceable service instance. Restart or replace services,
reconnect them to the same storage, and resume from persisted
state. Local caches can be rebuilt; there is no OpenMDTA-specific
backup-and-restore procedure for each service.
Reliable storage is the foundation of recovery. It must provide
the required consistency and conditional-write semantics.
Availability, retention, and data protection remain part of your
storage design. Keep configuration and credentials independently
available.
OPERATIONS / CHOOSE YOUR CONTROL LAYER
Your data plane. Optional managed control plane.
The data plane provides the services, schemas, connectors, SDK
tooling, and a static local configuration. Run those binaries with
the deployment system your team already operates. Our commercial
control plane adds the Kubernetes operator, declarative topology
reconciliation, coordinated rollouts, and managed operations.
Prefer bare metal? Run and manage the service binaries directly.
Kubernetes is an operating choice, not a requirement of the data
platform.
Our goal: independence you can demonstrate.
Give an external engineering team the data-plane release and its
documentation. Can they install it, run the static local
example, ingest a feed, generate an SDK, reconnect services to
existing storage, upgrade, and operate it without the commercial
Kubernetes control plane? That is our acceptance test for
independent self-hosting.
10 / MORE
More stuff
Different users. Different data access.
Your server creates and signs user-specific access tokens,
limiting each user to their licensed subset of data packages and
delivery quality. Signing keys stay on your server; clients
receive only short-lived, signed access tokens. The Gateway and
services enforce the applicable access rules.
For example, your real-time NASDAQ Level 2 license does not mean
every user should get that access. Grant premium exchanges or
specialist Datasets only to entitled users, while others keep
access to your standard offering.
Your application maps subscription plans to grants and handles
billing. OpenMDTA handles data access within the rights you hold.
Corporate actions. Consolidated and applied.
Consolidate dated corporate actions and distributions across
sources with deterministic source priorities and retained
provenance. Request raw values or opt into split adjustment
through the Gateway for supported snapshot, live, and time-series
fields. Confirmed split ratios adjust prices and quantities at
delivery; stored source data stays unchanged.
Raw is the default. Distributions are modeled and consolidated,
but are not yet applied as a total-return adjustment. Feed
coverage and the financial rules required by your product remain
part of the integration scope.