PinotStore — Apache Pinot real-time OLAP store

PinotStore — Apache Pinot real-time OLAP store Overview Apache Pinot is a real-time distributed OLAP datastore built for low-latency analytics on both streaming and batch data. Queries are issued against a broker over an HTTP/JSON API using Pinot SQL; results come back in a columnar resultTable envelope (a dataSchema plus positional rows). Pinot has no client-side row INSERT path — real-time ingestion is driven by Kafka/stream connectors configured server-side — so it does not fit the SQL-shaped Dialect interface. PinotStore implements the minimal NoSqlStore marker interface (src/nosql/store.ts) and exposes Pinot's read-only broker query API directly. Identity: Property | Value | --------- | --------- | name | 'pinot' | library | 'fetch' | Pinot has no single canonical npm client — the broker is just an HTTP/JSON API. Rather than depend on any package, the store speaks HTTP over the global fetch (hence library='fetch'). Lazy-built client — no hard dependency There is no external driver to install. When no client is injected, connect() builds a tiny fetch-based client (via the exported createFetchClient(endpoint, headers)) against endpoint. Importing this module pulls in nothing beyond fetch. Injected client PinotStoreOptions accepts a pre-built client implementing the small PinotHttpClient shape. When provided, it is used verbatim and endpoint is ignored — this is how the test suite injects a mock (no network), and how callers can supply their own transport. Connection Build a store from connection options and call connect(): All connection options are optional: Option | Type | Purpose | ---------- | --------------------------- | ----------------------------------------------------------------------------------------- | endpoint | string | Base HTTP endpoint of the broker. Defaults to 'http://localhost:8000'. | client | PinotHttpClient | Pre-built HTTP client. When set it is used verbatim and endpoint/headers are ignored. | headers | Record<string, string> | Extra HTTP headers sent with every internal-fetch-client request. | Injected-client form Supply your own client (or a mock) to bypass the internal fetch client: Methods Internally, both query methods go through a private runSql(sql) that POSTs { sql } to /query/sql; failures are wrapped in a DatabaseError (message prefixed Pinot query failed: ...). Using any method before connect() (or after disconnect()) throws a ConnectionError. This store is read-only — there is no ingest(). Lifecycle Method | Signature | Behavior | ------------- | -------------------------------- | ------------------------------------------------------------------------------------------- | connect | connect(): Promise<void> | Uses an injected client, otherwise builds a fetch-based client against endpoint. Idempotent. | disconnect | disconnect(): Promise<void> | Clears the client and connection state. | isConnected | isConnected(): boolean | true only when connected and a client is present. | getClient | getClient(): PinotHttpClient | Returns the underlying (internal or injected) client. Throws ConnectionError if not connected. | Query API (read-only) Method | Signature | Endpoint | Behavior | ------------ | ----------------------------------------------------------------- | ----------------- | ----------------------------------------------------------------------------------------------------------- | query | query<T = Record<string, unknown>>(sql: string): Promise<T[]> | POST /query/sql | Runs Pinot SQL and maps the columnar resultTable into an array of row objects keyed by dataSchema.columnNames. Returns [] when there is no resultTable. | queryTable | queryTable(sql: string): Promise<PinotResultTable> | POST /query/sql | Runs Pinot SQL and returns the raw resultTable envelope (dataSchema + positional rows) as the broker returns it. Falls back to { rows: [] } when absent. | Example Verification status Unit / mock-verified only. The tests in tests/nosql/pinot.test.ts are fully mock-driven: an in-memory client implementing PinotHttpClient is injected via PinotStoreOptions.client; it records every POST (path + body) and returns a canned Pinot resultTable envelope. There is no network and no live Pinot broker in the test run. What this proves: query() POSTs { sql } to /query/sql and correctly maps the columnar resultTable (dataSchema.columnNames + positional rows) into row objects. query() returns [] when the response has no resultTable. queryTable() returns the raw resultTable envelope unchanged. Lifecycle: connecting via an injected client, idempotent connect(), clean disconnect(), and ConnectionError before connect() (from both getClient() and query()). Error handling: client failures wrapped in DatabaseError. What this does not prove: live execution against a real Pinot broker. Request/response shapes are asserted against Pinot's documented broker API, but end-to-end execution over the wire has not been exercised here. Related reading All data stores — the full catalogue, grouped by purpose Database types — where this sits among the 22 categories