Reference
Commands &
HTTP API.
Exact arguments, fields, responses and defaults for the tools that ship in the evaluation preview.
The four tools and the launcher
Run commands from the extracted package directory. Paths beginning with ./bin/ select the supplied executable. All four tools accept --help, -h, --version and -V without opening a database or starting a service. Their application arguments are positional.
| Tool | Invocation | Purpose |
|---|---|---|
polybase | ./bin/polybase COMMAND DIR … | Open one local database, run a command, then close it. |
polybase-console | ./bin/polybase-console EXISTING_DB_DIR [PORT] | Own one local database and serve its browser console and HTTP API. Default port: 7340. |
polybase-node | ./bin/polybase-node CONFIG_JSON NODE_ID EXISTING_EMPTY_OR_REPLICA_DIR [--lab-controls] | Run one voter from the fixed three-node configuration. |
polybase-cluster-console | ./bin/polybase-cluster-console CONFIG_JSON [PORT] | Serve the cluster browser console and HTTP API. Default port: 7342. |
The database tools do not require Rust, Node, Docker or a cloud account. The optional launcher needs Python 3.10 or later:
python3 scripts/start-local-cluster.py ./my-lab --bin-dir ./bin
| Argument | Default | Meaning |
|---|---|---|
directory | Required | Lab directory. A new unbound directory must be empty. Existing configuration and node directories are retained. |
--bin-dir | Source-build location; pass ./bin for the download | Directory containing the node and cluster-console executables. |
--console-port | 7342 | Port for the cluster console. |
--node-ports | 7351 7352 7353 | Three distinct ports for a new lab. Existing configuration determines existing voter addresses. |
The launcher accepts four distinct ports between 1024 and 65535. It starts detached processes and records logs and observed process IDs. Closing its console leaves the nodes running. See how to start and stop your lab.
Every local CLI command
The CLI accepts UTF-8 strings as keys and values. Quote spaces and JSON in your shell. It does not parse JSON documents. Binary values are best read through the console's hex editor or the HTTP interface; CLI output converts non-UTF-8 bytes to display text.
| Command | Arguments | Result and behavior |
|---|---|---|
init | DIR | Create the directory if its parent exists, or open an existing directory. Print initialized head=N. Existing data is recovered, never reset. |
get | DIR KEY | Print the current value plus a newline, or (absent). An empty stored value prints an empty line. |
put | DIR KEY VALUE | Insert or replace one byte value. Print committed N. |
delete | DIR KEY | Submit a deletion. Print its commit position. A raw write can create a commit even when the key was already absent. |
batch | DIR (put KEY VALUE | delete KEY)… | Commit supplied mutations together, in supplied order. Invalid syntax submits no partial batch. |
snapshot | DIR | Print snapshot N, then current key/value entries in byte-key order. This is display output, not a backup format. |
events | DIR AFTER LIMIT | Read raw committed batches strictly after AFTER. LIMIT is an integer from 1 through 1024. Print commits, mutations and a next=N head=H footer. |
./bin/polybase init ./workspace-db
./bin/polybase batch ./workspace-db put tasks/1 open put notes/1 'A first note'
./bin/polybase get ./workspace-db notes/1
./bin/polybase batch ./workspace-db put tasks/1 done delete notes/1
./bin/polybase events ./workspace-db 0 10
./bin/polybase snapshot ./workspace-db
For this fresh database, the two batches print committed 1 and committed 2; the last snapshot starts with snapshot 2. Commands return exit status 0 on success and 1 on an error. Only init creates a missing directory; valid commands can initialize an existing empty directory when opening it. Stop the console before running the CLI against its directory.
Local HTTP conventions
Start polybase-console and use the exact printed address, normally http://127.0.0.1:7340. These are the preview console's local endpoints, not a hosted SaaS API. They are useful for evaluation scripts; there is no separately distributed SDK.
./bin/polybase-console ./workspace-db 7340
# From a second terminal:
curl --http1.1 http://127.0.0.1:7340/api/health
curl --http1.1 -H 'Content-Type: application/json' \
-H 'Origin: http://127.0.0.1:7340' \
--data '{"key":"7461736b732f31"}' \
http://127.0.0.1:7340/api/read
POST requests require exactly Content-Type: application/json and an Origin matching the console address, including its port. Use 127.0.0.1 consistently rather than substituting localhost. The HTTP/1.1 interface accepts ordinary Content-Length bodies; streamed bodies and Expect headers are unsupported. Each response closes the connection.
| Type | Representation | Example |
|---|---|---|
| Keys and values | Use lowercase, even-length hexadecimal strings representing bytes. Responses use lowercase hex. An empty string represents zero bytes. | tasks/1 → 7461736b732f31; open → 6f70656e. |
| Absent values / deletions | JSON null. This differs from the empty byte value "". | {"key":"7461736b732f31","value":null} |
| Positions and session IDs | Decimal strings, not JavaScript floating-point numbers. | "0", "42". Keep large positions as strings or arbitrary-precision integers. |
| Query and request tokens | Opaque strings returned by the server. Preserve the complete string. | Use the returned token or request_token; do not construct one. |
| Instance | Opaque string identifying this running console process. | Pair the returned instance with the session id. |
Unknown fields are rejected in the typed request bodies below. Always include required fields, even when their value is an empty string or null. JSON is just one possible value encoding: the server compares the encoded bytes you supply.
Complete local endpoint reference
| Method and path | Request body | Response / effect |
|---|---|---|
GET /api/health | None | Current local position, retained floor, identity, instance, state and subscription counters. Does not create a commit. |
POST /api/read | {key} | {value, position}. Value is hex or null. The read is validated at the returned local prefix. |
POST /api/query | {key_prefix, predicate, value} | {id, instance, token, position, rows}. Rows contain hex key and value. Registers a filtered snapshot and later changes. |
POST /api/resume | {token} | {id, instance}. Registers continuation from a saved token; does not send another snapshot. |
POST /api/poll | {id, instance, token} | {commits, token, position, head, floor}. Scans up to 64 retained commits. Apply complete changes before saving the returned token. |
POST /api/unsubscribe | {id, instance, token} | {closed:true}. Releases the matching query registration. |
POST /api/request | {} | {request_token}. Issue a named-write identity. Save it with the exact intended request before submitting. |
POST /api/commit | {reads, mutations, request_token?} | One conditional atomic batch. A named success returns {position,wrote:true,durable:true,replayed}. Without a request token there is no retained duplicate-request receipt. |
POST /api/outcome | {token} | {position}, with a decimal string for a retained outcome or null for no outcome observed. |
POST /api/checkpoint | {} | {generation,floor}. Full checkpoint, retaining the existing history floor. |
POST /api/checkpoint-incremental | {} | {generation,floor}. Optional incremental checkpoint; may use the full path when required by the current state or caps. |
POST /api/compact | {position} | {generation,floor}. Explicitly advance retained history through the supplied decimal position, protecting active subscriber cursors. |
POST /api/close | {} | {closed:true}. Stops the local console and releases its database owner after workers finish. |
Query fields
key_prefix is a hex-encoded byte prefix. Use 7461736b732f for tasks/, or "" for all keys. predicate is exactly any, equals or prefix. value is a hex string; supply "" with any. Equals compares the entire byte value, and prefix compares its starting bytes. Neither reads fields inside JSON.
curl --http1.1 -H 'Content-Type: application/json' \
-H 'Origin: http://127.0.0.1:7340' \
--data '{"key_prefix":"7461736b732f","predicate":"any","value":""}' \
http://127.0.0.1:7340/api/query
Conditional write fields
reads is an array of expected entries, each containing key and value. Use null to expect an absent key. There must be one unique expected key for every mutated key. Extra expected reads may check another dependency. mutations is a nonempty array of {key,value}: a hex string inserts or replaces; null deletes. Mutation order is retained, including repeated keys. The local console accepts at most 128 expected reads and 128 mutations per request.
A condition checks bytes, not a revision number. If a value changes away and later returns to the same bytes before your submission, that condition can still succeed. See the condition model. Use a named request for an outcome you can inspect after a missing reply. The working Python example constructs a complete named request without placeholder tokens.
Reading responses correctly
A query response establishes the starting result. Poll responses contain complete transaction groups. Each group has a decimal position and a changes array. Each change has key, before and after. Null before means an insertion; null after means a deletion. A replacement has both values.
One transaction yields at most one net change per matching key, even if it writes that key several times. A whole-value no-op can produce no query change, while the cursor still advances. Empty commits therefore does not mean the returned token should be ignored. head is the prefix captured for that response, and floor is the oldest retained position.
| Field | Type | Meaning |
|---|---|---|
head, floor | Decimal strings | Current local committed prefix and oldest resumable prefix. |
checkpoint, generation | Decimal strings | Position selected in the recovery checkpoint and selected storage generation. |
identity, instance | Opaque strings | Persistent database identity and running-console identity. A restart changes instance; replacing the database changes identity. |
keys, wal_bytes | Nonnegative JSON numbers | Live keys and bytes in the selected write-ahead log. |
version_bytes | Nonnegative JSON number | Logical charge for concurrent retained versions, rather than total process memory. |
subscribers, subscription_bytes | Nonnegative JSON numbers | Current subscriber count and aggregate logical subscription charge. |
work_cache_bytes, work_hits, work_misses | Nonnegative JSON numbers | Shared query-projection cache charge and reuse counters. |
checkpoint_layers | Nonnegative JSON number | Layers in the currently selected full/incremental checkpoint chain. |
These counters are observations, not one coordinated application snapshot or a promise about memory use. Use a query or validated read for record values.
Status codes and your next action
Local errors contain {code,message}. Check both the HTTP status and the structured code. A timeout or broken connection after submission may hide a successful write; keep the original named request.
| HTTP / code | Meaning | Next action |
|---|---|---|
400 invalid_request | Unsupported route, field, shape, number or byte encoding. | Correct the request. An invalid batch is not partly submitted. |
400 invalid_token | Token is malformed, from another history, ahead of the head, or incompatible with the query. | Use the token belonging to this database and exact query; do not edit its bytes. |
403 origin_rejected | Host/Origin does not match the exact printed loopback address. | Use the same 127.0.0.1 address and port in the URL and Origin header. |
409 conflict | An expected value changed; this local request did not commit. | Read the latest values and explicitly decide on new intent. A failed retry does not resolve an earlier uncertain attempt. |
409 request_mismatch | A retained request token was reused with different intent. | Inspect and preserve the original payload; do not change a submitted token's intent. |
409 history_pinned | Requested compaction would pass an active query cursor. | Poll or close the subscriber before choosing a new maintenance boundary. |
410 history_expired | Required query history is below the retained floor. | Start a fresh query and replace its result and cursor together. |
410 subscription_expired | The registration was idle, fell behind, or belongs to a prior console instance. | Resume the saved result/token if history remains, or obtain a fresh snapshot. |
410 request_expired | The named request's retained horizon ended. | Reconcile with application state/history. Do not automatically issue a new request identity. |
413 request_too_large | HTTP body exceeds 128 KiB. | Reduce the request. Remember that hex roughly doubles the underlying byte payload. |
429 capacity | A bounded queue, result, state or history allowance cannot admit this work. | Read the message; reduce query scope or batch size, release idle queries, or perform appropriate explicit maintenance. |
503 uncertain | Write/maintenance outcome may be unknown, or the instance cannot continue after an earlier failure. | Keep exact intent; stop and reopen when required, then inspect its named outcome. |
500 engine_error | Another engine error, including storage failures needing inspection. | Keep the directory intact, read the message and inspect the owning process. Do not assume a missing reply means no write. |
Defaults exposed by the preview
The supplied command-line servers use their default options. There are no CLI flags for increasing these limits. Counts and logical bytes bound individual data structures; they are not hard process-memory or timing budgets.
| Resource | Default | Implication |
|---|---|---|
| Encoded local transaction | 1 MiB / 1,024 mutations | Includes encoding overhead. Console requests have the smaller HTTP/128-mutation limits. |
| Key | 64 KiB | Count UTF-8 bytes, not characters. Empty byte keys are supported. |
| Live local state | 64 MiB key+value payload / 100,000 keys | All collection prefixes share the same database limits. |
| Retained local WAL | 256 MiB | New writes reject at the cap; old history is not automatically removed. |
| Console request / response | 128 KiB / 8 MiB | Hex and JSON representation add overhead. |
| Console query registrations | 32 / 60 seconds idle | Resume or start a fresh query after idle expiry. |
| Query result | 4,096 rows / 2 MiB logical charge | A broad prefix can fail rather than return a silently truncated result. |
| Query lag / poll | 4,096 commits / up to 64 scanned commits per HTTP poll | Keep polling; smaller results do not remove the retained-history horizon. |
| Named local receipts | 4,096 / 16 MiB logical charge | Whole canonical requests are retained, not just token strings. |
| Displayed browser changes | 200 changes | Display history is bounded; it is not the database's retained WAL. |
The bundled GUARANTEES.md lists further engine bounds for versions, leases, pages and checkpoints. The implemented scope explains the practical limits.
The separate cluster-console API
The cluster console normally runs at http://127.0.0.1:7342. Every API call is POST and requires node as a decimal string, read_mode as quorum or local, and an optional minimum of null or {index,term} with decimal strings. The same exact Origin and JSON conventions apply.
curl --http1.1 -H 'Content-Type: application/json' \
-H 'Origin: http://127.0.0.1:7342' \
--data '{"node":"1","read_mode":"local","minimum":null}' \
http://127.0.0.1:7342/api/health
The health call reports all configured nodes, their availability and roles, plus the selected context. It does not promise a quorum-confirmed record read. Select the observed leader explicitly for writes and quorum reads; do not assume node 1 is the leader. Node IDs are exactly 1, 2 and 3, with distinct IPv4 loopback addresses and one nonzero 16-byte hex partition identity in cluster.json. Keep that binding intact on restart.
| Route | Body in addition to context | Important difference |
|---|---|---|
/api/health | No additional fields | POST, including explicit context. Returns node status, selected availability and this console instance. |
/api/query | {key_prefix,predicate,value} | Returns snapshot rows, a term-bound position/token, registration id and instance. |
/api/resume | {token} | Checks the selected node/read mode, then returns id and instance without another snapshot. |
/api/poll, /api/unsubscribe | {id,instance,token} | Session and token belong to this partition/query. A valid cursor may continue on another eligible node. |
/api/read | {key} | Returns value, position, term and mode. Local reads can be stale; a retained minimum can reject a node that is behind. |
/api/request | No additional fields | Request issuance first confirms a quorum on the selected leader. |
/api/commit | {reads,mutations,request_token} | Named token is required. At most 64 expected keys and 64 mutations. Check the durable receipt's status, wrote, position and term. |
/api/outcome | {token,intent:{reads,mutations}} | The original canonical intent is required. Outcome lookup uses the selected node's quorum-confirmed behavior. |
/api/checkpoint | {confirm:"COMPACT"} | Select a physical snapshot and compact this node. Older query cursors can expire; this is not the local checkpoint's history-preserving behavior. |
/api/lab | {action,confirm} | With lab controls enabled, action is isolate, heal, campaign or stop; confirmation is that action in uppercase. These are explicit experiments on the selected owned node. |
/api/close | {confirm:"CLOSE CONSOLE"} | Close only the console; nodes continue running. |
/api/checkpoint-incremental and /api/compact are local-console routes and are not cluster-console routes. Replicated commits can return HTTP 200 with a durable non-write status such as conflict or capacity. A successful write has status=applied, wrote=true and durable=true; read the receipt rather than treating HTTP 200 alone as a mutation.
Additional cluster errors include 409 not_leader, 409 behind, 503 quorum_unavailable and 502 protocol. Keep the last applied result/cursor when a read cannot satisfy its chosen mode. A leader hint is a selection hint, not permission to replace a pending write's identity. See original-request reconciliation.
| Resource | Default |
|---|---|
| Application state | 512 KiB / 4,096 keys |
| Application command | 128 KiB / up to 64 mutations and 64 expected keys |
| Retained protocol entries | 512 / 4 MiB, including entries that do not change application records |
| Retained request outcomes | 256 / 512 KiB canonical intent plus receipt charge |
| Snapshot transfer | 2 MiB, one incoming transfer, 64 KiB chunks |
| Topology | Exactly three voters, one partition, one host in the supplied lab |
The local and cluster histories have different identities, position formats, request horizons and maintenance behavior. A local database directory is not a replica's inner store, and local tokens cannot be used as cluster tokens.
Continue reading
Put the reference into practice →