Tutorial

Build your first
record workspace.

From an empty directory to tasks, notes, readings and a working live-change client. Every example uses the supplied tools.

1. Open your extracted package

You will build a small workspace with tasks, notes and sensor readings, then make a change across collections and watch its committed result. Use a fresh disposable directory for this tutorial. The database tools need no compiler or cloud account. The optional script later needs Python 3.10+ and uses only its standard library.

Download the package for your machine, check and extract it, then change into the extracted directory. Keep bin/ available in your current working directory. The installation guide includes platform and opening steps.

./bin/polybase --version
./bin/polybase init ./learning-db

The version command prints polybase 0.1.0-preview.2. A fresh learning-db prints initialized head=0. If it instead reports an existing head, choose a new tutorial directory rather than resetting existing records. If the executable cannot run, check its Mac/Linux target and opening instructions before continuing.

2. Write and read your first record

./bin/polybase put ./learning-db tasks/1 '{"title":"Write a note","done":false}'
./bin/polybase get ./learning-db tasks/1

The put prints committed 1. The get prints {"title":"Write a note","done":false}. These are separate processes: the get reopened the directory and recovered the committed value. You have already stored a working record without registering a collection or providing any special business fields.

tasks/1 is the complete record key, and tasks/ is the collection prefix your application chose. The JSON is stored as supplied. You can use another name and value encoding for a different application.

3. Add two other kinds of records together

./bin/polybase batch ./learning-db \
  put notes/1 'A first note' \
  put sensors/room-a '{"temperature":21.5,"unit":"C"}'
./bin/polybase snapshot ./learning-db

The batch prints committed 2. The snapshot starts with snapshot 2 and shows all three records. There is a task with short ID 1 and a note with short ID 1; their complete keys differ. The sensor's name is another application-defined key.

The note and sensor share one commit, so they become visible together. A batch can cross collection prefixes within this local database. It does not create a transaction across separate database directories or partitions.

4. Update a task and its note in one change

./bin/polybase batch ./learning-db \
  put tasks/1 '{"title":"Write a note","done":true}' \
  put notes/1 'The task is complete'
./bin/polybase events ./learning-db 2 10
./bin/polybase get ./learning-db tasks/1

The batch prints committed 3. The event read starts strictly after snapshot position 2 and reports commit 3 with both puts, followed by next=3 head=3. The task get contains done:true. The database did not infer that relationship from JSON fields: you expressed it by putting both intended mutations in one batch.

5. Inspect your own collections in the browser

./bin/polybase-console ./learning-db

Open http://127.0.0.1:7340. In Collection / key prefix, enter tasks/, keep Any value selected and click Start query. You see the task. Change the prefix to notes/ for the note or sensors/ for the reading. Clear it to see all three.

To try a conditional edit, click the note's Edit button, replace its text, stage the mutation and inspect the staged list. Click Commit transaction to write it. The console includes the expected value you observed; if that expectation is stale, it reports a conflict rather than partly applying your staged transaction.

Do not run CLI commands against learning-db while the console owns it. Use the browser and its HTTP interface while it is running. Keep that console open for the next section; the HTTP client will use its exact localhost address.

6. Follow a named cross-collection write with Python

This optional working script evaluates the local HTTP API. It takes a task snapshot, saves its result/cursor together, submits one named write creating a task and note, inspects the original outcome, replays that same request and applies the query's later change. It then resumes from the saved view without obtaining another snapshot.

Download the two example files into the same directory. local_client.py supplies the standard-library HTTP and state-file helpers. live-workspace.py is the complete runnable example below. Neither file needs an installed SDK. This example targets the local console, not the cluster console's different request context.

curl --fail --location --output local_client.py \
  https://base.polymathlabs.xyz/examples/local_client.py
curl --fail --location --output live-workspace.py \
  https://base.polymathlabs.xyz/examples/live-workspace.py
python3 live-workspace.py --base http://127.0.0.1:7340 --state-dir ./client-state

Read or download local_client.py · Read or download live-workspace.py

#!/usr/bin/env python3
"""Run once on a disposable workspace to evaluate a named write and query resume."""
import argparse
import json
from pathlib import Path

from local_client import LocalClient, apply_page, hex_bytes, save_json


def main():
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--base", default="http://127.0.0.1:7340")
    parser.add_argument("--state-dir", type=Path, default=Path("./client-state"))
    args = parser.parse_args()
    args.state_dir.mkdir(parents=True, exist_ok=True)
    pending = args.state_dir / "pending-request.json"
    state_file = args.state_dir / "task-result.json"
    if pending.exists() or state_file.exists():
        raise SystemExit("existing client state was preserved; inspect its original request/view before a new run")
    client = LocalClient(args.base)
    snapshot = client.query("tasks/")
    rows = {row["key"]: row["value"] for row in snapshot["rows"]}
    token = snapshot["token"]
    save_json(state_file, {"rows": rows, "token": token})
    print(f"Task snapshot at #{snapshot['position']}: {len(rows)} rows")

    task_key, note_key = "tasks/stream-demo", "notes/stream-demo"
    if client.read(task_key)["value"] is not None or client.read(note_key)["value"] is not None:
        raise SystemExit("demo keys already exist; this example does not replace existing records")
    task = json.dumps({"title": "Follow live changes", "done": False}, separators=(",", ":"))
    exact = client.request([(task_key, None), (note_key, None)],
                           [(task_key, task), (note_key, "One request, two collections")])
    save_json(pending, exact)
    # If this call loses its reply, the exact submitted request stays in pending.
    receipt = client.commit(exact)
    print(f"Committed task and note together at #{receipt['position']}")
    observed = client.outcome(exact["request_token"])
    if observed["position"] != receipt["position"]:
        raise ValueError("original outcome did not match the received position")
    repeated = client.commit(exact)
    if not repeated["replayed"] or repeated["position"] != receipt["position"]:
        raise ValueError("same request did not return its original outcome")
    print("Original request replayed without a second write")

    while True:
        page = client.poll(snapshot, token)
        rows = apply_page(rows, page)
        token = page["token"]
        save_json(state_file, {"rows": rows, "token": token})
        if int(page["position"]) >= int(receipt["position"]):
            break
    if rows.get(hex_bytes(task_key)) != hex_bytes(task):
        raise ValueError("task query did not apply the committed task")
    if hex_bytes(note_key) in rows:
        raise ValueError("task query included a different collection")
    client.unsubscribe(snapshot, token)

    saved = json.loads(state_file.read_text(encoding="utf-8"))
    resumed = client.resume(saved["token"])
    next_page = client.poll(resumed, saved["token"])
    continued = apply_page(saved["rows"], next_page)
    save_json(state_file, {"rows": continued, "token": next_page["token"]})
    client.unsubscribe(resumed, next_page["token"])
    print(f"Saved task result resumed at #{next_page['position']}")
    print(f"Original outcome: #{observed['position']}; pending request kept for inspection")


if __name__ == "__main__":
    main()

The script prints its snapshot position, the shared task/note commit position, the original-request replay result and the resumed query position. The exact numbers depend on any browser edit you made. The task query includes tasks/stream-demo while the separately stored notes/stream-demo stays outside that query.

7. Inspect the saved view and request

The helper saves one complete state object using a file rename rather than saving result and cursor independently. It never modifies a saved request to create a fresh identity. If a temporary .next file remains after an interrupted client save, inspect that client state before continuing; the helper does not silently overwrite it.

To inspect the original request from another Python script in that example directory:

import json
from pathlib import Path
from local_client import LocalClient

client = LocalClient("http://127.0.0.1:7340")
exact = json.loads(Path("client-state/pending-request.json").read_text())
print(client.outcome(exact["request_token"]))
# Only after explicitly deciding to retry the original intent:
# print(client.commit(exact))

A retained outcome returns its original decimal position. Null means no outcome was observed; it does not cancel a possible submission. If a token expired, use application state/history to reconcile. After a console restart, a saved query needs a new resume registration; the persistent token and its corresponding rows are still the continuation input while history remains.

Finish your evaluation session

Stop the console with Ctrl-C when done. Reopen learning-db later to inspect the same task, note and sensor records. Keep its directory and the client-state directory distinct: the database owns committed history, while the client owns its view and submitted intent. Do not delete either to bypass an error you are still investigating.

What you built

You created a general record workspace with three collection prefixes, reopened committed records, changed related keys in one batch, read the matching event and used a browser query. The optional script evaluated a named request, exact retry and saved-result resume through the shipped local HTTP interface.

Use the how-to guides for binary records, explicit views, history maintenance, backup and the three-node lab. Use the reference for exact fields and limits, and the concept guide for the boundary between a collection prefix, a committed history and an application view.