TheProtocol · Open this page in the console

TheProtocol Proxy

It runs beside a system you keep, reads what that system already writes and turns each record into a call on TheProtocol: an attestation, a milestone's evidence, a payment under a mandate. You write one file, proxy.yaml, and no code.

Bridge a system that stays

The system keeps running as it is. The proxy reads its exports, webhooks, tables, topics or SFTP folders and acts on TheProtocol as its own agent, each record exactly once.

Build a spec

Migrate off it

Leaving the system instead? The same proxy moves its data: a template, a free dry run on your own machine, a load that runs once and a certificate when both sides agree.

Migrate a system

Install and run

From PyPI: theprotocol-sdk carries the proxy since 0.8.0. The proxy runs as its own agent: create it in the console and keep its client_id and client_secret where the spec can name them.

pip install "theprotocol-sdk[proxy]"
tp-proxy validate proxy.yaml
tp-proxy run proxy.yaml --dry-run --once   # map and print what it would send, send nothing
tp-proxy run proxy.yaml                    # for real, as a service

A connector that needs a driver has its own extra beside proxy: proxy-mqtt, proxy-sftp, proxy-s3, proxy-quantum, and proxy-sql with proxy-postgresql, proxy-mysql or proxy-sqlserver.

A dry run maps every record and prints what it would send, sending nothing. --once stops when every source has finished; a receiver (a webhook, MQTT, TheProtocol's events) never finishes.

The connectors

From theprotocol-sdk 0.10.0

Where the proxy reads the system's records.

  • filedropRows from the files a legacy system drops in a directory (CSV, JSON, JSONL, XML).
    FieldTypeWhat it is
    pathrequiredstringThe directory the legacy system writes into.
    formatcsv · json · jsonl · xmlHow one file's rows are read: csv, json, jsonl or xml.
    id_fieldstringA row field holding the legacy system's own id. Empty: the id is the file's path and the row's position, which is stable but local to this drop.
    poll_secondsnumberHow often the folder is listed again. 0 (the default, and what the source always did): list it once, read every settled file, then finish (tp-proxy run --once ends). A positive number keeps listing it every that many seconds: a file dropped later is read too, a row appended to a read file is read from where it stopped.
    require_markerstringIf set (e.g. '.done'), a file is read only when its marker file exists beside it, and settle_seconds is not consulted.
    settle_secondsnumberA file is read only once unmodified for this long: a drop is not atomic, and a file read mid-write loses its tail irrecoverably.
    suffixstringOnly files ending in this are read (default: every file, the marker aside).
  • logtailLines a legacy system appends to a log file, resumed by offset and file identity across rotation.
    FieldTypeWhat it is
    pathrequiredstringThe log file to tail.
    continuationstringA regex matching a CONTINUATION line (an indented stack-trace frame). Such a line joins the entry above it instead of starting a record.
    from_startbooleanWith no cursor yet, read the file from its start (default) or skip what is already there and take only what arrives next.
    patternstringA regex with named groups: a record's payload becomes those fields. Empty: the payload is {"line": ...}.
    skip_unmatchedbooleanWith a pattern, drop lines it does not match instead of carrying them as {"line": ...}. Off by default: a dropped line is invisible.
  • mqttMessages an MQTT broker delivers (MQTT 3.1.1 or 5, TLS with client certificates, QoS 0 and 1), acknowledged once in the outbox and resumed by the broker's persistent session
    FieldTypeWhat it is
    client_idrequiredstringThe name the broker keeps this source's session under: stable, and used by no other client.
    hostrequiredstringThe broker's host name or address.
    topicsrequiredarrayTopic filters (+ is one level, # the rest), each a string or {filter, qos}.
    finish_after_idle_snumberWith a value, finish once subscribed and quiet this long (tp-proxy run --once); 0 listens until stopped.
    formatjson · text · bytesjson (text when it is not JSON), text (base64 when it is not UTF-8) or bytes (base64).
    id_fieldstringA field of a JSON payload ($.a.b, or a top-level key) holding the publisher's own id for the message; the record id is topic#id.
    id_propertystringMQTT 5 only: a user property holding the publisher's own id (its first value, when the name is given more than once).
    keepalive_sinteger
    passworda referencestringA credential, given as env:NAME or file:/path.
    portintegerDefault 8883 with tls, 1883 without.
    protocol3.1.1 · 5 · 53.1.1 or 5 (YAML's protocol: 5, a number, is taken as "5").
    qos0 · 1The QoS a filter given as a string is subscribed with. 2 is refused: see the source's documentation (a QoS 2 message is handed over before it is on disk).
    retainedskip · takeWhat to do with the retained messages a broker sends at subscription.
    session_expiry_sintegerMQTT 5: how long the broker keeps the session while the proxy is away (default: it does not end).
    tlsobject
    usernamestring
  • quantum_resultsHand back each finished job once: its counts, its shots and its digests, never its circuit
    FieldTypeWhat it is
    backendrequiredobject
    dirrequiredstring
    meter_dirstring
  • sftpRows from the export files a legacy system leaves on an SFTP server (CSV, JSON, JSONL, XML), each file moved or marked once all its rows are in the outbox; key-based login, the host key checked.
    FieldTypeWhat it is
    hostrequiredstringThe SFTP server.
    known_hostsrequiredstringThe server's host key, as an OpenSSH known_hosts file: its absolute path (or ~/...), or a reference to its text (file:known_hosts is read from the spec's folder). A server missing from it, or presenting another key, is refused; a key a @revoked line names is never accepted.
    pathrequiredstringThe directory on the server the legacy system writes its exports into.
    private_keyrequireda referencestringThe login key (OpenSSH or PEM; Ed25519, ECDSA or RSA), a reference (file:/path or env:NAME). Key-based login only.
    usernamerequiredstring
    after_readmove · mark · leaveOnce every row of a file is in the outbox: move it into move_to, mark it (rename it with mark_suffix), or leave it (a read-only account).
    delimiterstringCSV only: the field separator (many exports use ';').
    encodingstringThe files' text encoding (the default reads UTF-8, with or without a BOM).
    formatcsv · json · jsonl · xmlHow one file's rows are read: csv, json, jsonl or xml. A CSV row with more fields than its header keeps the extra ones as a list under _extra.
    id_fieldstringA row field holding the legacy system's own id. Empty: the id is the file's name, its instance and the row's position.
    ignore_suffixesarrayNames an upload in progress carries; such files are never read.
    mark_suffixstringWhat mark appends to a read file's name.
    max_bytesintegerA larger file is not read, and the log says so.
    move_tostringWhere read files go: a directory relative to path, or an absolute one; the folders it names are made when they do not exist.
    passphrasea referencestringThe login key's passphrase, a reference, when it has one.
    poll_secondsnumberHow often the directory is listed. 0: list it once (tp-proxy run --once, or a schedule) and finish.
    portinteger
    require_markerstringIf set (e.g. '.ok'), a file is read only once its marker file is beside it, and settle_seconds is not consulted.
    settle_secondsnumberA file is read only once its size and time have stayed the same for this long: an upload is not atomic.
    suffixstringOnly files ending in this are read (default: every file).
    timeout_secondsnumber
  • sqlpollThe rows of a table or view (PostgreSQL, MySQL, SQL Server or SQLite), read in the order of a cursor column and resumed where they stopped, read only.
    FieldTypeWhat it is
    enginerequiredpostgresql · mysql · sqlserver · sqlitepostgresql, mysql or sqlserver; sqlite reads a database file.
    tablerequiredstringThe table or view, with its schema if it needs one (sales.order_events).
    batch_sizeintegerHow many rows one query reads.
    columnsarrayThe columns a record carries (default: all of them).
    connect_argsobjectMore of the driver's connect arguments (TLS and the like). A value may be a reference, and one whose key names a password, a passphrase or a secret must be. The login itself goes in user and password.
    cursorstringThe column the rows are read in the order of (leave it out, and name a key, to read a table once in the order of its key: a migration). Its values must only grow as rows are written: the database's own sequence, or a time the writer sets as it writes (now()); a date a person types in is not a cursor (a row dated behind the place is never read). Index it with the key, in that order: each poll asks for the rows after a place in that order.
    cursor_typeid · timestampid: an integer that only grows (an identity, a sequence). timestamp: a time set as the row is written (a date or a time column), carried as the database's own text for it.
    databasestringThe database to read (all but sqlite).
    from_startbooleanWith no place kept yet: read the rows already there (default), or only the rows that arrive next.
    hoststringThe database server (all but sqlite).
    keystringThe column (or columns) that tell apart rows with the same cursor value and name a record: unique as the database compares it (a case-insensitive column counts A and a as one). Default: the cursor itself (an id is unique). A timestamp cursor needs one.
    passworda referencestringThe login's password: a reference (env:NAME or file:/path).
    pathstringsqlite: the database file's name, taken as written (not a reference, not a file: URI), opened read only; a relative one is read from the proxy's working directory.
    poll_secondsnumberHow long to wait before reading again once every row is read. 0: read once and finish (for tp-proxy run --once).
    portintegerIts port (default: 5432, 3306 or 1433).
    settle_secondsnumberEach poll reads only up to the highest cursor value it saw this long before, so a transaction that took a lower id or an earlier time and commits later is still read. Longer than the legacy system's longest writing transaction. 0 reads at once and accepts that risk.
    timeout_secondsnumberHow long connecting, and each query, may take.
    userstringA login that may only SELECT the table or view.
  • theprotocol_eventsThe registry's webhooks: each signed delivery is one record of TheProtocol's events
    FieldTypeWhat it is
    listenrequiredstring
    pathrequiredstring
    secretrequireda referencestringA credential, given as env:NAME or file:/path.
    commit_timeout_snumber
    eventsarray
    max_bytesinteger
  • webhookAn HTTP receiver: each POSTed JSON object is a record, answered 202 once it is in the outbox
    FieldTypeWhat it is
    listenrequiredstring
    pathrequiredstring
    commit_timeout_snumber
    id_fieldstring
    max_bytesinteger
    secreta referencestringA credential, given as env:NAME or file:/path.
    signature_headerstring

A ready spec for eight of the blueprints, each with a fake of the system it connects, run end to end on the sandbox before it shipped.

  • BAZAAR

    Its blueprint

    inventory, and fulfilment only on a received payment

    The system
    shop and warehouse
    Read from
    webhook, TheProtocol's events
    On TheProtocol
    attestation
    Out to the system
    fulfilment releases to the WMS, only once paid
    How it runs

    The system's side. The shop posts inventory changes, HMAC-signed with SHOP_HMAC, to /in/shop; the warehouse system takes fulfilment releases on its API.

    On TheProtocol. Each inventory change becomes a bazaar.inventory_level attestation. TheProtocol's own payment.received webhook (signed by the registry) is forwarded to the WMS with the payer, the amount and the ledger transaction: the WMS fulfils only what has been paid, and a payment that never settles never reaches it.

    What it needs. An agent with the attestor role; a webhook for payment.received registered in the console (Developer, Webhooks) pointing at the proxy's /in/theprotocol, its secret as TP_WEBHOOK_SECRET, and the shop agent's DID in the paid-release mapping's when (the owner's webhook carries the income of every agent the owner has); SHOP_HMAC, WMS_HMAC, and the WMS endpoint in the sink's url. Credentials come from the agent's starter files (.env, .env.secret) by reference; none is written in the spec.

    Try it locally. python fake/mock_wms.py --listen 127.0.0.1:8721 is a WMS; python fake/fake_shop.py http://127.0.0.1:8711/in/shop --count 4 posts four inventory changes. Then:

    tp-proxy validate proxy.yaml
    tp-proxy run proxy.yaml --dry-run            # what it would send, sending nothing (Ctrl-C to stop: a receiver never finishes, so --once does not end here)
    tp-proxy run proxy.yaml                        # for real

    Check it. The WMS's file (wms-received.jsonl) holds one release per payment received.

    Its spec (proxy.yaml)
    # BAZAAR (the marketplace blueprint): the shop's inventory changes become signed attestations on TheProtocol, and the
    # warehouse (WMS) is told to fulfil only when a buyer's payment has arrived: TheProtocol's own payment.received webhook,
    # signed by the registry, is forwarded to the WMS with the payer, the amount and the ledger transaction, and the WMS
    # matches it to its open order. A payment that never settles never reaches the WMS.
    #
    # The shop side: POST /in/shop, HMAC-signed with SHOP_HMAC (fake/fake_shop.py). TheProtocol's side: register this
    # proxy's /in/theprotocol as a webhook for payment.received in the console (Developer, Webhooks) and give its secret as
    # TP_WEBHOOK_SECRET. The WMS side: the sink's url, checking X-Signature-256 with WMS_HMAC (fake/mock_wms.py).
    version: 1
    name: bazaar-shop
    registry: https://api.theprotocol.cloud
    identity:
      agent_did: env:AGENT_DID
      client_id: env:CLIENT_ID
      client_secret: env:CLIENT_SECRET
    state: /data/bazaar-state.db
    sources:
      shop:
        type: webhook
        config: {listen: "0.0.0.0:8711", path: /in/shop, id_field: event_id, secret: env:SHOP_HMAC}
      theprotocol:
        type: theprotocol_events
        config: {listen: "0.0.0.0:8712", path: /in/theprotocol, secret: env:TP_WEBHOOK_SECRET, events: [payment.received]}
    actions:
      attest:
        type: attestation
        config: {attestation_type: bazaar.inventory_level}
        limits: {per_minute: 300}
    sinks:
      wms:
        type: webhook
        # your warehouse system's fulfilment endpoint (a URL, written here; its signing secret by reference)
        config: {url: "https://wms.example.com/api/v1/fulfilment-releases", secret: env:WMS_HMAC, timeout_s: 15}
    mappings:
      - name: inventory
        version: 1
        from: shop
        to: attest
        when: {path: $.payload.kind, equals: inventory}
        fields:
          sku: $.payload.sku
          on_hand: $.payload.on_hand
          warehouse: $.payload.warehouse
          at: $.payload.at
      - name: paid-release
        version: 1
        from: theprotocol
        to: wms
        # the owner's webhook carries the income of every agent the owner has: release goods for the shop's own (its DID)
        when: {path: $.payload.agent_did, equals: "did:theprotocol:your-shop-agent"}
        fields:
          seller_agent: $.payload.agent_did
          payer: $.payload.from_did
          amount: $.payload.amount
          currency: $.payload.currency
          ledger_tx: $.payload.ledger_txid
    retry: {first_s: 2, max_s: 300, attempts: 8}
    service: {listen: "127.0.0.1:9108"}
    
  • DYNAMO

    Its blueprint

    meter readings over MQTT

    The system
    smart meters
    Read from
    MQTT
    On TheProtocol
    attestation
    How it runs

    The system's side. Smart meters publish JSON readings (seq, kwh, voltage, at) to meters/<meter>/reading on the plant's broker (QoS 1).

    On TheProtocol. Each reading becomes a dynamo.meter_reading attestation, the meter taken from the topic; a redelivered message is kept once (the meter's seq is the id).

    What it needs. An agent with the attestor role; the broker's address, and in production its CA and a client certificate (tls). Install the proxy-mqtt extra. Credentials come from the agent's starter files (.env, .env.secret) by reference; none is written in the spec.

    Try it locally. docker run -d -p 1883:1883 eclipse-mosquitto:2 mosquitto -c /mosquitto-no-auth.conf is a broker; python fake/fake_meters.py --meters 3 --readings 4 publishes twelve readings. Then:

    tp-proxy validate proxy.yaml
    tp-proxy run proxy.yaml --dry-run            # what it would send, sending nothing (Ctrl-C to stop: a receiver never finishes, so --once does not end here)
    tp-proxy run proxy.yaml                        # for real

    Check it. tp-proxy status shows the readings sent; the attestations are on the agent's record.

    Its spec (proxy.yaml)
    # DYNAMO (the energy blueprint): smart meters publish their readings to the plant's MQTT broker; every reading becomes
    # a signed attestation on TheProtocol (the meter is the topic's second level, meters/<meter>/reading), kept exactly once
    # however often the broker redelivers it (QoS 1, a persistent session, the meter's own sequence number as the id).
    #
    # The meter side: JSON {"seq", "kwh", "voltage", "at"} on meters/<meter>/reading (fake/fake_meters.py publishes such
    # readings). Production brokers use TLS with a client certificate: uncomment tls and name the files.
    version: 1
    name: dynamo-meters
    registry: https://api.theprotocol.cloud
    identity:
      agent_did: env:AGENT_DID
      client_id: env:CLIENT_ID
      client_secret: env:CLIENT_SECRET
    state: /data/dynamo-state.db
    sources:
      meters:
        type: mqtt
        config:
          host: broker.plant.local
          port: 1883
          client_id: tp-proxy-dynamo-meters      # the broker keeps this source's session under this name
          topics: [{filter: "meters/+/reading", qos: 1}]
          id_field: $.seq                        # each meter's own reading number
          # tls: {ca_file: /etc/tp-proxy/plant-ca.crt, cert_file: /etc/tp-proxy/meters.crt, private_key: file:/etc/tp-proxy/meters.key}
    actions:
      attest:
        type: attestation
        config: {attestation_type: dynamo.meter_reading}
        limits: {per_minute: 600}
    sinks: {}
    mappings:
      - name: reading
        version: 1
        from: meters
        to: attest
        fields:
          meter: $.payload.levels.1
          reading: $.payload.data.seq
          kwh: $.payload.data.kwh
          voltage: {path: $.payload.data.voltage, default: null}
          at: $.payload.data.at
    retry: {first_s: 2, max_s: 300, attempts: 8}
    service: {listen: "127.0.0.1:9108"}
    
  • HANSA

    Its blueprint

    gate-out events to attestations, the ERP receives the chain

    The system
    terminal operating system
    Read from
    webhook
    On TheProtocol
    attestation
    Out to the system
    the ERP's API
    How it runs

    The system's side. The terminal operating system posts each gate event, HMAC-signed with GATE_HMAC, to /in/gate.

    On TheProtocol. Each gate-OUT becomes a hansa.gate_out attestation, and the ERP's API receives one signed POST per event, the record's key as its Idempotency-Key.

    What it needs. An agent with the attestor role; GATE_HMAC shared with the TOS, ERP_HMAC with the ERP; the ERP's endpoint written in the sink's url. Credentials come from the agent's starter files (.env, .env.secret) by reference; none is written in the spec.

    Try it locally. python fake/mock_erp.py --listen 127.0.0.1:8720 is an ERP that checks the signature and keeps each event once; python fake/fake_tos.py http://127.0.0.1:8711/in/gate --count 6 posts six events (two gate-INs, not attested). Then:

    tp-proxy validate proxy.yaml
    tp-proxy run proxy.yaml --dry-run            # what it would send, sending nothing (Ctrl-C to stop: a receiver never finishes, so --once does not end here)
    tp-proxy run proxy.yaml                        # for real

    Check it. The ERP's file (erp-received.jsonl) holds each gate-out once; the attestations are on the agent's record.

    Its spec (proxy.yaml)
    # HANSA (the port-logistics blueprint): the terminal operating system (TOS) posts each gate event, HMAC-signed, to the
    # proxy; every gate-OUT becomes a signed attestation on TheProtocol, and the ERP receives the chain on its API: one
    # signed POST per event, the record's key as its Idempotency-Key, so the ERP keeps each event once.
    #
    # The TOS side: POST /in/gate with X-Signature-256: sha256=<hex HMAC of the body> (fake/fake_tos.py does it). The ERP
    # side: the sink's url, checking X-Signature-256 with ERP_HMAC (fake/mock_erp.py is such an ERP).
    version: 1
    name: hansa-gate
    registry: https://api.theprotocol.cloud
    identity:
      agent_did: env:AGENT_DID
      client_id: env:CLIENT_ID
      client_secret: env:CLIENT_SECRET
    state: /data/hansa-state.db
    sources:
      tos:
        type: webhook
        config: {listen: "0.0.0.0:8711", path: /in/gate, id_field: event_id, secret: env:GATE_HMAC}
    actions:
      attest:
        type: attestation
        config: {attestation_type: hansa.gate_out}
        limits: {per_minute: 300}
    sinks:
      erp:
        type: webhook
        # your ERP's endpoint for gate events (a URL, written here; its signing secret by reference)
        config: {url: "https://erp.example.com/api/v1/gate-events", secret: env:ERP_HMAC, timeout_s: 15}
        limits: {per_minute: 300}
    mappings:
      - name: gate-out
        version: 1
        from: tos
        to: attest
        when: {path: $.payload.status, equals: OUT}
        fields:
          container: $.payload.container_id
          gate: $.payload.gate
          truck: $.payload.truck_plate
          at: $.payload.timestamp
          seal: {path: $.payload.seal, default: null}
          terminal: HAM-CTA
      - name: gate-out-to-erp
        version: 1
        from: tos
        to: erp
        when: {path: $.payload.status, equals: OUT}
        fields:
          container: $.payload.container_id
          gate: $.payload.gate
          at: $.payload.timestamp
    retry: {first_s: 2, max_s: 300, attempts: 8}
    breaker: {refusals: 5, failures: 10, cooldown_s: 30}
    service: {listen: "127.0.0.1:9108"}
    
  • KEYSTONE

    Its blueprint

    a passed test run as a milestone's evidence

    The system
    site sensor logger
    Read from
    SFTP
    On TheProtocol
    milestone evidence
    How it runs

    The system's side. The site's sensor logger exports each test run as a CSV to an SFTP server (/exports, a .done marker once complete): measurement rows, then an acceptance row (passed) or a rejection row (failed) naming the contract and its milestone.

    On TheProtocol. A passed run's acceptance row is delivered as that milestone's evidence, by the contractor's agent (milestone_evidence), so the client releases the milestone payment on a measurement. A failed run delivers nothing.

    What it needs. A contract (contracts v2) whose provider is the proxy's agent; the SFTP account's key and the server's host key (known_hosts). Install the proxy-sftp extra. Credentials come from the agent's starter files (.env, .env.secret) by reference; none is written in the spec.

    Try it locally. python fake/fake_logger.py /exports --contract <id> --milestone 0 --strength 41.5 writes a passed run (35 MPa is the limit; --strength 30 writes a failed one). Then:

    tp-proxy validate proxy.yaml
    tp-proxy run proxy.yaml --dry-run --once     # what it would send, sending nothing
    tp-proxy run proxy.yaml                        # for real

    Check it. The contract shows the milestone delivered, its deliverable carrying the run; a failed run delivers nothing; each file read is moved into done/ on the server. The source polls every poll_seconds (60): for a one-off run (--once) set poll_seconds: 0, or --once never ends; the server account needs write access to the export folder for the move (without it the file stays, is named in the log, and is not read twice).

    Its spec (proxy.yaml)
    # KEYSTONE (the construction blueprint): the site's sensor logger exports each test run as a CSV to an SFTP server; the
    # run's acceptance row (the measured result against its limit, for one contract milestone) is delivered as that
    # milestone's evidence on TheProtocol, by the contractor's agent, so the client releases the milestone payment on a
    # measurement rather than an e-mail. The measurement rows of the run stay in the export; only the acceptance row is
    # evidence, and only a passed run has one (a failed run's last row is a rejection).
    #
    # The logger side: runs-<run>.csv in /exports on the SFTP server, each with a .done marker once complete
    # (fake/fake_logger.py writes such runs). The contract and its milestone are named in the export's acceptance row.
    version: 1
    name: keystone-sensors
    registry: https://api.theprotocol.cloud
    identity:
      agent_did: env:AGENT_DID
      client_id: env:CLIENT_ID
      client_secret: env:CLIENT_SECRET
    state: /data/keystone-state.db
    sources:
      logger:
        type: sftp
        config:
          host: sftp.site.local
          port: 22
          username: tp-proxy
          private_key: file:/run/secrets/sftp_key
          known_hosts: /run/secrets/sftp_known_hosts
          path: /exports
          suffix: .csv
          require_marker: .done          # a run is read only once the logger has written its .done marker
          id_field: row_id
    actions:
      deliver:
        type: milestone_evidence
        config: {}                       # the contract and the milestone come from the acceptance row
        limits: {per_minute: 30}
    sinks: {}
    mappings:
      - name: acceptance
        version: 1
        from: logger
        to: deliver
        when: {path: $.payload.data.kind, equals: acceptance}
        fields:
          contract_id: $.payload.data.contract_id
          seq: $.payload.data.milestone
          test: $.payload.data.test
          result: $.payload.data.result
          measured: $.payload.data.value
          limit: $.payload.data.limit
          unit: $.payload.data.unit
          run: $.payload.data.run_id
          at: $.payload.data.at
    retry: {first_s: 5, max_s: 600, attempts: 8}
    service: {listen: "127.0.0.1:9108"}
    
  • LYCEUM

    Its blueprint

    awarded credentials from the student-record export

    The system
    student records
    Read from
    file drop (CSV)
    On TheProtocol
    attestation
    How it runs

    The system's side. The student-record system exports the credentials it has awarded as a CSV (award_id, a pseudonymous holder_pseudonym, credential, eqf_level, ects, awarded_on, status) into /data/sis-out.

    On TheProtocol. Each AWARDED row becomes a lyceum.credential_awarded attestation a verifier can check without asking the school. A REVOKED row is left out: revocation is a deliberate act in the console.

    What it needs. An agent with the attestor role. The export names no student by name or birth date. Credentials come from the agent's starter files (.env, .env.secret) by reference; none is written in the spec.

    Try it locally. python fake/fake_sis.py ./sis-out --count 5 writes one export (its last row REVOKED). Then:

    tp-proxy validate proxy.yaml
    tp-proxy run proxy.yaml --dry-run            # what it would send, sending nothing (Ctrl-C to stop: the folder is watched, so --once does not end here)
    tp-proxy run proxy.yaml                        # for real

    Check it. The attestations are on the agent's record, one per awarded row. A file is read once it has not changed for settle_seconds (2): a --once run started while the export is still being written passes over it, and the next run reads it. The export folder is listed again every 30 seconds (poll_seconds: 30), so a file dropped while the proxy runs is read too; without it the source lists the folder once and finishes, which is what a --once run is. For a --once try, set poll_seconds: 0 in the spec for that run.

    Its spec (proxy.yaml)
    # LYCEUM (the education blueprint): the student-record system exports the credentials it has awarded as a CSV; each
    # awarded credential becomes a signed attestation on TheProtocol that a verifier can check later without asking the
    # school. A withdrawn award in the same export (status REVOKED) is left out: revocation is a deliberate act in the
    # console, never a side effect of an export.
    #
    # The student-record side: credentials-<date>.csv with the header line the fake writes (fake/fake_sis.py) into
    # /data/sis-out. The holder is named by a pseudonymous id, never by name or birth date.
    version: 1
    name: lyceum-credentials
    registry: https://api.theprotocol.cloud
    identity:
      agent_did: env:AGENT_DID
      client_id: env:CLIENT_ID
      client_secret: env:CLIENT_SECRET
    state: /data/lyceum-state.db
    sources:
      sis:
        type: filedrop
        config: {path: /data/sis-out, suffix: .csv, id_field: award_id, settle_seconds: 2, poll_seconds: 30}
    actions:
      issue:
        type: attestation
        config: {attestation_type: lyceum.credential_awarded}
        limits: {per_minute: 120}
    sinks: {}
    mappings:
      - name: credential
        version: 1
        from: sis
        to: issue
        when: {path: $.payload.data.status, equals: AWARDED}
        fields:
          award: $.payload.data.award_id
          holder: $.payload.data.holder_pseudonym
          credential: $.payload.data.credential
          level: $.payload.data.eqf_level
          ects: $.payload.data.ects
          awarded_on: $.payload.data.awarded_on
          institution: LYCEUM-DEMO
    retry: {first_s: 2, max_s: 300, attempts: 8}
    service: {listen: "127.0.0.1:9108"}
    
  • MERIDIAN

    Its blueprint

    order intents in, signed evidence to the ERP folder

    The system
    order management
    Read from
    file drop (JSON)
    On TheProtocol
    attestation
    Out to the system
    signed evidence to the ERP folder
    How it runs

    The system's side. The order-management system exports its order intents as JSON arrays into a folder (/data/oms-out).

    On TheProtocol. Each RELEASED intent becomes a meridian.order_intent attestation; the ERP's import folder (/data/erp-in) receives the hash-chained evidence of every intent, its manifest signed by the proxy's key.

    What it needs. An agent holding the attestor role (the attestation's reputation.signal), and an evidence signing key (tp-proxy keygen /run/secrets/evidence.pem). Credentials come from the agent's starter files (.env, .env.secret) by reference; none is written in the spec.

    Try it locally. python fake/fake_oms.py ./oms-out --batch 1 --count 5 writes one export (one HELD intent among them, not attested). Then:

    tp-proxy validate proxy.yaml
    tp-proxy run proxy.yaml --dry-run            # what it would send, sending nothing (Ctrl-C to stop: the folder is watched, so --once does not end here)
    tp-proxy run proxy.yaml                        # for real

    Check it. tp-proxy verify /data/erp-in --key <public key> checks the ERP's copy offline (tp-proxy keygen printed the public key); a changed line or another signer fails it with exit code 1 and names the line. A file is read once it has not changed for settle_seconds (2): a --once run started while the export is still being written passes over it, and the next run reads it. The export folder is listed again every 30 seconds (poll_seconds: 30), so a file dropped while the proxy runs is read too; without it the source lists the folder once and finishes, which is what a --once run is. For a --once try, set poll_seconds: 0 in the spec for that run.

    Its spec (proxy.yaml)
    # MERIDIAN (the industrial-procurement blueprint): the order-management system drops its order intents as JSON files;
    # each intent becomes a signed attestation on TheProtocol, and the ERP's import folder receives the hash-chained,
    # signed evidence of every intent the proxy carried (checked offline with tp-proxy verify).
    #
    # The OMS side: an export job writes intents-<batch>.json (a JSON array) into /data/oms-out; the fake in fake/ does
    # the same. The ERP side reads /data/erp-in. Credentials come from the agent's starter files, never this file.
    version: 1
    name: meridian-oms
    registry: https://api.theprotocol.cloud
    identity:
      agent_did: env:AGENT_DID
      client_id: env:CLIENT_ID
      client_secret: env:CLIENT_SECRET
    state: /data/meridian-state.db
    sources:
      oms:
        type: filedrop
        # settle_seconds: an export is read only once the file has stopped growing
        config: {path: /data/oms-out, format: json, suffix: .json, id_field: intent_id, settle_seconds: 2, poll_seconds: 30}
    actions:
      attest:
        type: attestation
        config: {attestation_type: meridian.order_intent}
        limits: {per_minute: 120}
    sinks:
      erp:
        type: evidence
        config: {dir: /data/erp-in, signing_key: file:/run/secrets/evidence.pem}
    mappings:
      - name: intent
        version: 1
        from: oms
        to: attest
        when: {path: $.payload.data.status, equals: RELEASED}
        fields:
          intent: $.payload.data.intent_id
          order: $.payload.data.order_no
          buyer: $.payload.data.buyer
          sku: $.payload.data.sku
          quantity: $.payload.data.qty
          released_at: $.payload.data.released_at
      - name: intent-evidence
        version: 1
        from: oms
        to: erp
        when: {path: $.payload.data.status, equals: RELEASED}
        fields:
          intent: $.payload.data.intent_id
          order: $.payload.data.order_no
          sku: $.payload.data.sku
          quantity: $.payload.data.qty
    retry: {first_s: 2, max_s: 300, attempts: 8}
    service: {listen: "127.0.0.1:9108"}
    
  • SCRIVENER

    Its blueprint

    matter events in, the DMS's evidence out

    The system
    practice management
    Read from
    webhook
    On TheProtocol
    attestation
    Out to the system
    signed evidence to the DMS
    How it runs

    The system's side. The practice-management system posts each matter event (opened, a filing with the filed document's SHA-256, a deadline met, closed), HMAC-signed with PMS_HMAC, to /in/matters.

    On TheProtocol. Each event becomes a scrivener.matter_event attestation, and the document-management system's import folder receives the signed, hash-chained history of the matter: a court or an auditor checks it offline.

    What it needs. An agent with the attestor role; an evidence signing key; PMS_HMAC. To mirror the evidence to an SFTP server, set the evidence sink's mirror (the proxy-sftp extra). Credentials come from the agent's starter files (.env, .env.secret) by reference; none is written in the spec.

    Try it locally. python fake/fake_pms.py http://127.0.0.1:8711/in/matters --matter M-2026-0042 posts one matter's four events. Then:

    tp-proxy validate proxy.yaml
    tp-proxy run proxy.yaml --dry-run            # what it would send, sending nothing (Ctrl-C to stop: a receiver never finishes, so --once does not end here)
    tp-proxy run proxy.yaml                        # for real

    Check it. tp-proxy verify /data/dms-import --key <public key> names any changed, removed or moved line.

    Its spec (proxy.yaml)
    # SCRIVENER (the legal-services blueprint): the practice-management system posts each matter event (opened, a filing
    # made, a deadline met, closed) to the proxy; every event becomes a signed attestation on TheProtocol, and the document
    # management system's import folder receives the hash-chained, signed evidence of the matter's history, checked offline
    # with tp-proxy verify (a court or an auditor needs no network and no trust in the firm's own systems).
    #
    # The practice-management side: POST /in/matters, HMAC-signed with PMS_HMAC (fake/fake_pms.py). The DMS side reads
    # /data/dms-import; mirror it to an SFTP server with the evidence sink's mirror setting (the proxy-sftp extra).
    version: 1
    name: scrivener-matters
    registry: https://api.theprotocol.cloud
    identity:
      agent_did: env:AGENT_DID
      client_id: env:CLIENT_ID
      client_secret: env:CLIENT_SECRET
    state: /data/scrivener-state.db
    sources:
      pms:
        type: webhook
        config: {listen: "0.0.0.0:8711", path: /in/matters, id_field: event_id, secret: env:PMS_HMAC}
    actions:
      attest:
        type: attestation
        config: {attestation_type: scrivener.matter_event}
        limits: {per_minute: 120}
    sinks:
      dms:
        type: evidence
        config: {dir: /data/dms-import, signing_key: file:/run/secrets/evidence.pem}
    mappings:
      - name: matter-event
        version: 1
        from: pms
        to: attest
        fields:
          matter: $.payload.matter_id
          event: $.payload.type
          document: {path: $.payload.document_sha256, default: null}
          by: $.payload.fee_earner
          at: $.payload.at
      - name: matter-evidence
        version: 1
        from: pms
        to: dms
        fields:
          matter: $.payload.matter_id
          event: $.payload.type
          document: {path: $.payload.document_sha256, default: null}
          at: $.payload.at
    retry: {first_s: 2, max_s: 300, attempts: 8}
    service: {listen: "127.0.0.1:9108"}
    
  • TRUNK

    Its blueprint

    per-event settlement of interconnect records

    The system
    mediation
    Read from
    SQL poll
    On TheProtocol
    payment under a mandate
    Out to the system
    the finance journal
    How it runs

    The system's side. The mediation system rates each interconnect event into its settlement table (settlement_events, cursor event_seq): the partner carrier's agent, the minutes and the charge.

    On TheProtocol. Each RATED event is settled as it happens, a payment to the partner's agent under the operator's Cockpit mandate, never above the action's max_amount or per_day, nor the mandate's limits; finance gets a CSV journal (or ISO 20022 camt.053) of what settled, booked from the registry's own payment.settled notice (each line names its settlement), so a payment the proxy refused or the registry held back has no line. A DISPUTED event is neither paid nor booked.

    What it needs. A Cockpit mandate (a2a.pay) issued by the operator's owner, named in mandate:; a webhook in the console (Developer, Webhooks) for payment.settled to this proxy's receiver (/in/theprotocol, its secret as TP_WEBHOOK_SECRET), and the proxy agent's DID in the book-settled mapping's when (the owner's webhook carries every payment of the owner's agents); the agent's IRONKEY spend cap at or under it; the payee agents bonded where the registry enforces the bond. SQLite needs no extra; PostgreSQL, MySQL and SQL Server take the proxy-sql extras. Credentials come from the agent's starter files (.env, .env.secret) by reference; none is written in the spec.

    Try it locally. python fake/fake_mediation.py ./mediation.db --partner-did <the partner's agent> --count 5 rates five events (one DISPUTED). Then:

    tp-proxy validate proxy.yaml
    tp-proxy run proxy.yaml --dry-run            # what it would send, sending nothing (Ctrl-C to stop: a receiver never finishes, so --once does not end here)
    tp-proxy run proxy.yaml                        # for real

    Check it. Each payment's settlement is on both agents' records; the journal holds one debit per settled payment, its text the settlement's transaction id; a refused one is in tp-proxy dead and not in the journal; the ledgers' supply delta stays 0.

    Its spec (proxy.yaml)
    # TRUNK (the telecom blueprint): the mediation system rates each interconnect event and writes it to its settlement
    # table; every rated event is settled on TheProtocol as it happens, a payment to the partner carrier's agent under the
    # operator's Cockpit mandate (nothing is paid without one, and never above the action's max_amount or the mandate's
    # limits), and finance gets a CSV journal (or camt.053, for systems that read account statements) of what SETTLED: each
    # line comes from TheProtocol's own payment.settled notice (the registry's webhook) and names its settlement, so a
    # payment refused or held back (over max_amount, outside the mandate) has no line. An event the mediation system
    # disputes (status DISPUTED) is neither paid nor booked.
    #
    # The mediation side: the table settlement_events, its cursor column event_seq (fake/fake_mediation.py writes such
    # rows into an SQLite file; PostgreSQL, MySQL and SQL Server work the same with the proxy-sql extras). The mandate: the
    # operator's owner issues it in the console (Cockpit, a2a.pay) and names it here; the agent's IRONKEY spend cap must
    # sit at or under it, or the payment action does not start.
    version: 1
    name: trunk-settlement
    registry: https://api.theprotocol.cloud
    identity:
      agent_did: env:AGENT_DID
      client_id: env:CLIENT_ID
      client_secret: env:CLIENT_SECRET
    state: /data/trunk-state.db
    sources:
      mediation:
        type: sqlpoll
        config: {engine: sqlite, path: /data/mediation.db, table: settlement_events, cursor: event_seq, poll_seconds: 5}
      theprotocol:
        # the registry's webhook (console: Developer, Webhooks; event payment.settled) for the journal: it books what
        # settled, never what was only rated (a payment refused or held back has no line)
        type: theprotocol_events
        config: {listen: "0.0.0.0:8712", path: /in/theprotocol, secret: env:TP_WEBHOOK_SECRET, events: [payment.settled]}
    actions:
      settle:
        type: payment
        # your mandate's id (lm_...), from the console; max_amount: the most one event may settle
        config: {mandate: lm_yourmandate, max_amount: 25, per_day: 2000}
        limits: {per_minute: 120}
    sinks:
      finance:
        type: journal
        config: {dir: /data/finance-journal, format: csv, account: {id: TRUNK-INTERCONNECT, currency: ECU}}
    mappings:
      - name: settle-event
        version: 1
        from: mediation
        to: settle
        when: {path: $.payload.status, equals: RATED}
        fields:
          payee: $.payload.partner_agent_did
          amount: $.payload.charge
          event: $.payload.event_id
          partner: $.payload.partner_code
          minutes: $.payload.minutes
      - name: book-settled
        version: 1
        from: theprotocol
        to: finance
        # the owner's webhook carries every payment of the owner's agents: book this proxy's agent's own (its DID here)
        when: {path: $.payload.caller_did, equals: "did:theprotocol:your-proxy-agent"}
        fields:
          amount: $.payload.amount
          direction: debit
          currency: $.payload.currency
          counterparty: $.payload.target_agent_did
          text: $.payload.settlement_tx_id
    retry: {first_s: 5, max_s: 600, attempts: 8}
    breaker: {refusals: 3, failures: 10, cooldown_s: 60}
    service: {listen: "127.0.0.1:9108"}
    

The proxy's chapter in the docs