Metsuke Details

This page describes what the agent sends, which key signs, and every config setting there is. The four steps to get running are on the quickstart, and nothing here is needed to follow them.

Your application code on chain

The rewards program emails you an application code when you apply. That code has to be in your pool registration transaction's metadata, under label 674:

{"674": {"musashinet_incentives_application_code": "YOUR-CODE"}}

Only your cold key can sign a pool registration, so the code on chain matching the one you were emailed is what shows the application came from you. Until they match, this server refuses your submissions regardless of what key signs them. This is program registration rather than agent setup, so it is normally done before you reach the quickstart.

Keep it on every registration you submit afterwards. Changing your relays, pledge, margin or reward account submits a new registration certificate, and one that carries no metadata is read as a registration with no code. Attaching the same code again is what keeps your pool on the list; the code itself does not change unless the program says so.

What leaves your machine

One submission is a plain JSON header, then your scrapes zstd compressed, one JSON object per line, with a detached Ed25519 signature over the whole byte sequence. The header rides in a zstd skippable frame, so zstd -d on a submission hands back the lines and nothing else. This is an example of the whole thing: two scrapes, the first cut down to two metrics where a real line carries every metric your node exposes, the second a scrape that failed.

{
  "agent_id": "relay-1",
  "agent_version": "0.2.1",
  "counter": 42,
  "pool_id": "pool1ewf4s55a73efcvjx52sr8jucyx4ml5tdujygqpvsf27yza5rpqn",
  "schema_version": 1,
  "timestamp": "2026-05-28T20:26:40Z"
}

{"scraped_at":"2026-05-28T20:26:40Z","clock_offset_ms":-3,"failure":null,"metrics":[{"name":"cardano_node_metrics_blockNum_int","labels":{},"value":12318442,"declared_type":"gauge"},{"name":"cardano_node_metrics_tipBlock","labels":{"hash":"0e2b4b1a0e2b4b1a0e2b4b1a0e2b4b1a0e2b4b1a0e2b4b1a0e2b4b1a0e2b4b1a"},"value":1,"declared_type":"info"}],"metsuke":{"pool_id":"pool1ewf4s55a73efcvjx52sr8jucyx4ml5tdujygqpvsf27yza5rpqn","agent_id":"relay-1"}}
{"scraped_at":"2026-05-28T20:31:40Z","clock_offset_ms":null,"failure":{"reason":"unreachable","detail":"the endpoint did not answer: connection refused"},"metrics":[],"metsuke":{"pool_id":"pool1ewf4s55a73efcvjx52sr8jucyx4ml5tdujygqpvsf27yza5rpqn","agent_id":"relay-1"}}

There is one line per scrape, and its metrics are every metric your node exposes whose value JSON can hold. Those are the same metric lines curl prints against the metrics endpoint. Each entry carries the metric's name, its labels and its value as your node stated them, plus the declared_type its # TYPE line gave if it had one. The agent reads nothing else on your machine. It contributes two facts of its own: scraped_at, the time it scraped, and clock_offset_ms, the offset its own NTP query measured.

A scrape that failed is itself a signal, so the submission is sent either way: no metrics, and failure names what stopped it. Its reason is one of unreachable, refused, too_large, unreadable, and its detail is the message the agent had: the port, the status, or the size limit it was configured with.

Every line names the pool and the machine that wrote it under metsuke, as pool_id and agent_id, which is the name you configure. One line read out of the archive on its own still says where it came from.

The header carries those same two, and four more: agent_version, the build that scraped; counter, which submission of yours this is; timestamp, when the submission was sealed; and schema_version, which shape its lines are.

Trace lines travel as their own submissions: the same header with schema_version 2, and then the lines your agent config selects, one per line, each containing the object your node wrote plus that same metsuke JSON key.

The signature travels beside the body in two headers: x-metsuke-vkey and x-metsuke-signature. Anything between your agent and this server has to pass both through unchanged, or the signature will not verify.

Which pool a submission is from is settled from those rather than taken on trust. A cold key hashes to your pool id, so this server derives it from the verification key in the first header. A Leios key hashes to nothing, so the submission names its pool in a third header, x-metsuke-pool, and this server believes it only where its key roster lists that key for that pool.

Which key signs

Use your Leios key. Your node already forges and votes with it, so it is already on the machine the agent runs on and nothing new has to be put there. Signing telemetry is a poor reason to move a cold key onto a reporting host, which is what using a cold key here comes to.

The same goes if you report from your relays as well as your block producer. Every host that reports needs one of your keys on it, and a relay is the machine taking connections from strangers, so a cold key there is the worse version of the same mistake. Which host a submission came from is in it either way, under agent_id.

A cold key does work and this server accepts it, so it is an option if you have a reason to prefer it. If you use one it has to hash to the pool_id in your config, or the agent refuses to start.

The agent reads the bls.skey or pool.skey you already have. The path the unit loads it from names neither, so either one installs there and the unit runs unedited.

The node's metrics endpoint

The published MusashiNet configuration already serves this on loopback, and the config the quickstart hands you already points there, so this section is a check and a repair kit rather than a step:

curl -s http://127.0.0.1:12798/metrics

If you receive metric lines back, there is nothing to do here. If nothing answers, your node's tracing has diverged from the published configuration, and this is the backend to add to its TraceOptions, bound to loopback so it is not reachable from anywhere else:

{
  "TraceOptions": {
    "": { "backends": ["Stdout MachineFormat", "PrometheusSimple 127.0.0.1 12798"] }
  }
}

If your node configuration has no TraceOptions at all, paste that as it stands. If it has one, merge into it rather than pasting over it. The "" key is your node's root entry, so keep every other key it has and add to its backends list.

Each backend replaces one of its own kind rather than sitting alongside it. If your root already names a PrometheusSimple or a Stdout backend, replace that one instead of keeping both. Where this is wrong, metrics still work, the trace configuration below looks applied, and not one trace line is ever collected.

Stdout MachineFormat is what both trace sources, journald and pipe, read, so it has to be in that list for either of them. Metrics alone do not need it.

Restart the node and run the check again.

What the node has to emit for trace lines

The metrics endpoint carries no per-event timestamps, so it cannot answer when an announcement arrived, when a block body and its closure were received, or when a quorum was reached. Those live in the node's trace stream, and the agent ships every field of the lines you select from it. It reads one field to decide whether to ship a line, and computes nothing from any of them.

The published MusashiNet configuration already emits every one of them, so again there is nothing to do unless your node's tracing has diverged from it. These are the namespaces, as keys to merge into your TraceOptions:

{
  "TraceOptions": {
    "Consensus.LeiosKernel": { "severity": "Info", "maxFrequency": 0 },
    "Consensus.LeiosPeer": { "severity": "Info", "maxFrequency": 0 },
    "Forge.Loop.AdoptedBlock": { "severity": "Info", "maxFrequency": 0 },
    "ChainDB.AddBlockEvent.AddedToCurrentChain": { "severity": "Info", "maxFrequency": 0 }
  }
}

Check before you add them. Your configuration may already set some of these, and merging replaces a key's whole entry, so these take the place of whatever severity or rate limit you had on those namespaces. It may also set rate limits on namespaces this snippet does not name; pasting over the object rather than merging into it would drop those too.

Each namespace carries its own severity, so your node's root threshold is left as you have it and nothing here depends on where you set it. There is no "" key in this snippet, so it cannot disturb the root entry the backend above touched. maxFrequency: 0 is not a typo; it means no rate limit, and leaving it out silently caps the stream.

Restart the node if you changed anything. What it emits is the same regardless of trace source; how the agent gets it is the choice below.

The pipe

In the pipe setup the node runs as cardano-node run ... | /usr/local/bin/metsuke --config /etc/metsuke/config.toml. The agent reads the node's stdout on its own stdin, adds no group of its own, and writes every line through to its own stdout unchanged, so whatever read the node's output before still reads it.

What this setup costs is not nothing. The agent is a process of your node's unit rather than one of its own, and two things follow from that. It runs as your node's user under your node's sandbox, so it keeps none of the confinement metsuke.service sets for it and inherits every group your node's unit already had. And the drop-in loads your signing key as a credential of that unit, so $CREDENTIALS_DIRECTORY/signing-key is readable by every process the unit runs, cardano-node among them.

What the credential costs depends on the host. On a block producer signing with its Leios key it costs nothing new: your node already forges and votes with that key, so a credential it can read is a key it already holds. On a relay it is a real difference. The key is on that host only because the agent needs it, and under the journald setup only the agent's own unit can read it, so a cardano-node compromise there reaches no key of yours. Under the drop-in it reaches this one.

So adding no group is a real saving over the journald setup, but it is not the same as costing nothing, and an operator picking the pipe because it grants no group is picking the less confined of the two. Use your Leios key either way. A cold key works and this server accepts it, but a cold key cannot be replaced: your pool id is its hash, so a compromised one cannot be rotated; it is the end of that pool and the start of an unrelated one your delegators have to find.

The agent runs downstream of the node, so it needs no unit of its own. That makes this a change to the node's unit, and systemd has no pipelines, so the command is wrapped in a shell. This drop-in is that change, and the empty ExecStart= is what clears the command the node's unit already had: node-pipe.conf, with config.pipe.toml beside it.

Read your node's unit before you use it. The shell is what systemd supervises once the drop-in is in place, not your node, so three directives change meaning: Type=notify times the start out and systemd kills your node, WatchdogSec= kills it every interval, and ExecReload= signals the shell instead of your node. Two more things concern the command itself: it sits inside single quotes, and systemd expands % in it. The drop-in's own header lists what to do about each. None of this applies to the journald setup, which leaves your node's unit alone.

A node whose output goes to a file rather than the journal, by StandardOutput=append: with logrotate over it, needs nothing different. The drop-in replaces the command and leaves that directive alone, and the agent writes every line it reads straight back to its own stdout, so the same file receives the same content. Rotation works exactly as it did before the agent was there: systemd still opens the file and hands the descriptor down. Redirecting the pipeline yourself is the case where that is not true.

There is one thing to know if you redirect stdout to a file. StandardError= follows StandardOutput= unless you set it, so stderr goes to that file as well, and journald receives neither stream: journalctl -u <your-node> will show the unit starting and stopping and nothing from the node or the agent. Everything is in your file, where the agent's prose is easy to pick out from the node's JSON. If you would rather keep the file to the node alone, the pipeline is a shell command and takes a redirect of its own: … | metsuke --config … 2>>/var/log/metsuke.log.

Where your node's command lives somewhere other than a unit, the pipeline goes there instead: a wrapper script, a runit or s6 run file, a supervisord program, or a container entrypoint. In a shell or a terminal it works as it stands. Wrapped around the command that starts a container, it works only while that command stays attached: a detached one prints an id rather than the node's output. So in a container the pipeline goes inside, as the entrypoint.

If you redirect that pipeline to a file yourself, rotation is yours to get right. Writing cardano-node run ... | /usr/local/bin/metsuke --config /etc/metsuke/config.toml >> /var/log/node.log puts the descriptor on that file in the agent's hands rather than your node's, and the agent installs no signal handler, so it never reopens. logrotate's default create renames the file and leaves the agent writing to the renamed one: the new file stays empty and the old one grows. Use copytruncate, which keeps the same file and so needs no reopen. A postrotate that signals your node to reopen does nothing here either, because your node is no longer what holds that descriptor.

And nothing restarts either half. Under a unit the drop-in's Restart=always does that; here it is your supervisor's job, and it has to treat the pipeline as one thing. If the agent exits, your node's next write to the pipe fails, and what your node does about that is your node's to decide.

The agent is a static binary, which is what makes a container setup cheap: it needs no image of its own and nothing in the node's image has to match it. Mount it in beside the node, with its config and its key. Nothing has to be installed on the host first, so a container setup can skip the install and configure steps of the quickstart and keep all three files in the directory the node already runs from. Against the MusashiNet node image:

# Run this from the directory holding the node's data and these three files
docker run -d --name leios-relay --init \
  -p 3010:3010 \
  -v "$PWD:/data" \
  -v "$PWD/metsuke:/usr/local/bin/metsuke:ro" \
  -v "$PWD/config.pipe.toml:/etc/metsuke/config.toml:ro" \
  -v "$PWD/bls.skey:/etc/metsuke/signing-key:ro" \
  -v metsuke-spool:/var/lib/metsuke \
  --entrypoint /bin/sh \
  ghcr.io/input-output-hk/ouroboros-leios/cardano-node-testnet:<tag> \
  -c '/app/run-node.sh | /usr/local/bin/metsuke \
        --config /etc/metsuke/config.toml \
        --signing-key /etc/metsuke/signing-key'

<tag> is the node image's version, which the testnet getting started guide names. This page prints no tag of its own: the node and this agent are released apart, so one written here would be a second place for that version to live and the wrong one to trust when they disagree.

Nothing of the node's is mounted in. That image ships the pinned MusashiNet configuration already, and it is one this agent reads unchanged: it serves Prometheus on loopback and writes its trace lines to stdout in the machine format, which are the two backends the metrics endpoint and the trace lines above ask for. Mount your own over /app/config only if you have changed the node's tracing.

/app/run-node.sh is that image's own launcher, which is what it runs when you do not override the entrypoint. On any other image, docker inspect prints the command to put there instead. Every path left of a colon is on this host and every path right of one is inside the container, which is why the agent's three are mounted at the same places its unit would have put them. Write the host side as $PWD/... rather than ./...: docker reads a source that does not start with a slash as the name of a volume, so a relative one either fails or quietly creates a volume instead of mounting your file. The port and the two mounts under it are the node's. The three below those are the agent's, all read-only because it only reads them, and the spool volume is what survives a restart. The tag is the one thing left to fill in, and the testnet documentation is what pins it.

Keep --init. A shell running a pipeline as PID 1 does not pass SIGTERM on to what it started, so without an init the runtime waits out the stop timeout and kills the node instead of letting it close its databases. That is a slow replay on the next start, and on a block producer it is the expensive kind of mistake.

The spool volume is what makes a restart cheap rather than lossy, and metrics_url needs no change: the agent is in the node's container network namespace, so the node's loopback is its own. That holds however the node binds its endpoint, since a wildcard bind includes loopback.

Write that metrics_url endpoint as an IP literal regardless of how the node binds it. Nothing resolves that host, so localhost is refused and it has to parse as a loopback address, meaning 127.0.0.1 or [::1].

The pipeline can also run on the host instead of inside the container, as docker logs -f --tail 0 <container> | /usr/local/bin/metsuke --config /etc/metsuke/config.toml. That decouples the agent from the node's lifecycle and leaves the image alone, but it costs more than it appears to. Reading the docker socket is root-equivalent on that host, whereas the entrypoint approach above needs no privilege at all. It also needs the node's metrics endpoint bound to a wildcard, and the MusashiNet image bakes in PrometheusSimple suffix 127.0.0.1 12798, so you would need to override its configuration to get one. The EXPOSE 12798 in that image does not help: a published port forwards to the container's interface, and nothing is listening there. And without --tail 0 a restart re-reads the whole log and sends it again.

None of that applies to the entrypoint above, which shares the node's container network namespace and so reaches the loopback the image already binds. The image ships Stdout MachineFormat in its backends either way, so the trace lines themselves need nothing.

The NixOS module only renders the journald setup, not the pipe one.

The journal

In the journald setup the agent reads the node's journal under its own unit, so it needs a node that runs as a systemd unit and logs to the journal. That costs it membership of the systemd-journal group, which reads every unit's journal on the host and is the one privilege it holds beyond scraping loopback. Two directives are the whole difference from the quickstart's unit, and journalctl needs both: metsuke-journald.service, with config.journald.toml beside it.

That config is the one place the agent cannot guess anything: neither your node's unit name nor where journalctl lives on your distribution.

Every setting

The quickstart ships the values that have to be set and leaves the rest at their defaults. config.example.toml is the annotated form of the same file, with every default written out.

On NixOS

Add github:input-output-hk/metsuke as a flake input and import its nixosModules.metsuke. That writes the configuration and the unit for you, including the two directives the journal source needs.

Every setting in config.example.toml is an option under services.metsuke.settings, named the same, and the three required ones have to be set. The exception is signing_key, which is not an option: the module hands the key over as a systemd credential, and you point signingKeyFile at the path instead.

services.metsuke = {
  enable = true;
  signingKeyFile = "/run/secrets/metsuke-signing-key";
  settings = {
    pool_id = "pool1…";
    metrics_url = "http://127.0.0.1:12798/metrics";
    upload_url = "https://metsuke-leios.play.dev.cardano.org/v1/submit";
    log.journal_unit = "cardano-node.service";
  };
};

Every option, with its type and its default, is in the options reference.

Where the builds come from

The builds this server offers are also attached to the releases in the repository, with a checksum file each. Check whichever copy you download against the .sha256 beside it.