Skip to content

feat: egress tracking commands and a shared journal and reporter - #92

Open
alanshaw wants to merge 4 commits into
mainfrom
ash/feat/egress-tracking
Open

alanshaw wants to merge 4 commits into
mainfrom
ash/feat/egress-tracking

Conversation

@alanshaw

@alanshaw alanshaw commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

The first step of the Ingot egress tracking RFC: the commands and the shared machinery that Ingot, and later Piri, will use to report what they serve to the egress tracking service.

  • Commands. Adds /content/serve, which a gateway invokes on itself for each read it serves. Its receipt carries two counts: the stored bytes of the blob that were read, and the bytes sent to the client, which differ for an encrypted blob. The tracking commands drop the /space prefix left over from UCAN 0.9 and become /egress/track and /egress/consolidate; the second is new here.
  • Journal. A new egress package appends each receipt, with its invocation and proofs, to a batch on local disk. A batch is sealed at 100 MiB or after an hour, and is a CAR file in which each delegation appears once.
  • Reporting. Each sealed batch is submitted to the tracker, and removed once the tracker's consolidation receipt has arrived and its total has been compared with the batch's own.
  • Serving batches. An HTTP handler serves a sealed batch by CID for the tracker to fetch.

This is a breaking change for the delegator and smelt, which import the tracking commands from their old package path.

A batch the tracker fails to consolidate is submitted again on every pass, with no limit, so the tracker must count a batch once however many times it is submitted.

refs https://app.shortcut.com/filone/story/108/trial-egress-cap-works-in-forge-regions

🤖 Generated with Claude Code

alanshaw and others added 2 commits November 18, 2026 15:38
The tracking commands lose the /space prefix of their UCAN 0.9 names and
move from commands/space/egress to commands/egress. /egress/consolidate
joins /egress/track there.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 18, 2026 15:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@alanshaw alanshaw left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the hand-written code in full (commands, journal, batch, reporter, handler) and ran go test -race ./egress/... ./commands/... and go vet on 085db29; all pass. The generated codecs I only skimmed.

The batch format, the recovery of a batch left open and the track/settle split all read correctly to me. Six comments inline. The two I would fix before Ingot depends on this are the {cid} placeholder, which goes out percent-encoded, and the journal staying broken after one failed write. The rest are ways a batch can get stuck or be submitted twice.

Comment thread egress/reporter.go Resolved
Comment thread egress/journal.go Resolved
Comment thread egress/reporter.go Resolved
Comment thread egress/reporter.go Resolved
Comment thread egress/reporter.go Resolved
Comment thread egress/journal.go Resolved
alanshaw and others added 2 commits November 18, 2026 16:01
- Send the batch endpoint as a string template, so its {cid} placeholder
  is not percent-encoded, and resolve it with TrackArguments.BatchURL.
- Drop the open batch after a failed write and recover it on the next
  append, so one failure does not stop the journal until a restart.
- Seal before a record that would take a batch past its size limit.
- Remove a consolidated batch that cannot be measured, reporting why.
- Track a batch again when its track record cannot be read, sync the
  record before renaming it, and sweep records whose batch is gone.
- Say that the tracker must count a batch CID once.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Read is the stored bytes of the range that were read, in the unit of the
invocation's range. Sent is the bytes that went to the client from them,
which is what the default measure sums. The two differ for an encrypted
blob, where a small ranged read takes whole ciphertext chunks.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

@alanshaw alanshaw left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 43aed19: read both new commits and re-ran go test -race on egress and the two command packages, plus go vet. All pass, and CI is green on all three platforms.

All six comments are resolved in the code:

  • The endpoint is a string on the wire, and ResolveEndpoint gives the tracker and the reporter the same check.
  • A failed write, flush, seal or rename leaves the file for recovery, and O_EXCL means a batch on disk cannot be truncated. I traced each failure path and found none that leaves the hash and the file out of step.
  • A batch that cannot be measured settles with CountErr.
  • The repeat-submission rule is stated on Track and Reporter, and the RFC now says the tracker answers a repeat with the consolidation it already has, which also keeps the total comparison meaningful after a re-track.
  • An unreadable track record is dropped and the batch tracked again; the sweep clears orphans.
  • The size limit is checked before the write.

The read and sent counts on /content/serve match the RFC text, including the two checks MeasureContent shares with the tracker.

One small follow-up inline, which need not hold up the merge.

Comment thread egress/journal.go
func (j *Journal) tick() error {
j.mu.Lock()
defer j.mu.Unlock()
if j.closed || j.file == nil {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking. A batch set aside by a failed write is recovered only by the next Append or a restart. With j.file nil, tick returns here and Seal reports nothing to seal, so on a quiet appliance the records that reached the disk wait until the next read arrives, however long that is. Recovering from tick when the last write failed (a flag set in abandon would avoid a stat every second) would keep the hourly bound.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants