Skip to content

aitools: throttle the native Claude Code hint on its own shorter clock - #7692

Merged
anton-107 merged 2 commits into
mainfrom
aitools/claude-code-native-hint-throttle
Nov 17, 2026
Merged

anton-107 merged 2 commits into
mainfrom
aitools/claude-code-native-hint-throttle

Conversation

@anton-107

@anton-107 anton-107 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Changes

The native <claude-code-hint> line now has its own throttle (a separate cache bucket, 10-minute TTL), keyed by the hint's throttle key from #7586 and written only when the line actually prints. The text hint keeps its one-hour marker, and other agents' durable claim (claimHintMarker) now goes through the same text throttle. The native throttle applies only to Claude Code, the one agent with a native line. All throttles are still checked before the plugin/skills filesystem probe.

The CLAUDE_CODE_NATIVE hint-shown event stays on the text hint's hourly clock. It is logged only for the first native line printed since the text line last printed, so a native-only repeat logs no event.

Why

This addresses simonfaltum's review comment on hint.go in #7585. The single shared marker was written whenever the text hint printed. That included commands where the native line was withheld (no CLAUDE_CODE_CHILD_SESSION, CI, a terminal attached), and commands where Claude Code strips the tag without prompting (subagents, hooks). After that, the main loop got neither line for an hour.

The CLI can't tell a subagent from the main loop, because both carry the same session id and child-session marker. So a separate key alone doesn't fix the subagent case; the shorter TTL is what bounds it. Claude Code shows the dialog at most once per session, so repeating the line is cheap: a raw tag reaches a 2>&1 | jq reader at most once per 10 minutes. Telemetry costs nothing extra, because a native-only repeat logs no event and so triggers no exit-time upload (hintOnlyUploadTimeout). Native-event counts stay at most one per text window instead of growing with session length. In return, a session waits at most 10 minutes for the dialog.

Tests

  • New unit tests:
    • A withheld native line (no child session, CI) leaves the next command's native line due. Only that line prints, and it logs its event.
    • The native line reappears after its TTL while the text hint stays throttled, and that repeat logs no event. A new text window logs both events again.
    • A text-only print starts a new event window, so the next native print logs its event.
  • New acceptance/cmd/aitools/claude-config-dir case for the withheld-then-native sequence. The existing throttle case's title now describes both clocks.
  • libs/telemetry/protos/README.md describes both clocks and the hourly native event.
  • go test ./libs/aitools/... ./cmd/root/..., go test ./acceptance -run 'TestAccept/cmd/aitools', ./task test-exp-aitools, ./task checks and ./task lint-q all pass.

This pull request and its description were written by Isaac.

@anton-107
anton-107 requested review from a team and lennartkats-db as code owners November 16, 2026 16:05
@anton-107
anton-107 requested a review from misha-db November 16, 2026 16:05
@anton-107
anton-107 force-pushed the aitools/claude-code-native-hint-throttle branch from 329003c to cfed319 Compare November 16, 2026 16:06
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Commit: feeecc54f | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

1 tests: regression=1 unreleased=0 coverage=0 failing=0
test branch main (bdc5f0e) latest (v1.26.0)
TestAccept/cmd/aitools/claude-config-dir ✅ ❌ ❌
TestAccept/cmd/aitools/claude-config-dir: branch ✅ | main (bdc5f0e) ❌ | latest (v1.26.0) ❌

main (bdc5f0e):

acceptance_test.go:1380: Diff:
        --- cmd/aitools/claude-config-dir/output.txt
        +++ /tmp/TestAcceptcmdaitoolsclaude-config-dir3447945116/002/output.txt
        @@ -65,7 +65,6 @@
         9876  some-other-cluster  
         
         >>> [CLI] clusters list
        -<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
         ID    Name                State
         4321  some-test-cluster   
         9876  some-other-cluster  
        
FAIL

latest (v1.26.0):

acceptance_test.go:1380: Diff:
        --- cmd/aitools/claude-config-dir/output.txt
        +++ /tmp/TestAcceptcmdaitoolsclaude-config-dir535994751/002/output.txt
        @@ -9,7 +9,6 @@
         === Custom directory takes precedence over the default installation
         
         >>> CLAUDE_CONFIG_DIR=missing [CLI] clusters list
        -<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
         Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         ID    Name                State
         4321  some-test-cluster   
        @@ -18,6 +17,7 @@
         === Exempt commands never hint, even without the tooling
         
         >>> CLAUDE_CONFIG_DIR=missing [CLI] current-user me
        +Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         json.userName = "[USERNAME]";
         
         === Empty override uses the default installation
        @@ -45,7 +45,6 @@
         === Within one session the text line prints at most once per hour and the native line at most once per 10 minutes, so a 2>&1 pipeline on the next command sees neither
         
         >>> [CLI] clusters list
        -<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
         Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         ID    Name                State
         4321  some-test-cluster   
        @@ -65,7 +64,6 @@
         9876  some-other-cluster  
         
         >>> [CLI] clusters list
        -<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
         ID    Name                State
         4321  some-test-cluster   
         9876  some-other-cluster  
        
FAIL

@anton-107 anton-107 added aitools AI tools (databricks aitools) related changes ai-auto PRs made by AI loops & factories, labels Nov 16, 2026
Base automatically changed from aitools/claude-code-native-hint to main November 16, 2026 16:40
@anton-107
anton-107 requested review from a team as code owners November 16, 2026 16:40
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: feeecc5

Run: 44103597426

Env 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 16 8:51
💚​ aws-windows-latest-4core-16gb 1 272 14 6:34
🔄​ azure linux-2core-8gb 1 1 268 16 11:46
💚​ azure-windows-latest-4core-16gb 1 271 14 5:55
💚​ gcp linux-2core-8gb 1 270 16 9:49
💚​ gcp-windows-latest-4core-16gb 1 272 14 6:20
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestTelemetryEndpoint ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
Top 5 slowest tests (at least 2 minutes):
duration env testname
4:02 azure-windows-latest-4core-16gb TestAccept
3:46 gcp-windows-latest-4core-16gb TestAccept
3:41 aws-windows-latest-4core-16gb TestAccept
2:22 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsRead
2:04 aws linux-2core-8gb TestFilerReadWrite/workspace_files

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at cfed319. Each finding was verified against the code; simonfaltum may follow up.

Not approving yet: 1 blocking issue(s), tagged [blocking].

This PR gives Claude Code's native <claude-code-hint> line its own 10-minute throttle per session, written only when the line prints. A withheld line, or a subagent making the session's first call, no longer hides the install dialog for an hour. The throttle logic is correct and its tests pass locally. However, the branch conflicts with main: #7586 has since rewritten MaybeHint around agent-prefixed throttle keys and a durable claim, and main's hint_agents.go uses the constants this PR removes. So it needs a careful rebase and a fresh CI run. The other comments are about the stale telemetry README, the extra exit-time telemetry uploads from native-only repeats, and two nits on the changelog and test helpers.

Accepted answers

  • Why repeat the native line every 10 minutes instead of keeping its own key on the text hint's hour?

Comment on lines +188 to +213
// Check the throttles before the filesystem probe, so a throttled session
// skips the plugin/skills detection. A throttle marker is written only
// after its line prints, so an already-installed environment never burns
// one, and neither does a native line withheld from this invocation.
session := sessionID(ctx, hint.sessionEnv)
native := hint.nativeLine(ctx)
if native != "" && nativeHintThrottle.seen(ctx, session) {
native = ""
}
textDue := !textHintThrottle.seen(ctx, session)
if native == "" && !textDue {
return
}
if claudeToolingInstalled(ctx) {
if hint.installed(ctx) {
return
}
cmdio.LogString(ctx, hintText)
logHintShown(ctx, protos.AitoolsHintKindInstallText)
recordSessionHint(ctx)
if native != "" {
cmdio.LogString(ctx, native)
logHintShown(ctx, hint.nativeKind)
nativeHintThrottle.record(ctx, session)
}
if textDue {
cmdio.LogString(ctx, hint.text)
logHintShown(ctx, hint.textKind)
textHintThrottle.record(ctx, session)
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] This branch conflicts with main, and the conflict isn't mechanical. After this commit, #7586 (d285909) rewrote MaybeHint on main in three ways. It keys the throttle by throttleKey: an agent-prefixed session id, or a per-machine fallbackKey. It prints non-Claude hints only after claim/claimHintMarker has persisted the marker. And it takes a third pluginRecorded argument. Main's hint_agents.go:59 (claimHintMarker) still uses hintCacheComponent and hintSessionTTL, which this PR deletes. The new tests also call the two-argument MaybeHint. If the conflict is resolved by taking this branch's hint.go, the build breaks and the non-Claude throttle is lost.

Please rebase onto main and then:

  • key hintThrottle by the throttle key;
  • have claimHintMarker use textHintThrottle, or keep the constants;
  • keep the native throttle for Claude Code only;
  • update the new tests and the acceptance output.

The green CI ran before #7585 and #7586 landed, so it needs a fresh run after the rebase.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebased onto main as one commit, 9d0ae56. hintThrottle is now keyed by throttleKey, and claimHintMarker uses textHintThrottle, so non-Claude hints still print only after the claim persists. The native 10-minute throttle applies only to Claude Code. The tests use the three-argument MaybeHint, and the acceptance output is regenerated. Fresh CI is running.

Comment thread libs/telemetry/protos/README.md Outdated
Comment on lines 14 to 17
uploads at exit only when the command resolved auth. The install hint prints at most once per
Claude Code session per hour, so when the command that printed it resolves no auth (e.g.
`databricks version`), that session logs no install hint event for the next hour even though the
agent saw the hint.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] This paragraph still says the install hint prints at most once per Claude Code session per hour, and that a command that resolves no auth means no event "for the next hour". With this PR, the native line and its CLAUDE_CODE_NATIVE event can repeat on their own every 10 minutes per session. Anyone who reads native-event counts with this doc in mind can be off by up to 6x.

Please say that the text line prints at most once per hour and the native line at most once per 10 minutes. Main reworded this sentence in #7586, so update that version after the rebase. The acceptance title at acceptance/cmd/aitools/claude-config-dir/script:31 ("both lines print once, so 2>&1 pipelines see them only on the first command") has the same problem.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 9d0ae56. The README now says the text line prints at most once per hour and the native line at most once per 10 minutes, while the native event stays at most hourly. The claude-config-dir title now describes both clocks.

Comment on lines +204 to +208
if native != "" {
cmdio.LogString(ctx, native)
logHintShown(ctx, hint.nativeKind)
nativeHintThrottle.record(ctx, session)
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] The description counts the cost of the 10-minute clock as a raw tag in a 2>&1 | jq pipeline. It leaves out telemetry. Each native-only repeat logs a CLAUDE_CODE_NATIVE event, and a command that resolved auth then uploads telemetry at exit. That upload waits up to hintOnlyUploadTimeout (1s); without the event, the command wouldn't upload at all.

Take a long session without the plugin, where an agent runs databricks jobs list every few minutes. It now makes up to 6 of these uploads per hour instead of 1, and native-event counts grow with session length rather than with the number of dialog chances. Consider keeping the native event on the text hint's hourly clock. Otherwise, mention the extra uploads in the description and the README. (in reply to #7692)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went with keeping the native event on the text hint's hourly clock (9d0ae56). The CLAUDE_CODE_NATIVE event is logged only for the first native line since the text last printed, so a native-only repeat logs nothing and triggers no exit-time upload. TestMaybeHint_NativeOnlyRepeatLogsNoEvent covers this, and the description's cost paragraph is updated.

@@ -0,0 +1 @@
* Repeat Claude Code's native plugin install hint every 10 minutes per session instead of hourly with the text hint, so the install dialog isn't held back for an hour when a subagent or hook runs the session's first `databricks` command. ([#7692](https://github.com/databricks/cli/pull/7692))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] No release contains #7585 yet (v1.26.0 predates it), so both fragments ship in the same release. Readers will first see "Print the native hint alongside the text hint", then "Repeat it every 10 minutes ... instead of hourly with the text hint". That describes an hourly behaviour no release ever had. "Every 10 minutes" also reads like a timer, but the line prints at most once per 10 minutes, on the next command.

Please fold this into .nextchanges/cli/claude-code-native-hint.md: drop "alongside the text hint", describe the 10-minute throttle and link both PRs. Then delete this file.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworded in 9d0ae56 so it reads as a follow-on to #7585's line in the same release. It now says the native hint is throttled separately and prints at most once per 10 minutes per session, with no hourly baseline. I kept it as its own fragment instead of folding it into #7585's.

Comment thread libs/aitools/agents/hint_test.go Outdated

// sessionCtx returns a context for one command in session-abc, sharing home
// and cacheDir with the session's other commands.
func sessionCtx(t *testing.T, home, cacheDir string, childSession bool) (context.Context, *bytes.Buffer) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] sessionCtx duplicates the newCtx closures in TestMaybeHint_NativeHintThrottledPerSession (line 463) and TestMaybeHint_LogsEventsOncePerSession (line 625, which is identical). .agents/rules/style-guide-go.md asks for a named function instead of closures like these. Since this PR adds the helper, please use it in both tests and drop the closures.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 9d0ae56. A single named sessionCtx(t, home, cacheDir, childSession) replaces the closures in TestMaybeHint_ThrottledPerSession, TestMaybeHint_NativeHintThrottledPerSession and TestMaybeHint_LogsEventsOncePerSession, and the new tests use it too.

The native <claude-code-hint> line shared the text hint's one-hour session
marker, which was written whenever the text printed, including when the
native line was withheld (no child-session marker, CI, terminal attached)
or printed where Claude Code strips it without prompting (subagents,
hooks). The main loop could then go an hour without the install dialog.

Throttle the native line under its own cache bucket with a 10-minute TTL,
keyed by the hint's throttle key and written only when it prints. The text
hint keeps its one-hour marker, and other agents' durable claim now goes
through the same text throttle.

The CLAUDE_CODE_NATIVE event stays on the text hint's hourly clock: it is
logged only for the first native line printed since the text last printed,
so a native-only repeat logs no event and triggers no telemetry upload at
exit.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107
anton-107 force-pushed the aitools/claude-code-native-hint-throttle branch from cfed319 to 9d0ae56 Compare November 17, 2026 10:55

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Follow-up review from simonfaltum's review agent (pr-review skill) at 9d0ae56, covering the changes since cfed319. Each finding was verified against the code; simonfaltum may follow up.

No blocking issues found, approving. 2 non-blocking comment(s) below.

4 earlier comment(s) fixed since cfed319.

The rebase onto main carries the separate 10-minute native throttle over cleanly to #7586's throttle keys and durable claim. The new event marker keeps CLAUDE_CODE_NATIVE on the text hint's hourly window, so native-only repeats no longer trigger telemetry uploads. That addresses all five earlier comments: four are fixed, and the reworded, separate changelog fragment is fine. CI is green and nothing blocking is left. Two small follow-ups remain: a test for the textDue || branch, and README wording that overstates how rarely the native event is logged.

Accepted answers

nativeHintThrottle.record(ctx, key)
// A text print starts a new text window, in which this is the first
// native print.
if textDue || !nativeEventThrottle.seen(ctx, key) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] No test covers the textDue || half of this condition. With it removed, go test ./libs/aitools/agents/ still passes, because every test that prints the text and native lines together starts with the event marker missing or aged out.

It matters when the previous window's first native line printed late. Say a text-only command runs at T0 and the first native line at T0+55m, which writes a true event marker that lasts until T0+115m. A command at T0+61m then prints both lines. Without textDue ||, that command logs only INSTALL_TEXT, so the new window's CLAUDE_CODE_NATIVE event is lost.

Please add a test for this sequence:

  1. Run a text-only command.
  2. Age the text marker by 55 minutes and run a Claude Code command.
  3. Age the text marker past 1 hour and the native marker past 10 minutes, leaving the event marker fresh.
  4. Run another Claude Code command and expect both lines and both events.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added TestMaybeHint_TextAndNativePrintStartsNewNativeEventWindow in feeecc5, following your sequence. It fails with textDue || removed.

Comment thread libs/telemetry/protos/README.md Outdated
(e.g. `databricks bundle schema`), no install hint event is logged for that hour even though the
resolved auth. The text line prints at most once per hour per agent session (or per agent on the
machine, for agents without a session id). Claude Code's native line prints at most once per 10
minutes per session, but its event is logged at most once per hour: only for the first native line

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] "its event is logged at most once per hour" is stronger than what the code does. The bound is one native event per text-line print, and two can land about one native TTL apart. Here is a sequence:

  • T0: a command outside a Claude Code subprocess prints only the text.
  • T0+55m: a Claude Code command prints the native line and logs the event.
  • T0+61m: the text prints again while the native line is still throttled, which resets the marker.
  • T0+66m: the native line prints and logs the event again.

That's two CLAUDE_CODE_NATIVE events 11 minutes apart. Since this paragraph is what people read when they interpret the counts, consider "its event is logged at most once per text-line print: only for the first native line printed since the text line last printed".

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworded in feeecc5 to "logged at most once per text-line print", using your wording. The closing sentence now says no event is uploaded "until the text line prints again" instead of "for that hour".

… bound

Cover the textDue half of the native-event condition: a command that
prints both lines starts a new text window and logs the native event,
even when the previous window's late native print left a fresh event
marker. The README now states the real bound, one native event per
text-line print, rather than once per hour.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107
anton-107 enabled auto-merge November 17, 2026 14:28
@anton-107
anton-107 added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit baf6a33 Nov 17, 2026
29 checks passed
@anton-107
anton-107 deleted the aitools/claude-code-native-hint-throttle branch November 17, 2026 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-auto PRs made by AI loops & factories, aitools AI tools (databricks aitools) related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants