Repository navigation
aitools: throttle the native Claude Code hint on its own shorter clock - #7692
Conversation
329003c to
cfed319
Compare
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 1 tests: regression=1 unreleased=0 coverage=0 failing=0
TestAccept/cmd/aitools/claude-config-dir: branch ✅ | main (bdc5f0e) ❌ | latest (v1.26.0) ❌main (bdc5f0e): latest (v1.26.0): |
✅ Integration test reportCommit: feeecc5
Top 5 slowest tests (at least 2 minutes):
|
simonfaltum
left a comment
There was a problem hiding this comment.
🤖 Automated review from simonfaltum's review agent (pr-review skill) at cfed319. Each finding was verified against the code; simonfaltum may follow up.
Not approving yet: 1 blocking issue(s), tagged [blocking].
This PR gives Claude Code's native <claude-code-hint> line its own 10-minute throttle per session, written only when the line prints. A withheld line, or a subagent making the session's first call, no longer hides the install dialog for an hour. The throttle logic is correct and its tests pass locally. However, the branch conflicts with main: #7586 has since rewritten MaybeHint around agent-prefixed throttle keys and a durable claim, and main's hint_agents.go uses the constants this PR removes. So it needs a careful rebase and a fresh CI run. The other comments are about the stale telemetry README, the extra exit-time telemetry uploads from native-only repeats, and two nits on the changelog and test helpers.
Accepted answers
- Why repeat the native line every 10 minutes instead of keeping its own key on the text hint's hour?
| // Check the throttles before the filesystem probe, so a throttled session | ||
| // skips the plugin/skills detection. A throttle marker is written only | ||
| // after its line prints, so an already-installed environment never burns | ||
| // one, and neither does a native line withheld from this invocation. | ||
| session := sessionID(ctx, hint.sessionEnv) | ||
| native := hint.nativeLine(ctx) | ||
| if native != "" && nativeHintThrottle.seen(ctx, session) { | ||
| native = "" | ||
| } | ||
| textDue := !textHintThrottle.seen(ctx, session) | ||
| if native == "" && !textDue { | ||
| return | ||
| } | ||
| if claudeToolingInstalled(ctx) { | ||
| if hint.installed(ctx) { | ||
| return | ||
| } | ||
| cmdio.LogString(ctx, hintText) | ||
| logHintShown(ctx, protos.AitoolsHintKindInstallText) | ||
| recordSessionHint(ctx) | ||
| if native != "" { | ||
| cmdio.LogString(ctx, native) | ||
| logHintShown(ctx, hint.nativeKind) | ||
| nativeHintThrottle.record(ctx, session) | ||
| } | ||
| if textDue { | ||
| cmdio.LogString(ctx, hint.text) | ||
| logHintShown(ctx, hint.textKind) | ||
| textHintThrottle.record(ctx, session) | ||
| } |
There was a problem hiding this comment.
[blocking] This branch conflicts with main, and the conflict isn't mechanical. After this commit, #7586 (d285909) rewrote MaybeHint on main in three ways. It keys the throttle by throttleKey: an agent-prefixed session id, or a per-machine fallbackKey. It prints non-Claude hints only after claim/claimHintMarker has persisted the marker. And it takes a third pluginRecorded argument. Main's hint_agents.go:59 (claimHintMarker) still uses hintCacheComponent and hintSessionTTL, which this PR deletes. The new tests also call the two-argument MaybeHint. If the conflict is resolved by taking this branch's hint.go, the build breaks and the non-Claude throttle is lost.
Please rebase onto main and then:
- key
hintThrottleby the throttle key; - have
claimHintMarkerusetextHintThrottle, or keep the constants; - keep the native throttle for Claude Code only;
- update the new tests and the acceptance output.
The green CI ran before #7585 and #7586 landed, so it needs a fresh run after the rebase.
There was a problem hiding this comment.
Rebased onto main as one commit, 9d0ae56. hintThrottle is now keyed by throttleKey, and claimHintMarker uses textHintThrottle, so non-Claude hints still print only after the claim persists. The native 10-minute throttle applies only to Claude Code. The tests use the three-argument MaybeHint, and the acceptance output is regenerated. Fresh CI is running.
| uploads at exit only when the command resolved auth. The install hint prints at most once per | ||
| Claude Code session per hour, so when the command that printed it resolves no auth (e.g. | ||
| `databricks version`), that session logs no install hint event for the next hour even though the | ||
| agent saw the hint. |
There was a problem hiding this comment.
[nice to have] This paragraph still says the install hint prints at most once per Claude Code session per hour, and that a command that resolves no auth means no event "for the next hour". With this PR, the native line and its CLAUDE_CODE_NATIVE event can repeat on their own every 10 minutes per session. Anyone who reads native-event counts with this doc in mind can be off by up to 6x.
Please say that the text line prints at most once per hour and the native line at most once per 10 minutes. Main reworded this sentence in #7586, so update that version after the rebase. The acceptance title at acceptance/cmd/aitools/claude-config-dir/script:31 ("both lines print once, so 2>&1 pipelines see them only on the first command") has the same problem.
There was a problem hiding this comment.
Done in 9d0ae56. The README now says the text line prints at most once per hour and the native line at most once per 10 minutes, while the native event stays at most hourly. The claude-config-dir title now describes both clocks.
| if native != "" { | ||
| cmdio.LogString(ctx, native) | ||
| logHintShown(ctx, hint.nativeKind) | ||
| nativeHintThrottle.record(ctx, session) | ||
| } |
There was a problem hiding this comment.
[nice to have] The description counts the cost of the 10-minute clock as a raw tag in a 2>&1 | jq pipeline. It leaves out telemetry. Each native-only repeat logs a CLAUDE_CODE_NATIVE event, and a command that resolved auth then uploads telemetry at exit. That upload waits up to hintOnlyUploadTimeout (1s); without the event, the command wouldn't upload at all.
Take a long session without the plugin, where an agent runs databricks jobs list every few minutes. It now makes up to 6 of these uploads per hour instead of 1, and native-event counts grow with session length rather than with the number of dialog chances. Consider keeping the native event on the text hint's hourly clock. Otherwise, mention the extra uploads in the description and the README. (in reply to #7692)
There was a problem hiding this comment.
Went with keeping the native event on the text hint's hourly clock (9d0ae56). The CLAUDE_CODE_NATIVE event is logged only for the first native line since the text last printed, so a native-only repeat logs nothing and triggers no exit-time upload. TestMaybeHint_NativeOnlyRepeatLogsNoEvent covers this, and the description's cost paragraph is updated.
| @@ -0,0 +1 @@ | |||
| * Repeat Claude Code's native plugin install hint every 10 minutes per session instead of hourly with the text hint, so the install dialog isn't held back for an hour when a subagent or hook runs the session's first `databricks` command. ([#7692](https://github.com/databricks/cli/pull/7692)) | |||
There was a problem hiding this comment.
[nit] No release contains #7585 yet (v1.26.0 predates it), so both fragments ship in the same release. Readers will first see "Print the native hint alongside the text hint", then "Repeat it every 10 minutes ... instead of hourly with the text hint". That describes an hourly behaviour no release ever had. "Every 10 minutes" also reads like a timer, but the line prints at most once per 10 minutes, on the next command.
Please fold this into .nextchanges/cli/claude-code-native-hint.md: drop "alongside the text hint", describe the 10-minute throttle and link both PRs. Then delete this file.
There was a problem hiding this comment.
|
|
||
| // sessionCtx returns a context for one command in session-abc, sharing home | ||
| // and cacheDir with the session's other commands. | ||
| func sessionCtx(t *testing.T, home, cacheDir string, childSession bool) (context.Context, *bytes.Buffer) { |
There was a problem hiding this comment.
[nit] sessionCtx duplicates the newCtx closures in TestMaybeHint_NativeHintThrottledPerSession (line 463) and TestMaybeHint_LogsEventsOncePerSession (line 625, which is identical). .agents/rules/style-guide-go.md asks for a named function instead of closures like these. Since this PR adds the helper, please use it in both tests and drop the closures.
There was a problem hiding this comment.
Done in 9d0ae56. A single named sessionCtx(t, home, cacheDir, childSession) replaces the closures in TestMaybeHint_ThrottledPerSession, TestMaybeHint_NativeHintThrottledPerSession and TestMaybeHint_LogsEventsOncePerSession, and the new tests use it too.
The native <claude-code-hint> line shared the text hint's one-hour session marker, which was written whenever the text printed, including when the native line was withheld (no child-session marker, CI, terminal attached) or printed where Claude Code strips it without prompting (subagents, hooks). The main loop could then go an hour without the install dialog. Throttle the native line under its own cache bucket with a 10-minute TTL, keyed by the hint's throttle key and written only when it prints. The text hint keeps its one-hour marker, and other agents' durable claim now goes through the same text throttle. The CLAUDE_CODE_NATIVE event stays on the text hint's hourly clock: it is logged only for the first native line printed since the text last printed, so a native-only repeat logs no event and triggers no telemetry upload at exit. Co-authored-by: Isaac <no-reply@databricks.com>
cfed319 to
9d0ae56
Compare
simonfaltum
left a comment
There was a problem hiding this comment.
🤖 Follow-up review from simonfaltum's review agent (pr-review skill) at 9d0ae56, covering the changes since cfed319. Each finding was verified against the code; simonfaltum may follow up.
No blocking issues found, approving. 2 non-blocking comment(s) below.
4 earlier comment(s) fixed since cfed319.
The rebase onto main carries the separate 10-minute native throttle over cleanly to #7586's throttle keys and durable claim. The new event marker keeps CLAUDE_CODE_NATIVE on the text hint's hourly window, so native-only repeats no longer trigger telemetry uploads. That addresses all five earlier comments: four are fixed, and the reworded, separate changelog fragment is fine. CI is green and nothing blocking is left. Two small follow-ups remain: a test for the textDue || branch, and README wording that overstates how rarely the native event is logged.
Accepted answers
- Fold the throttle changelog into #7585's fragment?: #7692 (comment)
| nativeHintThrottle.record(ctx, key) | ||
| // A text print starts a new text window, in which this is the first | ||
| // native print. | ||
| if textDue || !nativeEventThrottle.seen(ctx, key) { |
There was a problem hiding this comment.
[nice to have] No test covers the textDue || half of this condition. With it removed, go test ./libs/aitools/agents/ still passes, because every test that prints the text and native lines together starts with the event marker missing or aged out.
It matters when the previous window's first native line printed late. Say a text-only command runs at T0 and the first native line at T0+55m, which writes a true event marker that lasts until T0+115m. A command at T0+61m then prints both lines. Without textDue ||, that command logs only INSTALL_TEXT, so the new window's CLAUDE_CODE_NATIVE event is lost.
Please add a test for this sequence:
- Run a text-only command.
- Age the text marker by 55 minutes and run a Claude Code command.
- Age the text marker past 1 hour and the native marker past 10 minutes, leaving the event marker fresh.
- Run another Claude Code command and expect both lines and both events.
There was a problem hiding this comment.
Added TestMaybeHint_TextAndNativePrintStartsNewNativeEventWindow in feeecc5, following your sequence. It fails with textDue || removed.
| (e.g. `databricks bundle schema`), no install hint event is logged for that hour even though the | ||
| resolved auth. The text line prints at most once per hour per agent session (or per agent on the | ||
| machine, for agents without a session id). Claude Code's native line prints at most once per 10 | ||
| minutes per session, but its event is logged at most once per hour: only for the first native line |
There was a problem hiding this comment.
[nit] "its event is logged at most once per hour" is stronger than what the code does. The bound is one native event per text-line print, and two can land about one native TTL apart. Here is a sequence:
- T0: a command outside a Claude Code subprocess prints only the text.
- T0+55m: a Claude Code command prints the native line and logs the event.
- T0+61m: the text prints again while the native line is still throttled, which resets the marker.
- T0+66m: the native line prints and logs the event again.
That's two CLAUDE_CODE_NATIVE events 11 minutes apart. Since this paragraph is what people read when they interpret the counts, consider "its event is logged at most once per text-line print: only for the first native line printed since the text line last printed".
There was a problem hiding this comment.
Reworded in feeecc5 to "logged at most once per text-line print", using your wording. The closing sentence now says no event is uploaded "until the text line prints again" instead of "for that hour".
… bound Cover the textDue half of the native-event condition: a command that prints both lines starts a new text window and logs the native event, even when the previous window's late native print left a fresh event marker. The README now states the real bound, one native event per text-line print, rather than once per hour. Co-authored-by: Isaac <no-reply@databricks.com>
Changes
The native
<claude-code-hint>line now has its own throttle (a separate cache bucket, 10-minute TTL), keyed by the hint's throttle key from #7586 and written only when the line actually prints. The text hint keeps its one-hour marker, and other agents' durable claim (claimHintMarker) now goes through the same text throttle. The native throttle applies only to Claude Code, the one agent with a native line. All throttles are still checked before the plugin/skills filesystem probe.The
CLAUDE_CODE_NATIVEhint-shown event stays on the text hint's hourly clock. It is logged only for the first native line printed since the text line last printed, so a native-only repeat logs no event.Why
This addresses simonfaltum's review comment on
hint.goin #7585. The single shared marker was written whenever the text hint printed. That included commands where the native line was withheld (noCLAUDE_CODE_CHILD_SESSION, CI, a terminal attached), and commands where Claude Code strips the tag without prompting (subagents, hooks). After that, the main loop got neither line for an hour.The CLI can't tell a subagent from the main loop, because both carry the same session id and child-session marker. So a separate key alone doesn't fix the subagent case; the shorter TTL is what bounds it. Claude Code shows the dialog at most once per session, so repeating the line is cheap: a raw tag reaches a
2>&1 | jqreader at most once per 10 minutes. Telemetry costs nothing extra, because a native-only repeat logs no event and so triggers no exit-time upload (hintOnlyUploadTimeout). Native-event counts stay at most one per text window instead of growing with session length. In return, a session waits at most 10 minutes for the dialog.Tests
acceptance/cmd/aitools/claude-config-dircase for the withheld-then-native sequence. The existing throttle case's title now describes both clocks.libs/telemetry/protos/README.mddescribes both clocks and the hourly native event.go test ./libs/aitools/... ./cmd/root/...,go test ./acceptance -run 'TestAccept/cmd/aitools',./task test-exp-aitools,./task checksand./task lint-qall pass.This pull request and its description were written by Isaac.