Skip to content

aitools: install AI tools after the first command of a new CLI version - #7702

Open
anton-107 wants to merge 6 commits into
mainfrom
aitools/auto-install-first-run
Open

anton-107 wants to merge 6 commits into
mainfrom
aitools/auto-install-first-run

Conversation

@anton-107

@anton-107 anton-107 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #7701, the PR that stores the auto-install state and adds the opt-out commands and the cli-compat gate.

After a command succeeds, the CLI installs (or updates) Databricks AI tools for the coding agents on the machine, once per new CLI version. Ships off: nothing sets the gate yet.

Summary

root PersistentPostRunE (cmd/cmd.go; cobra skips it when the command failed)
  cmdio.Wait
  aitools.AutoInstall(cmd)                         # cmd/aitools/autoinstall_run.go; recovers panics, returns nothing
    Decide(command, version)                       # libs/aitools/autoinstall/decide.go (pure)
      skipped command | Databricks Runtime
      | cache disabled | development build         -> return (no state read, no log)
    Load state (under auto-install.json.lock)
    Decide again
      settled | retries exhausted
      | older than a CLI that already ran here     -> debug line, return
    TryLock auto-install.lock (busy -> skip); reload state
    Decide with the gate off
      CI                 -> settle, report SKIPPED_CI       # agents.InCI: CI=<anything but false>, or a provider the SDK detects
      opt-out            -> settle, report SKIPPED_OPTOUT   # env var, stored opt-out, or "no" in this process
      gate_off           -> look the gate up (cli-compat auto_install, 3s), Decide again
        still off        -> settle silently
        lookup failed    -> record attempt N, report nothing  # only the embedded manifest was reachable
      holdout (10%)      -> settle, report SKIPPED_HOLDOUT
      run
        record attempt N
        planTools
          global skills on disk, no install state,
            not left by our own unfinished install  -> leave them alone, settle, report KEPT
          no global install state:  mark "installing", buildPlan + executePlan  # as `aitools install --scope=global`,
                                                               # no prompts, agents whose config dir exists
          global install state:     runUpdate(--scope=global)  # as `aitools update`, edited skills kept,
                                                               # no raw-skills-to-plugin migration
          Windows: drop plugin installs; leave an install alone if its update touches plugins -> SKIPPED_PLATFORM
        (10s budget, installer output discarded, warnings demoted to debug)
        installed / changed -> settle, report INSTALLED, stderr: "Installed Databricks AI tools for Claude Code, Cursor. Disable: databricks aitools auto-install disable"
        update, no change -> settle, report UP_TO_DATE
        no agents -> settle, report SKIPPED_NO_AGENTS
        refused downgrade -> keep the installed tools, settle, report KEPT
        error     -> debug line, report FAILED (RETRIES_EXHAUSTED on attempt 10)
  • "Settle" writes last_version = current, attempts = 0 and clears installing: each machine reports at most one outcome per version, plus one per retry. A new version resets the retry budget.
  • Locks are OS file locks (flock / LockFileEx) on files that are never removed. They are only tried, never waited on. The state lock is retried for up to 1 s while another process holds it; any other lock error returns at once. A busy run lock skips the attempt.
  • Agent CLIs run in their own process group (process.BackgroundInGroup), so a timeout stops their descendants too. That takes them out of the terminal's foreground group, so while one runs, the first SIGINT/SIGTERM is caught, stops the group, and is raised again in the CLI: Ctrl-C ends aitools install/update as before, without orphans. Windows has no such stop, so there auto-install runs no agent CLI: a fresh install drops its plugin installs (skills are still installed), and an existing install whose update would touch plugins is left alone. A Windows machine with only plugin-capable agents settles as SKIPPED_NO_AGENTS.
  • Answering "no" at the apps init skills prompt opts out, and says so with the auto-install enable command. It is stored, honored in the process even if the write fails, and passed as DATABRICKS_AITOOLS_AUTO_INSTALL=false to the apps deploy / apps dev-remote children that apps init starts, next to their DATABRICKS_CLI_SELF_EXEC=1.
  • Skipped commands are the version notice's (version, help, completion), the credential helpers (auth token, auth docker token) and the aitools trees. Non-interactive and -o json runs are not skipped, unlike for the notice: coding agents run the CLI that way, and auto-install writes nothing to stdout. Databricks Runtime is skipped (ephemeral home, platform-managed CLI).
  • The gate is unknown, not off, when only the embedded manifest is reachable (clicompat.ResolveAutoInstall fails): the version is not settled, and the lookup uses up one of the version's 10 attempts, so an offline machine stops paying for it.
  • The holdout uses a random id stored in the state (the CLI has no machine id); only the arm (HOLDOUT/TREATMENT) is reported.
  • Development and snapshot builds never auto-install. Acceptance tests set the undocumented DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION to simulate releases and upgrades.
  • Telemetry: aitools_install_event.trigger = AUTO_FIRST_RUN (unset for the command's own installs, so existing goldens do not change) and a new aitools_auto_install_event {outcome, holdout_arm, interactive, error_category}, documented in libs/telemetry/protos/README.md. Outcomes: INSTALLED, UP_TO_DATE, KEPT, SKIPPED_CI, SKIPPED_OPTOUT, SKIPPED_HOLDOUT, SKIPPED_NO_AGENTS, SKIPPED_PLATFORM, FAILED, RETRIES_EXHAUSTED; the lumberjack proto needs the same values.
  • Includes a no-behavior-change refactor: aitools update's RunE body moves to runUpdate; agents.InCI, agents.IsCredentialHelperCommand and versioncheck.IsExemptCommand are exported for reuse; the acceptance helper print_telemetry moves up a directory.
  • No changelog fragment: nothing is user-visible while the gate is off. The PR that turns it on should add one.

Known limitations

Both must be fixed before the gate is turned on:

  • A global aitools uninstall that runs while an automatic install is in flight can see that attempt recreate the tools. Running uninstall again removes them.
  • A partial install (one agent fails) settles as INSTALLED, and later updates never add the missing agent. The same applies to the agents the Windows rule above leaves out. One fix is to record the targeted agents in auto-install.json. (An install cut short by the timeout is different: its installing marker makes the next run finish it.)

Review order

  1. libs/aitools/autoinstall/decide.go: the decision, pure.
  2. cmd/aitools/autoinstall_run.go: the hook, planTools, reporting.
  3. libs/aitools/autoinstall/run.go, lock_unix.go, lock_windows.go, state.go: locks, attempts, Decline / ChildEnv.
  4. cmd/cmd.go, cmd/apps/init.go, libs/aitools/agents/recommend.go: the wiring.
  5. Supporting: cmd/aitools/update.go (refactor), libs/aitools/installer/update.go (KeepModified, HasUntrackedSkills), libs/aitools/installer/plugin.go (process group), libs/telemetry/protos/.

Evidence

New acceptance tests in acceptance/cmd/aitools/auto-install/ run databricks jobs list against the fake server and a fake skills source, with a temporary HOME and agent dirs:

  • first-run:
    >>> [CLI] jobs list
    Installed Databricks AI tools for Claude Code, Cursor. Disable: databricks aitools auto-install disable
    {"aitools_install_event":{"agents":["CLAUDE_CODE","CURSOR"],...,"trigger":"AUTO_FIRST_RUN"}}
    {"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
    === second run of the same version: nothing
    === uninstall turns it off: the next version installs nothing   -> SKIPPED_OPTOUT
    
  • upgrade: 1.30.0 installs skills release 0.3.0; 1.31.0 with release 0.4.0 prints Updated Databricks AI tools for Cursor., updates one skill and keeps a skill the user edited; 1.32.0 while the latest release is 0.2.0 refuses the downgrade (files unchanged, nothing printed, settled as KEPT, not retried); a 1.31.0 CLI run afterwards does nothing.
  • skipped: CI=true (reported once per version), DATABRICKS_AITOOLS_AUTO_INSTALL=false, auto-install disable, version/aitools/completion/__complete/help, DATABRICKS_CACHE_ENABLED=false, gate off (nothing reported, version settled), holdout machine, no agent config dir.
  • retries: nine FAILED attempts, RETRIES_EXHAUSTED on the tenth, nothing on the eleventh, a fresh install on the next version.
  • Unit tests: TestDecide (table-driven over every input); TestSkipsAutoInstall; TestPlanToolsRunsNoAgentCLIWhereTimeoutsCannotStopThem, TestPlanToolsReportsWhyNothingIsInstalled and TestPlanToolsKeepsSkillsWithoutInstallState (also: an unfinished install of ours is finished); TestChangedAgents; TestTryLockExcludesOtherRuns, TestLoadWaitsForStateLock, TestLoadFailsWhenStateLockStaysHeld, TestLockReturnsOtherErrorsAtOnce, TestRecordKeepsConcurrentOptOut, TestSetOptedOutWithoutLocks, TestRecordClearsInstallingWhenSettled; TestChildEnvCarriesTheRefusal, TestSelfExecEnvCarriesBothMarkers (the apps init children's env); TestResolveAutoInstallFailsOnTheEmbeddedFallback; TestBackgroundInGroupForwardsInterrupt (a helper process interrupted mid-run dies by SIGINT and its grandchild is reaped); TestRunAgentCmdTimeoutStopsDescendants; TestUpdateKeepModifiedKeepsEditedSkill.
  • Unit tests of cmd/... and libs/..., the aitools and apps acceptance suites, lint and whitespace checks pass on Linux; GOOS=windows/darwin build and vet pass. The gate-lookup failure has no acceptance test: the manifest URL has no override, so it is covered by the clicompat and state unit tests.

Merge Danger

Door: two-way while the gate is off

With the gate off it reads one small file and settles the version on the first run of each release, sends SKIPPED_CI/SKIPPED_OPTOUT events when those apply, and does nothing else. Turning the gate on (a cli-compat.json edit on main) is the one-way step: installs it causes are not rolled back by turning it off again.

Blast Radius: every command (post-run)

  • Runs after every successful command, except the skipped ones and on Databricks Runtime. Off the first run of a version it is one state-file read under a lock (at most 1 s more if another CLI holds the lock); on the first run, with the gate off, a cli-compat manifest lookup (usually the 1 h local cache, at most 3 s). A machine that cannot fetch the manifest repeats the lookup on up to 10 runs of the version.
  • Also changes the explicit aitools install/update today: agent CLIs run in their own process group, with Ctrl-C forwarded to it.
  • Two CLI versions on one machine: only the newest auto-installs; an older one does nothing.
  • With the gate on: up to about 20 s more after the first command of a version (the 10 s install budget plus up to 10 s for a timed-out agent CLI to release its output pipes), plus retries after failures; skill files in global agent dirs; and plugin installs through agent CLIs on PATH (not on Windows).
  • Writes ~/.databricks/aitools/auto-install.json and two lock files next to it.
  • Windows and macOS behavior is built and vetted, not run.

This pull request and its description were written by Isaac.

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 15cf94d23 | Workflow run

Regression test report

Selected 5 changed tests (limit=20, 0 not selected)

5 tests: regression=1 unreleased=0 coverage=0 failing=4
test branch main (4c8b7a1) latest (v1.26.0)
TestAccept/cmd/aitools/auto-install/enable-disable ✅ ❌ ❌
TestAccept/cmd/aitools/auto-install/first-run ❌ ❌ ❌
TestAccept/cmd/aitools/auto-install/retries ❌ ❌ ❌
TestAccept/cmd/aitools/auto-install/skipped ❌ ❌ ❌
TestAccept/cmd/aitools/auto-install/upgrade ❌ ❌ ❌
TestAccept/cmd/aitools/auto-install/enable-disable: branch ✅ | main (4c8b7a1) ❌ | latest (v1.26.0) ❌

main (4c8b7a1):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:34525" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:34525" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/enable-disable/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installenable-disable844361765/002/output.txt
        @@ -7,71 +7,106 @@
         === disable stores the opt-out
         
         >>> [CLI] aitools auto-install disable
        -Automatic installation of Databricks AI tools is disabled.
        +Install Databricks skills and plugins into your coding agent so it can work
        +effectively with Databricks resources (bundles, jobs, SQL, and more).
         
        ->>> auto_install_state
        -{
        -  "opted_out": true
        -}
        +Supported agents: Claude Code, Cursor, Codex CLI, OpenCode, GitHub Copilot, Antigravity, Pi, Gemini CLI, Goose, Kiro.
         
        -=== enable clears it
        +Skills and plugins are sourced from
        +https://github.com/databricks/databricks-agent-skills
         
        ->>> [CLI] aitools auto-install enable
        -Automatic installation of Databricks AI tools is enabled.
        +Usage:
        +  databricks aitools [command]
         
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
        +
        +Flags:
        +  -h, --help[=true|false]   help for aitools
        +
        +Global Flags:
        +      --debug[=true|false]   enable debug logging
        +  -o, --output type          output type: text or json (default text)
        +  -p, --profile string       ~/.databrickscfg profile
        +  -t, --target string        bundle target to use (if applicable)
        +
        +Use "databricks aitools [command] --help" for more information about a command.
        +
         >>> auto_install_state
        -{}
        +(no auto-install state)
         
        -=== enable says when the environment still turns it off
        +=== enable clears it
         
        ->>> DATABRICKS_AITOOLS_AUTO_INSTALL=false [CLI] aitools auto-install enable
        -Automatic installation of Databricks AI tools is enabled.
        -DATABRICKS_AITOOLS_AUTO_INSTALL still disables it in this environment; unset it to let it run.
        +>>> [CLI] aitools auto-install enable
        +Install Databricks skills and plugins into your coding agent so it can work
        +effectively with Databricks resources (bundles, jobs, SQL, and more).
         
        -=== disable replaces a state file it cannot read
        +Supported agents: Claude Code, Cursor, Codex CLI, OpenCode, GitHub Copilot, Antigravity, Pi, Gemini CLI, Goose, Kiro.
         
        ->>> [CLI] aitools auto-install disable
        -Automatic installation of Databricks AI tools is disabled.
        +Skills and plugins are sourced from
        +https://github.com/databricks/databricks-agent-skills
         
        ->>> auto_install_state
        -{
        -  "opted_out": true
        -}
        +Usage:
        +  databricks aitools [command]
         
        -=== a global uninstall that removes nothing leaves it on
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
         
        ->>> [CLI] aitools uninstall --scope=global
        -Error: no skills or plugins installed
        +Flags:
        +  -h, --help[=true|false]   help for aitools
         
        -Exit code: 1
        +Global Flags:
        +      --debug[=true|false]   enable debug logging
        +  -o, --output type          output type: text or json (default text)
        +  -p, --profile string       ~/.databrickscfg profile
        +  -t, --target string        bundle target to use (if applicable)
         
        +Use "databricks aitools [command] --help" for more information about a command.
        +
         >>> auto_install_state
        -{}
        +(no auto-install state)
         
        -=== a global uninstall of a mistyped skill leaves it on
        +=== enable says when the environment still turns it off
         
        ->>> [CLI] aitools uninstall --scope=global --skills databricks-tset
        -Error: skill "databricks-tset" is not installed
        +>>> DATABRICKS_AITOOLS_AUTO_INSTALL=false [CLI] aitools auto-install enable
        +Install Databricks skills and plugins into your coding agent so it can work
        +effectively with Databricks resources (bundles, jobs, SQL, and more).
         
        -Exit code: 1
        +Supported agents: Claude Code, Cursor, Codex CLI, OpenCode, GitHub Copilot, Antigravity, Pi, Gemini CLI, Goose, Kiro.
         
        ->>> auto_install_state
        -{}
        +Skills and plugins are sourced from
        +https://github.com/databricks/databricks-agent-skills
         
        -=== a global uninstall turns it off
        +Usage:
        +  databricks aitools [command]
         
        ->>> [CLI] aitools uninstall --scope=global
        -Uninstalled 1 skill.
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
         
        ->>> auto_install_state
        -{
        -  "opted_out": true
        -}
        +Flags:
        +  -h, --help[=true|false]   help for aitools
         
        -=== a project uninstall leaves it on
        +Global Flags:
        +      --debug[=true|false]   enable debug logging
        +  -o, --output type          output type: text or json (default text)
        +  -p, --profile string       ~/.databrickscfg profile
        +  -t, --target string        bundle target to use (if applicable)
         
        ->>> [CLI] aitools uninstall --scope=project
        -Uninstalled 1 skill.
        +Use "databricks aitools [command] --help" for more information about a command.
         
        ->>> auto_install_state
        -{}
        +=== disable replaces a state file it cannot read
        +script: line 301: [TEST_TMP_DIR]/home/.databricks/aitools/auto-install.json: No such file or directory
        +
        +Exit code: 1

latest (v1.26.0):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:40197" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:40197" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/enable-disable/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installenable-disable3483840470/002/output.txt
        @@ -7,71 +7,106 @@
         === disable stores the opt-out
         
         >>> [CLI] aitools auto-install disable
        -Automatic installation of Databricks AI tools is disabled.
        +Install Databricks skills and plugins into your coding agent so it can work
        +effectively with Databricks resources (bundles, jobs, SQL, and more).
         
        ->>> auto_install_state
        -{
        -  "opted_out": true
        -}
        +Supported agents: Claude Code, Cursor, Codex CLI, OpenCode, GitHub Copilot, Antigravity, Pi, Gemini CLI, Goose, Kiro.
         
        -=== enable clears it
        +Skills and plugins are sourced from
        +https://github.com/databricks/databricks-agent-skills
         
        ->>> [CLI] aitools auto-install enable
        -Automatic installation of Databricks AI tools is enabled.
        +Usage:
        +  databricks aitools [command]
         
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
        +
        +Flags:
        +  -h, --help   help for aitools
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks aitools [command] --help" for more information about a command.
        +
         >>> auto_install_state
        -{}
        +(no auto-install state)
         
        -=== enable says when the environment still turns it off
        +=== enable clears it
         
        ->>> DATABRICKS_AITOOLS_AUTO_INSTALL=false [CLI] aitools auto-install enable
        -Automatic installation of Databricks AI tools is enabled.
        -DATABRICKS_AITOOLS_AUTO_INSTALL still disables it in this environment; unset it to let it run.
        +>>> [CLI] aitools auto-install enable
        +Install Databricks skills and plugins into your coding agent so it can work
        +effectively with Databricks resources (bundles, jobs, SQL, and more).
         
        -=== disable replaces a state file it cannot read
        +Supported agents: Claude Code, Cursor, Codex CLI, OpenCode, GitHub Copilot, Antigravity, Pi, Gemini CLI, Goose, Kiro.
         
        ->>> [CLI] aitools auto-install disable
        -Automatic installation of Databricks AI tools is disabled.
        +Skills and plugins are sourced from
        +https://github.com/databricks/databricks-agent-skills
         
        ->>> auto_install_state
        -{
        -  "opted_out": true
        -}
        +Usage:
        +  databricks aitools [command]
         
        -=== a global uninstall that removes nothing leaves it on
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
         
        ->>> [CLI] aitools uninstall --scope=global
        -Error: no skills or plugins installed
        +Flags:
        +  -h, --help   help for aitools
         
        -Exit code: 1
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
         
        +Use "databricks aitools [command] --help" for more information about a command.
        +
         >>> auto_install_state
        -{}
        +(no auto-install state)
         
        -=== a global uninstall of a mistyped skill leaves it on
        +=== enable says when the environment still turns it off
         
        ->>> [CLI] aitools uninstall --scope=global --skills databricks-tset
        -Error: skill "databricks-tset" is not installed
        +>>> DATABRICKS_AITOOLS_AUTO_INSTALL=false [CLI] aitools auto-install enable
        +Install Databricks skills and plugins into your coding agent so it can work
        +effectively with Databricks resources (bundles, jobs, SQL, and more).
         
        -Exit code: 1
        +Supported agents: Claude Code, Cursor, Codex CLI, OpenCode, GitHub Copilot, Antigravity, Pi, Gemini CLI, Goose, Kiro.
         
        ->>> auto_install_state
        -{}
        +Skills and plugins are sourced from
        +https://github.com/databricks/databricks-agent-skills
         
        -=== a global uninstall turns it off
        +Usage:
        +  databricks aitools [command]
         
        ->>> [CLI] aitools uninstall --scope=global
        -Uninstalled 1 skill.
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
         
        ->>> auto_install_state
        -{
        -  "opted_out": true
        -}
        +Flags:
        +  -h, --help   help for aitools
         
        -=== a project uninstall leaves it on
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
         
        ->>> [CLI] aitools uninstall --scope=project
        -Uninstalled 1 skill.
        +Use "databricks aitools [command] --help" for more information about a command.
         
        ->>> auto_install_state
        -{}
        +=== disable replaces a state file it cannot read
        +script: line 301: [TEST_TMP_DIR]/home/.databricks/aitools/auto-install.json: No such file or directory
        +
        +Exit code: 1
TestAccept/cmd/aitools/auto-install/first-run: branch ❌ | main (4c8b7a1) ❌ | latest (v1.26.0) ❌

branch:

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:38257" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:38257" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/first-run/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installfirst-run3962934680/002/output.txt
        @@ -2,14 +2,12 @@
         === first run of a new version: installs for the detected agents, one stderr line
         
         >>> [CLI] jobs list
        -Installed Databricks AI tools for Claude Code, Cursor. Disable: databricks aitools auto-install disable
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE","CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED","agent_results":[{"agent":"CLAUDE_CODE","error_category":"CLI_NOT_ON_PATH"}],"trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
        +{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_CI","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> cursor_skills
        -databricks-test
        +(no skills)
         
         >>> auto_install_state
         {
        @@ -26,14 +20,6 @@
         === uninstall turns it off: the next version installs nothing
         
         >>> [CLI] aitools uninstall
        -Uninstalled 1 skill.
        +Error: no skills or plugins installed
         
        ->>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.31.0 [CLI] jobs list
        -
        ->>> print_telemetry
        -{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"STARTED","interactive":false}}
        -{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"scope":"GLOBAL"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_OPTOUT","holdout_arm":"TREATMENT","interactive":false}}
        -
        ->>> cursor_skills
        -(no skills)
        +Exit code: 1

main (4c8b7a1):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:37475" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:37475" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/first-run/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installfirst-run1902566470/002/output.txt
        @@ -2,20 +2,14 @@
         === first run of a new version: installs for the detected agents, one stderr line
         
         >>> [CLI] jobs list
        -Installed Databricks AI tools for Claude Code, Cursor. Disable: databricks aitools auto-install disable
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE","CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED","agent_results":[{"agent":"CLAUDE_CODE","error_category":"CLI_NOT_ON_PATH"}],"trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> cursor_skills
        -databricks-test
        +(no skills)
         
         >>> auto_install_state
        -{
        -  "last_version": "1.30.0",
        -  "id": "treatment-machine"
        -}
        +{"id": "treatment-machine"}
         
         === second run of the same version: nothing
         
        @@ -26,14 +14,6 @@
         === uninstall turns it off: the next version installs nothing
         
         >>> [CLI] aitools uninstall
        -Uninstalled 1 skill.
        +Error: no skills or plugins installed
         
        ->>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.31.0 [CLI] jobs list
        -
        ->>> print_telemetry
        -{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"STARTED","interactive":false}}
        -{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"scope":"GLOBAL"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_OPTOUT","holdout_arm":"TREATMENT","interactive":false}}
        -
        ->>> cursor_skills
        -(no skills)
        +Exit code: 1
        
FAIL

latest (v1.26.0):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:35975" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:35975" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/first-run/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installfirst-run2356677262/002/output.txt
        @@ -2,20 +2,14 @@
         === first run of a new version: installs for the detected agents, one stderr line
         
         >>> [CLI] jobs list
        -Installed Databricks AI tools for Claude Code, Cursor. Disable: databricks aitools auto-install disable
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE","CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED","agent_results":[{"agent":"CLAUDE_CODE","error_category":"CLI_NOT_ON_PATH"}],"trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> cursor_skills
        -databricks-test
        +(no skills)
         
         >>> auto_install_state
        -{
        -  "last_version": "1.30.0",
        -  "id": "treatment-machine"
        -}
        +{"id": "treatment-machine"}
         
         === second run of the same version: nothing
         
        @@ -26,14 +14,6 @@
         === uninstall turns it off: the next version installs nothing
         
         >>> [CLI] aitools uninstall
        -Uninstalled 1 skill.
        +Error: no skills or plugins installed
         
        ->>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.31.0 [CLI] jobs list
        -
        ->>> print_telemetry
        -{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"STARTED","interactive":false}}
        -{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"scope":"GLOBAL"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_OPTOUT","holdout_arm":"TREATMENT","interactive":false}}
        -
        ->>> cursor_skills
        -(no skills)
        +Exit code: 1
        
FAIL
TestAccept/cmd/aitools/auto-install/retries: branch ❌ | main (4c8b7a1) ❌ | latest (v1.26.0) ❌

branch:

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:39823" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:39823" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/retries/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installretries3393866413/002/output.txt
        @@ -2,22 +2,17 @@
         === a failed attempt is silent and retried on the next run, up to 10 attempts
         
         >>> print_telemetry
        -9 {"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"FAILED","holdout_arm":"TREATMENT","interactive":false,"error_category":"SKILLS_FETCH_FAILED"}}
        -9 {"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILLS_FETCH_FAILED","trigger":"AUTO_FIRST_RUN"}}
        +1 {"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_CI","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> auto_install_state
         {
           "last_version": "1.30.0",
        -  "attempts": 9,
        -  "installing": true,
           "id": "treatment-machine"
         }
         
         >>> DATABRICKS_SKILLS_REF=broken-ref [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILLS_FETCH_FAILED","trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"RETRIES_EXHAUSTED","holdout_arm":"TREATMENT","interactive":false,"error_category":"SKILLS_FETCH_FAILED"}}
         
         === after 10 attempts the version gets no more
         
        @@ -31,11 +16,9 @@
         === a new version starts over
         
         >>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.31.0 [CLI] jobs list
        -Installed Databricks AI tools for Cursor. Disable: databricks aitools auto-install disable
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED","trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
        +{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_CI","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> auto_install_state
         {

main (4c8b7a1):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:37923" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:37923" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/retries/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installretries175678551/002/output.txt
        @@ -2,22 +2,13 @@
         === a failed attempt is silent and retried on the next run, up to 10 attempts
         
         >>> print_telemetry
        -9 {"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"FAILED","holdout_arm":"TREATMENT","interactive":false,"error_category":"SKILLS_FETCH_FAILED"}}
        -9 {"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILLS_FETCH_FAILED","trigger":"AUTO_FIRST_RUN"}}
         
         >>> auto_install_state
        -{
        -  "last_version": "1.30.0",
        -  "attempts": 9,
        -  "installing": true,
        -  "id": "treatment-machine"
        -}
        +{"id": "treatment-machine"}
         
         >>> DATABRICKS_SKILLS_REF=broken-ref [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILLS_FETCH_FAILED","trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"RETRIES_EXHAUSTED","holdout_arm":"TREATMENT","interactive":false,"error_category":"SKILLS_FETCH_FAILED"}}
         
         === after 10 attempts the version gets no more
         
        @@ -31,14 +16,8 @@
         === a new version starts over
         
         >>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.31.0 [CLI] jobs list
        -Installed Databricks AI tools for Cursor. Disable: databricks aitools auto-install disable
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED","trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> auto_install_state
        -{
        -  "last_version": "1.31.0",
        -  "id": "treatment-machine"
        -}
        +{"id": "treatment-machine"}

latest (v1.26.0):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:46355" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:46355" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/retries/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installretries70233284/002/output.txt
        @@ -2,22 +2,13 @@
         === a failed attempt is silent and retried on the next run, up to 10 attempts
         
         >>> print_telemetry
        -9 {"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"FAILED","holdout_arm":"TREATMENT","interactive":false,"error_category":"SKILLS_FETCH_FAILED"}}
        -9 {"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILLS_FETCH_FAILED","trigger":"AUTO_FIRST_RUN"}}
         
         >>> auto_install_state
        -{
        -  "last_version": "1.30.0",
        -  "attempts": 9,
        -  "installing": true,
        -  "id": "treatment-machine"
        -}
        +{"id": "treatment-machine"}
         
         >>> DATABRICKS_SKILLS_REF=broken-ref [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILLS_FETCH_FAILED","trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"RETRIES_EXHAUSTED","holdout_arm":"TREATMENT","interactive":false,"error_category":"SKILLS_FETCH_FAILED"}}
         
         === after 10 attempts the version gets no more
         
        @@ -31,14 +16,8 @@
         === a new version starts over
         
         >>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.31.0 [CLI] jobs list
        -Installed Databricks AI tools for Cursor. Disable: databricks aitools auto-install disable
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED","trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> auto_install_state
        -{
        -  "last_version": "1.31.0",
        -  "id": "treatment-machine"
        -}
        +{"id": "treatment-machine"}

2 more failure details omitted, see the full report in the run.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 15cf94d

Run: 44106860327

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 20 7:53
💚​ aws-windows-latest-4core-16gb 1 272 18 5:47
💚​ azure linux-2core-8gb 1 269 20 9:36
💚​ azure-windows-latest-4core-16gb 1 271 18 7:17
💚​ gcp linux-2core-8gb 1 270 20 8:39
💚​ gcp-windows-latest-4core-16gb 1 272 18 7:34
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 8 slowest tests (at least 2 minutes):
duration env testname
3:13 gcp-windows-latest-4core-16gb TestAccept
3:12 azure-windows-latest-4core-16gb TestAccept
3:03 aws-windows-latest-4core-16gb TestAccept
2:45 aws linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
2:35 azure linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
2:34 azure-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsStat
2:31 azure linux-2core-8gb TestFilerRecursiveDelete/workspace_files
2:16 azure linux-2core-8gb TestTelemetryEndpoint

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at 019d40c. Each finding was verified against the code; simonfaltum may follow up.

Not approving yet: 2 blocking issue(s), tagged [blocking]; 1 open question about a change users could be hurt by.

This PR installs or updates Databricks AI tools after the first successful command of each new CLI version. It is gated off behind cli-compat and has opt-out, CI, holdout, retry and lock handling. The decision logic, locking and tests are careful, and CI is green. Two regressions ship even with the gate off: apps init drops the DATABRICKS_CLI_SELF_EXEC marker on its children, and the new process group stops Ctrl-C from reaching agent CLIs in the explicit aitools install/update. Several gaps should be closed before the gate is turned on: the skip list (auth docker token, CI providers, DBR/non-interactive runs), the gate-failure settle, a stuck partial first install, plugin migration hiding edits, and telemetry outcomes.

Questions

  1. [blocking] Auto-install skips only CI (CI=true), but the version-check notice and MaybeHint also skip Databricks Runtime, non-interactive runs and -o json. Scripts, IDE extensions and ephemeral homes (containers, DBR clusters) pay the gate lookup now: up to 3 s, once per version per fresh home. Once the gate is on, they also pay up to about 20 s, plus retries, after their command's output. Should auto-install reuse those skips, or is running there intended? If it is intended, a comment saying why would help.

Accepted answers

  • Changelog fragment for this change

Comment thread cmd/apps/init.go Outdated
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.Stdin = os.Stdin
cmd.Env = autoinstall.ChildEnv()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] cmd.Env = autoinstall.ChildEnv() replaces the cmd.Env = agents.SelfExecEnv(os.Environ()) set four lines above (line 1968). The same happens in runPostCreateDev (lines 1998 and 2002). The apps deploy / apps dev-remote children therefore lose DATABRICKS_CLI_SELF_EXEC=1, and MaybeHint (libs/aitools/agents/hint.go:196) no longer suppresses the coding-agent hint in them. Under Claude Code and other agents, the child prints the hint again inside the parent's output and logs a second hint event. This ships even while the gate is off. Let ChildEnv take the environ slice and compose the two, e.g. cmd.Env = agents.SelfExecEnv(autoinstall.ChildEnv(os.Environ())), at both call sites. That also stops it from reading os.Environ() in library code. A test that checks both markers would cover these call sites, which currently have none.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f. ChildEnv now takes the environ, and both call sites use a new selfExecEnv(environ) in cmd/apps/init.go = agents.SelfExecEnv(autoinstall.ChildEnv(environ)), so the children keep DATABRICKS_CLI_SELF_EXEC=1. TestSelfExecEnvCarriesBothMarkers checks both markers, before and after a "no".

Comment thread libs/aitools/installer/plugin.go Outdated
out, err := process.Background(cctx, argv)
// A timed-out agent CLI is stopped with its descendants: one left running
// would hold the output pipes open and block past the timeout.
out, err := process.Background(cctx, argv, process.WithProcessGroup())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] WithProcessGroup moves the agent CLI out of the terminal's foreground process group (here and in libs/aitools/agents/detect.go:115), so Ctrl-C no longer reaches it. Nothing in cmd/root or cmd/aitools catches SIGINT. Ctrl-C kills the CLI, the context is never cancelled, and claude plugin install / marketplace add keep running as orphans that modify ~/.claude. Before this PR, they were interrupted too. This changes the explicit aitools install / update today, for a feature that is off. cmd/environments/sync.go:92-101 documents the same effect for uv and adds a signal handler for that reason. With the gate on, it also releases the run lock while the orphan is still installing, which is the hazard used to exclude Windows. Either cancel the context on SIGINT/SIGTERM while agent CLIs run (see cmd/air/run.go:350 for restoring the default disposition), or use the process group only on the auto-install path.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f, with the signal route so the explicit commands and auto-install both get it. New process.BackgroundInGroup (libs/process/interrupt_unix.go) runs the child with WithProcessGroup and, while it runs, catches the first SIGINT/SIGTERM. It calls signal.Stop first, so a second signal kills at once (the same order as cmd/air/run.go), then cancels, which stops the group, and then raises the signal again in the CLI. The CLI therefore ends as it did before the group, and nothing is orphaned or keeps running past the run lock. runAgentCmd and ProbePlugin use it. On Windows the child stays in the console group, so nothing is forwarded there. TestBackgroundInGroupForwardsInterrupt interrupts a helper process mid-run, then checks that it died by SIGINT and that its grandchild was reaped.

Comment thread libs/aitools/autoinstall/decide.go Outdated
return true
}
}
return slices.Equal(path, []string{"auth", "token"})

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] isSkippedCommand exempts only the exact path auth token. docker-credential-databricks runs databricks auth docker token (libs/dockercredentials/shim.go:68) on pulls and pushes. So auto-install runs inside Docker's credential lookup: up to 3 s for the gate now, and an install plus a stderr line once the gate is on. agents.isCredentialHelperCommand (hint.go:268-279) already covers both paths, and versioncheck.isExemptCommand covers the rest of this list. Reuse them instead of a parallel list. That also replaces the hard-coded "__complete" strings with cobra's constants.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f. The command list is gone from Decide; it takes a SkippedCommand bool. skipsAutoInstall(cmd) in cmd/aitools is versioncheck.IsExemptCommand || agents.IsCredentialHelperCommand || in an aitools tree. Both helpers are now exported, so auth docker token is covered and the cobra constants are used. Test: TestSkipsAutoInstall.

Comment thread cmd/aitools/autoinstall_run.go Outdated
Command: path,
CacheDisabled: autoinstall.CacheDisabled(ctx),
Version: version,
CI: versioncheck.InCI(ctx),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] versioncheck.InCI is env.GetBool(ctx, "CI"). It misses CI=<provider name>, which GetBool maps to false. It also misses providers that don't set CI, such as Azure Pipelines (TF_BUILD), Jenkins and CodeBuild, which useragent.CiCdProvider() detects. libs/aitools/agents/hint.go:253-266 (inCI) handles both cases, and its comment explains why GetBool can't be used. Such build agents are treated as user machines: the gate lookup today, and installs once the gate is on. Reuse inCI.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f. It now uses agents.InCI (exported), so CI=<provider> and SDK-detected providers such as TF_BUILD count. versioncheck.InCI is reverted, so notice.go only has the IsExemptCommand export now.

Comment thread cmd/aitools/autoinstall_run.go Outdated
}
// Skills without install state have no checksums to tell the user's edits
// from ours, and an install would replace them.
if installer.HasUntrackedSkills(ctx, installer.ScopeGlobal) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] InstallSkillsForAgents writes skill directories one by one and saves .state.json only after the loop (installer.go:478). A first attempt that hits the 10 s installTimeout partway leaves skills on disk with no state, and counts as FAILED. On the next run, HasUntrackedSkills is true, so planTools takes the keep path and the run settles INSTALLED. Every later version takes the same path. The machine keeps a partial skill set forever and reports success. Tell the CLI's own interrupted install apart from a legacy one (for example, save state incrementally, or write a marker before the first skill), and retry it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f with a marker. Before a fresh install, the state gets installing: true, which only settling clears. planTools takes the keep path for untracked skills only when that marker is not set, so an install cut short by the timeout is finished (and retried) on a later run, while hand-made or legacy skills are still left alone. Tests: TestPlanToolsKeepsSkillsWithoutInstallState (second half) and TestRecordClearsInstallingWhenSettled; the retries golden shows the marker.

Comment thread cmd/aitools/autoinstall_run.go Outdated
return installResult{}, err
}
result := installResult{updated: true}
if after != nil && toolsChanged(before, after) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] With KeepModified, UpdateSkills still advances state.Release to the new tag (update.go:225) even when every outdated skill was kept. toolsChanged then sees a different Release, and the notice says the tools were updated when no file changed. It also lists every agent with a global install, not just the ones that changed. Compare skills and plugin versions only, or have UpdateSkills report whether anything changed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f. toolsChanged became changedAgents, which ignores Release and names an agent only if it has skills and a skill version changed, or if its own plugin version changed. With no changed agent, the run reports UP_TO_DATE and prints nothing. Test: TestChangedAgents.

// explicit choice matters more than a broken record.
func SetOptedOut(ctx context.Context, optedOut bool) error {
s, err := Load(ctx)
unlock, err := lock(ctx, stateLockFileName, stateLockWait)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] SetOptedOut now requires the state lock. On a home directory whose filesystem rejects flock (some NFS/FUSE mounts), lock fails at once, so databricks aitools auto-install disable exits 1. In the base it just wrote the file. Auto-install can't run there either, because Load fails. Consider letting the explicit opt-out write without the lock when locking is unsupported (not merely held).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 879027f. SetOptedOut still returns errLockHeld when another process holds the lock. On any other lock error it writes without the lock (logged at debug), since auto-install cannot run there anyway. Test: TestSetOptedOutWithoutLocks.

Comment thread cmd/aitools/autoinstall_run.go Outdated
}
// Commands and builds that never auto-install skip before the state is
// read, and without a log line: they are most runs.
if reason := autoinstall.Decide(in); reason == autoinstall.ReasonSkippedCommand || reason == autoinstall.ReasonDevelopmentBuild {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] The early return covers only skipped commands and development builds. With the cache disabled, the run still calls Load, which creates ~/.databricks/aitools/ and auto-install.json.lock. Add ReasonCacheDisabled to the early return.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f. The early return now covers skipped command, Databricks Runtime, cache disabled and development build (stateFreeSkips), so none of them touches ~/.databricks/aitools/.

state, err := autoinstall.Load(ctx)
require.NoError(t, err)
assert.True(t, state.OptedOut)
assert.True(t, autoinstall.Declined())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] This test leaves the package-global autoinstall.declined flag set, and the agents package can't reset it. Separately, TestChildEnvCarriesTheRefusal (libs/aitools/autoinstall/run_test.go:102) fails when the developer has DATABRICKS_AITOOLS_AUTO_INSTALL=false set, because ChildEnv reads os.Environ(). Either export a test reset or pass the environ in, and clear EnvVar in that test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f. Added autoinstall.ResetDeclined(), which TestRecommendSkillsInstallInteractiveDecline and the new apps test pass to t.Cleanup. ChildEnv takes the environ, so TestChildEnvCarriesTheRefusal no longer reads the developer's environment.

Comment on lines +28 to +30
$CLI completion bash > /dev/null 2>&1
$CLI __complete jobs > /dev/null 2>&1
$CLI help jobs > /dev/null 2>&1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] .agents/rules/testing.md asks to route noisy output to LOG.<name> rather than /dev/null, and to capture a tolerated non-zero exit with errcode rather than || true. cursor_skills in script.prepare:63 uses || true; a [ -d ... ] check, as auto_install_state already does, avoids it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f. cursor_skills checks [ -d ] instead of using || true, and the completion/__complete/help runs in skipped/script go to LOG.* instead of /dev/null.

First of three changes that install Databricks AI tools automatically on
the first run of a new CLI version. This one stores the state and the
opt-out and adds the remote gate; nothing installs yet.

- ~/.databricks/aitools/auto-install.json holds last_version, attempts
  and opted_out. It is a dedicated file, not a libs/cache entry, so an
  opt-out survives upgrades and `databricks cache clear`.
- `databricks aitools auto-install disable|enable` toggles the opt-out;
  enable warns when DATABRICKS_AITOOLS_AUTO_INSTALL=false is set.
- A global-scope `databricks aitools uninstall` (including --skills and
  --agents) opts out.
- cli-compat.json entries get an optional auto_install field, resolved by
  clicompat.ResolveAutoInstall. No entry sets it, and
  TestManifestLeavesAutoInstallOff keeps it off.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107
anton-107 force-pushed the aitools/auto-install-state-opt-out branch from 258418b to c5eef9c Compare November 17, 2026 10:30
After a command succeeds, the root PersistentPostRunE installs (or
updates) Databricks AI tools for the coding agents on the machine, once
per new CLI version. It ships off: no cli-compat entry sets the gate.

- autoinstall.Decide is a pure decision over the command, version, state,
  CI, opt-out, gate and holdout; skipped commands and development builds
  return before any state read.
- A fresh install runs as `aitools install --scope=global` without
  prompts for agents whose config dir exists; an existing global install
  runs `aitools update --scope=global`, keeping edited skills. The run has
  a 10s budget, discards installer output and prints one notice line.
- State is updated under OS file locks that are only tried, never waited
  on beyond 1s. Attempts are retried up to 10 times per version.
- Agent CLIs run in their own process group so a timeout stops their
  descendants. Windows cannot do that, so auto-install skips plugin
  installs there.
- Answering "no" at the `apps init` skills prompt opts out and is passed
  to the CLI children `apps init` starts.
- Telemetry: aitools_install_event.trigger = AUTO_FIRST_RUN and a new
  aitools_auto_install_event.
- Refactor: `aitools update`'s RunE body moves to runUpdate,
  versioncheck.InCI is exported, and the print_telemetry acceptance
  helper moves up a directory.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107
anton-107 force-pushed the aitools/auto-install-first-run branch from 019d40c to e5662ad Compare November 17, 2026 12:29
anton-107 and others added 2 commits November 17, 2026 14:26
- A global uninstall opts out only when it removed something, so a
  mistyped --skills or an empty scope no longer turns auto-install off.
- `aitools auto-install <typo>` now fails instead of printing help and
  exiting 0.
- Document auto_install in internal/build/README.md (format, the
  copy-forward step for new entries, the trust model).
- Say that any non-true DATABRICKS_AITOOLS_AUTO_INSTALL value opts out,
  and move EnvVar next to the other constant.
- Add unit tests for the state file; pin the env var in the acceptance
  test and log the reset steps instead of discarding them.

Co-authored-by: Isaac <no-reply@databricks.com>
- apps init children keep DATABRICKS_CLI_SELF_EXEC: ChildEnv takes the
  environ and composes with agents.SelfExecEnv.
- Agent CLIs run through process.BackgroundInGroup, which forwards the first
  SIGINT/SIGTERM to the child's process group and then re-raises it, so
  Ctrl-C stops `claude plugin install` again in aitools install/update.
- Skips reuse versioncheck.IsExemptCommand, agents.IsCredentialHelperCommand
  (covers `auth docker token`) and agents.InCI (CI=<provider>, TF_BUILD...),
  and add Databricks Runtime. Cache-disabled runs no longer read the state.
- A gate lookup that only reaches the embedded manifest fails instead of
  reading as off; it uses up an attempt and does not settle the version.
- An unfinished first install is marked in the state, so its skills are
  finished on a later run instead of being kept as untracked forever.
- Auto-install updates no longer migrate or remove raw skills.
- New outcomes UP_TO_DATE, KEPT and SKIPPED_PLATFORM; the update notice
  names only the agents whose skills or plugin changed.
- Saying "no" in apps init says that auto-install is off too.
- The explicit opt-out writes without the lock where locks are unsupported.
- Tests: ResetDeclined, environ passed in; acceptance helpers use LOG.* and
  no `|| true`.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107

Copy link
Copy Markdown
Contributor Author

Thanks for the review. All 14 threads are addressed in 879027f; there's a reply on each one. The PR description is updated too.

Re question 1 [blocking], the skips:

  • Databricks Runtime: now skipped (dbr.RunsOnRuntime). It is checked before the state is read, and nothing is reported. Your reasoning holds: the home doesn't outlive the cluster, and the platform owns the CLI version.
  • CI: broadened to agents.InCI (thread above), so CI=<provider>, Azure Pipelines, Jenkins and CodeBuild are covered.
  • Non-interactive and -o json: intentionally not skipped. Coding agents run the CLI that way, without a TTY and often with -o json, and they are the audience for the tools. Skipping those runs would leave only humans at a terminal, and aitools: ask before auto-installing AI tools in a terminal #7703 asks them before installing. Auto-install never writes to stdout, so JSON output stays clean. This is now a comment on skipsAutoInstall.
  • Lookup cost when the manifest can't be fetched: the lookup no longer settles the version. It uses one of the version's 10 attempts, so an offline machine, container or fresh home pays the 3 s lookup on at most 10 runs, not on every command.

Once the gate is on, the cost after a script's first command of a version is still there: up to about 20 s, plus retries. I've left that as is, because a script can't be told apart from an agent here. DATABRICKS_AITOOLS_AUTO_INSTALL=false turns it off, and so does CI detection. If you'd rather drop -o json runs anyway, I can add that.

Also: the lumberjack proto needs the new outcomes UP_TO_DATE, KEPT and SKIPPED_PLATFORM.

…nstall-first-run

Co-authored-by: Isaac <no-reply@databricks.com>
Base automatically changed from aitools/auto-install-state-opt-out to main November 17, 2026 14:59
…ests

agents.InCI also detects providers such as GITHUB_ACTIONS, so on CI runners
every case reported SKIPPED_CI.

Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions github-actions Bot added the aitools AI tools (databricks aitools) related changes label Nov 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aitools AI tools (databricks aitools) related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants