Skip to content

acc: share the CI env unset in an unset_ci_env helper - #7691

Open
anton-107 wants to merge 1 commit into
mainfrom
aitools/claude-code-native-hint-ci-env
Open

anton-107 wants to merge 1 commit into
mainfrom
aitools/claude-code-native-hint-ci-env

Conversation

@anton-107

Copy link
Copy Markdown
Contributor

Changes

Adds an unset_ci_env helper to acceptance/script.prepare. It replaces the copies of the CI and CI/CD provider unset line in cmd/aitools/claude-config-dir, cmd/aitools/telemetry/hint and air/image-push. The helper's comment points at listCiCdProviders in the Go SDK's useragent/cicd.go so the list stays in sync. The helper is also listed in the built-in helpers section of .agents/rules/testing.md.

Stacked on #7585.

Why

Addresses simonfaltum's review comment on #7585 about the duplicated line. Only the root script.prepare covers both cmd/aitools and air, so the helper lives there. The Go-side copy (hintEnvVars in libs/aitools/agents/hint_test.go) is left as is.

Tests

  • go test ./acceptance -run 'TestAccept/cmd/aitools|TestAccept/air' passes with no golden changes, both in a clean env and with CI=true GITHUB_ACTIONS=true exported. The native hint lines still match.
  • Negative control: with the helper turned into a no-op and the CI vars exported, all three tests fail.
  • ./task ws and ./task checks pass.

This pull request and its description were written by Isaac.

@anton-107
anton-107 requested review from a team as code owners November 16, 2026 16:03
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 193c48085 | Workflow run

Regression test report

Selected 3 changed tests (limit=20, 0 not selected)

3 tests: regression=0 unreleased=3 coverage=0 failing=0
test branch main (4c8b7a1) latest (v1.26.0)
TestAccept/air/image-push ✅ ✅ ❌
TestAccept/cmd/aitools/claude-config-dir ✅ ✅ ❌
TestAccept/cmd/aitools/telemetry/hint ✅ ✅ ❌
TestAccept/air/image-push: branch ✅ | main (4c8b7a1) ✅ | latest (v1.26.0) ❌

latest (v1.26.0):

acceptance_test.go:1380: Diff:
        --- air/image-push/output.txt
        +++ /tmp/TestAcceptairimage-push1283609857/002/output.txt
        @@ -150,18 +150,18 @@
           databricks air images push [flags]
         
         Flags:
        -      --artifact string     Destination ARTIFACT[:TAG]; defaults to the source image name and tag (latest if untagged); maximum 255 characters for catalog.schema.artifact and 128 for the tag
        -      --catalog string      Destination Unity Catalog catalog; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        -  -h, --help[=true|false]   help for push
        -      --pull[=true|false]   Pull the source image even when it is already available locally
        -      --schema string       Destination Unity Catalog schema; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        -  -s, --source string       Source container image to push (NAME[:TAG] or NAME@DIGEST)
        +      --artifact string   Destination ARTIFACT[:TAG]; defaults to the source image name and tag (latest if untagged); maximum 255 characters for catalog.schema.artifact and 128 for the tag
        +      --catalog string    Destination Unity Catalog catalog; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        +  -h, --help              help for push
        +      --pull              Pull the source image even when it is already available locally
        +      --schema string     Destination Unity Catalog schema; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        +  -s, --source string     Source container image to push (NAME[:TAG] or NAME@DIGEST)
         
         Global Flags:
        -      --debug[=true|false]   enable debug logging
        -  -o, --output type          output type: text or json (default text)
        -  -p, --profile string       ~/.databrickscfg profile
        -  -t, --target string        bundle target to use (if applicable)
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
         
         
         >>> musterr databricks air images push --image artifact
        @@ -171,18 +169,18 @@
           databricks air images push [flags]
         
         Flags:
        -      --artifact string     Destination ARTIFACT[:TAG]; defaults to the source image name and tag (latest if untagged); maximum 255 characters for catalog.schema.artifact and 128 for the tag
        -      --catalog string      Destination Unity Catalog catalog; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        -  -h, --help[=true|false]   help for push
        -      --pull[=true|false]   Pull the source image even when it is already available locally
        -      --schema string       Destination Unity Catalog schema; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        -  -s, --source string       Source container image to push (NAME[:TAG] or NAME@DIGEST)
        +      --artifact string   Destination ARTIFACT[:TAG]; defaults to the source image name and tag (latest if untagged); maximum 255 characters for catalog.schema.artifact and 128 for the tag
        +      --catalog string    Destination Unity Catalog catalog; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        +  -h, --help              help for push
        +      --pull              Pull the source image even when it is already available locally
        +      --schema string     Destination Unity Catalog schema; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        +  -s, --source string     Source container image to push (NAME[:TAG] or NAME@DIGEST)
         
         Global Flags:
        -      --debug[=true|false]   enable debug logging
        -  -o, --output type          output type: text or json (default text)
        -  -p, --profile string       ~/.databrickscfg profile
        -  -t, --target string        bundle target to use (if applicable)
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
         
         
         === reject oversized catalog.schema.artifact before Docker operations
        @@ -217,10 +213,10 @@
         === in a Claude Code session the auth docker configure child adds no install hint
         
         >>> databricks air images push -p docker-test -s example/image:v1 --catalog main --schema training
        -<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
         Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         Warning: This feature is in Preview. APIs may change, and the workspace must enable the Preview features.
         Pushing example/image:v1 as main.training.image:v1 using profile docker-test
        +Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         Configured Docker credential helper for [WORKSPACE_ID].container.us-west-2.localhost
         Updated Docker config: [TEST_TMP_DIR]/home/docker/config.json
         Installed Docker credential helper: [TEST_TMP_DIR]/home/bin/docker-credential-databricks
        @@ -232,8 +225,8 @@
         Now you can set `environment.unity_catalog_image = main.training.image:v1` in your YAML config and create a workload with `databricks air run -f config.yaml`.
         
         >>> musterr databricks air images push -p pat-test -s example/image:v1 --catalog main --schema training
        -<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
         Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         Warning: This feature is in Preview. APIs may change, and the workspace must enable the Preview features.
         Pushing example/image:v1 as main.training.image:v1 using profile pat-test
        -Error: configure Docker authentication: run auth docker configure: exit status 1: Error: profile "pat-test" uses auth_type "pat". Docker credential helper requires a profile created by databricks auth login
        +Error: configure Docker authentication: run auth docker configure: exit status 1: Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
        +Error: profile "pat-test" uses auth_type "pat". Docker credential helper requires a profile created by databricks auth login
        
FAIL
TestAccept/cmd/aitools/claude-config-dir: branch ✅ | main (4c8b7a1) ✅ | latest (v1.26.0) ❌

latest (v1.26.0):

acceptance_test.go:1380: Diff:
        --- cmd/aitools/claude-config-dir/output.txt
        +++ /tmp/TestAcceptcmdaitoolsclaude-config-dir2735723887/002/output.txt
        @@ -9,7 +9,6 @@
         === Custom directory takes precedence over the default installation
         
         >>> CLAUDE_CONFIG_DIR=missing [CLI] clusters list
        -<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
         Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         ID    Name                State
         4321  some-test-cluster   
        @@ -18,6 +17,7 @@
         === Exempt commands never hint, even without the tooling
         
         >>> CLAUDE_CONFIG_DIR=missing [CLI] current-user me
        +Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         json.userName = "[USERNAME]";
         
         === Empty override uses the default installation
        @@ -45,7 +45,6 @@
         === Within one session both lines print once, so 2>&1 pipelines see them only on the first command
         
         >>> [CLI] clusters list
        -<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
         Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         ID    Name                State
         4321  some-test-cluster
TestAccept/cmd/aitools/telemetry/hint: branch ✅ | main (4c8b7a1) ✅ | latest (v1.26.0) ❌

latest (v1.26.0):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:38157" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:38157" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1380
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1121
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:666
        	Error:      	Not equal: 
        	            	expected: "\n=== hint printed on an authenticated command: one hint-shown event per printed line is uploaded\n\n>>> [CLI] jobs list\n<claude-code-hint v=\"1\" type=\"plugin\" value=\"databricks@claude-plugins-official\" />\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n{\"command\":\"jobs_list\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"CLAUDE_CODE_NATIVE\"}}\n{\"command\":\"jobs_list\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"INSTALL_TEXT\"}}\n\n=== hint printed on a command that resolves no auth: nothing is uploaded\n\n>>> [CLI] bundle schema\n<claude-code-hint v=\"1\" type=\"plugin\" value=\"databricks@claude-plugins-official\" />\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no event\n\n>>> [CLI] jobs list\n\n>>> print_telemetry\n"
        	            	actual  : "\n=== hint printed on an authenticated command: one hint-shown event per printed line is uploaded\n\n>>> [CLI] jobs list\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n\n=== hint printed on a command that resolves no auth: nothing is uploaded\n\n>>> [CLI] bundle schema\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no event\n\n>>> [CLI] jobs list\n\n>>> print_telemetry\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,3 +4,2 @@
        	            	 >>> [CLI] jobs list
        	            	-<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
        	            	 Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
        	            	@@ -8,4 +7,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"jobs_list","exit_code":0,"aitools_hint_event":{"hint_kind":"CLAUDE_CODE_NATIVE"}}
        	            	-{"command":"jobs_list","exit_code":0,"aitools_hint_event":{"hint_kind":"INSTALL_TEXT"}}
        	            	 
        	            	@@ -14,3 +11,2 @@
        	            	 >>> [CLI] bundle schema
        	            	-<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
        	            	 Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
        	Test:       	TestAccept/cmd/aitools/telemetry/hint
        	Messages:   	cmd/aitools/telemetry/hint/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryhint2768237970/002/output.txt

@anton-107 anton-107 added aitools AI tools (databricks aitools) related changes ai-auto PRs made by AI loops & factories, labels Nov 16, 2026
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 193c480

Run: 44072493989

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 18 5:50
💚​ aws-windows-latest-4core-16gb 1 272 16 4:23
💚​ azure linux-2core-8gb 1 269 18 6:22
💚​ azure-windows-latest-4core-16gb 1 271 16 5:17
💚​ gcp linux-2core-8gb 1 270 18 6:18
💚​ gcp-windows-latest-4core-16gb 1 272 16 5:00
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:44 azure-windows-latest-4core-16gb TestAccept
3:14 gcp-windows-latest-4core-16gb TestAccept
2:59 aws-windows-latest-4core-16gb TestAccept

Base automatically changed from aitools/claude-code-native-hint to main November 16, 2026 16:40
@anton-107
anton-107 requested review from a team and lennartkats-db as code owners November 16, 2026 16:40

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at 0486b26. Each finding was verified against the code; simonfaltum may follow up.

Not approving yet: 1 blocking issue(s), tagged [blocking].

This PR moves the CI / CI/CD-provider unset line, which was copied into three acceptance scripts, into a shared unset_ci_env helper in the root script.prepare. The helper commit itself is correct and follows the repo's conventions: its variable list matches listCiCdProviders in the pinned SDK, and the root prepare file is the right home for it. However, the branch still contains an older, pre-review revision of #7585 rather than the merged one, so it conflicts with main and its diff drags in stale #7585 code. Resolving those conflicts toward the branch would revert merged fixes, such as accepting CI=no/off. Please rebase so that only the helper commit sits on main, and refresh the description; note that #7692 edits the same three scripts.

Comment thread libs/aitools/agents/hint_test.go Outdated
"github.com/stretchr/testify/require"
)

// hintEnvVars are the non-detection env vars MaybeHint reads. They are cleared

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] This branch is built on 07a831d, an older revision of #7585 from before its review feedback, not on the squash-merged #7585 (10fe968). GitHub reports the PR as not mergeable, with conflicts in six files: .nextchanges/cli/claude-code-native-hint.md, the three acceptance scripts, hint.go and hint_test.go. The PR diff also shows 21 files instead of the 5 the helper touches. If a conflict is resolved toward this branch, merged #7585 fixes come back undone. For example, inCI would go back to strconv.ParseBool, so CI=no or CI=off would count as CI again and drop the native hint. Also returning: hintEnvVars/unsetEnv (main now relies on testutil.SetAgentEnv clearing the env), the inline closure in place of the shared noRecordedPlugin, the extra plugin-manifest read, and the older .nextchanges entry. The description is also stale: #7585 is already merged, and main has no hintEnvVars, so after a rebase unset_ci_env is the only copy of the list. Please rebase with git rebase --onto origin/main 07a831d8a, which keeps only 0486b26 (it applies cleanly), and update the description.

Three acceptance scripts unset the same list of CI and CI/CD provider
variables so the native Claude Code hint line also prints in CI. Move the
list into one unset_ci_env helper in the root script.prepare, the only
prepare file that covers both cmd/aitools and air, with a pointer to the
SDK's useragent/cicd.go so it stays in sync.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107
anton-107 force-pushed the aitools/claude-code-native-hint-ci-env branch from 0486b26 to 193c480 Compare November 17, 2026 11:15
@github-actions github-actions Bot added the AIR Databricks AI Runtime CLI label Nov 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-auto PRs made by AI loops & factories, AIR Databricks AI Runtime CLI aitools AI tools (databricks aitools) related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants