Repository navigation
refactor: read Python's MSIs from the installer, drop vendored WiX - #61
Merged
Merged
Conversation
The Windows install path extracted the bundle's MSI packages with a vendored copy of WiX 3.11 (dark.exe). The copy was 17 binary files, about 8.9 MiB uncompressed, and the only binary content in the repository. Ask the Python bootstrapper to write its packages to a local layout with `/quiet /layout <dir>` instead. The MSIs are installed from that layout as before, and the layout is removed afterwards. The vendored tool, the bin/ directory and its entry in the CI plugin archive are gone. The layout may also download components that are not attached to the bundle. Every MSI in it is installed except the four that were already skipped (appendpath, launcher, path, pip). Unit tests: tests/windows_install_test.lua and tests/hooks_test.lua pass with gopher-lua v1.1.1. The Windows installer job in CI is the real check for the layout behaviour.
linghengqian
marked this pull request as draft
November 13, 2026 17:48
main moved the Windows EXE path onto its own dark.exe error handling and split CI into test, lint, license and e2e workflows. Keep that structure and port this branch's change onto it: - The EXE route writes the bundle's MSIs with `/quiet /layout` instead of running dark.exe, and removes the layout after the MSIs are installed. The WiX tools and bin/ are gone, so the dark.exe quarantine check goes with them. - windows_install_test.lua is removed on main; its layout checks move to windows_install_layouts.lua. The cleanup adds one command, so the alias failure case moves from call 6 to call 7. - test.yml's Windows job packages metadata.lua, hooks and lib only. - The e2e plugin archive, the license ignore list and the docs no longer mention bin/WiX. Checked locally with the same tools as CI: stylua --check ., LuaLS --checklevel=Warning, actionlint, vfox 1.1.0 plugin test (30 passed) and the GopherLua layouts test.
Burn expands %VARIABLES% in the /layout argument (core.cpp parses it with PATH_EXPAND_ENVIRONMENT; /log is not expanded). The CI install path contains %USERNAME%, so the layout was written under the expanded name, and the MSI listing then looked at the literal path and failed. The E2E installer cases fail the same way with a plain path, so the layout step now also passes /log and prints the end of Burn's log when the layout or the MSI listing fails. The layout moves to TEMP, which has no percent signs, and is removed with its log after the MSIs are installed.
The CI log for the layout shows Burn v3.14.1 starting a clean-room child (-burn.clean.room=...) about 1.5 seconds after the bundle process returned. The layout was still running when the MSI listing looked for it, so the listing found nothing. The E2E "Access is denied" on removing the installer is the same child still holding the original bundle. windowsCommand.installerLayout runs the installer, then waits for the child that carries the layout's unique name on its command line, and returns the child's exit code. The layout stays one command, so the call order in the GopherLua tests does not change.
Burn's /layout does not split out the packages attached to the bundle. The copied EXE keeps them in its attached cabinet, and the layout only downloads the packages that are not attached: debug, symbol, free-threaded and UCRT packages that the default install does not use. The CI log showed exactly that, so the layout could not supply the MSIs the install needs. The installer is now read directly. windowsCommand.bundlePackages locates the .wixburn section, unpacks the UX cabinet to read the bundle manifest, unpacks the attached package cabinet with expand.exe (built into Windows), and copies each MSI the manifest names under its own name. The MSIs installed are the same as with the WiX extraction: the attached set minus appendpath, launcher, path and pip. No vendored tools and no extra downloads. windowsCommand's base64 now builds its output in chunks: GopherLua runs out of registry space when one table holds a whole long PowerShell script. Checked locally: the generated script runs under PowerShell 7.7 against the python-3.14.5 installer, with expand.exe replaced by a stand-in that decodes the same cabinets. The 11 MSIs it writes match the SHA1 values in the bundle manifest. The Windows CI job is still the real check for expand.exe.
Burn's /layout copies the bundle with its attached cabinet inside and downloads only the packages that are not attached: debug, symbol, free-threaded and UCRT packages the default install does not use. The packages the install needs were never split out, so the layout could not supply them. The installer is read directly now. lib/bundle.lua finds the .wixburn section, the attached containers and the bundle manifest. extractBundlePackages writes the UX cabinet and the attached package cabinet to the scratch folder and unpacks each with expand.exe, which ships with Windows. The MSIs are renamed to the names the manifest gives them. The installed set is the same as with the WiX extraction: the attached MSIs minus appendpath, launcher, path and pip. No vendored tools and no extra downloads. PowerShell commands stay short: the earlier script exceeded cmd's 8191 character limit once base64 encoded. windowsCommand's base64 also builds its output in chunks, since GopherLua runs out of registry space on long input. Checked locally: the production code, run by GopherLua against the python-3.14.5 installer, unpacks the 11 MSIs whose SHA1 values match the bundle manifest. pwsh 7.7 and a stand-in for expand.exe ran the commands. The Windows CI job is still the real check for expand.exe.
linghengqian
changed the title
linghengqian
marked this pull request as ready for review
November 14, 2026 00:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
bin/WiX/vendors WiX 3.11'sdark.exeand its extension DLLs: 17 files, about 8.9 MiB uncompressed and 2.9 MiB compressed in git. They are the only binary content in the repository. The tool only extracted the MSI packages from the python.org installer, which is a WiX Burn bundle.What changed
lib/bundle.lua(new) reads the.wixburnsection of the installer, its attached cabinets and the bundle manifest. It follows WiX v3's Burn container format (src/burn/engine/section.cpp,container.cpp).extractBundlePackages(lib/util.lua) writes the UX cabinet and the attached package cabinet underTEMP, unpacks each withexpand.exe(built into Windows), and names the MSIs as the manifest does.lib/windows_command.lua:expandandmakeDirectoryhelpers. Base64 now builds its output in chunks, because GopherLua runs out of registry space on long input.bin/WiX/is removed. The Windows CI plugin archive and the e2e plugin archive no longer includebin, and the license ignore entry for it is gone.tests/windows_install_layouts.luacovers the install flow (with the packages step stubbed) and the bundle parsing on a synthetic installer.Behaviour
appendpath,launcher,pathandpip./layout, which downloads the non-attached debug, symbol, free-threaded and UCRT packages and does not split out the attached ones. The commit history shows that attempt; the final approach reads the attached cabinet directly.Verification
8e8507d, includingWindows installer paths(a real install of CPython 3.14.5 through the plugin on Windows) andE2E on Windows.--checklevel=Warning), actionlint, vfox 1.1.0plugin test .(30 passed), the GopherLua layouts test, and the production extraction code run by GopherLua against the python-3.14.5 installer. The 11 MSIs it writes match the SHA1 values in the bundle manifest.