Skip to content

Fix hangs in off-thread WebAssembly compilation - #522

Draft
wesgarland wants to merge 1 commit into
mainfrom
wes/fix/wasm-async-compile
Draft

wesgarland wants to merge 1 commit into
mainfrom
wes/fix/wasm-async-compile

Conversation

@wesgarland

@wesgarland wesgarland commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Pythonmonkey can hang in off-thread WASM compilation. This can happen running dcp-client because of the makeSignature() code in the wallet API.

  • Deadlock with two compiles in flight. dispatchToEventLoop took the GIL on the SpiderMonkey helper thread, which holds the helper-thread lock; the JS thread, holding the GIL, waits for that lock in OffThreadPromiseTask::init when it starts another compile. It now starts a thread without the GIL, which takes the GIL and enqueues the job (or hands the task back with ReleaseFailedTask when no event-loop is running).
  • pm.wait() returns while a compile runs. Nothing counted the in-flight task, so the event-loop finished, the promise never settled, and the process hung in JS_DestroyContext at exit. The asyncTaskStarted/asyncTaskFinished callbacks now count off-thread tasks as pending jobs.

Diagnosed on 1.5.1 (same src as main) with py-spy dump --native: the main thread in OffThreadPromiseTask::init and a helper thread in JobQueue::dispatchToEventLoop waiting for the GIL; and the main thread in js::DestroyContext waiting for the helper-thread lock.

New tests tests/js/wasm-compile-alone.simple and wasm-compile-concurrent.simple both hang (timeout) on 1.5.1. Not built locally (no SpiderMonkey build here); CI is the first build.

Behaviour change: a pending Atomics.waitAsync with no timeout now also keeps pm.wait() waiting.

- dispatchToEventLoop no longer takes the GIL. SpiderMonkey calls it on a helper thread holding the
  helper-thread lock; the JS thread, holding the GIL, can be waiting for that lock (starting another
  WebAssembly.compile), so the two deadlocked. It now starts a thread, without the GIL, that takes
  the GIL and enqueues the job; with no event-loop running it hands the task back to SpiderMonkey.
- Off-thread tasks count as pending jobs (asyncTaskStarted/Finished callbacks), so pm.wait() waits
  for them. It returned while a compile ran; the promise never settled and the process hung in
  JS_DestroyContext at exit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant