Skip to content

perf: share repeated source message IDs when reading PO catalogs - #1395

Open
SAY-5 wants to merge 1 commit into
python-babel:masterfrom
SAY-5:fix/po-message-id-sharing-20261018
Open

SAY-5 wants to merge 1 commit into
python-babel:masterfrom
SAY-5:fix/po-message-id-sharing-20261018

Conversation

@SAY-5

@SAY-5 SAY-5 commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

read_po (and PoFileParser) now take an optional shared_msgids dict. Identical message IDs within one file always share a string object, and passing the same dict to several read_po calls shares them between catalogs too, which is what #571 is after, without sys.intern or any free-threading check.

Refs #571.

@akx akx left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of using sys.intern, what about just having a temporary dict of these strings within the parser? That would work for free-threaded builds too.

@SAY-5

SAY-5 commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

Switched to a dict on the parser, dropped sys.intern and the build check, and cut the tests down to a single case in test_pofile_read.py. It only dedups within one read_po call now (e.g. the same msgid under different contexts), so separately loaded catalogs no longer share IDs.

@akx

akx commented Oct 30, 2026

Copy link
Copy Markdown
Member

Hmmmn, sorry, I had forgotten to actually read the context of #571 before suggesting the per-parser dict. I'd just rather not like to have to probe around for whether we're doing free-threading...

@codecov

codecov Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.51%. Comparing base (9a3ef47) to head (4da5393).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1395   +/-   ##
=======================================
  Coverage   93.51%   93.51%           
=======================================
  Files          27       27           
  Lines        4932     4936    +4     
=======================================
+ Hits         4612     4616    +4     
  Misses        320      320           
Flag Coverage Δ
macos-14-3.10 92.60% <100.00%> (+<0.01%) ⬆️
macos-14-3.11 92.54% <100.00%> (+<0.01%) ⬆️
macos-14-3.12 92.74% <100.00%> (+<0.01%) ⬆️
macos-14-3.13 92.74% <100.00%> (+<0.01%) ⬆️
macos-14-3.14 92.72% <100.00%> (+<0.01%) ⬆️
macos-14-3.15 92.72% <100.00%> (+<0.01%) ⬆️
macos-14-3.8 92.47% <100.00%> (+<0.01%) ⬆️
macos-14-3.9 92.54% <100.00%> (+<0.01%) ⬆️
macos-14-pypy3.10 92.60% <100.00%> (+<0.01%) ⬆️
ubuntu-24.04-3.10 92.62% <100.00%> (+<0.01%) ⬆️
ubuntu-24.04-3.11 92.56% <100.00%> (+<0.01%) ⬆️
ubuntu-24.04-3.12 92.76% <100.00%> (+<0.01%) ⬆️
ubuntu-24.04-3.13 92.76% <100.00%> (+<0.01%) ⬆️
ubuntu-24.04-3.14 92.74% <100.00%> (+<0.01%) ⬆️
ubuntu-24.04-3.15 92.74% <100.00%> (+<0.01%) ⬆️
ubuntu-24.04-3.8 92.49% <100.00%> (+<0.01%) ⬆️
ubuntu-24.04-3.9 92.56% <100.00%> (+<0.01%) ⬆️
ubuntu-24.04-pypy3.10 92.62% <100.00%> (+<0.01%) ⬆️
windows-2022-3.10 92.61% <100.00%> (+<0.01%) ⬆️
windows-2022-3.11 92.55% <100.00%> (+<0.01%) ⬆️
windows-2022-3.12 92.76% <100.00%> (+<0.01%) ⬆️
windows-2022-3.13 92.76% <100.00%> (+<0.01%) ⬆️
windows-2022-3.14 92.73% <100.00%> (+<0.01%) ⬆️
windows-2022-3.15 92.73% <100.00%> (+<0.01%) ⬆️
windows-2022-3.8 92.59% <100.00%> (+<0.01%) ⬆️
windows-2022-3.9 92.55% <100.00%> (+<0.01%) ⬆️
windows-2022-pypy3.10 92.61% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed

codspeed Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 88 untouched benchmarks


Comparing SAY-5:fix/po-message-id-sharing-20261018 (79b1b5a) with master (3072a70)1

Open in CodSpeed

Footnotes

  1. No successful run was found on master (19d490d) during the generation of this report, so 3072a70 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

Signed-off-by: Sai Asish Y <say.apm35@gmail.com>
@SAY-5
SAY-5 force-pushed the fix/po-message-id-sharing-20261018 branch from 4da5393 to 79b1b5a Compare November 12, 2026 23:27
@SAY-5

SAY-5 commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

No worries. read_po now takes an optional shared_msgids dict (the parser still uses its own one by default), so passing the same dict for every catalog shares the IDs across them like the snippet in #571, with no sys.intern or free-threading check.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants