[pawank925]

HtmlRewriter feeds HTMLParser with convert_charrefs=False, so that entity and character references are re-emitted verbatim in the rewritten content. That hands the text of a <title> over in several calls, and handle_data kept only the first piece:

<title>	before	after
Tom &amp; Jerry	Tom	Tom & Jerry
Rock &amp; Roll &#8211; My Blog	Rock	Rock & Roll – My Blog
L&#039;Illustre Maurin	L	L'Illustre Maurin

warc2zim uses this value as the ZIM entry title (Rewriter.rewrite_html()), so a title with an ampersand or an escaped apostrophe ends up truncated in title search and suggestions.

The pieces are now collected as they arrive, with references decoded, and joined when the title element closes. Nothing is added or changed in the output stream, so content is exactly what it was; only the extracted title changes.

This keeps both behaviours the current implementation has, which is why I went for it rather than the two BeautifulSoup variants discussed in the issue (a second parse of the document costs ~15% on the rewrite, per the measurements in #401):

• the first <title> which has text still wins: with an empty <title></title> in head, the title of a later inline SVG is used;
• whitespace around the title is still stripped, and only around it;
• a nested tag in a title still cuts it, since html_rewrite_context is per-tag and I did not touch that.

One case changes beyond the bug itself: a document whose title element is never closed (<title>Unclosed &amp; title) yielded its first piece, and now yields its whole text, so _save_title() also runs at close().

Tests cover named (é), decimal (&, '), hex (') and bare-& references, whitespace stripping, a second title in body, an empty head title followed by an inline SVG, an unclosed title, and that the references survive in the rewritten content.

ruff check / ruff format and pyright (strict) are clean, and tests/rewriting passes (635).

Fixes #401

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (d1deb81) to head (ba82863).

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #402   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           42        42           
  Lines         2947      2961   +14     
  Branches       452       454    +2     
=========================================
+ Hits          2947      2961   +14     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[benoit74]

@pawank925 I appreciate you are working hard to fix things, but opening a PR on an issue which has been reported by someone which is only waiting for confirmation on how we want to implement the fix, did the hard work of reporting the issue cleanly, analyzing it and already proposed to push a PR is rude.

It is the second time I warn you about this is not the way opensource contributions works (at least at openZIM / Kiwix). I will now have to be more strict with you: please do not push a PR (and ideally do not work on an issue, but this is your call) before the issue is assigned to you.

I will be very strict with this rule from now and close any PR which does not follow this rule.