Skip to content

load_web_page returns an empty string for Chinese and Japanese pages #7956

Description

@Fruiterer333

Describe the Bug:

load_web_page drops every line of a page written in Chinese or Japanese, so the tool returns an empty string and the agent gets no content.

It keeps only lines with more than three words, and counts words with len(line.split()) (src/google/adk/tools/load_web_page.py). Chinese and Japanese text has no spaces between words, so a whole line counts as one word and is filtered out. Thai, Lao, Khmer and Myanmar are written without spaces too.

Steps to Reproduce:

  1. Install google-adk 2.13.0 (also reproduced on main @ 3565ad3) with beautifulsoup4 and lxml.
  2. Run the script under "Minimal Reproduction Code". It replaces _fetch_content with the same small page in three languages, so no network access is needed.

Expected Behavior:

The paragraphs of the Chinese and Japanese pages are returned, as they are for the English page. Only the one-word navigation item is dropped.

Observed Behavior:

--- English: 136 characters returned
2024 Model Y price
The 2024 Model Y starts at 249,900 yuan after subsidies.
Deliveries for orders placed this month begin in four weeks.
--- Chinese: 0 characters returned
(empty)
--- Japanese: 0 characters returned
(empty)

Environment Details:

  • ADK Library Version (pip show google-adk): 2.13.0 (also on main @ 3565ad3)
  • Desktop OS: macOS 26.6.2
  • Python Version (python -V): 3.12.7

Model Information:

  • Are you using LiteLLM: N/A (no model call needed to reproduce)
  • Which model is being used: N/A

Minimal Reproduction Code:

"""Repro: load_web_page drops every line of a Chinese or Japanese page."""

from unittest import mock

from google.adk.tools import load_web_page as load_web_page_module

PAGES = {
    "English": """<html><body>
<nav>Home</nav>
<h1>2024 Model Y price</h1>
<p>The 2024 Model Y starts at 249,900 yuan after subsidies.</p>
<p>Deliveries for orders placed this month begin in four weeks.</p>
</body></html>""",
    "Chinese": """<html><body>
<nav>首页</nav>
<h1>2024款Model Y价格</h1>
<p>2024款Model Y补贴后起售价为24.99万元。</p>
<p>本月下单的车辆预计四周后开始交付。</p>
</body></html>""",
    "Japanese": """<html><body>
<nav>ホーム</nav>
<h1>2024年モデルYの価格</h1>
<p>2024年モデルYの補助金適用後の価格は249万円からです。</p>
<p>今月の注文分は4週間後に納車が始まります。</p>
</body></html>""",
}

for language, html in PAGES.items():
    with mock.patch.object(
        load_web_page_module, "_fetch_content", return_value=html.encode()
    ):
        text = load_web_page_module.load_web_page("https://example.com/")
    print(f"--- {language}: {len(text)} characters returned")
    print(text or "(empty)")

Additional Context:

I'm opening a PR with a fix and regression tests.

How often has this issue occurred?:

  • Always (100%)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions