Skip to content

MCP Server Part 6: Format callback results for LLM consumption#3748

Open
KoolADE85 wants to merge 9 commits intomcpfrom
feature/mcp-formatted-tool-results
Open

MCP Server Part 6: Format callback results for LLM consumption#3748
KoolADE85 wants to merge 9 commits intomcpfrom
feature/mcp-formatted-tool-results

Conversation

@KoolADE85
Copy link
Copy Markdown
Contributor

Summary

  • Add formatters that enrich callback responses with LLM-friendly content (images, markdown tables) for certain data types
  • PlotlyFigureResult — renders Graph.figure outputs as PNG images via kaleido, returned as ImageContent
  • DataFrameResult — renders DataTable.data and AgGrid.rowData outputs as markdown tables, returned as TextContent

Manual verification:

import plotly.express as px
from dash import Dash, html, dcc, Input, Output, dash_table

df = px.data.gapminder().query("year == 2007 and continent == 'Europe'")

app = Dash(__name__)
app.layout = html.Div([
    dcc.Dropdown(id="metric", options=["gdpPercap", "lifeExp", "pop"], value="gdpPercap"),
    dcc.Graph(id="chart"),
    dash_table.DataTable(id="table"),
])

@app.callback(Output("chart", "figure"), Input("metric", "value"))
def update_chart(metric):
    """Bar chart of the selected metric across European countries."""
    return px.bar(df, x="country", y=metric, title=f"{metric} in Europe (2007)")

@app.callback(Output("table", "data"), Input("metric", "value"))
def update_table(metric):
    """Top 5 countries for the selected metric."""
    return df.nlargest(5, metric)[["country", metric]].to_dict("records")

with app.server.app_context():
    from dash.mcp.primitives.tools.callback_adapter_collection import CallbackAdapterCollection
    from dash.mcp.primitives.tools.callback_utils import run_callback
    from dash.mcp.primitives.tools.results import format_callback_response

    collection = CallbackAdapterCollection(app)
    for adapter in collection:
        response = run_callback(adapter, {"metric": "lifeExp"})
        result = format_callback_response(response, adapter)
        for item in result.content:
            if item.type == "text":
                print(f"[text] {item.text[:200]}...")
            elif item.type == "image":
                print(f"[image] {item.mimeType}, {len(item.data)} chars base64")
        # update_chart: [text] JSON + [image] PNG
        # update_table: [text] JSON + [text] markdown table

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 16, 2026

Thank you for your contribution to Dash! 🎉

This PR is exempt from requiring a linked issue due to its labels.

@KoolADE85 KoolADE85 force-pushed the feature/mcp-pattern-matching-schemas branch from 9b3063f to 7451883 Compare April 21, 2026 17:27
@KoolADE85 KoolADE85 force-pushed the feature/mcp-formatted-tool-results branch from 0e54d74 to 8a41b76 Compare April 21, 2026 17:28
@KoolADE85 KoolADE85 force-pushed the feature/mcp-pattern-matching-schemas branch from 7451883 to 5fbf497 Compare April 22, 2026 21:13
@KoolADE85 KoolADE85 force-pushed the feature/mcp-formatted-tool-results branch from 8a41b76 to 4fb9d4a Compare April 22, 2026 21:31
@KoolADE85 KoolADE85 force-pushed the feature/mcp-pattern-matching-schemas branch from 5fbf497 to b564279 Compare April 23, 2026 20:04
@KoolADE85 KoolADE85 force-pushed the feature/mcp-formatted-tool-results branch from 4fb9d4a to 10a544b Compare April 23, 2026 20:15
@KoolADE85 KoolADE85 force-pushed the feature/mcp-pattern-matching-schemas branch from b564279 to e2aec3d Compare April 30, 2026 15:05
@KoolADE85 KoolADE85 force-pushed the feature/mcp-formatted-tool-results branch 3 times, most recently from 2acb39d to 04ca2fe Compare April 30, 2026 16:32
@KoolADE85 KoolADE85 changed the base branch from feature/mcp-pattern-matching-schemas to mcp May 6, 2026 21:29
@KoolADE85 KoolADE85 force-pushed the feature/mcp-formatted-tool-results branch from 04ca2fe to 8f9d5a0 Compare May 6, 2026 21:31
Copy link
Copy Markdown
Contributor

@camdecoster camdecoster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I add a few minor suggestions.

formatters are called per output property and may add additional
content items (images, markdown, etc.).
"""
content: list[Any] = [
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would using a dict instead of a list make more sense here? Then you wouldn't have to iterate through the list down below (assuming you could figure out which formatter to use before calling it).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about this further, I'd like to keep the current structure for two reasons:

  1. consistency: the pattern of "iterate providers, each decides if it applies" is already used for input schemas, tool descriptions, and resource providers; I think it's valuable to keep a consistent pattern here too.
  2. flexibility: iterating lets each formatter decide dynamically whether it applies. While it's fairly static today, we are leaving room for multiple formatters to enrich the same output without restructuring later.

Comment thread dash/mcp/primitives/tools/results/result_dataframe.py Outdated
Comment thread dash/mcp/primitives/tools/results/result_dataframe.py Outdated
Comment thread dash/mcp/primitives/tools/results/result_dataframe.py Outdated
)

GENERIC_FIGURE = PropRole(
PLOTLY_FIGURE = PropRole(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this change necessary?

Copy link
Copy Markdown
Contributor Author

@KoolADE85 KoolADE85 May 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a style choice: I wanted it to read more clearly when it's subsequently used in the result formatter, that we are actually concerned with formatting Plotly figures, not more "generic" figures.

Comment thread dash/mcp/primitives/tools/results/result_dataframe.py Outdated
Comment thread dash/mcp/primitives/tools/results/result_plotly_figure.py Outdated
Comment thread dash/mcp/primitives/tools/results/result_plotly_figure.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants