v0.7.14: perf improvements, code hygiene, GitLab private host support#5201
Merged
Conversation
…s) (#5193) * improvement(mistral): update OCR pricing to OCR 4 rate ($4/1,000 pages) * docs(mistral): document mistral-ocr-latest alias resolves to OCR 4
…rimitive (#5195) Replace four hand-rolled client SSE decode loops with two layered primitives in lib/core/utils/sse.ts: - readSSELines: the single byte-stream decode engine. Splits on \n, strips trailing \r, tolerates data: with/without a leading space, skips the [DONE] sentinel, honors an AbortSignal before each chunk and between events, and releases the reader lock only when it acquired it. - readSSEEvents<T>: a thin JSON layer that parses each payload and routes unparseable lines to onParseError (default: skip). An SSESource union accepts a Response, a ReadableStream, or an already-acquired reader so callers that must stash the reader for external cancellation keep ownership of the lock. Migrates use-execution-stream, chat use-chat-streaming, home use-chat (via readSSELines for schema-validated decode), and the workflow chat panel. Legacy server/wand exports (encodeSSE, SSE_HEADERS, readSSEStream) are untouched. Behavior is preserved across abort, RAF batching, TTS, [DONE], delimiter tolerance, and reader-lock ownership. Tests in sse.test.ts pin the prior behavior: \n and \n\n framing, mid-chunk splits, [DONE], data: with/without leading space, \r\n stripping, sync/async early-stop, pre-aborted and mid-stream abort, lock release/non-release per source, lock release on a throwing handler, and Response/stream/reader sources.
…riggers, webhook, and connector (#5200) * feat(gitlab): support self-managed GitLab host across tools, block, triggers, webhook, and connector Add an optional `host` so the GitLab integration can target a self-managed instance (e.g. gitlab.example.com) instead of gitlab.com. Defaults to gitlab.com everywhere, so existing workflows, blocks, triggers, and stored webhooks are unchanged. - Shared host helper (normalizeGitLabHost/getGitLabApiBase) used by all 19 tools, the block, triggers, the webhook provider, and the connector - SSRF hardening: reject structurally unsafe hosts (userinfo `@`, whitespace, control chars, embedded path/query, empty labels) before the token-bearing request is built; allow self-managed hosts, ports, and IDN punycode - Route the webhook provider's previously-raw fetches through secureFetchWithValidation (DNS + private-IP rejection + IP pinning), matching the tool and connector paths - Add regression tests for the host validator * fix(gitlab): handle unsafe-host errors gracefully in webhook provider Address review feedback: - Validate the optional self-managed host up front in createSubscription and deleteSubscription so a structurally unsafe value surfaces as a clear error (create) or a graceful non-strict skip (delete) instead of an unhandled UnsafeGitLabHostError, mirroring the connector's handling - Document the layered SSRF defense: bare IP literals pass the structural host guard by design and are rejected at the fetch layer (validateUrlWithDNS); add a confirming test group making that intent explicit * fix(slack): drop assistant:write scope pending app review approval Requesting assistant:write before Slack approved it fails the OAuth/install flow for users. Remove it from both request paths until approval lands: - Remove from the user OAuth scope list (oauth.ts), matching the existing users:read.email TODO pattern - Remove the action_assistant capability from the bot manifest generator (capabilities.ts), leaving a TODO to restore it after approval The set_status/set_title/set_suggested_prompts tools remain and surface their existing graceful "reconnect with assistant:write" message until re-enabled.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
PR SummaryMedium Risk Overview GitLab gains optional Workspace files & public shares add workspace- and token-scoped inline image routes with referenced-by-doc, same-workspace, and byte-sniff gates; shared Data loading server-prefetches home, files, tables, and knowledge lists via cookie-forwarded internal API fetches and Streaming replaces ad-hoc SSE parsing in chat, workflow chat, mothership, and execution streams with shared Sandbox Runtime Reviewed by Cursor Bugbot for commit cff7a49. Bugbot is set up for automated code reviews on this repo. Configure here. |
Contributor
Greptile SummaryThis release bundles four independent improvements: GitLab self-managed host support wired across tools, blocks, triggers, webhook provider and connector; a consolidated typed SSE primitive (
Confidence Score: 4/5Safe to merge after confirming the tool executor routes GitLab API calls through the SSRF-protected fetch layer; all other changes are clean refactors with good test coverage. Every changed surface — SSE primitive, store enums, prefetches — is well-tested and the logic is correct. The one open question is whether the 20 GitLab tool configs, which now build URLs from a user-supplied hostname, are executed through the same secureFetchWithValidation chokepoint that the webhook provider and connector explicitly use. The structural guard in normalizeGitLabHost prevents authority-confusion attacks, but bare IPs pass through and would reach internal infrastructure carrying the user's PRIVATE-TOKEN header if the tool executor uses plain fetch. apps/sim/tools/gitlab/utils.ts and all 20 tool files under apps/sim/tools/gitlab/ — confirm the tool executor applies DNS/IP SSRF protection when resolving ToolConfig.request URLs.
|
| Filename | Overview |
|---|---|
| apps/sim/lib/core/utils/sse.ts | New typed SSE primitive: readSSELines (raw) + readSSEEvents (JSON). Handles both \n and \n\n framing, \r stripping, [DONE] sentinel, abort signal, and correct lock-ownership semantics. Comprehensive tests accompany it. |
| apps/sim/tools/gitlab/utils.ts | New shared normalizeGitLabHost/getGitLabApiBase utilities with structural host validation (UnsafeGitLabHostError). Correctly rejects userinfo, embedded paths, whitespace; intentionally allows bare IPs delegating DNS/SSRF blocking to the fetch layer — needs confirmation the tool executor honours the same chokepoint. |
| apps/sim/lib/webhooks/providers/gitlab.ts | Host parameter threaded through subscribe/unsubscribe; all outbound GitLab API calls switched to secureFetchWithValidation and UnsafeGitLabHostError handled gracefully in both strict and non-strict modes. |
| apps/sim/stores/execution/types.ts | Introduces ExecutionStatus enum ('idle' |
| apps/sim/stores/workflow-diff/types.ts | Introduces WorkflowDiffStatus ('none' |
| apps/sim/app/workspace/[workspaceId]/lib/prefetch-internal-fetch.ts | Server-side cookie-forwarding prefetch helper; workspaceId comes from Next.js router params, getInternalApiBaseUrl is a fixed value. Safe pattern, well-documented rationale for routing through API routes instead of the data layer directly. |
| apps/sim/app/chat/hooks/use-chat-streaming.ts | Replaced manual SSE decode loop with readSSEEvents; adds terminated flag to gate post-stream TTS/flush only when no final/error event was received. Also fixes a latent double-finalizeMessageStream bug from the old success-final path that didn't return. |
| apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/chat/chat.tsx | Replaced manual SSE decode loop with readSSEEvents. Minor: flushChunks() is called redundantly after readSSEEvents returns in the error-final path (harmless no-op but slightly inconsistent with use-chat-streaming.ts pattern). |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[User supplies optional host] --> B{normalizeGitLabHost}
B -->|empty / undefined| C[default: gitlab.com]
B -->|structurally unsafe\nuserinfo / path / whitespace| D[UnsafeGitLabHostError]
B -->|bare IP / valid hostname| E[getGitLabApiBase]
E --> F[https://host/api/v4]
F --> G{Call surface}
G -->|Webhook provider subscribe / unsubscribe| H[secureFetchWithValidation\nDNS + IP blocklist applied]
G -->|Connector sync| I[secureFetchWithValidation\nDNS + IP blocklist applied]
G -->|Tools ToolConfig.request| J[Tool executor\nSSRF protection unconfirmed]
D --> K[Surfaced as user-facing validation error]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[User supplies optional host] --> B{normalizeGitLabHost}
B -->|empty / undefined| C[default: gitlab.com]
B -->|structurally unsafe\nuserinfo / path / whitespace| D[UnsafeGitLabHostError]
B -->|bare IP / valid hostname| E[getGitLabApiBase]
E --> F[https://host/api/v4]
F --> G{Call surface}
G -->|Webhook provider subscribe / unsubscribe| H[secureFetchWithValidation\nDNS + IP blocklist applied]
G -->|Connector sync| I[secureFetchWithValidation\nDNS + IP blocklist applied]
G -->|Tools ToolConfig.request| J[Tool executor\nSSRF protection unconfirmed]
D --> K[Surfaced as user-facing validation error]
Comments Outside Diff (1)
-
apps/sim/tools/gitlab/get_project.ts, line 31-36 (link)Tool executor SSRF gap for user-controlled host
The webhook provider and connector both explicitly call
secureFetchWithValidationwhen usinggetGitLabApiBase(host), routing the request through the DNS/IP blocklist. The GitLab tools (this file and all 20 sibling tools) build a URL fromgetGitLabApiBase(params.host)but the HTTP call is made by the tool executor framework — which is not visible in the diff and may use plainfetch. If the tool executor doesn't go throughsecureFetchWithValidation, a value like127.0.0.1or169.254.169.254would pass the structural check and the request (carryingPRIVATE-TOKEN) would reach internal infrastructure.The test file explicitly documents this assumption: "SSRF to private/loopback/metadata addresses is the responsibility of
validateUrlWithDNS/secureFetchWithValidationat fetch time, the single SSRF chokepoint" — but unlikecleanupGitLabHookByUrlor the connector, nothing in the diff shows the tool executor enforcing that chokepoint. Please confirm thatToolConfig.requestURLs are processed throughsecureFetchWithValidation(or equivalent) before this merges.
Reviews (1): Last reviewed commit: "feat(gitlab): support self-managed GitLa..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
…ning into ECS taskdef (#5189) * feat(secrets): ingest env secrets at container runtime instead of fanning into ECS taskdef The app/socket ECS taskdefs were ~42KB, ~93% of which was the secrets[] array: 268 pointer entries each restating the full ~78-char secret ARN, marching toward the 64KB taskdef limit and growing ~150 bytes per hosted key added. The secret blob itself is only ~18KB/268 keys. Move secret delivery to container boot: new @sim/runtime-secrets loadRuntimeSecrets() reads SIM_ENV_SECRET_ID, fetches the combined secret once, and hydrates process.env (no-clobber, no-op when unset, fail-fast). Bootstrap entrypoints for app + realtime await it before importing the real server (env-flags reads env at module load). The app bootstrap is bun-bundled in the Dockerfile builder stage since it runs outside the Next standalone bundle; realtime keeps full node_modules and runs the TS entry. Backward-compatible: with the current fan-out taskdef the loader no-ops and the app reads the injected env vars unchanged. The matching infra change (empty secrets[] + SIM_ENV_SECRET_ID) ships separately, after this image is live. * fix(runtime-secrets): address review feedback - Move the binary-secret guard outside the retry loop (sendWithRetry) so a missing SecretString throws immediately instead of burning 3 attempts + backoff. - Bound each Secrets Manager request with AbortSignal.timeout(5s) so a stalled response can't hang boot indefinitely. - Drop the redundant @aws-sdk/client-secrets-manager pin from apps/realtime; it resolves transitively via @sim/runtime-secrets. - Add a test for the non-retriable binary-secret path.
…of buffering bytes (#5202) * improvement(sandbox): mount workspace files by presigned URL instead of buffering bytes Files and directories mounted into the function_execute sandbox were downloaded into the web process, re-encoded, and shipped inline. Mirror the table-snapshot path: under cloud storage, presign each file and let the sandbox curl it directly (no web-heap transit). Local storage keeps the buffered fallback. Add a count cap on the inputFiles list and a generous aggregate URL-mount byte ceiling so oversized requests fail fast instead of filling sandbox disk. * improvement(sandbox): use mount path in size-limit errors, display GB, add directory local-fallback test
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 5a938e5. Configure here.
… redundant flushChunks (#5204) * fix(tables): move tableKeys to a non-client module so the SSR prefetch works The tables list page crashed at SSR ('tableKeys.list is not a function') because tables/prefetch.ts (a server component) imported tableKeys from hooks/queries/tables.ts — a 'use client' module whose exports resolve to client-reference stubs on the server. Extract the key factory into hooks/queries/utils/table-keys.ts (no 'use client'), mirroring folder-keys.ts, and import it from there in the prefetch, hook, trigger, and consumers. * refactor(chat): drop redundant flushChunks on the SSE error path On an error 'final' event the reader stops via return true, so the post-loop flush is the single flush point. Defer the error append to after that flush (single flush, correct ordering) instead of flushing inside onEvent and again post-loop. No behavior change. * fix(sse): process the final unterminated line on stream end readSSELines broke out of the read loop on 'done' without flushing the TextDecoder or processing the trailing buffer, so a final 'data:' line not terminated by a newline (and any buffered multi-byte character) was dropped. Flush the decoder on end-of-stream and process the remaining buffer. Addresses a Cursor Medium finding on the consolidated SSE reader.
…ion fixes (#5205) * feat(gitlab): add repository, code-review, and CI job tools + validation fixes Expand the GitLab integration with 12 new tools (all host-aware via getGitLabApiBase, wired through types/index/registry/block): - Repository: list_repository_tree, get_file, create_file, update_file, create_branch, list_branches, list_commits - Code review: get_merge_request_changes, approve_merge_request - CI jobs: list_pipeline_jobs, get_job_log, play_job Validation fixes from /validate-integration: - Correct the block inputs key (credential -> accessToken) so it matches the subBlock id and the params the block reads - Trim projectId before encoding in all tool request URLs (input hygiene) /validate-connector and /validate-trigger passed clean against the GitLab REST API v4 docs — no changes required. * fix(gitlab): address review feedback + regen docs - get_merge_request_changes: use the /diffs endpoint (/changes was removed in GitLab 18.0); return the diff array + count (drops the MR envelope that /diffs no longer provides), fetch max page size in a single call - create_file/update_file: send explicit `encoding: 'text'` for clarity - Remove `// =====` separator comments from types.ts (repo convention) - Regenerate GitLab integration docs + catalog for the 12 new tools
) * feat(file): workspace-scoped inline images + public-share cascade Embedded markdown images now resolve only within the document's workspace, and public file shares cascade to the images the shared document embeds. - New /api/workspaces/[id]/files/inline (in-app, workspace-scoped) and /api/files/public/[token]/inline (public cascade) routes; the public one serves an embed only when it is referenced-by-doc, same-workspace, and passes a magic-byte image sniff - Embed srcs (serve-key and view-id forms) rewrite through one scoped inline route; one shared isomorphic parser owns the embed grammar for both the frontend renderer and the server doc scan - Accept wf_ file ids on the view/export routes (were 400ing on .uuid()) * feat(file): add Image command to the markdown editor slash menu - New /Image slash command uploads an image via a file picker and inserts it at the caret (same upload+insert path as paste/drop) - Inserted src is the workspace serve URL, so it renders in-app and cascades to public shares like any other embed - Per-editor handler wired through slash-command storage (the extension set is a shared singleton); only active when the editor is editable * fix(file): export rewrites all embed forms; cap embedded refs combined Addresses PR review: - Markdown export now rewrites the in-app `/workspace/<ws>/files/<id>` embed form too (not just `/api/files/view/<id>`), so a bundled asset never leaves a broken link in an offline export (Bugbot) - extractEmbeddedFileRefs bounds total references (keys + ids) to 50 combined rather than 50 each, matching MAX_EMBEDDED_IMAGES intent
…'use client' stub bug (#5206) * fix(ssr): move credential query-key factory + fetchers to non-client modules Preventively closes the same 'use client' SSR client-reference-stub class that crashed the tables page. Server-evaluated modules (the credential block def, the workflow-comparison helpers) imported workspaceCredentialKeys / fetchWorkspaceCredentialList / fetchCredentialSetById from 'use client' hook modules, where they resolve to client-reference stubs on the server (a future server call path would throw 'X is not a function'). Extract them into non-client hooks/queries/utils/{credential-keys, fetch-workspace-credentials,fetch-credential-set}.ts (mirroring folder-keys.ts / fetch-workflow-envelope.ts) and import from there. No behavior change — these values were only ever called from browser paths. * docs+ci: codify the 'use client' server-import rule + add check:client-boundary Document the Next.js rule that server code can only render a 'use client' export as a component, never call it (server imports resolve to client-reference stubs that throw — the tables-page crash). Add the rule to .claude/rules/sim-queries.md + a cross-ref in sim-architecture.md. Add scripts/check-client-boundary-imports.ts (wired into CI as check:client-boundary) that flags any value import from a 'use client' module in a server-evaluated, non-JSX surface (prefetch / route handler / trigger / block definition), so this class can't silently recur. Escape hatch: // client-boundary-allow: <reason>.
…ments (#5207) Adds error.tsx (reusing the shared ErrorState) to home, integrations, knowledge/[id], skills, settings, scheduled-tasks, and chat/[chatId] so a crash in any of these panels stays scoped to the panel and offers a retry, instead of bubbling to the generic workspace-level boundary.
…realtime-protocol (#5208) * refactor(realtime): type the socket event-handler boundary with @sim/realtime-protocol Replace the (data: any) event-handler types in socket-provider.tsx with precise broadcast types that mirror the exact payloads emitted by the realtime Socket.IO server (apps/realtime/src/handlers/** and rooms/**). Add @sim/realtime-protocol/events with the canonical wire types for the broadcast/confirmation events the server emits: WorkflowOperationBroadcast, SubblockUpdateBroadcast, VariableUpdateBroadcast, CursorUpdateBroadcast, SelectionUpdateBroadcast, the four workflow-lifecycle broadcasts, and OperationConfirmed/Failed. Typing change only; zero runtime/logic changes. Store-internal any (rehydrate state, subblock map, emit payloads) is left untouched as out of scope. * fix(realtime): type cursor-update broadcast cursor as nullable The client emits 'cursor-update' with { cursor: null } when a remote user's cursor leaves the canvas, and the server re-broadcasts it verbatim, so receivers genuinely get cursor: null. Type CursorUpdateBroadcast.cursor as CursorPosition | null to match the wire. (selection stays non-null — it signals absence via type: 'none', never null.)
…+ metadata query (#5209) * feat(salesforce): add Tooling API schema tools (custom field/object) + metadata query Add salesforce_create_custom_field, salesforce_update_custom_field, salesforce_delete_custom_field, salesforce_create_custom_object, and salesforce_tooling_query so the connector can make schema/metadata changes (e.g. create a custom field on Account). Previously the integration only did record CRUD via the REST Data API. Existing `api` OAuth scope covers the Tooling API; metadata creation is profile-permission gated, so no scope change. Also: fix Opportunity closeDate being wrongly required on update_opportunity, make list_reports/list_dashboards descriptions honest (recently-viewed scope), and document run_report's includeDetails default. * improvement(salesforce): non-destructive custom field update + align metadata param types - update_custom_field now does a read-modify-write (GET existing Metadata, overlay only provided changes, PATCH) so omitted properties are preserved instead of being reset by the Tooling API's full-metadata PATCH; no more fabricated label or injected create-time defaults on update - fieldType is now optional on update (kept from the existing field unless changed) - widen length/precision/scale/visibleLines param types to number | string to match the tool param configs (type: number) * improvement(salesforce): preserve picklist values and clear stale metadata on field type change - custom field update now unions provided picklist values with the field's existing values instead of replacing the whole valueSet (no data loss) - when fieldType changes on update, drop the prior type's type-specific metadata (length/precision/scale/visibleLines/valueSet/defaultValue/unique/ externalId) and backfill the new type's required defaults * improvement(salesforce): scope custom field update to attributes, never the type update_custom_field no longer changes a field's data type: Salesforce treats a type change as a separate conversion operation, and a stale forwarded fieldType could otherwise trigger an unintended destructive migration. The merge keeps the field's existing type and overlays only the other provided properties, dropping the type-change/stale-metadata-stripping logic entirely.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.