fix: write repack launcher with LF endings so pipelines built on Windows work (v2) - #6326
Merged
mohamedzeidan2021 merged 1 commit intoSep 28, 2026
Conversation
…ows work _RepackModelStep._inject_repack_script_and_launcher writes the bash launcher _repack_script_launcher.sh from a Python string using text-mode open(..., "w"). On Windows, text mode translates every \n to os.linesep (\r\n); the resulting CRLF bash script then breaks in the Linux repack container (a trailing \r corrupts arguments, e.g. model.tar.gz\r / model.tar.gz#015), so the repack cannot find the model artifact. Open both launcher writes (S3 and local source_dir branches) with newline="\n" so LF is written verbatim on every host. Output is byte-identical on Linux/mac; only Windows behavior changes. Adds host-independent guard tests plus a raw-bytes CRLF check. Fixes aws#3762
jam-jee
approved these changes
Sep 28, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context (please read first)
This addresses #3762 (v2 backport — companion to #6313 which fixes the same bug in v3/
sagemaker-mlops).In that thread a maintainer (@qidewenwhen) confirmed the root-cause analysis is valid but noted the SDK officially supports Unix/Linux/Mac only (supported OS) and re-labeled the issue from bug → Windows-support feature request.
Opening this as a low-risk hardening change, not a claim of full Windows support: the fix is a one-argument change that is byte-for-byte identical on the supported platforms and only alters behavior when the SDK happens to run on Windows. It costs supported users nothing and unblocks the (unofficial but common) "author pipeline from a local Windows IDE" workflow. Happy to close if the team would rather keep Windows fixes out entirely.
Problem
When a pipeline is built/upserted from a Windows host, the RepackModel step fails in execution.
_RepackModelStep._inject_repack_script_and_launcherwrites the bash launcher_repack_script_launcher.shfrom a Python string using text-modeopen(..., "w"). On Windows, text mode translates every\ntoos.linesep(\r\n). The resulting CRLF bash script then breaks in the Linux repack container — a trailing\rcorrupts arguments (e.g.model.tar.gz\r/model.tar.gz#015), so the repack cannot find the model artifact. The same code run from Linux/mac works becauseos.linesep == "\n"there.Fix
Open both launcher writes — the S3
source_dirbranch and the localsource_dirbranch — withnewline="\n", which disables newline translation so LF is written verbatim on every host. On Linux/mac the output is byte-identical to before; only Windows behavior changes.The sibling
_repack_model.pyis unaffected: it's copied byte-for-byte viashutil.copy2from an LF-only checked-in file and executed bypython(universal-newline tolerant). The launcher was the only shell script written from a string.Testing
tests/unit/sagemaker/workflow/test_utils.py:test_inject_repack_launcher_opened_with_lf_newlineasserts the launcher is opened withnewline="\n"— a host-independent guard that fails on the old code and passes with the fix (verified by reverting the source change).test_inject_repack_launcher_has_lf_endingsreads the written launcher raw and asserts no\r\n(a Windows-only guard, labeled as such).All tests in the module pass;
blackandflake8clean.Backwards compatibility
_inject_repack_script_and_launcheris private; no public signature/return change. Output is unchanged on Linux/mac (already LF). No other readers/writers of the launcher exist in the module.