Skip to content

brew update followed by brew upgrade fails for Python's 2to3 conflict (December 2022) #11545

Closed as duplicate
apache/casbin-sqlx-adapter
#87

Description

@traversaro

Description

If one has on a job:

brew update
brew upgrade

the job fails with error:

 ==> Upgrading python@3.10
  3.10.8 -> 3.10.9 

==> Pouring python@3.10--3.10.9.big_sur.bottle.tar.gz
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/2to3
Target /usr/local/bin/2to3
already exists. You may want to remove it:
  rm '/usr/local/bin/2to3'

To force the link and overwrite all conflicting files:
  brew link --overwrite python@3.10

To list all files that would be deleted:
  brew link --overwrite --dry-run python@3.10

Possible conflicting files are:
/usr/local/bin/2to3 -> /Library/Frameworks/Python.framework/Versions/3.11/bin/2to3
/usr/local/bin/idle3 -> /Library/Frameworks/Python.framework/Versions/3.11/bin/idle3
/usr/local/bin/pydoc3 -> /Library/Frameworks/Python.framework/Versions/3.11/bin/pydoc3
/usr/local/bin/python3 -> /Library/Frameworks/Python.framework/Versions/3.11/bin/python3
/usr/local/bin/python3-config -> /Library/Frameworks/Python.framework/Versions/3.11/bin/python3-config

This is similar to old issues:

Platforms affected

  • Azure DevOps
  • GitHub Actions - Standard Runners
  • GitHub Actions - Larger Runners

Runner images affected

  • Ubuntu 18.04
  • Ubuntu 20.04
  • Ubuntu 22.04
  • macOS 11
  • macOS 12
  • Windows Server 2019
  • Windows Server 2022

Image version and build link

Image Version: 20221215.1
Link: https://github.com/traversaro/github-actions-brew-update-upgrade-check/actions/runs/3742142787

Is it regression?

20221211.1

Expected behavior

The brew upgrade should exit fine.

Actual behavior

The brew upgrade fails with the error message in the top of the issue.

Repro steps

See https://github.com/traversaro/github-actions-brew-update-upgrade-check/blob/2fa847c627c643716ba10d681ea18c86c0b54dec/.github/workflows/brew-update.yml for a MWE .

Activity

  1. traversaro commented on Dec 20, 2022

    @traversaro
    Author

    Possibly related to #6459 .

  2. igorboskovic3 commented on Dec 20, 2022

    @igorboskovic3
    Contributor

    Hi @traversaro , thank you, we will take a look.

  3. david-engelmann commented on Dec 21, 2022

    @david-engelmann
  4. ryancurrah commented on Dec 21, 2022

    @ryancurrah

    Also observed this here https://github.com/ryancurrah/lima-and-qemu/actions/runs/3745432855/jobs/6359863993 and here https://github.com/mook-as/lima-and-qemu/actions/runs/3743467832/jobs/6356414606.

    EDIT:

    I was able to get it working by removing the existing symlinks.

    rm /usr/local/bin/2to3
    rm /usr/local/bin/idle3
    rm /usr/local/bin/pydoc3
    rm /usr/local/bin/python3
    rm /usr/local/bin/python3-config
    
  5. mikhailkoliada commented on Dec 21, 2022

    @mikhailkoliada
    Contributor

    image

    It looks like it the universal python's package used in actions/setup-python is installing its symlinks to /usr/local, while I think it should not.

    @marko-zivic-93 @dsame @dmitry-shibanov could you please take a look?

  6. ni4 commented on Dec 22, 2022

    @ni4

    Got the same issue, and find out that something similar we had previously. Now it looks like:

      # homebrew fails to update python 3.9.1 to 3.9.1.1 due to unlinking failure
      rm /usr/local/bin/2to3 || true
      # homebrew fails to update python from 3.9 to 3.10 due to another unlinking failure
      rm /usr/local/bin/idle3 || true
      rm /usr/local/bin/pydoc3 || true
      rm /usr/local/bin/python3 || true
      rm /usr/local/bin/python3-config || true
  7. Flamefire commented on Dec 22, 2022

    @Flamefire

    It looks like it the universal python's package used in actions/setup-python is installing its symlinks to /usr/local, while I think it should not.

    That is not the case for us and the OP:

    /usr/local/bin/2to3 -> /Library/Frameworks/Python.framework/Versions/3.11/bin/2to3

    There is seemingly a non-brew installed Python from somewhere else

  8. dsame commented on Dec 22, 2022

    @dsame
    Contributor

    @Flamefire the workflow does not use action/setup-python. It seems the links are created by the macos python package - i see them on the "almost" clean machine with the XCode installed.

  9. dsame commented on Dec 22, 2022

    @dsame
    Contributor

    @traversaro actully it is a normal behaviour. As you see from the log the installation continues normally with just a warning the links were not overwritten. The problem is brew upgrade exits with non-zero exit code and bash shell runs with set -e option. Thus the step signals the error condition despite brew upgrade is the very last command or not.

    The easiest way to make the step to success is to add true after brew upgrade:

          - run: |
              brew update
              brew upgrade || true
    
  10. self-assigned this
    on Dec 22, 2022
  11. 121 remaining items

  12. added a commit that references this issue on Apr 13, 2024
  13. v-aparnajyothi-y commented on Dec 4, 2024

    @v-aparnajyothi-y

    Hello Everyone, This behavior is expected. The installation proceeds normally with just a warning that the links were not overwritten, which is typical. The issue arises because the brew upgrade command exits with a non-zero exit code, and since the bash shell runs with the set -e option, it causes the step to fail even though brew upgrade is the final command.

    Regarding the versions, Python 2.7, 3.10.8, and 3.11.0 symlinks are created during image generation. Specifically:

    Python 2.7 is linked to macOS Library/Frameworks
    Python 3.10 is linked to the Homebrew cellar
    Python 3.11 is linked to macOS Library/Frameworks
    The possible resolution of this issue would be updating the image with latest patches of Python 3.10 and Python 3.11. instead of 3.10.8 and 3.11.0. All macOS-12, 13, and 14 images have been updated with the latest patches for these versions, which resolved the issue.

    Please let us know if any further assistance or clarification!

  14. eli-schwartz commented on Dec 4, 2024

    @eli-schwartz

    @aparnajyothi-y,

    You seem to have fundamentally failed to understand the entire reason anyone is unhappy with GitHub Actions.

    The behavior is detrimental and bad. We know that GitHub expects this to happen and we don't wish for further clarification.

    We wish for GitHub to repent, to acknowledge that the expected behavior is detrimental and bad, and for GitHub to redesign the Actions image such that the Actions image works robustly, which it currently does not.

    It is an antipattern to respond to an issue saying that some behavior is bad and causes troubles, by responding "we know it's bad and causes troubles, but that was intentional".

  15. poshul commented on Dec 12, 2024

    @poshul

    The following works for us:

       - name: Force update all brew python
          run: |
            brew update
            # GHA messes with links https://github.com/actions/runner-images/issues/11545 so we just force replace their installs' with brew's
            for pkg in $(brew list --formula | grep -E '^python(@[0-9]+\.[0-9]+)?$'); do
                brew upgrade --force --overwrite "$pkg"
            done
            brew cleanup
  16. erik-bershel commented on Feb 7, 2025

    @erik-bershel
    Contributor

    Heads up! 👋

    Yes, this is a long-known configuration conflict. 😞 We have had several different incidents related to this topic. The work is currently underway within the framework of this task:

    Once we come to a solution, it will be announced in the specified issue. I am closing this one as a duplicate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions