Skip to content

Emit slices as constants in the bytecode compiler #125063

Description

@mdboom

Feature or enhancement

Proposal:

Constant slices are currently emitted as multiple bytecodes by the Python bytecode compiler.

For example, x[:-2, 1:-1] generates:

              LOAD_CONST               4 (0)
              LOAD_CONST               5 (-2)
              BUILD_SLICE              2
              LOAD_CONST               1 (1)
              LOAD_CONST               2 (-1)
              BUILD_SLICE              2
              BUILD_TUPLE              2

Then tuples of slices are constant like this, this could instead be compiled down to a single LOAD_CONST instruction:

              LOAD_CONST                1 ((slice(0, -2, None), slice(1, -1, None))

According to @fberlakovich and Stefan Brunthaler's paper on Cross module quickening, this can have a significant impact on Numpy benchmarks.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

Activity

  1. self-assigned this
    on Oct 7, 2024
  2. added a commit that references this issue on Oct 8, 2024
  3. nineteendo commented on Oct 8, 2024

    @nineteendo
    Contributor

    Shouldn't this be closed now?

  4. mdboom commented on Oct 8, 2024

    @mdboom
    ContributorAuthor

    Shouldn't this be closed now?

    Yep -- sorry, I forgot that step.

  5. added a commit that references this issue on Oct 9, 2024
  6. encukou commented on Nov 13, 2024

    @encukou
    Member

    Hi @mdboom,
    This is missing a documentation update, since marshal docs have a list of the types that pyc can handle.

    But, I wanted to ask: would it make sense to do the folding in AST, rather than in codegen? That seems like a more appropriate place, even though it probably won't help common code (e.g. folding "abc"[:1] to "a" isn't too relevant).

    I quickly tried moving it, and tests pass, but I don't know if I'm missing something: main...encukou:cpython:const_slice

  7. mdboom commented on Nov 13, 2024

    @mdboom
    ContributorAuthor

    his is missing a documentation update, since marshal docs have a list of the types that pyc can handle.

    Thanks for pointing that out. I'll handle that part.

    But, I wanted to ask: would it make sense to do the folding in AST, rather than in codegen?

    I don't know if I have enough historical to say which is better, it does seem like doing it earlier makes sense. I'd want a second opinion about whether it's the right thing to do, though (Cc: @iritkatriel?), and if so, you should submit your branch as a PR.

  8. iritkatriel commented on Nov 13, 2024

    @iritkatriel
    Member

    But, I wanted to ask: would it make sense to do the folding in AST, rather than in codegen?

    Yes, I think this makes sense.

  9. added a commit that references this issue on Nov 13, 2024
  10. added 2 commits that reference this issue on Nov 14, 2024
  11. added a commit that references this issue on Dec 8, 2024
  12. added a commit that references this issue on Jan 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

performancePerformance or resource usagetype-featureA feature request or enhancement

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions