Skip to content

Add parsing of type alias statements i.e. the type keyword - #97

Merged
charliermarsh merged 19 commits into
RustPython:mainfrom
zanieb:parse/typealias
Jul 17, 2023
Merged

Add parsing of type alias statements i.e. the type keyword#97
charliermarsh merged 19 commits into
RustPython:mainfrom
zanieb:parse/typealias

Conversation

@zanieb

@zanieb zanieb commented Jul 13, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

Extends #95
Closes #82

Adds parsing of new type soft keyword for defining type aliases.

Supports type alias statements as defined in PEP 695 e.g.

# A non-generic type alias
type IntOrStr = int | str

# A generic type alias
type ListOrSet[T] = list[T] | set[T]

# A type alias that includes a forward reference
type AnimalOrVegetable = Animal | "Vegetable"

# A generic self-referential type alias
type RecursiveList[T] = T | list[RecursiveList[T]]

All type parameter kinds are supported as in #95.

Builds on soft keyword abstractions introduced in RustPython/RustPython#4519

@zanieb
zanieb force-pushed the parse/typealias branch 2 times, most recently from d04c741 to 52d1a41 Compare July 14, 2023 15:18
@zanieb
zanieb marked this pull request as ready for review July 14, 2023 15:18
zanieb
zanieb commented Jul 14, 2023
View reviewed changes
Comment thread parser/src/soft_keywords.rs Outdated
@MichaReiser
MichaReiser requested a review from youknowone July 14, 2023 16:35
MichaReiser
MichaReiser reviewed Jul 14, 2023
View reviewed changes
Comment thread parser/src/parser.rs
Comment thread parser/src/parser.rs
Comment thread parser/src/python.lalrpop
Comment thread parser/src/soft_keywords.rs Outdated
Comment thread parser/src/parser.rs
Comment thread parser/src/soft_keywords.rs Outdated
// 2. The type token is followed by a name token.
// 3. The name token is followed by an equality token.
else if matches!(tok, Tok::Type) {
if !self.start_of_line {

@MichaReiser MichaReiser Jul 14, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll look at this on Monday. I need a fresh mind to process this

charliermarsh
charliermarsh reviewed Jul 14, 2023
View reviewed changes
Comment thread parser/src/soft_keywords.rs Outdated
Comment thread parser/src/soft_keywords.rs Outdated
@charliermarsh

charliermarsh commented Jul 14, 2023

Copy link
Copy Markdown
Contributor

Nice job figuring this out. A couple questions to confirm my understanding of the soft-keyword heuristics.

zanieb
zanieb commented Jul 14, 2023
View reviewed changes
Comment thread parser/src/soft_keywords.rs Outdated
charliermarsh
charliermarsh reviewed Jul 14, 2023
View reviewed changes
Comment thread parser/src/soft_keywords.rs Outdated
@zanieb zanieb mentioned this pull request Jul 14, 2023
Open
youknowone
youknowone approved these changes Jul 15, 2023
View reviewed changes

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thank you so much.

@zanieb zanieb changed the title Add parsing of type alias statements Add parsing of type alias statements i.e. the type keyword Jul 15, 2023
zanieb added 16 commits July 17, 2023 09:12
@zanieb
zanieb force-pushed the parse/typealias branch from e7ac2ef to 6050b5e Compare July 17, 2023 14:12
MichaReiser
MichaReiser reviewed Jul 17, 2023
View reviewed changes
Comment thread parser/src/soft_keywords.rs Outdated
Comment on lines +116 to +117
Tok::Lpar | Tok::Lsqb | Tok::Lbrace => nesting += 1,
Tok::Rpar | Tok::Rsqb | Tok::Rbrace => nesting -= 1,

@MichaReiser MichaReiser Jul 17, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to support parentheses for the type case? I didn't see any test case that uses a parenthesized name.

I think we can be more strict in the type implementation and only skip over NonLogicalNewline, Continuation, and Comment. Whatever token comes after must be the = or it isn't a type assignment

@charliermarsh charliermarsh Jul 17, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is valid: type Foo[T] = ....

@zanieb zanieb Jul 17, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I cannot find a case where parens are allowed, although we do need to support brackets for the generic definition

@MichaReiser MichaReiser Jul 17, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I guess we then still need to support skipping over arbitrary tokens.

@zanieb zanieb Jul 17, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are generally syntax errors and won't result in an identifier at runtime so I'm actually not sure if we want to change the type token to an identifier.

@zanieb zanieb Jul 17, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

b2d67c7
af508c2

We can add more complex parsing in a follow-up that includes tests that assert syntax errors if we want.

MichaReiser
MichaReiser approved these changes Jul 17, 2023
View reviewed changes

@MichaReiser MichaReiser left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ready to merge when you are

@charliermarsh
charliermarsh merged commit 704eb40 into RustPython:main Jul 17, 2023
This was referenced Jul 17, 2023
Closed
Closed
Merged
zanieb added a commit to astral-sh/RustPython-Parser that referenced this pull request Jul 17, 2023
@zanieb
Add parsing of type alias statements i.e. the type keyword (RustPyt…
1cb6925 
…hon#97)

Extends RustPython#95
Closes RustPython#82

Adds parsing of new `type` soft keyword for defining type aliases.

Supports type alias statements as defined in PEP 695 e.g.

```python
type IntOrStr = int | str

type ListOrSet[T] = list[T] | set[T]

type AnimalOrVegetable = Animal | "Vegetable"

type RecursiveList[T] = T | list[RecursiveList[T]]
```

All type parameter kinds are supported as in RustPython#95.

Builds on soft keyword abstractions introduced in RustPython/RustPython#4519
zanieb added a commit to astral-sh/RustPython-Parser that referenced this pull request Jul 17, 2023
@zanieb
Add parsing of type alias statements i.e. the type keyword (RustPyt…
a843a00 
…hon#97)

Extends RustPython#95
Closes RustPython#82

Adds parsing of new `type` soft keyword for defining type aliases.

Supports type alias statements as defined in PEP 695 e.g.

```python
type IntOrStr = int | str

type ListOrSet[T] = list[T] | set[T]

type AnimalOrVegetable = Animal | "Vegetable"

type RecursiveList[T] = T | list[RecursiveList[T]]
```

All type parameter kinds are supported as in RustPython#95.

Builds on soft keyword abstractions introduced in RustPython/RustPython#4519
@charliermarsh charliermarsh mentioned this pull request Mar 16, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@charliermarsh charliermarsh charliermarsh approved these changes
@youknowone youknowone youknowone approved these changes
@MichaReiser MichaReiser MichaReiser approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

PEP 695 – Type Parameter Syntax

4 participants

@zanieb @charliermarsh @youknowone @MichaReiser