Skip to content

Can't parse openapi.yml as valid YAML #3

Open
Open
Can't parse openapi.yml as valid YAML#3
Assignees
raphinoclat
Labels
bugSomething isn't working

Description

@MikeRalphson
MikeRalphson
opened on May 4, 2023
boast https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml
GET https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml
YAMLSyntaxError: All collection items must start at the same column at line 1, column 1:

curl --location --request POST 'https://api.placekit.co/search' \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^…

    at Collection.parse (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/node_modules/yaml/dist/parse-cst.js:331:24)
    at /Users/mikeralphson/node/oas-kit/packages/oas-resolver/node_modules/yaml/dist/parse-cst.js:1605:29
    at Document.parseContents (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/node_modules/yaml/dist/parse-cst.js:622:26)
    at Document.parse (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/node_modules/yaml/dist/parse-cst.js:685:19)
    at Object.parse (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/node_modules/yaml/dist/parse-cst.js:1723:18)
    at parseDocument (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/node_modules/yaml/dist/index.js:43:24)
    at Object.parse (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/node_modules/yaml/dist/index.js:55:15)
    at mainPass (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/index.js:171:21)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.resolve (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/index.js:239:3) {
  nodeType: 'MAP',
  range: Range { start: 0, end: 390 },
  linePos: { start: { line: 1, col: 1 }, end: { line: 11, col: 7 } }
}
All collection items must start at the same column at line 1, column 1:

curl --location --request POST 'https://api.placekit.co/search' \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^…

Activity

  1. noclat
    added
    bugSomething isn't working
    on May 4, 2023

  2. noclat commented on May 4, 2023

    @noclat
    noclat
    on May 4, 2023
    Member

    Hi @MikeRalphson, could you share a codesnippet? I've tried to first check with online validators and it seems ok, redocly linter is all green too, and I've tried to parse it with YAML.parse() from the yaml node package from both local file and distant from raw.githubusercontent.com and get no error so far.

  3. noclat
    self-assigned this
    on May 4, 2023

  4. MikeRalphson commented on May 4, 2023

    @MikeRalphson
    MikeRalphson
    on May 4, 2023
    Author

    It's in one of the included code-snippet files...

    boast --verbose --verbose https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml
GET https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml
Resolving a Symbol(docType-openapi) parent none at https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml Previous []
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/curl/search.sh Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/go/search.go Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/js/search.js Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/python/search.py Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/ruby/search.rb Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/curl/reverse.sh Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/go/reverse.go Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/js/reverse.js Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/python/reverse.py Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/ruby/reverse.rb Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
YAMLSyntaxError: All collection items must start at the same column at line 1, column 1:

curl --location --request POST 'https://api.placekit.co/search' \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^…

    Ok - the problem is using $ref within an extension is not guaranteed to work, it's at best undefined behaviour, at worst $ref is just another property and doesn't get resolved. You're using plain text (markdown) files as the target of $refs too, rather than the more normal JSON or YAML (things with defined semantics in the OpenAPI world).

    ReDoc is very lenient here, but it doesn't necessarily help people write interoperable OpenAPI definitions (cc: @lornajane)

  5. noclat
    assigned
    raphi
    on May 4, 2023

  6. lornajane commented on May 4, 2023

    @lornajane
    lornajane
    on May 4, 2023

    Thanks for the ping, this is interesting! Redocly tooling will usually expand a $ref in a context where it's clear what the user intended - but the OpenAPI specification actually only technically supports it in a limited set of locations. This usually works out pretty well for rendering docs, but I can see that it could confuse other tools that trip up on the unexpected fields. (see also: docs for the x-codeSample support in Redocly https://redocly.com/docs/api-reference-docs/specification-extensions/x-code-samples/) I wonder if we could add a linting rule to catch this sort of thing for situations where there are other tools in the chain that need a stricter interpretation of the spec.

  7. MikeRalphson commented on May 4, 2023

    @MikeRalphson
    MikeRalphson
    on May 4, 2023
    Author

    That sounds excellent, thanks for pitching in on this issue!

  8. noclat commented on May 10, 2023

    @noclat
    noclat
    on May 10, 2023
    Member

    @MikeRalphson we've investigated a bit further, and we noticed a few things:

    • boast is kind of an unmaintained tool, and indeed we got the same results, but doesn't mean our model is invalid.
    • We've tried with oas-validate (from OAS Kit) and failed because our spec is 3.1 and the package only supports up to 3.0. It then passed with 3.0.3 so we downgraded to that version.
    • We've had indeed some warnings so we've added tags props to make it 100% valid.
    npx oas-validate -v --validateSchema=first --lint openapi.yml
Gathering...
openapi.yml
  PlaceKit API Reference 1.0.0
  https://api.placekit.co 

Tests: 1 passing, 0 failing, 0 warnings

    Anything else we can do on our side?

  9. MikeRalphson commented on May 10, 2023

    @MikeRalphson
    MikeRalphson
    on May 10, 2023
    Author

    Thanks. BTW boast isn't unmaintained, the last commit to it was within the last 10 days. It just needs a new release.

  10. noclat
    closed this as completedon May 10, 2023

  11. lornajane commented on Jun 12, 2023

    @lornajane
    lornajane
    on Jun 12, 2023

    To close the loop on this, we've added a rule to check for the more restricted/correct set of $ref usage. I'd be interested to know if this helps catch the problem, so feedback is welcome! Docs: https://redocly.com/docs/cli/rules/spec-strict-refs/

  12. raphi commented on Jun 15, 2023

    @raphi
    raphi
    on Jun 15, 2023
    Member

    Hi @lornajane , thanks for following up! I've added the new rule and the linter passed successfully.

    Taking a closer look at the issue and as @MikeRalphson said in comments, the problem is referencing a non-yaml file for the sample's source. Redocly extract and display the code sample correctly, but the syntax is not strictly valid. (BTW there is no example in the documentation about referencing a code sample file, just a simple inline example. Any idea how to do it properly @lornajane ?)

    I've changed the samples to be valid yaml reference files and boast validation is now passing (#4). It's not ideal because code samples aren't in there original files extension (same source files used for formatting, linting and testing) but at least both boast and redocly now pass!

  13. raphi
    reopened this on Jun 15, 2023

  14. lornajane commented on Jun 16, 2023

    @lornajane
    lornajane
    on Jun 16, 2023

    I did some digging on this, and found an example in the Rebilly API (Redocly and Rebilly are sister companies, lots of Redocly functionality serves Rebilly's needs!) https://github.com/Rebilly/api-definitions/blob/main/openapi/paths/customers%40%7Bid%7D.yaml#L32 It probably won't validate in any other tooling (perhaps that's less of a problem when it's inside an x- extension), but it does mean that the code can be in its own file format, and be linted or even executed there to test it, which is why we did it this way.

    I'm not sure why it's not in the docs but I'll add it!

  15. raphi commented on Jun 19, 2023

    @raphi
    raphi
    on Jun 19, 2023
    Member

    Thanks for the many examples!

    I ran Mike's validator on your specs and indeed the $ref targeting non-yaml/json files is invalid:

    $ npx boast https://raw.githubusercontent.com/Rebilly/api-definitions/main/openapi/openapi.yaml
[...]
GET https://raw.githubusercontent.com/Rebilly/api-definitions/main/openapi/code_samples/PHP-SDK-3.0/customers/get.php 
YAMLSyntaxError: All collection items must start at the same column at line 1, column 1:

$service = new Rebilly\Sdk\CoreService($client);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^…
[...]

    @MikeRalphson anything else I can do on my side to pass APIGuru's validation?

  16. raphi commented on Sep 18, 2023

    @raphi
    raphi
    on Sep 18, 2023
    Member

    @MikeRalphson up?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions