Bibliography in YAML only accept relative path?

[andreatitolo]

Bug description

I use the same .bib file as bibliographic source for many quarto documents, and for easier managing, I have this file in a folder outside my quarto projects. I updated recently to quarto 1.4.553, and noticed that the bibliography field in the YAML frontmatter of a .qmd file (or in the quarto.yml) does not accept absolute paths anymore.
Given the following example .qmd:

---
title: "My Document"
bibliography: /path/to/references.bib
---

Using quarto render, pandoc complains that the file does not exist because instead of looking it in /path/to/references.bib it looks for the file in /path/to/project/path/to/references.bib. This was not the case before. The same happens also if the absolute path points to a file inside the project folder.
Note that references and citation are indeed picked up by the citation picker (I am using the VSCode plugin), only the render part fails.

Steps to reproduce

From a fresh quarto manuscript project (from quarto create project), this will result in an error.

---
title: testing
authors:
  - name: Norah Jones
    affiliation: The University
    roles: writing
    corresponding: true
bibliography: /Users/user/testing/references.bib
---

## Section
This is a simple placeholder for the manuscript's main document [@knuth84].

Quarto.yml

project:
  type: manuscript

manuscript:
  article: index.qmd

format:
  html:
    comments:
      hypothesis: true
  docx: default
  jats: default

  # (other formats)
  # pdf: default

execute:
  freeze: true

Expected behavior

I should be able to use an absolute path in the bibliography field of the YAML frontmatter.

Actual behavior

The error is the following

ERROR: Failed to generate bibliography, rendering failed with code 1
pandoc: pandoc: /path/to/references.bib: withBinaryFile: does not exist (No such file or directory)


Stack trace:
pandoc: /path/to/references.bib: withBinaryFile: does not exist (No such file or directory)

    at renderToCSLJSON (file:///Applications/quarto/bin/quarto.js:92045:19)
    at eventLoopTick (ext:core/01_core.js:183:11)
    at async bibliographyCslJson (file:///Applications/quarto/bin/quarto.js:91968:28)
    at async googleScholarReferences (file:///Applications/quarto/bin/quarto.js:92554:24)
    at async file:///Applications/quarto/bin/quarto.js:92416:39
    at async file:///Applications/quarto/bin/quarto.js:70242:33
    at async withTimingAsync (file:///Applications/quarto/bin/quarto.js:19466:25)
    at async runHtmlPostprocessors (file:///Applications/quarto/bin/quarto.js:70235:9)
    at async Object.complete (file:///Applications/quarto/bin/quarto.js:70075:43)
    at async Object.onPostProcess (file:///Applications/quarto/bin/quarto.js:93802:38)

Your environment

• IDE: VSCodium 1.87.2
• OS: MacOS Ventura 13.6.5

Quarto check output

quarto check
Quarto 1.4.553
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.11: OK
      Dart Sass version 1.69.5: OK
      Deno version 1.37.2: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.4.553
      Path: /Applications/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: v2024.03.13
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: TinyTex
      Path: /Users/andreatitolo/Library/TinyTeX/bin/universal-darwin
      Version: 2024

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.12.2
      Path: /opt/homebrew/opt/python@3.12/bin/python3.12
      Jupyter: (None)

      Jupyter is not available in this Python installation.
      Install with python3 -m pip install jupyter

[✓] Checking R installation...........OK
      Version: 4.3.1
      Path: /Library/Frameworks/R.framework/Resources
      LibPaths:
        - /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library
      knitr: 1.45
      rmarkdown: 2.25

[✓] Checking Knitr engine render......OK

[mcanouil]

I should be able to use an absolute path in the bibliography field of the YAML frontmatter.

There is not a bug.
Quarto is designed to work as a self-contained project.
Paths starting with / means at the root of the projects.

You can find many issues and discussions about this.

This was not the case before.

What do you mean by "before"?

Note that if it was "working" before, it was most likely a bug that was fixed.

[andreatitolo]

With "Before" I meant before updating to version 1.4.553. But yes it was likely a bug fixed now and I should have read some discussions before about it, thanks. It seems the only way other than maintaining a local .bib file just for that project would be to use Zotero, I guess I'll have to do that.

[mcanouil]

With "Before" I meant before updating to version 1.4.553

From 1.4.552 or 1.0? That's not the same thing.

It seems the only way other than maintaining a local .bib file just for that project would be to use Zotero, I guess I'll have to do that.

Nota that using Zotero and the Visual Editor, you get a small bibliography file in your project which makes your project portable/sharable (which is/was not the case with absolute path outside the project).

[andreatitolo]

From 1.4.552 or 1.0? That's not the same thing.

From v1.4.551, sorry, should have been clearer :)

Nota that using Zotero and the Visual Editor, you get a small bibliography file in your project which makes your project portable/sharable (which is/was not the case with absolute path outside the project).

Yep that's true and it is actually a great feature, but I am using JabRef as my reference manager and not Zotero, that's why I was using an external bib file.
No big deal tough, I'll see if it's worth switching software or find some other way around it :)

[cscheid]

Some possible workaround for your workflow, in increasing levels of hackiness:

• If you're on Unix, you can use a hard link
• If you're inside a project, you can use a prerender script to copy the file to the right place (so that it's always up to date)
• If you're in a single-file render, you can create a Lua filter that copies the file to the right place (again, so that it's always up to date)

[AlloriMD]

A symlink to your .bib file works great, btw.

If absolute paths won't be supported in the future, then the Quarto documentation might mention three options for references:

1. Manually create a new project-specific references.bib file that is stored in the root directory of the project;
2. Make a copy of your library.bib file from the Zotero (or other) directory and place the copy in the root directory of the Quarto project; or
3. Create a symlink to an external library.bib file, and place the symlink in the Quarto project root directory.

Going further, Quarto might even make #2 (copy library.bib) or #3 (symlink to library.bib) be options that can be specified during a quarto create myproject command.

[mcanouil]

Quarto does create a project bibliography file as documented, see https://quarto.org/docs/visual-editor/vscode/index.html#zotero-citations.

Quarto can use references directly from Zotero to create a project bibliography file.

Quarto won't document "solution" which goes against self-contained project and reproducibility.