You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
The diagram exporter 404s on the dev box: download/current/diagram was deleted #230
GET /ContentService/exporter/diagram/<id>.{png,svg,jpeg} returns 404 on the dev
box for every pathway, while production returns 200. It is not an EHLD or
illustration problem, and not an application bug — the Java ContentService reads
these from disk and the files are gone.
The diagnosis, which took three steps
The error names its own path, which is the useful part:
Could not read diagram json for pathway
/usr/local/reactomes/Reactome/production/Website/static/download/current/diagram/R-HSA-109606.json
download/current is a symlink to release 97, which is what this box serves
97/ still holds ehld/ and the 528 MB reactome.graphdb.dump
97/diagram/ had been deleted to reclaim space
Caching the .json files moved the error rather than clearing it, because each
diagram needs two files:
With both present the exporter answers 200 immediately.
What was done on the box, and what it is not
Cached the 11 pathways the e2e suite uses — 1,262 KB of .json plus 506 KB of .graph.json — from https://download.reactome.org/97/diagram/, which still
publishes release 97. The full diagram/ directory for a release is gigabytes
and this host has 11 GB free, so restoring all of it is not an option.
This is a local workaround. It lives on one machine and disappears the next
time the directory is cleaned. It is recorded here so the next person does not
rediscover .graph.json from a 404.
Why it matters before curator testing
Downloading an illustration as SVG, PNG or JPEG is broken on beta right now
and works on production. A curator will find that and report it as a website bug.
Ordinary diagram downloads are unaffected because the app exports those
client-side from cytoscape — an EHLD has no cytoscape graph, so it falls through
to this endpoint.
Worth a line in CURATOR-REPORT.md either way, since the cache covers only the
pathways our tests touch.
The actual fix
Serve these from S3 rather than from a local directory, with a local cache for
what is used. That fits the existing plan for node fronting the exporters, and it
removes a class of failure where the site is fine and the backend quietly is not.
Re-measured today. Three things have changed since this was written, one of them making it worse.
ehld/ is gone too. The issue records 97/ as still holding ehld/. It is now empty — 0 files. diagram/ holds 11 of the 9,559 pathways that have one, and those 11 are the ids our own tests use, which is a trap worth naming: R-HSA-109606 is one of them, so the first spot check anyone runs returns 200 and the exporter looks healthy.
The image formats no longer matter, because nothing asks the exporter for them. Diagram downloads go to the render service, and illustrated pathways are now made in the page (#271). The detail-diagram fallback still points here and is inoperative on this box; its comment now says so rather than claiming the exporter "still draws something for every pathway".
SBGN is the part that is still broken, and it 500s rather than 404s. That is a download button on the detail bar and in the download tab, and it tracks the missing files exactly:
R-HSA-73894 sbgn 500 (diagram file on disk: no)
R-HSA-1640170 sbgn 500 (diagram file on disk: no)
R-HSA-168256 sbgn 500 (diagram file on disk: no)
R-HSA-69306 sbgn 200 (diagram file on disk: yes)
R-HSA-109581 sbgn 200 (diagram file on disk: yes)
SBML and PDF are fine without the files — SBML answers in 27s, which is its own question. The site itself is unaffected throughout, because the app fetches diagram JSON from download.reactome.org directly rather than from this disk; only the Java exporter reads locally.
What restoring would cost. Sampled 30 pathways spread across all 9,559 (mean 56 KB per file, two files each): 1.02 GB. The box has 5.6 GB free, so it fits, but not comfortably — and another deploy here is already waiting on 6 GB. Removing the one superseded docker image reclaimed nothing, its layers being shared with the current one.
So the decision is a resource one rather than a technical one: 1 GB to make SBGN work for 9,548 pathways it currently fails for, on a disk that is the binding constraint. Per the port plan SBGN stays with Java, so it needs these files — the alternative is teaching that exporter to read from the download host, which is a change in a different repository.
Correcting my earlier comment, and answering why the exporter needs these files at all.
The empty ehld/ is not a fault. I listed it as making this worse; it does not. Illustrations live on download.reactome.org and the app fetches them from there. The only things that read a local ehld/ were the Java image exporters — exporter/diagram/<id>.{png,svg,jpeg,pptx} — and nothing asks for those any more: diagram figures come from the render service and illustrations are made in the page (#271). An empty ehld/ on this box is correct.
Why SBGN needs the diagram JSON on the fly, from ExportManager.getDiagram():
SBGN carries layout, and a pathway's layout is a curated artefact that exists only as diagram JSON — it cannot be derived from the graph. A reaction's layout is computed from the graph, by the else branch, which is why reaction SBGN works here and pathway SBGN does not.
The 500 is not a "file not found" path either. A missing file makes getDiagramreturn null, and that null goes straight into new SbgnConverter(null).
Production is fast because it is not generating anything.getCachedFile reads ContentService/exporter/<version>/sbgn/, and its comment says that folder "will be created during release phase". Production answers in 0.15s. Here the folder held one file from 3 Sep — the two beside it now are from 04:47 today, written by my own test requests, so the cache did not pre-exist and the local diagram JSON really is what decides.
Sizing, measured rather than extrapolated: 9,559 pathways have their own diagram — 1,382 human and 8,152 computationally inferred into other species. All of them: 1.02 GB. Human only: 252 MB, mean 94 KB per file over a 25-pathway sample.
~/restore-diagram-files.mjs fetches the missing ones into the download directory, atomically and resumably. Since nothing readable on the box says which directory the exporter is configured to read — service.properties is tomcat:tomcat 0640 — it takes --only <stId>, so one pathway can prove the path for 90 KB before committing to the rest.
Fixed. The files are restored and SBGN works again.
19,092 files fetched from the release bucket into download/current/diagram/, 0.96 GB, bringing it to 19,118 — two per pathway for all 9,559. Checked across 14 pathways spanning 13 species, human and computationally inferred, including an illustrated one:
What this was, in the end. Not the image exporters — nothing asks those for anything now. It was /ContentService/exporter/event/<id>.sbgn, a button on the download panel and the detail bar, returning 500 for 9,548 of 9,559 pathways. A pathway's SBGN carries layout, that layout is a curated artefact stored as diagram json, and ExportManager.getDiagram returns null when the file is missing, which then goes into new SbgnConverter(null). A reaction needs no file, which is why reaction SBGN never broke.
Why it went unnoticed for weeks, which is the part worth keeping. The eleven pathways whose files survived are the ids our own fixtures use. Every spot check anyone ran — including my first one here, on R-HSA-109606 — came back healthy. e2e/release/sbgn-export.spec.ts now asserts on Cell Cycle, chosen precisely because it is not a fixture, and asserts the pathway and reaction cases separately so a future failure says which half is wrong.
And the ehld/ directory is fine empty, as noted above — illustrations live on the download host, and the only things that read a local copy were the image exporters we no longer use.
~/restore-diagram-files.sh is on the box if this happens again after a release; it fetches only what is absent, so it is safe to re-run.
Activity
docs(curators): bring both documents up to today's work (#240)
adamjohnwright commentedon Sep 20, 2026
Re-measured today. Three things have changed since this was written, one of them making it worse.
ehld/is gone too. The issue records97/as still holdingehld/. It is now empty — 0 files.diagram/holds 11 of the 9,559 pathways that have one, and those 11 are the ids our own tests use, which is a trap worth naming: R-HSA-109606 is one of them, so the first spot check anyone runs returns 200 and the exporter looks healthy.The image formats no longer matter, because nothing asks the exporter for them. Diagram downloads go to the render service, and illustrated pathways are now made in the page (#271). The
detail-diagramfallback still points here and is inoperative on this box; its comment now says so rather than claiming the exporter "still draws something for every pathway".SBGN is the part that is still broken, and it 500s rather than 404s. That is a download button on the detail bar and in the download tab, and it tracks the missing files exactly:
SBML and PDF are fine without the files — SBML answers in 27s, which is its own question. The site itself is unaffected throughout, because the app fetches diagram JSON from
download.reactome.orgdirectly rather than from this disk; only the Java exporter reads locally.What restoring would cost. Sampled 30 pathways spread across all 9,559 (mean 56 KB per file, two files each): 1.02 GB. The box has 5.6 GB free, so it fits, but not comfortably — and another deploy here is already waiting on 6 GB. Removing the one superseded docker image reclaimed nothing, its layers being shared with the current one.
So the decision is a resource one rather than a technical one: 1 GB to make SBGN work for 9,548 pathways it currently fails for, on a disk that is the binding constraint. Per the port plan SBGN stays with Java, so it needs these files — the alternative is teaching that exporter to read from the download host, which is a change in a different repository.
adamjohnwright commentedon Sep 20, 2026
Correcting my earlier comment, and answering why the exporter needs these files at all.
The empty
ehld/is not a fault. I listed it as making this worse; it does not. Illustrations live ondownload.reactome.organd the app fetches them from there. The only things that read a localehld/were the Java image exporters —exporter/diagram/<id>.{png,svg,jpeg,pptx}— and nothing asks for those any more: diagram figures come from the render service and illustrations are made in the page (#271). An emptyehld/on this box is correct.Why SBGN needs the diagram JSON on the fly, from
ExportManager.getDiagram():SBGN carries layout, and a pathway's layout is a curated artefact that exists only as diagram JSON — it cannot be derived from the graph. A reaction's layout is computed from the graph, by the
elsebranch, which is why reaction SBGN works here and pathway SBGN does not.The 500 is not a "file not found" path either. A missing file makes
getDiagramreturn null, and that null goes straight intonew SbgnConverter(null).Production is fast because it is not generating anything.
getCachedFilereadsContentService/exporter/<version>/sbgn/, and its comment says that folder "will be created during release phase". Production answers in 0.15s. Here the folder held one file from 3 Sep — the two beside it now are from 04:47 today, written by my own test requests, so the cache did not pre-exist and the local diagram JSON really is what decides.Sizing, measured rather than extrapolated: 9,559 pathways have their own diagram — 1,382 human and 8,152 computationally inferred into other species. All of them: 1.02 GB. Human only: 252 MB, mean 94 KB per file over a 25-pathway sample.
~/restore-diagram-files.mjsfetches the missing ones into the download directory, atomically and resumably. Since nothing readable on the box says which directory the exporter is configured to read —service.propertiesistomcat:tomcat 0640— it takes--only <stId>, so one pathway can prove the path for 90 KB before committing to the rest.adamjohnwright commentedon Sep 20, 2026
Fixed. The files are restored and SBGN works again.
19,092 files fetched from the release bucket into
download/current/diagram/, 0.96 GB, bringing it to 19,118 — two per pathway for all 9,559. Checked across 14 pathways spanning 13 species, human and computationally inferred, including an illustrated one:What this was, in the end. Not the image exporters — nothing asks those for anything now. It was
/ContentService/exporter/event/<id>.sbgn, a button on the download panel and the detail bar, returning 500 for 9,548 of 9,559 pathways. A pathway's SBGN carries layout, that layout is a curated artefact stored as diagram json, andExportManager.getDiagramreturns null when the file is missing, which then goes intonew SbgnConverter(null). A reaction needs no file, which is why reaction SBGN never broke.Why it went unnoticed for weeks, which is the part worth keeping. The eleven pathways whose files survived are the ids our own fixtures use. Every spot check anyone ran — including my first one here, on R-HSA-109606 — came back healthy.
e2e/release/sbgn-export.spec.tsnow asserts on Cell Cycle, chosen precisely because it is not a fixture, and asserts the pathway and reaction cases separately so a future failure says which half is wrong.And the
ehld/directory is fine empty, as noted above — illustrations live on the download host, and the only things that read a local copy were the image exporters we no longer use.~/restore-diagram-files.shis on the box if this happens again after a release; it fetches only what is absent, so it is safe to re-run.