Skip to content

[wip][ENG-9706] migrate_osfmetrics_6to8#11699

Draft
aaxelb wants to merge 17 commits intoCenterForOpenScience:feature/9691-osfmetrics-migrationfrom
aaxelb:9706-metrics-migration
Draft

[wip][ENG-9706] migrate_osfmetrics_6to8#11699
aaxelb wants to merge 17 commits intoCenterForOpenScience:feature/9691-osfmetrics-migrationfrom
aaxelb:9706-metrics-migration

Conversation

@aaxelb
Copy link
Copy Markdown
Contributor

@aaxelb aaxelb commented Apr 17, 2026

Ticket

Purpose

migrate osf-metrics data from elasticsearch6 to elasticsearch8

Changes

  • add management command migrate_osfmetrics_6to8 that:
    • (unless --no-counts) displays counts of documents in osf-metrics indexes in both elastic6 and elastic8
    • (if --start) schedules tasks to copy osf-metrics data from elastic6 to elastic8 (making slight changes along the way)
  • add/use background_migration task queue, as another way to check progress
  • update djelme dependency (parallel [ENG-9706] support osfmetrics migration django-elasticsearch-metrics#96 )
  • add Elastic6To8State index in es8 to store when the migration was started
  • tidy/improve djelme recordtypes in osf.metrics.es8_metrics
    • loosen some validation to fit actual usage
    • omit "es8" from es8 index names
    • set a lower timeseries_index_timedepth for reports (e.g. once-a-day or once-a-month reports would be fine in yearly indexes)

for any testing locally, if you haven't been using your local osf, might want to run management commands to get fake data: fake_metrics_reports, make_dummy_pageviews_for_metrics, populate_impact_preprint_metrics, ...

Side Effects

QE Notes

CE Notes

Documentation

@aaxelb aaxelb changed the title [wip][ENG-9706] osfmetrics migration [wip][ENG-9706] migrate_osfmetrics_6to8 Apr 17, 2026
@aaxelb aaxelb force-pushed the 9706-metrics-migration branch from 6cd013e to 999dc86 Compare April 21, 2026 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant