[wip][ENG-9706] migrate_osfmetrics_6to8#11699
Draft
aaxelb wants to merge 17 commits intoCenterForOpenScience:feature/9691-osfmetrics-migrationfrom
Draft
[wip][ENG-9706] migrate_osfmetrics_6to8#11699aaxelb wants to merge 17 commits intoCenterForOpenScience:feature/9691-osfmetrics-migrationfrom
aaxelb wants to merge 17 commits intoCenterForOpenScience:feature/9691-osfmetrics-migrationfrom
Conversation
6cd013e to
999dc86
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket
Purpose
migrate osf-metrics data from elasticsearch6 to elasticsearch8
Changes
migrate_osfmetrics_6to8that:--no-counts) displays counts of documents in osf-metrics indexes in both elastic6 and elastic8--start) schedules tasks to copy osf-metrics data from elastic6 to elastic8 (making slight changes along the way)background_migrationtask queue, as another way to check progressElastic6To8Stateindex in es8 to store when the migration was startedosf.metrics.es8_metricstimeseries_index_timedepthfor reports (e.g. once-a-day or once-a-month reports would be fine in yearly indexes)for any testing locally, if you haven't been using your local osf, might want to run management commands to get fake data:
fake_metrics_reports,make_dummy_pageviews_for_metrics,populate_impact_preprint_metrics, ...Side Effects
QE Notes
CE Notes
Documentation