Skip to content

Upsert support#263

Merged
frankmcsherry merged 5 commits intomasterfrom
upsert
Apr 1, 2020
Merged

Upsert support#263
frankmcsherry merged 5 commits intomasterfrom
upsert

Conversation

@frankmcsherry
Copy link
Copy Markdown
Member

This PR introduces a prototype "upsert" arrangement operator. Instead of ingesting standard update triples, it receives keyed optional values, where the current value for a key is the most recently received value.

This is substantially less general than differential dataflow allows, but lots of people use this format. The reason to do this here is that if the plan is to arrange the results anyhow, then we can re-use the arrangement and extract the "prior" values associated with keys, and undo the damage done at nominal incremental overhead.

cc @wangandi

@frankmcsherry
Copy link
Copy Markdown
Member Author

The code is mostly a copy/paste of the Arrange operator, and much of its generality isn't required here, where a total order on timestamps is required. That should probably be cleaned up, but it seemed most likely to work correctly starting from something else that was known (believed) to be correct.

@frankmcsherry frankmcsherry changed the title WIP Upsert support Upsert support Mar 26, 2020
@frankmcsherry frankmcsherry merged commit 60b5e16 into master Apr 1, 2020
@rjnn
Copy link
Copy Markdown

rjnn commented Apr 1, 2020

🎉!

This was referenced Oct 29, 2024
@antiguru antiguru deleted the upsert branch February 13, 2025 08:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants