fix(search): make openSearch and bleve behave the same by fschade · Pull Request #3408 · opencloud-eu/opencloud

fschade · 2026-08-26T14:37:08Z

Description

since we have support for bleve and openSearch as search backends, we also have to take care that those engines behave the same (at least in all basic cases), which was not the case ash shown in the table.

Query	before bleve	before OpenSearch	both now
`new`	-	`new-folder`	`new-folder`
`report`	-	-	`Report.txt`
`name:new`	-	`new-folder`	`new-folder`
`name:"new"`	-	`new-folder`	-
`name:"*report"`	-	-	`Report.txt`
`name:"übung"`	`Übung.txt`	-	`Übung.txt`
`Title:quarterly`	-	`q1.html`	`q1.html`
`Title:QUARTERLY`	-	`q1.html`	`q1.html`
`title:quarterly`	-	-	`q1.html`
`Content:report`	`monthly.txt`	-	-
`Content:"reports monthly"`	`monthly.txt`	-	-
`mediatype:TEXT/MARKDOWN`	-	`notes.md`	`notes.md`
`path:"./parent"`	nur `parent`	`parent`, `child.pdf`	`parent`, `child.pdf`
`type:file`	-	error 400	jede Datei
`type:folder`	-	error 400	jeder Ordner
`size>100`	-	-	filtert nach Größe
`hidden:true`	-	`hidden.txt`	`hidden.txt`
`Favorites:"A1B2-Upper"`	`starred.txt`	-	`starred.txt`

or more detailed here

for that reason, the pr contains a new test suite which runs the same tests against each search backend which proves the compatibility between the two.

The suite produces a markdown document at the end which lists the test cases and shows the similarity between bleve and openSearch.

while writing that engine i stumbled across some inconsistencies which are fixed now too, in a nutshell:

field matching on OpenSearch (name, title, tags, path), boolean and negated group queries on bleve, cased values (type:File, hidden:TRUE, ids), trailing slash in paths, ? as wildcard, missing favorite: alias, a broken mimetype entry
KQL: quoted strings are marked in the AST, number restrictions (size>100) are parsed as numbers
Index names carry a generation (opencloud-resource-v3, bleve-v2): a changed mapping starts on a fresh index instead of failing, filling it is manual
Some new acceptance tests

unfortunately all that also means, we have to create a new index mapping for openSearch but also for bleve, so the PR can be considered breaking and should not to into the next minor version. For details please read services/search/MIGRATION.md.

it also acts as a transition point to:

Motivation and Context

Searching for a name, a tag or inside a folder returned wrong or no results.

How Has This Been Tested?

unit & acceptance tests
lab installation

Types of changes

Bug fix
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Technical debt
Tests only (no source changes)

Checklist:

Code changes
Unit tests added
Acceptance tests added
Documentation added

codacy-production · 2026-08-26T14:39:08Z

Not up to standards ⛔

🔴 Issues 1 minor

Alerts:
⚠ 1 issue (≤ 0 issues of at least minor severity)

Results:
1 new issue

Category Results

CodeStyle 1 minor

View in Codacy

🟢 Metrics 26 complexity · 57 duplication

Metric Results

Complexity 26

Duplication 57

View in Codacy

🟢 Coverage 72.33% diff coverage · +0.35% coverage variation

Metric Results

Coverage variation ✅ +0.35% coverage variation (-1.00%)

Diff coverage ✅ 72.33% diff coverage

View coverage diff in Codacy

Coverage variation details

Coverable lines Covered lines Coverage

Common ancestor commit (ffbfe17) 84501 18925 22.40%

Head commit (1bde3d1) 84816 (+315) 19295 (+370) 22.75% (+0.35%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details

Coverable lines Covered lines Diff coverage

Pull request (#3408) 506 366 72.33%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer}
_{TIP This summary will be updated as you push new changes.}

dschmidt

LGTM

@butonic

Good questions raised by @butonic, wouldn't approve merging it currently

both engines now agree on names, titles, tags, paths, types, sizes, dates, hidden flags, facet values and wildcards. quotes only delimit phrases and the equals operator matches the whole field value, following the kql spec. the index name carries a generation so a changed mapping starts on a fresh index, MIGRATION.md says how to fill it.

Adopts the parity-pinned semantics from #3408: 'report' does not match 'reports'; the porter fulltext analyzer is gone.

…routing From #3408: hidden takes bool words only, type categories map to the stored value in the shared pass, ? counts as a wildcard, a non-suffix wildcard on a word-broken field forgives the extension, = matches the whole value on the lowercased sibling, paths lose their trailing slash. Dead compiler helpers removed.

Adopts the parity-pinned semantics from #3408: 'report' does not match 'reports'; the porter fulltext analyzer is gone.

…routing From #3408: hidden takes bool words only, type categories map to the stored value in the shared pass, ? counts as a wildcard, a non-suffix wildcard on a word-broken field forgives the extension, = matches the whole value on the lowercased sibling, paths lose their trailing slash. Dead compiler helpers removed.

fschade self-assigned this Aug 26, 2026

fschade added the Type:Bug label Aug 26, 2026

fschade added this to OpenCloud Team Board Aug 26, 2026

github-project-automation Bot moved this to Qualification in OpenCloud Team Board Aug 26, 2026

fschade moved this from Qualification to In Progress in OpenCloud Team Board Aug 26, 2026

fschade changed the title ~~fix(search): match names, tags and paths as indexed~~ fix(search): match names, tags and paths as indexed [opensearch] Aug 26, 2026

fschade closed this Aug 26, 2026

github-project-automation Bot moved this from In Progress to Done in OpenCloud Team Board Aug 26, 2026

fschade reopened this Aug 26, 2026

butonic reviewed Aug 26, 2026

View reviewed changes

Comment thread services/search/MIGRATION.md Outdated

dschmidt previously approved these changes Aug 26, 2026

View reviewed changes

butonic reviewed Aug 26, 2026

View reviewed changes

Comment thread services/search/MIGRATION.md Outdated

butonic reviewed Aug 26, 2026

View reviewed changes

Comment thread services/search/MIGRATION.md Outdated

butonic reviewed Aug 26, 2026

View reviewed changes

Comment thread services/search/README.md Outdated

fschade changed the title ~~fix(search): match names, tags and paths as indexed [opensearch]~~ fix(search): make openSearch and bleve behave the same Aug 28, 2026

fschade moved this from Done to In Progress in OpenCloud Team Board Aug 28, 2026

fschade added the Type:Feature-Request label Aug 28, 2026

fschade requested review from butonic and dschmidt August 28, 2026 09:49

fschade enabled auto-merge August 28, 2026 12:10

dschmidt mentioned this pull request Aug 28, 2026

test(search): engine parity suite for bleve and opensearch #3418

Merged

fschade added 3 commits August 31, 2026 10:44

test(search): follow the versioned index and pin the equals operator

0106a70

test(search): drop the engine overrides, both engines answer alike

5c153ff

fschade force-pushed the fix-search-field-matching branch from e6cf23e to 5c153ff Compare August 31, 2026 09:23

butonic reviewed Aug 31, 2026

View reviewed changes

Comment thread services/search/internal/opensearchtest/suite.go

butonic reviewed Aug 31, 2026

View reviewed changes

Comment thread services/search/internal/opensearchtest/suite.go

butonic reviewed Aug 31, 2026

View reviewed changes

Comment thread services/search/pkg/opensearch/index.go

butonic approved these changes Aug 31, 2026

View reviewed changes

fix(search): bring back rebase removals

1bde3d1

dschmidt added a commit that referenced this pull request Aug 31, 2026

feat(search): analyze Content by words on both engines, no stemming

be9c9c8

Adopts the parity-pinned semantics from #3408: 'report' does not match 'reports'; the porter fulltext analyzer is gone.

dschmidt mentioned this pull request Aug 31, 2026

refactor: reflection-based search mapping + location geopoint #3345

Merged

fschade merged commit a9da258 into opencloud-eu:main Aug 31, 2026
64 of 66 checks passed

github-project-automation Bot moved this from In Progress to Done in OpenCloud Team Board Aug 31, 2026

openclouders mentioned this pull request Aug 31, 2026

🎉 Release 8.0.0 #3405

Open

dschmidt added a commit that referenced this pull request Aug 31, 2026

feat(search): analyze Content by words on both engines, no stemming

ba21023

Adopts the parity-pinned semantics from #3408: 'report' does not match 'reports'; the porter fulltext analyzer is gone.

fschade added the Type:Breaking-Change label Sep 3, 2026

Navigation Menu

Uh oh!

Uh oh!

fix(search): make openSearch and bleve behave the same - #3408

fix(search): make openSearch and bleve behave the same#3408
fschade merged 4 commits into
opencloud-eu:mainfrom
fschade:fix-search-field-matching

fschade commented Aug 26, 2026 •

edited

Loading

Uh oh!

codacy-production Bot commented Aug 26, 2026 •

edited

Loading

Uh oh!

Uh oh!

dschmidt left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

fschade commented Aug 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

How Has This Been Tested?

Types of changes

Checklist:

Uh oh!

codacy-production Bot commented Aug 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Not up to standards ⛔

Uh oh!

Uh oh!

dschmidt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

fschade commented Aug 26, 2026 •

edited

Loading

codacy-production Bot commented Aug 26, 2026 •

edited

Loading