Skip to content

feat(search): extract more data from tika 4 (if available) - #3198

Merged
dschmidt merged 8 commits into
mainfrom
feat/search-tika-facets
Sep 1, 2026
Merged

feat(search): extract more data from tika 4 (if available)#3198
dschmidt merged 8 commits into
mainfrom
feat/search-tika-facets

Conversation

@dschmidt

@dschmidt dschmidt commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Stacked on #3197 (search schema-change handling).

Splits the tika extractor and its tests into per-facet files and fills the facet properties Tika 4.0.0 exposes:

  • Audio: bitrate, isVariableBitrate, hasDrm, trackCount, discCount, copyright, composers
  • Location: altitude (EXIF GPS, metres → feet)
  • GPS now also resolves for videos (MP4/MOV)

Works against Tika 3 and 4 at the same time: full text is read from tk:content with the legacy X-TIKA:content as fallback (4.0.0 renamed the meta prefix), and language detection for stop-word cleaning asks /language first with the legacy /language/string as fallback (4.0.0 moved the endpoint, go-tika only knows the old path). The new facet keys only exist on Tika 4, an older Tika simply leaves those facets empty. The acceptance suite runs against the released apache/tika:4.0.0-full; CI sets the tika service workdir to /opt/tika-server since Tika 4 discovers its plugins relative to it (TIKA-4864).

@codacy-production

codacy-production Bot commented Jul 29, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 4 duplication

Metric Results
Duplication 4

View in Codacy

🟢 Coverage 97.18% diff coverage

Metric Results
Coverage variation Report missing for 49ebe071
Diff coverage 97.18% diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (49ebe07) Report Missing Report Missing Report Missing
Head commit (5e4f124) 85849 20133 23.45%

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#3198) 213 207 97.18%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

1 Codacy didn't receive coverage data for the commit, or there was an error processing the received data. Check your integration for errors and validate that your coverage setup is correct.

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@dschmidt dschmidt changed the title feat(search): more tika facets (audio bitrate/drm/counts, copyright, composer, gps altitude) + libregraph bump feat(search): extract more data from tika Jul 29, 2026
@dschmidt
dschmidt force-pushed the feat/search-tika-facets branch from c7c0551 to 35231b1 Compare August 18, 2026 16:28
@dschmidt
dschmidt force-pushed the feat/search-tika-facets branch from 35231b1 to add256c Compare August 31, 2026 14:33
@dschmidt dschmidt changed the title feat(search): extract more data from tika feat(search): extract more data from tika 4, keep tika 3 working Aug 31, 2026
@dschmidt dschmidt changed the title feat(search): extract more data from tika 4, keep tika 3 working feat(search): extract more data from tika 4 Aug 31, 2026
@dschmidt dschmidt changed the title feat(search): extract more data from tika 4 feat(search): extract more data from tika 4 (if available) Aug 31, 2026
@dschmidt
dschmidt marked this pull request as ready for review August 31, 2026 15:13
@dschmidt
dschmidt requested review from butonic and fschade August 31, 2026 15:13
@dschmidt
dschmidt force-pushed the feat/search-tika-facets branch 2 times, most recently from 19fb88d to a9c7079 Compare August 31, 2026 17:00
Base automatically changed from feat/search-schema-change-handling to main September 1, 2026 05:17
@fschade
fschade force-pushed the feat/search-tika-facets branch from 2329c62 to 240239d Compare September 1, 2026 05:17
@dschmidt
dschmidt force-pushed the feat/search-tika-facets branch from 240239d to 5e4f124 Compare September 1, 2026 07:10
@dschmidt
dschmidt merged commit df8d0e8 into main Sep 1, 2026
66 checks passed
@dschmidt
dschmidt deleted the feat/search-tika-facets branch September 1, 2026 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants