Skip to content

fix glossary status frontend filtering logic to move to backend#25428

Merged
siddhant1 merged 52 commits intomainfrom
sid/fix-status-glossary
Mar 5, 2026
Merged

fix glossary status frontend filtering logic to move to backend#25428
siddhant1 merged 52 commits intomainfrom
sid/fix-status-glossary

Conversation

@siddhant1
Copy link
Copy Markdown
Member

@siddhant1 siddhant1 commented Jan 21, 2026

Fixes #25377

image

I worked on ... because ...

Type of change:

  • Bug fix
  • Improvement
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.


Summary by Gitar

  • Database schema: Added generated column entityStatus with index to glossary_term_entity table for efficient filtering
  • Backend filtering: Modified /api/v1/glossaryTerms/search endpoint to accept entityStatus query parameter; updated repository methods to filter at database level with SQL injection prevention
  • API refactor: Restructured searchGlossaryTermsPaginated() to accept single params object; added entityStatus parameter to list endpoints
  • Frontend removal: Eliminated client-side status filtering from GlossaryTermTab.component; now passes status filters to backend API
  • Test coverage: Added 3 comprehensive e2e test suites (1400+ lines) validating single/multiple status filters, large datasets, and nested term hierarchies

This will update automatically on new commits.

@github-actions
Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@github-actions
Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@github-actions
Copy link
Copy Markdown
Contributor

The Java checkstyle failed.

Please run mvn spotless:apply in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Java code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

@siddhant1
Copy link
Copy Markdown
Member Author

@gitar-bot fix the SQL injection, add a commit

@github-actions
Copy link
Copy Markdown
Contributor

The Java checkstyle failed.

Please run mvn spotless:apply in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Java code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

@gitar-bot
Copy link
Copy Markdown

gitar-bot bot commented Jan 21, 2026

@gitar-bot fix the SQL injection, add a commit

I've fixed the SQL injection vulnerability in GlossaryTermRepository.buildEntityStatusCondition.

The fix validates input values against the EntityStatus enum before constructing SQL queries. Invalid status values are now filtered out, preventing SQL injection attacks.

Changes in GlossaryTermRepository.java:1633

  • Added validation logic that checks each status value against the allowed EntityStatus enum values
  • Only legitimate status values (APPROVED, DRAFT, IN_REVIEW, DEPRECATED, REJECTED, UNPROCESSED) can be used
  • Ran mvn spotless:apply to format the code

This complements the earlier fix in ListFilter.java, ensuring both locations are now protected against SQL injection.

…e against the enum, in case if an invalid value like "Bogus" is passed
siddhant1 and others added 3 commits March 1, 2026 13:55
Remove rejected terms from visible list when status filter excludes
them, and fix reused waitForResponse promise in Playwright test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 2, 2026

Jest test Coverage

UI tests summary

Lines Statements Branches Functions
Coverage: 65%
65.94% (57235/86795) 45.55% (30223/66339) 48.37% (9072/18752)

@siddhant1
Copy link
Copy Markdown
Member Author

@gitar-bot do a frontend review

@gitar-bot
Copy link
Copy Markdown

gitar-bot bot commented Mar 2, 2026

@gitar-bot do a frontend review

Frontend Review Summary

I've reviewed the frontend changes in this PR. Overall, the implementation is solid and well-architected. Here are my findings:

✅ Strengths

1. Clean State Management

  • Proper use of React hooks (useState, useCallback, useMemo, useEffect)
  • Status filter state properly initialized with [EntityStatus.Approved, EntityStatus.Draft, EntityStatus.InReview] instead of 'all'
  • Dependency arrays correctly include selectedStatus where needed (line 698)

2. Smart API Integration

  • Status filtering properly integrated into both search and list APIs
  • entityStatus parameter correctly formatted as comma-separated string
  • Conditional API usage: search API when searching, list API otherwise
  • Both APIs receive entityStatusParam for consistent filtering

3. Excellent UX Implementation

  • Critical fix in updateTaskData (lines 671-680): When a term is approved/rejected, if the new status is not in the current filter, the term is removed from the visible list. This prevents rejected terms from staying visible when "Rejected" is not selected.
  • Proper handling of expanded rows during status updates
  • Search term properly triggers offset reset

4. Type Safety

  • Proper TypeScript types: SearchGlossaryTermsParams interface includes entityStatus?: string
  • Correct use of EntityStatus enum from generated types

💡 Suggestions for Improvement

1. Magic String "all" Pattern

// Current approach uses string literal 'all'
if (!selectedStatus.includes('all'))

// Consider: Define as constant for better maintainability
const STATUS_FILTER_ALL = 'all';
if (!selectedStatus.includes(STATUS_FILTER_ALL))

2. API Parameter Clarity

// Good: entityStatusParam is computed once
const entityStatusParam = isStatusFilterActive
  ? selectedStatus.filter((s) => s !== 'all').join(',')
  : undefined;

// Suggestion: Add JSDoc comment explaining the format
/**
 * Comma-separated list of entity statuses for filtering.
 * undefined = no filter (show all)
 * Example: "Approved,Draft,InReview"
 */

3. Error Boundary Consideration

  • Status filter changes trigger data fetching
  • Consider adding loading states or error handling for filter changes
  • Currently relies on isTableLoading which is good

4. Accessibility

  • Status dropdown should have proper ARIA labels
  • Consider adding aria-label="Filter by status" to the dropdown

🐛 Potential Issues

None identified! The critical bug where rejected terms stayed visible after rejection was fixed in commit 8d5f3cce.

📊 Code Quality Metrics

  • Readability: 9/10 - Clear variable names, logical flow
  • Maintainability: 8/10 - Some magic strings, but overall well-structured
  • Performance: 9/10 - Proper use of memoization, efficient filtering
  • Type Safety: 10/10 - Full TypeScript coverage
  • Testing: ✅ Playwright E2E tests included

🎯 Overall Assessment

Status: Approved ✅

The frontend implementation is production-ready. The status filtering feature is well-integrated, type-safe, and provides excellent UX. The recent fix for removing rejected terms from the visible list shows good attention to detail.

Key Achievement: Seamless integration of status filtering without disrupting existing glossary term navigation and hierarchy features.

@siddhant1
Copy link
Copy Markdown
Member Author

@gitar-bot do a frontend review

karanh37
karanh37 previously approved these changes Mar 2, 2026
siddhant1 and others added 5 commits March 4, 2026 11:12
Pass entityStatus parameter in fetchExpadedTree so Expand All respects
the active status filter. Add E2E test suite to verify the behavior.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…atterns

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
anuj-kumary
anuj-kumary previously approved these changes Mar 4, 2026
Scoped unscoped `tbody .ant-table-row` locators to `glossary-terms-table`
testid, and replaced unreliable row count assertion in empty state test
with visibility checks on `no-data-placeholder`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
aji-aju
aji-aju previously approved these changes Mar 4, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 4, 2026

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@gitar-bot
Copy link
Copy Markdown

gitar-bot bot commented Mar 4, 2026

Code Review ✅ Approved 13 resolved / 13 findings

All 10 findings have been resolved, including API documentation inconsistencies, SQL query issues, code duplication, memory management concerns, and UI filter logic problems. The code is approved and ready.

✅ 13 resolved
Bug: API doc says "InReview" but valid value is "In Review"

📄 openmetadata-service/src/main/java/org/openmetadata/service/resources/glossary/GlossaryTermResource.java:343
The @Parameter description on the search endpoint documents the status values as Approved,Draft,InReview,Rejected,Deprecated, but the actual EntityStatus enum values (returned by EntityStatus.value()) use "In Review" (with a space), not "InReview".

Since the parseEntityStatusValues() method validates against EntityStatus.values() using the value() string, passing InReview as documented will be silently filtered out as invalid, effectively becoming a no-op filter. Users following the API documentation will get unfiltered results when trying to filter by "In Review" status.

Additionally, Unprocessed is a valid status that's missing from the documented list.

Edge Case: Missing newline at end of migration SQL files

📄 bootstrap/sql/migrations/native/1.11.7/mysql/schemaChanges.sql:9 📄 bootstrap/sql/migrations/native/1.11.7/postgres/schemaChanges.sql:9 📄 bootstrap/sql/migrations/native/1.11.7/mysql/schemaChanges.sql:15
Both the MySQL and PostgreSQL migration SQL files are missing a newline at the end of the file. While this is a minor issue, many code conventions and tools (like POSIX standards, git diffs, and various linters) expect files to end with a newline character.

The files currently end with the CREATE INDEX statement without a trailing newline, which can cause issues with file concatenation tools and may result in "\ No newline at end of file" warnings in diffs.

Suggested fix: Add a newline at the end of both:

  • bootstrap/sql/migrations/native/1.11.7/mysql/schemaChanges.sql
  • bootstrap/sql/migrations/native/1.11.7/postgres/schemaChanges.sql
Quality: Duplicated parseEntityStatusValues method across classes

📄 openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/ListFilter.java:165-176 📄 openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/GlossaryTermRepository.java:1632-1643
The parseEntityStatusValues method is duplicated identically in two places:

  1. ListFilter.java (lines 165-176)
  2. GlossaryTermRepository.java (lines 1632-1643)

Both methods have the exact same logic:

  • Check for null/empty input
  • Validate against EntityStatus enum values
  • Split comma-separated values, trim, and filter valid statuses

This violates DRY (Don't Repeat Yourself) principle and creates maintenance burden - if the parsing logic needs to change, it must be updated in multiple places.

Suggested fix: Extract this common method to a shared utility class (e.g., EntityStatusUtils) that both classes can reference. This would centralize the validation logic and make future changes easier to maintain.

Performance: Fallback search path loads ALL glossary terms into memory

📄 openmetadata-service/src/main/java/org/openmetadata/service/resources/glossary/GlossaryTermResource.java:384-385
When no glossaryId, glossaryFqn, parentId, or parentFqn is provided, the search endpoint falls back to loading ALL glossary terms with Integer.MAX_VALUE as the limit (line 385), then performs in-memory filtering and pagination.

ResultList<GlossaryTerm> allTerms =
    repository.listAfter(uriInfo, fields, filter, Integer.MAX_VALUE, null);

For large deployments with thousands of glossary terms, this loads every term's full JSON into memory on every request. While the entityStatus filter is applied at the DB level via ListFilter, the search query filtering still happens in-memory (lines 390-409), and the full result set is materialized before pagination is applied (line 412).

This is an existing pattern predating this PR, but the PR adds a new code path that routes more requests through it (status-only filtering without a search term and without a glossary/parent identifier). Previously this path was only hit for the less common "search all terms" case.

Quality: CollectionDAO comment references wrong migration version (1.11.7)

📄 openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/CollectionDAO.java:3923
The comment on line 3923 of CollectionDAO.java says "entityStatus filtering uses generated column added in migration 1.11.7" but the migration was moved to 1.12.2 (as confirmed by the PR author's feedback). This stale reference could mislead developers trying to trace the migration history.

Should be updated to: // entityStatus filtering uses generated column added in migration 1.12.2

...and 8 more resolved from earlier reviews

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Mar 4, 2026

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Mar 4, 2026

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 5, 2026

Failed to cherry-pick changes to the 1.11.12 branch.
Please cherry-pick the changes manually.
You can find more details here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend safe to test Add this label to run secure Github workflows on PRs To release Will cherry-pick this PR into the release branch UI UI specific issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Status Filter (Draft / In Review) does not work with 3000+ Glossary Terms

9 participants