Fixes #27001: Delete pipeline execution history on hard delete#27037
Open
RajdeepKushwaha5 wants to merge 3 commits intoopen-metadata:mainfrom
Open
Fixes #27001: Delete pipeline execution history on hard delete#27037RajdeepKushwaha5 wants to merge 3 commits intoopen-metadata:mainfrom
RajdeepKushwaha5 wants to merge 3 commits intoopen-metadata:mainfrom
Conversation
…delete When a Pipeline entity was permanently deleted (hardDelete=true), the execution status records in entity_extension_time_series were left behind. Recreating a pipeline with the same name would surface stale historical execution data. Add entityExtensionTimeSeriesDao().delete() call in entitySpecificCleanup() to remove all pipeline status time series records keyed by the pipeline FQN. This mirrors the existing pattern in IngestionPipelineRepository.postDelete() and runs within the same transaction as the rest of the cleanup.
Contributor
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes orphaned pipeline execution history when a Pipeline entity is hard deleted, preventing stale Task Execution data from reappearing if a pipeline is recreated with the same FQN.
Changes:
- Delete
pipeline.pipelineStatustime-series rows fromentity_extension_time_seriesduring hard-delete cleanup. - Keep existing lineage cleanup behavior intact.
harshach
added
the
safe to test
harshach
approved these changes
Apr 4, 2026
6 tasks
Contributor
🟡 Playwright Results — all passed (22 flaky)✅ 3665 passed · ❌ 0 failed · 🟡 22 flaky · ⏭️ 89 skipped
🟡 22 flaky test(s) (passed on retry)
How to debug locally# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip # view trace |
Code Review ✅ ApprovedHard delete operations now remove associated pipeline execution history, resolving the orphaned records issue. No issues found. OptionsDisplay: compact → Showing less information. Comment with these commands to change:
Was this helpful? React with 👍 / 👎 | Gitar |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe your changes:
Fixes #27001
When a Pipeline entity is permanently deleted (
hardDelete=true), the execution status records in theentity_extension_time_seriestable were left behind. If a pipeline with the same fully qualified name was recreated later, the stale historical execution data would resurface in the UI's Task Execution tab.Root cause:
PipelineRepository.entitySpecificCleanup()only cleaned up lineage relationships but did not delete time series records. The baseEntityRepository.cleanup()deletes fromentity_extensionbut not fromentity_extension_time_series— these are separate tables/DAOs.Fix: Added
entityExtensionTimeSeriesDao().delete(fqn, PIPELINE_STATUS_EXTENSION)toentitySpecificCleanup(), which runs within the same transaction as the rest of the hard-delete cleanup. This mirrors the existing pattern inIngestionPipelineRepository.postDelete()(line 487-489) which already handles this correctly for ingestion pipelines.Testing: The deletion call uses the existing
EntityTimeSeriesDAO.delete(entityFQNHash, extension)method which executes a simpleDELETE FROM entity_extension_time_series WHERE entityFQNHash = :entityFQNHash AND extension = :extension. This is a well-established DAO method already used byIngestionPipelineRepository,TestCaseRepository, anddeletePipelineStatus()within this same class.Type of change:
Checklist:
Fixes <issue-number>: <short explanation>