feat: [6598098782] Support indefrows funcs in window query. by Simon9997 · Pull Request #35140 · taosdata/TDengine

Simon9997 · 2026-04-15T00:26:02Z

Description

Issue(s)

Close/close/Fix/fix/Resolve/resolve: Issue Link

Checklist

Please check the items in the checklist if applicable.

Is the user manual updated?
Are the test cases passed and automated?
Is there no significant decrease in test coverage?

Copilot

Pull request overview

Adds support for “indefinite rows” (by-row/vector) time-series functions inside window queries, extending parser/planner/executor behavior and updating tests/docs accordingly.

Changes:

Enable/validate indefinite-rows functions in window clauses (and refine related rejection rules/messages in parser/planner).
Extend executor window operators (interval/session/state/event/count) with an SIndefRowsRuntime path to accumulate and return variable-row results.
Add/adjust test coverage and update error-code/docs to reflect new behavior and new internal error code.

Reviewed changes

Copilot reviewed 30 out of 32 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`source/libs/parser/src/parTranslater.c`	Allows certain indefinite-rows funcs in window queries; refines error messages and adds FILL restriction.
`source/libs/planner/src/planLogicCreater.c`	Marks window nodes as `indefRowsFunc` and collects appropriate function set for window execution.
`source/libs/planner/src/planSpliter.c`	Prevents certain interval split optimizations when `indefRowsFunc` is present.
`source/libs/function/src/functionMgt.c` / `include/libs/function/functionMgt.h`	Adds `fmIsWindowIndefRowsFunc` classification helper.
`source/libs/executor/inc/executorInt.h`	Introduces `SIndefRowsRuntime`/state structs + APIs for indefinite-rows window execution.
`source/libs/executor/src/executorInt.c`	Implements indefinite-rows window runtime (state management, block sealing, result delivery).
`source/libs/executor/src/timewindowoperator.c`	Hooks interval window aggregation into indefinite-rows runtime path.
`source/libs/executor/src/eventwindowoperator.c`	Hooks event window aggregation into indefinite-rows runtime path and adjusts lifecycle.
`source/libs/executor/src/countwindowoperator.c`	Hooks count window aggregation into indefinite-rows runtime path and adjusts buffering/state.
`include/util/taoserror.h` / `source/util/src/terror.c` / `source/util/test/errorCodeTable.ini`	Adds new internal error code `TSDB_CODE_QRY_WINDOW_STATE_NOT_EXIST`.
`docs/*/14-reference/09-error-code.md`	Documents the new error code (ZH/EN).
`docs//03-taos-sql/` and `docs/*/05-basic/03-query.md`	Updates window-clause/function docs to reflect new window support semantics.
`test/cases/...` / `test/ci/cases.task`	Adds new window test suite, updates existing expectations and error messages.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

gemini-code-assist

Code Review

This pull request introduces support for time-series functions like LAG, LEAD, MAVG, and CSUM within window queries, ensuring they are evaluated locally within each window. It implements a new state management framework (SIndefRowsRuntime) and updates the executor's window operators, parser, and planner logic. The feedback highlights a need to update data block window metadata to the full logical window range before processing sealed blocks to maintain consistency for upstream components.

Copilot

Pull request overview

Copilot reviewed 41 out of 43 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…ons in interval queries Allow indefRows functions (csum, diff, derivative, mavg, statecount, stateduration) to work with FILL clause in interval queries. Changes: - parser: lift restriction blocking FILL with indefRows functions - planner: propagate indefRowsMode flag through plan nodes - nodes: serialize/deserialize/clone indefRowsMode field - executor: implement multi-row window pass-through in fill engine, reset indefWindowActive on fill reset, validate fill values Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 41 out of 43 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

dapan1121

[MAJOR] countwindowoperator.c — Count Window Sliding 场景下 indefRows 状态残留风险
问题描述: 在 doCountWindowAggImpl 中, 当 windowCount != windowSliding 时, count window 使用多个 buffer slots 交替工作。在 indefRows 模式下, clearWinStateBuff 会重置 winRows 和 winSKey, 但到下一个 sliding point 时, getCountWinStateInfo 返回的 buffer 可能遗留旧的 SIndefRowsWindowState（从上个 window period）的引用在 hash map 中。

原因分析: indefRows hash map 的 key 是 (groupId, skey)。当 count window sliding < count 时, 同一 buffer slot 被复用, 但如果前一个窗口未完整（winRows < windowCount）, 对应的 SIndefRowsWindowState 仅在 buildCountResult 时才被 close。如果 pBuffInfo->winSKey 被 clearWinStateBuff 重置为 0, 而旧的 state 未 close, 可能导致 hash map 中残留 stale state。

建议修改: 在 clearWinStateBuff 时检测是否有关联的 IndefRows state 需要先 drop/close。或在注释中说明为何这不会发生（例如 sliding 场景下 window 总是完成后才 clear）。

[MAJOR] closeIndefRowsWindowState — _cleanup 路径可能返回错误但 readyBlocks 中有部分结果
问题描述: executorInt.c closeIndefRowsWindowState 函数中, _cleanup 路径在处理 sealed blocks 时如果某个 block 的 fillIndefRowsWindowPseudoCols 或 doFilter 失败, 已成功 push 到 pReadyBlocks 的 block 依然存在, 但函数返回的是非零 error code。

原因分析: 上游调用方收到错误后可能直接报错退出, 而 pReadyBlocks 中已有的正确结果 blocks 不会被消费。虽然这些 blocks 最终会被 cleanupIndefRowsRuntime 回收（不会泄漏）, 但错误处理语义不够明确——部分正确结果被丢弃, error code 可能掩盖了实际只是个别 block 失败。

建议修改: 在 _cleanup 注释中补充说明此设计意图: "保守策略, 任何单 block 失败即中止整个窗口的结果输出"。或者考虑更精细的处理（继续处理剩余 blocks, 仅丢弃失败的 block）。

[MAJOR] planLogicCreater.c — hasAggFuncs = false 行为变更
问题描述: createWindowLogicNodeFinalize 重构后, pSelect->hasAggFuncs = false 和 pSelect->hasIndefiniteRowsFunc = false 仅在成功路径执行。旧代码中 hasAggFuncs = false 是无条件执行的（即使前面步骤失败也会执行）。

原因分析: 这是一个行为变更。虽然在当前代码流中, createWindowLogicNodeFinalize 失败后整个计划创建会中止, 所以改变不太可能引发实际问题。但如果未来有代码在此函数返回错误后仍然检查 hasAggFuncs 标记, 可能产生不一致。

建议修改: 确认这是有意的行为变更。如果需要保持向后兼容, 将 pSelect->hasAggFuncs = false 移至 _return 标签之前（无论成功失败都执行）。

dapan1121 · 2026-04-20T00:58:21Z


 1. 使用 FILL 语句的时候可能生成大量的填充输出，务必指定查询的时间区间。针对每次查询，系统可返回不超过 1 千万条具有插值的结果。
 2. FILL 具有连续性，例如一列数据中仅第一条不为 NULL，则 FILL(PREV) 会为后续所有行填充该值。
+3. 当时序函数（如 CSUM、DIFF、DERIVATIVE、MAVG、STATECOUNT、STATEDURATION、IRATE、TWA、LAG、LEAD、FILL_FORWARD）与 INTERVAL 查询一起使用时，仅支持 FILL(NONE)、FILL(NULL)、FILL(NULL_F)、FILL(VALUE) 和 FILL(VALUE_F)。不支持 FILL(PREV)、FILL(NEXT)、FILL(LINEAR) 和 FILL(NEAR)。


非不定行的函数不需要有这个限制吧？

Simon9997 · 2026-04-20T02:40:42Z

[MAJOR] countwindowoperator.c — Count Window Sliding 场景下 indefRows 状态残留风险问题描述: 在 doCountWindowAggImpl 中, 当 windowCount != windowSliding 时, count window 使用多个 buffer slots 交替工作。在 indefRows 模式下, clearWinStateBuff 会重置 winRows 和 winSKey, 但到下一个 sliding point 时, getCountWinStateInfo 返回的 buffer 可能遗留旧的 SIndefRowsWindowState（从上个 window period）的引用在 hash map 中。

原因分析: indefRows hash map 的 key 是 (groupId, skey)。当 count window sliding < count 时, 同一 buffer slot 被复用, 但如果前一个窗口未完整（winRows < windowCount）, 对应的 SIndefRowsWindowState 仅在 buildCountResult 时才被 close。如果 pBuffInfo->winSKey 被 clearWinStateBuff 重置为 0, 而旧的 state 未 close, 可能导致 hash map 中残留 stale state。

建议修改: 在 clearWinStateBuff 时检测是否有关联的 IndefRows state 需要先 drop/close。或在注释中说明为何这不会发生（例如 sliding 场景下 window 总是完成后才 clear）。

[MAJOR] closeIndefRowsWindowState — _cleanup 路径可能返回错误但 readyBlocks 中有部分结果问题描述: executorInt.c closeIndefRowsWindowState 函数中, _cleanup 路径在处理 sealed blocks 时如果某个 block 的 fillIndefRowsWindowPseudoCols 或 doFilter 失败, 已成功 push 到 pReadyBlocks 的 block 依然存在, 但函数返回的是非零 error code。

原因分析: 上游调用方收到错误后可能直接报错退出, 而 pReadyBlocks 中已有的正确结果 blocks 不会被消费。虽然这些 blocks 最终会被 cleanupIndefRowsRuntime 回收（不会泄漏）, 但错误处理语义不够明确——部分正确结果被丢弃, error code 可能掩盖了实际只是个别 block 失败。

建议修改: 在 _cleanup 注释中补充说明此设计意图: "保守策略, 任何单 block 失败即中止整个窗口的结果输出"。或者考虑更精细的处理（继续处理剩余 blocks, 仅丢弃失败的 block）。

[MAJOR] planLogicCreater.c — hasAggFuncs = false 行为变更问题描述: createWindowLogicNodeFinalize 重构后, pSelect->hasAggFuncs = false 和 pSelect->hasIndefiniteRowsFunc = false 仅在成功路径执行。旧代码中 hasAggFuncs = false 是无条件执行的（即使前面步骤失败也会执行）。

原因分析: 这是一个行为变更。虽然在当前代码流中, createWindowLogicNodeFinalize 失败后整个计划创建会中止, 所以改变不太可能引发实际问题。但如果未来有代码在此函数返回错误后仍然检查 hasAggFuncs 标记, 可能产生不一致。

建议修改: 确认这是有意的行为变更。如果需要保持向后兼容, 将 pSelect->hasAggFuncs = false 移至 _return 标签之前（无论成功失败都执行）。

clearWinStateBuff 调用之前都调用了closeIndefRowsWindowState清理状态，所以不会出现遗留的问题。
push 到 pReadyBlocks 的 block 会在 task failed 之后调用 destroy operator 的时候调用 cleanupIndefRowsRuntime 清理
失败之后赋值 pSelect->hasAggFuncs = false 也没什么意义，后面都走失败退出的逻辑了。所以不用移动。

Copilot

Pull request overview

Copilot reviewed 42 out of 44 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-20T02:54:00Z

+    def format_ts(self, ts_ms: int) -> str:
+        return datetime.fromtimestamp(ts_ms / 1000).strftime("%Y-%m-%d %H:%M:%S.%f")[:-3]
+


format_ts() uses datetime.fromtimestamp(...), which depends on the local timezone of the test runner. Since these tests compare against string timestamps, results can differ across environments if TZ differs. Consider using an explicit timezone (e.g., UTC) or using the same timestamp-to-string conversion logic as the test framework/driver to make expectations deterministic.

Copilot · 2026-04-20T02:54:01Z

 | 0x80000741 | VSTB slotId not found for column                                            | 查询执行时未能将源列映射到虚拟表的 slotId                                                                   | 保留现场和日志，github 上报 issue                                                                                                 |
+| 0x80000742 | Window state not found                                                      | 查询执行时未能按窗口键找到对应的窗口状态，通常表示执行器内部窗口状态管理出现异常。                                              | 保留现场和日志，github 上报 issue                                                                                                 |


Copilot AI review requested due to automatic review settings April 15, 2026 00:26

Simon9997 requested review from a team, dapan1121, guanshengliang and zitsen as code owners April 15, 2026 00:26

taosdata-bot bot added the team engine label Apr 15, 2026

Copilot started reviewing on behalf of Simon9997 April 15, 2026 00:26 View session

Copilot AI reviewed Apr 15, 2026

View reviewed changes

Comment thread source/libs/executor/src/executorInt.c Outdated

Comment thread source/libs/executor/src/timewindowoperator.c

Comment thread source/libs/executor/src/timewindowoperator.c

gemini-code-assist bot reviewed Apr 15, 2026

View reviewed changes

Comment thread source/libs/executor/src/executorInt.c

Simon9997 force-pushed the feat/3.0/6598098782 branch from 8b0bb18 to ab30519 Compare April 15, 2026 00:52

Simon9997 added 2 commits April 16, 2026 15:18

feat: [6598098782] Support indefrows funcs in window query.

32cee6d

feat: [6598098782] Modify doc.

1ae4504

Copilot AI review requested due to automatic review settings April 16, 2026 07:18

Simon9997 force-pushed the feat/3.0/6598098782 branch from ab30519 to e956367 Compare April 16, 2026 07:18

Copilot started reviewing on behalf of Simon9997 April 16, 2026 07:19 View session

Copilot AI reviewed Apr 16, 2026

View reviewed changes

Comment thread source/libs/parser/src/parTranslater.c

Simon9997 force-pushed the feat/3.0/6598098782 branch from e956367 to f508210 Compare April 16, 2026 08:34

Copilot AI review requested due to automatic review settings April 16, 2026 10:10

Simon9997 force-pushed the feat/3.0/6598098782 branch from f508210 to 5972c20 Compare April 16, 2026 10:10

Copilot started reviewing on behalf of Simon9997 April 16, 2026 10:11 View session

Copilot AI reviewed Apr 16, 2026

View reviewed changes

Comment thread test/cases/11-Functions/04-Timeseries/test_fun_ts_with_window.py

Comment thread docs/en/05-basic/03-query.md

feat: [6598098782] Fix unit test.

30b9b74

dapan1121 reviewed Apr 20, 2026

View reviewed changes

feat: [6598098782] Modify doc.

6fe6bfa

Copilot AI review requested due to automatic review settings April 20, 2026 02:47

Copilot started reviewing on behalf of Simon9997 April 20, 2026 02:47 View session

Copilot AI reviewed Apr 20, 2026

View reviewed changes

		def format_ts(self, ts_ms: int) -> str:
		return datetime.fromtimestamp(ts_ms / 1000).strftime("%Y-%m-%d %H:%M:%S.%f")[:-3]

		\| 0x80000741 \| VSTB slotId not found for column \| 查询执行时未能将源列映射到虚拟表的 slotId \| 保留现场和日志，github 上报 issue \|
		\| 0x80000742 \| Window state not found \| 查询执行时未能按窗口键找到对应的窗口状态，通常表示执行器内部窗口状态管理出现异常。 \| 保留现场和日志，github 上报 issue \|

Conversation

Simon9997 commented Apr 15, 2026

Description

Issue(s)

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

dapan1121 left a comment

Choose a reason for hiding this comment

Uh oh!

dapan1121 Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Simon9997 Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Simon9997 commented Apr 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants