Skip to content

Upgrade CloudLogger to aepipe-sdk v0.1.1 with D1 payload support#91

Merged
loadchange merged 5 commits intomainfrom
claude/upgrade-logging-system-uGytM
Mar 29, 2026
Merged

Upgrade CloudLogger to aepipe-sdk v0.1.1 with D1 payload support#91
loadchange merged 5 commits intomainfrom
claude/upgrade-logging-system-uGytM

Conversation

@loadchange
Copy link
Copy Markdown
Collaborator

Summary

Upgrade the CloudLogger module from a custom HTTP-based implementation to use the official aepipe-sdk (v0.1.1), introducing D1 payload support for storing complete data without truncation and adding comprehensive event types for the full trading lifecycle.

Key Changes

Core SDK Integration

  • Replace custom urllib-based HTTP implementation with official aepipe-sdk client
  • Use Aepipe client for all cloud communication with built-in blob size validation
  • Import DataPoint, LogEntry, AepipeError, and ValidationError from aepipe SDK
  • Remove manual URL construction and Bearer token header management (handled by SDK)

Data Model Improvements

  • Change queue types from dict to SDK types: LogEntry for logs, DataPoint for events
  • Introduce D1 payload support for storing complete data (AI responses, prompts, market snapshots)
  • Implement payload TTL configuration (default 90 days) for automatic cleanup
  • Separate concerns: blobs for lightweight indexing, doubles for metrics, payload for full data

Decision Logging Enhancement

  • Store complete AI responses and prompts in payload (no truncation)
  • Add market data snapshot storage via payload
  • Include additional decision context: debate summary, regime, validation result
  • Truncate error messages in blobs to 200 chars while preserving full version in payload
  • Blobs now contain: [symbol, decision, status, error_msg_truncated, regime]

New Event Types

  • send_system_event(): System lifecycle events (startup, shutdown, config changes)
  • send_risk_event(): Risk management events (drawdown, daily loss, stop loss triggers)
  • send_grid_event(): Grid trading events (sync, rebuild, cancel)
  • send_cycle_event(): Trading cycle events (start, end, skip)
  • send_account_snapshot(): Periodic account state snapshots with positions
  • send_alert(): Market alerts with severity mapping (extreme/high → error, elevated → warn)

Trade Event Expansion

  • Add leverage, take_profit_price, stop_loss_price, order_type, and fee fields
  • Store complete trade data in payload alongside indexed blobs

Error Handling

  • Distinguish between ValidationError (client-side validation, skip without retry) and AepipeError (server errors, retry)
  • Gracefully handle network errors without impacting main trading flow

Integration Points

  • Update main.py to send system startup event with configuration details
  • Add account snapshot logging in trading cycle
  • Add market alert logging in volatility detection
  • Add risk event logging for stop loss failures and protection triggers
  • Add grid event logging for grid rebuilds
  • Update grid_main.py and related modules to use new cloud logger API

Configuration

  • Add payload_ttl parameter to CloudLogger initialization
  • Update config.yaml example with v2 documentation
  • Add aepipe-sdk dependency to pyproject.toml

Implementation Details

  • Payload TTL defaults to 90 days (7,776,000 seconds), matching Analytics Engine retention
  • Blobs limited to 15 items (aepipe-sdk 0.1.1 constraint), doubles to 20 items
  • Queue overflow handling: discard oldest message when full (maintains FIFO for important data)
  • Batch size capped at 200 items per request (aepipe limit is 250)
  • Retry logic: 2 attempts for transient errors, skip on validation errors
  • All cloud operations are non-blocking; failures don't affect trading logic

https://claude.ai/code/session_01McaLk3VSxwmNnMVybEsbW9

核心升级:
- 使用官方 aepipe-sdk 0.1.1 替代手写 HTTP 客户端,客户端防截断校验
- 利用 D1 payload 存储完整 AI 响应、Prompt、市场数据快照,突破 16KB blob 限制
- blobs 仅存轻量索引字段(symbol/action/status),大数据全走 payload
- AI 响应不再截断到 500 字符,决策推理过程完整可追溯

新增事件类型覆盖完整交易生命周期:
- system: 启动/关闭/配置变更
- cycle: 交易周期开始/结束/跳过
- account: 账户余额/权益/持仓快照
- risk: 回撤保护/止损失败/安全平仓等风控事件
- grid: 网格重建/同步/撤单等操作
- alert: 异常波动/API 故障/订单失败等告警

关键模块日志增强:
- main.py: 启动/关闭/周期异常/波动告警/账户快照
- grid_main.py: 启动/周期异常
- client.py: 止损失败自动平仓/安全平仓失败严重告警
- account_protector.py: 保护机制触发事件
- grid_manager.py: 网格重建完整参数
- order_manager.py: TPSL 设置失败紧急平仓

配置新增:
- cloud_logging.payload_ttl: D1 payload 过期时间(默认 90 天)

测试:39 个测试全部通过,431 个完整测试套件通过

https://claude.ai/code/session_01McaLk3VSxwmNnMVybEsbW9
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request upgrades the cloud logging infrastructure to version 2 by integrating the aepipe-sdk and implementing D1 payload support for storing large datasets like AI responses and prompts. It significantly expands logging coverage across the application, adding specific event types for system lifecycle, risk management, grid operations, and account snapshots. Review feedback recommends enhancing error reporting in main.py by replacing repr(e) with traceback.format_exc() to provide more detailed stack traces for debugging decision and cycle errors.

Comment thread main.py Outdated
alert_type="decision_error",
severity="high",
message=str(e),
details={"traceback": repr(e), "cycle": self.cycle_counter},
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

For better debuggability, it's recommended to use traceback.format_exc() instead of repr(e) to capture the full stack trace of the exception. This provides more context when an error occurs. Other parts of the codebase, like in grid_main.py, already use this approach, and maintaining consistency is good practice.

You'll also need to add import traceback at the top of the file.

Suggested change
details={"traceback": repr(e), "cycle": self.cycle_counter},
details={"traceback": traceback.format_exc(), "cycle": self.cycle_counter},

Comment thread main.py Outdated
alert_type="cycle_error",
severity="extreme",
message=str(e),
details={"traceback": repr(e), "cycle": self.cycle_counter},
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to my other comment, using traceback.format_exc() here instead of repr(e) would provide a more detailed traceback for easier debugging of cycle errors. This will also require importing the traceback module if it's not already present.

Suggested change
details={"traceback": repr(e), "cycle": self.cycle_counter},
details={"traceback": traceback.format_exc(), "cycle": self.cycle_counter},

claude and others added 4 commits March 29, 2026 09:13
根据 PR review 建议,在云端日志的异常记录中使用 traceback.format_exc()
替代 repr(e),提供完整的堆栈信息便于远程调试。

https://claude.ai/code/session_01McaLk3VSxwmNnMVybEsbW9
- cloud_logger.py: 移除未使用的 json import(已由 aepipe-sdk 处理序列化)
- test_cloud_logger.py: 移除未使用的 patch import

https://claude.ai/code/session_01McaLk3VSxwmNnMVybEsbW9
@loadchange loadchange merged commit cd88632 into main Mar 29, 2026
3 checks passed
@loadchange loadchange deleted the claude/upgrade-logging-system-uGytM branch March 29, 2026 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants