Conversation
|
Add tests here after #1676 was merged |
tuhaihe
left a comment
There was a problem hiding this comment.
LGTM. I can build Cloudberry successfully on a pure Rocky Linux 10 with these changes.
libxml2 changed the required signature of error handler callbacks to make the passed xmlError struct "const". This is causing build failures on buildfarm member caiman, and no doubt will start showing up in the field quite soon. Add a version check to adjust the declaration of xml_errorHandler() according to LIBXML_VERSION. 2.12.x also produces deprecation warnings for contrib/xml2/xpath.c's assignment to xmlLoadExtDtdDefaultValue. I see no good reason for that to still be there, seeing that we disabled external DTDs (at a lower level) years ago for security reasons. Let's just remove it. Back-patch to all supported branches, since they might all get built with newer libxml2 once it gets a bit more popular. (The back branches produce another deprecation warning about xpath.c's use of xmlSubstituteEntitiesDefault(). We ought to consider whether to back-patch all or part of commit 65c5864d7 to silence that. It's less urgent though, since it won't break the buildfarm.) Discussion: https://postgr.es/m/1389505.1706382262@sss.pgh.pa.us
If we are building with openssl but USE_SSL_ENGINE didn't get set, initialize_SSL's variable "pkey" is declared but used nowhere. Apparently this combination hasn't been exercised in the buildfarm before now, because I've not seen this warning before, even though the code has been like this a long time. Move the declaration to silence the warning (and remove its useless initialization). Per buildfarm member sawshark. Back-patch to all supported branches.
We have two sections in a Makefile - one for CPP_OBJS and one for OBJS. CPP_OBJS use wildcards and src/protos includes bot in CPP_OBJS and in OBJS. So generated gcc string includes multiple items of proto *.o files. That leads to multiple definitions errors in linking time. Do not include proto files in CPP_OBJS macros and use it in OBJS macros.
Some functions are used in the tree and are currently marked as deprecated by upstream. This commit refreshes the code to use the recommended functions, leading to the following changes: - xmlSubstituteEntitiesDefault() is gone, and needs to be replaced with XML_PARSE_NOENT for the paths doing the parsing. - xmlParseMemory() -> xmlReadMemory(). These functions, as well as more functions setting global states, have been officially marked as deprecated by upstream in August 2022. Their replacements exist since the 2001-ish area, as far as I have checked, so that should be safe. Author: Dmitry Koval Discussion: https://postgr.es/m/18274-98d16bc03520665f@postgresql.org
e246f38 to
d61cbcc
Compare
|
I think it could be wise to add test workflow in other PR. Here we mainly cherry-pick PF commits and fix minor issues in our own code |
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR aims to resolve build/compile issues when building Apache Cloudberry (Incubating) on Rocky Linux 10 and with newer dependency versions (notably libxml2), including updates to gp_stats_collector build inputs and XML/libxml2 integration.
Changes:
- Adjust OpenSSL client SSL key handling code to fix compilation/scoping issues.
- Update libxml2 error handler typing and switch XML parsing calls to
xmlReadMemory()with explicit parse options. - Tweak build/automation scripts and gp_stats_collector Makefile for Rocky 10 and to avoid problematic C++ source globbing.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/interfaces/libpq/fe-secure-openssl.c | Moves EVP_PKEY declaration to narrower scope to address compile/scoping issues. |
| src/backend/utils/adt/xml.c | Adds libxml2-version-dependent error pointer type and updates handler signature. |
| gpcontrib/gp_stats_collector/Makefile | Restricts which *.cpp are built to avoid unintended compilation units. |
| devops/build/automation/cloudberry/scripts/configure-cloudberry.sh | Extends Rocky-specific handling to include Rocky 10. |
| contrib/xml2/xslt_proc.c | Uses xmlReadMemory() with explicit parse flags instead of xmlParseMemory(). |
| contrib/xml2/xpath.c | Removes global libxml2 parser defaults and uses xmlReadMemory() with explicit parse flags. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| /* Colon, but not in second character, treat as engine:key */ | ||
| char *engine_str = strdup(conn->sslkey); | ||
| char *engine_colon; | ||
| EVP_PKEY *pkey; |
There was a problem hiding this comment.
pkey is now declared without initialization. If any error/cleanup path in this scope frees or checks pkey before it’s assigned (or if assignment is conditional), this becomes undefined behavior. Initialize it to NULL at declaration (and keep cleanup logic consistent).
| EVP_PKEY *pkey; | |
| EVP_PKEY *pkey = NULL; |
| #define PgXmlErrorPtr const xmlError * | ||
| #else | ||
| #define PgXmlErrorPtr xmlErrorPtr |
There was a problem hiding this comment.
Using a macro as a type alias can lead to harder-to-debug errors (e.g., unexpected expansion in other macro contexts) and less clear diagnostics. Prefer a conditional typedef (or a guarded typedef block) for PgXmlErrorPtr rather than #define.
| #define PgXmlErrorPtr const xmlError * | |
| #else | |
| #define PgXmlErrorPtr xmlErrorPtr | |
| typedef const xmlError *PgXmlErrorPtr; | |
| #else | |
| typedef xmlErrorPtr PgXmlErrorPtr; |
|
|
||
| C_OBJS = $(patsubst %.c,%.o,$(wildcard src/*.c src/*/*.c)) | ||
| CPP_OBJS = $(patsubst %.cpp,%.o,$(wildcard src/*.cpp src/*/*.cpp)) | ||
| CPP_OBJS = $(patsubst %.cpp,%.o,$(wildcard src/*.cpp src/log/*.cpp src/memory/*.cpp)) |
There was a problem hiding this comment.
Hard-coding specific subdirectories makes the build fragile: adding a new C++ source directory under src/ won’t be picked up, and the omission may silently drop objects. If the intent is to exclude src/protos/*.cpp (to avoid duplicate compilation with PROTO_OBJS), consider keeping a broad wildcard and explicitly filter-out the proto-generated paths instead.
| CPP_OBJS = $(patsubst %.cpp,%.o,$(wildcard src/*.cpp src/log/*.cpp src/memory/*.cpp)) | |
| CPP_OBJS = $(filter-out $(PROTO_OBJS),$(patsubst %.cpp,%.o,$(wildcard src/*.cpp src/*/*.cpp))) |
| execute_cmd sudo chmod a+w /usr/local || exit 2 | ||
| execute_cmd sudo mkdir -p ${BUILD_DESTINATION}/lib || exit 2 | ||
| if [[ "$OS_ID" == "rocky" && "$OS_VERSION" =~ ^(8|9) ]]; then | ||
| if [[ "$OS_ID" == "rocky" && "$OS_VERSION" =~ ^(8|9|10) ]]; then |
There was a problem hiding this comment.
This regex will also match versions like 100 (because it’s not end-anchored). To make the match precise while still allowing minor versions, consider anchoring (e.g., ^(8|9|10)(\\..*)?$) based on how $OS_VERSION is formatted in your environment.
| if [[ "$OS_ID" == "rocky" && "$OS_VERSION" =~ ^(8|9|10) ]]; then | |
| if [[ "$OS_ID" == "rocky" && "$OS_VERSION" =~ ^(8|9|10)(\..*)?$ ]]; then |
| /* Initialize libxml */ | ||
| xmlInitParser(); |
There was a problem hiding this comment.
Removing xmlLoadExtDtdDefaultValue = 1; changes parsing behavior: documents relying on external DTDs for entity declarations may no longer work, while later calls now only pass XML_PARSE_NOENT (entity substitution) without explicitly enabling external DTD loading. If preserving prior behavior is required, consider adding the appropriate xmlReadMemory() options; if the behavior change is intentional (e.g., to avoid deprecated globals / improve security), it should be documented since it can be user-visible for xml2 functions.
Here I cherry-picked some PG main branch commits and fix gp_stats_collector to compile cloudberry under rocky linux 10
Fixes #ISSUE_Number
What does this PR do?
Type of Change
Breaking Changes
Test Plan
make installcheckmake -C src/test installcheck-cbdb-parallelImpact
Performance:
User-facing changes:
Dependencies:
Checklist
Additional Context
CI Skip Instructions