Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,11 +61,10 @@ cargo +nightly fmt \

### Feature Flags

- `default-engine` / `default-engine-rustls` / `default-engine-native-tls` -- async
Arrow/Tokio engine (pick one TLS backend)
- `default-engine-rustls` / `default-engine-native-tls` -- async Arrow/Tokio engine (pick a TLS backend)
- `arrow`, `arrow-XX`, `arrow-YY` -- Arrow version selection (kernel tracks the latest two
major Arrow releases; `arrow` defaults to latest). Kernel itself does not depend on Arrow,
but default-engine does.
but the default engine does.
- `arrow-conversion`, `arrow-expression` -- Arrow interop (auto-enabled by default engine)
- `prettyprint` -- enables Arrow pretty-print helpers (primarily test/example oriented)
- `clustered-table` -- clustered table write support (experimental)
Expand Down
13 changes: 7 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,16 +55,17 @@ consumer's own `Engine` trait, the kernel has a feature flag to enable a default
delta_kernel = "0.21.0"

# or turn on the default engine, based on latest arrow
delta_kernel = { version = "0.21.0", features = ["default-engine", "arrow"] }
delta_kernel = { version = "0.21.0", features = ["default-engine-rustls", "arrow"] }
```

### Feature flags
There are more feature flags in addition to the `default-engine` flag shown above. Relevant flags
include:
There are more feature flags in addition to the `default-engine-rustls` flag shown above. Relevant
flags include:

| Feature flag | Description |
| ------------- | ------------- |
| `default-engine` | Turn on the 'default' engine: async, arrow-based `Engine` implementation |
| `default-engine-rustls` | Turn on the 'default' engine with rustls TLS backend |
| `default-engine-native-tls`| Turn on the 'default' engine with native-tls TLS backend |
| `arrow-conversion` | Conversion utilities for arrow/kernel schema interoperation |
| `arrow-expression` | Expression system implementation for arrow |

Expand All @@ -74,8 +75,8 @@ are still unstable. We therefore may break APIs within minor releases (that is,
we will not break APIs in patch releases (`0.1.0` -> `0.1.1`).

## Arrow versioning
If you enable the `default-engine` feature, you get an implementation of the `Engine` trait that
uses [Arrow] as its data format.
If you enable a default engine feature (`default-engine-rustls` or `default-engine-native-tls`),
you get an implementation of the `Engine` trait that uses [Arrow] as its data format.

The [`arrow crate`](https://docs.rs/arrow/latest/arrow/) tends to release new major versions rather
frequently. To enable engines that already integrate arrow to also integrate kernel and not force
Expand Down
1 change: 0 additions & 1 deletion ffi/cbindgen.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ namespace = "ffi"
cpp_compat = true

[defines]
"feature = default-engine" = "DEFINE_DEFAULT_ENGINE"
"feature = default-engine-rustls" = "DEFINE_DEFAULT_ENGINE_RUSTLS"
"feature = default-engine-base" = "DEFINE_DEFAULT_ENGINE_BASE"

Expand Down
4 changes: 2 additions & 2 deletions kernel/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,8 @@ arrow-expression = ["need-arrow"]
# Schema diffing functionality (experimental)
schema-diff = []

# this is an 'internal' feature flag which has all the shared bits from default-engine and
# default-engine-rustls
# this is an 'internal' feature flag which has all the shared bits from default-engine-native-tls
# and default-engine-rustls
default-engine-base = [
"arrow-conversion",
"arrow-expression",
Expand Down
4 changes: 2 additions & 2 deletions kernel/src/engine/mod.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
//! Provides an engine implementation that implements the required traits. The engine can optionally
//! be built into the kernel by setting the `default-engine` feature flag. See the related module
//! for more information.
//! be built into the kernel by setting the `default-engine-rustls` or `default-engine-native-tls`
//! feature flag. See the related module for more information.

#[cfg(feature = "arrow-expression")]
use crate::parquet::arrow::arrow_reader::ArrowReaderOptions;
Expand Down
Loading