Skip to content

FEATURES.md

Latest commit

 

History

History
37 lines (31 loc) · 5.47 KB

FEATURES.md

File metadata and controls

37 lines (31 loc) · 5.47 KB

Features:

qsv has several features:

  • jemallocator (default) - use the jemalloc allocator (see Memory Allocator for more info).
  • mimalloc - use the mimalloc allocator (see Memory Allocator for more info).
  • apply - enable apply command. This swiss-army knife of CSV transformations is very powerful, but it has a lot of dependencies that increases both compile time and binary size.
  • fetch - enables the fetch & fetchpost commands.
  • foreach - enable foreach command.
  • geocode - enable geocode & geoconvert commands.
  • clipboard - enable clipboard command.
  • color - enable color command. Enables colorized tabular output. When the polars feature is also enabled, supports multiple non-CSV formats.
  • lens - enable lens command.
  • luau - enable luau command. Embeds a Luau interpreter into qsv. Luau has type-checking, sandboxing, additional language operators, increased performance & other improvements over Lua. Luau is the DSL of qsv - as its statically linked, has a MUCH smaller footprint (in both file size and memory without having to deal with Python's infamous Global Interpreter Lock) & is faster (in both startup & execution time) than Python.
  • polars - enables all Polars-powered commands (currently, joinp, pivotp, scoresql and sqlp and enables polars mode in count). It also enables support for reading .parquet, .ipc/.arrow and .json/.jsonl formats as well as .csv.gz, .csv.zst and .csv.zlib compressed files.
    Note that Polars is a very powerful library, but it has a lot of dependencies that drastically increases both compile time and binary size.
  • prompt - enable prompt command.
  • python - enable py command. Note that qsv will look for the shared library for the Python version (Python 3.10 & above supported) it was compiled against & will abort on startup if the library is not found, even if you're NOT using the py command. Check Python section for more info. Though Luau is the preferred DSL for qsv for all the reasons stated above, Python is still the lingua franca of data wrangling.
  • magika - enable Magika-powered, AI-based file type detection in the sniff command for enhanced MIME type identification. Falls back to the file-format crate in qsvlite/qsvdp variants. Not available on MUSL builds.
  • mcp - enable log command and MCP (Model Context Protocol) skill JSON generation for AI agent integration.
  • to - enables the to command.
  • self_update - enable self-update engine, checking GitHub for the latest release. Note that if you manually built qsv, self-update will only alert you about new releases (it checks GitHub for the latest release 10% of the time upon startup unless the QSV_NO_UPDATE environment variable is set). It will NOT offer the choice to update itself to the prebuilt binaries published on GitHub.
    You need not worry that your manually built qsv will be overwritten by a self-update.
    To check if your qsv build will have the option to self-update, run qsv --version. If you see self_update in the enabled features, and QSV_KIND is prebuilt* at the end, then you have the option to self-update.
  • ui - enables commands that require linking UI libraries - clipboard, color, lens and prompt. Disable this feature if you're building for a headless environment. Note that qsvdp and qsvlite does not enable the ui feature by default.

Special Features for building qsv binary variants:

  • feature_capable - enable to build qsv binary variant which is feature-capable. Also used by qsvmcp. (mutually exclusive with lite and datapusher_plus)

    • all_features - shortcut to build qsv binary variant with all features enabled (apply,fetch,foreach,geocode,luau,magika,mcp,polars,python,to,self_update,ui).

  • qsvmcp - enable to build qsvmcp binary variant - optimized for MCP server use with geocode, luau, mcp, polars, self_update, and to features. Shares src/main.rs with qsv. (mutually exclusive with lite and datapusher_plus)

  • lite - enable to build qsvlite binary variant with all features disabled. (mutually exclusive with feature_capable and datapusher_plus)

  • datapusher_plus - enable to build qsvdp binary variant - the DataPusher+ optimized qsv binary. (mutually exclusive with feature_capable and lite)

  • nightly - enable to turn on nightly/unstable features in the crc32fast, pyo3, rand, simd-json & foldhash crates when building with Rust nightly/unstable. Note that Polars has its own separate nightly-polars feature.

  • distrib_features - enable to build qsv binary variant with the core distribution features enabled (apply, fetch, foreach, geocode, luau, mcp, polars, python, to) - i.e. all features except self_update, ui and magika. This should make it easier for distro packagers to build qsv as qsv removes and adds features over time.

ℹ️ NOTE: qsvlite, as the name implies, always has non-default features disabled. qsv can be built with any combination of the above features using the cargo --features & --no-default-features flags. The prebuilt qsv binaries has all applicable features valid for the target platform.