Skip to content

feat(plugin-import-export): add support for collection-level and field-level hooks#16252

Open
paulpopus wants to merge 8 commits intomainfrom
feat/import-export-add-support-for-hooks
Open

feat(plugin-import-export): add support for collection-level and field-level hooks#16252
paulpopus wants to merge 8 commits intomainfrom
feat/import-export-add-support-for-hooks

Conversation

@paulpopus
Copy link
Copy Markdown
Contributor

@paulpopus paulpopus commented Apr 11, 2026

Why

Field-level hooks are essential because:

  1. Co-location — the transform behavior lives next to the field definition, not in a separate collection config
  2. Reusability — a shared field config carries its hooks to every collection that uses it, zero duplication
  3. Deeply nested fields — transforming group.tabs.namedTab.array[0].field in a collection-level hook requires manually navigating the full document structure; field-level hooks handle this transparently

This PR keeps both levels: field-level hooks for per-field transforms, collection-level hooks for batch-wide operations (masking, logging, filtering).

Field-level hooks

Defined per-field via custom['plugin-import-export'].hooks:

import type { FieldBeforeExportHook } from '@payloadcms/plugin-import-export'

const authorField = {
  name: 'author',
  type: 'relationship',
  relationTo: 'users',
  custom: {
    'plugin-import-export': {
      hooks: {
        // Runs before the field value is written to the export file
        beforeExport: (({ value, columnName, row, format }) => {
          if (format === 'csv' && value && typeof value === 'object' && 'id' in value) {
            row[`${columnName}_id`] = value.id
            row[`${columnName}_email`] = value.email
            return undefined // let row mutation take effect
          }
          return value
        }) satisfies FieldBeforeExportHook,

        // Runs before the field value is written to the database
        beforeImport: ({ value, columnName, data, format }) => {
          if (format === 'csv') {
            return data[`${columnName}_id`] ?? undefined
          }
          return value
        },
      },
    },
  },
}

Collection-level hooks

Defined in the plugin config:

importExportPlugin({
  collections: [
    {
      slug: 'users',
      export: {
        hooks: {
          before: ({ data, format }) => {
            // Mask sensitive fields from the entire batch
            return data.map(({ passwordHash, ssn, ...safe }) => safe)
          },
          after: ({ batchNumber, totalBatches, req }) => {
            req.payload.logger.info(`Export batch ${batchNumber}/${totalBatches} written`)
          },
        },
      },
    },
  ],
})

Execution order

field-level hooks.beforeExport / hooks.beforeImport run first (per-field, per-document), then collection-level hooks.before / hooks.after run on the already-transformed batch.

Migration from toCSV / fromCSV

toCSV and fromCSV remain fully functional but are deprecated — removed in v4.0. Migration is a 1:1 rename plus the new format parameter:

// Before (deprecated)
custom: {
  'plugin-import-export': {
    toCSV: ({ value, row }) => { ... },
    fromCSV: ({ value, data }) => { ... },
  },
}

// After
custom: {
  'plugin-import-export': {
    hooks: {
      beforeExport: ({ value, row, format }) => { ... },
      beforeImport: ({ value, data, format }) => { ... },
    },
  },
}

What changed

New types

FieldBeforeExportHook — field-level export hook type (same args as ToCSVFunction + format)
FieldBeforeImportHook — field-level import hook type (same args as FromCSVFunction + format)
ToCSVFunction / FromCSVFunction — now deprecated aliases

JSON format support (new)

  • Field-level hooks now fire for JSON exports/imports, not just CSV
  • New applyFieldExportHooks / applyFieldImportHooks utilities traverse nested JSON documents and apply field hooks at each position
  • Preview handlers also apply field hooks for both CSV and JSON

Bug fixes

  • Fixed parentPath key computation in getExportFieldFunctions / getImportFieldFunctions — named tabs inside groups now produce correct underscore-separated keys (pre-existing bug)
  • Fixed slice(-0) bug in import batchProcessor.ts — ImportAfterHook was receiving all accumulated errors instead of per-batch errors when a batch had zero failures
  • Removed unused fs import from hooks.int.spec.ts

Renamed internals

  • toCSVFunctions → exportFieldHooks throughout all source files
  • fromCSVFunctions → importFieldHooks throughout all source files
  • Error messages updated from "toCSVFunction" to "field export hook"

Checklist

  • Documentation updated with new API, migration guide, execution order
  • Test collection PostsWithFieldHooks with field-level hooks
  • 12 new integration tests (CSV export, JSON export, format parameter, deeply nested fields, CSV import, JSON import, backward compatibility with toCSV, backward compatibility with fromCSV, execution order, reusable field config, priority, default behaviour)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 11, 2026

📦 esbuild Bundle Analysis for payload

This analysis was generated by esbuild-bundle-analyzer. 🤖

Meta File Out File Size (raw) Note
packages/next/meta_index.json esbuild/index.js 985.21 KB 🆕 Added
packages/payload/meta_index.json esbuild/index.js 1.39 MB 🆕 Added
packages/payload/meta_shared.json esbuild/exports/shared.js 191.27 KB 🆕 Added
packages/richtext-lexical/meta_client.json esbuild/exports/client_optimized/index.js 289.51 KB 🆕 Added
packages/ui/meta_client.json esbuild/exports/client_optimized/index.js 1.18 MB 🆕 Added
packages/ui/meta_shared.json esbuild/exports/shared_optimized/index.js 16.32 KB 🆕 Added
Largest paths These visualization shows top 20 largest paths in the bundle.

Meta file: packages/next/meta_index.json, Out file: esbuild/index.js

Path Size
../../node_modules ${{\color{Goldenrod}{ ████████████████████▌ }}}$ 82.4%, 807.63 KB
dist/views/Version ${{\color{Goldenrod}{ █▎ }}}$ 5.3%, 51.49 KB
dist/views/Dashboard ${{\color{Goldenrod}{ ▌ }}}$ 2.2%, 21.37 KB
dist/views/Document ${{\color{Goldenrod}{ ▍ }}}$ 1.7%, 16.59 KB
dist/views/List ${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 11.38 KB
dist/views/Root ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 9.90 KB
dist/views/Versions ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 6.17 KB
dist/views/API ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 6.13 KB
dist/elements/Nav ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 5.96 KB
dist/views/Account ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 5.55 KB
dist/elements/DocumentHeader ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 4.81 KB
dist/views/Login ${{\color{Goldenrod}{ }}}$ 0.4%, 4.40 KB
dist/layouts/Root ${{\color{Goldenrod}{ }}}$ 0.3%, 3.20 KB
dist/views/ForgotPassword ${{\color{Goldenrod}{ }}}$ 0.3%, 3.13 KB
dist/views/CreateFirstUser ${{\color{Goldenrod}{ }}}$ 0.3%, 2.81 KB
dist/templates/Default ${{\color{Goldenrod}{ }}}$ 0.3%, 2.64 KB
dist/views/BrowseByFolder ${{\color{Goldenrod}{ }}}$ 0.3%, 2.61 KB
dist/views/CollectionFolders ${{\color{Goldenrod}{ }}}$ 0.2%, 2.44 KB
dist/views/ResetPassword ${{\color{Goldenrod}{ }}}$ 0.2%, 2.40 KB
dist/views/Logout ${{\color{Goldenrod}{ }}}$ 0.2%, 1.94 KB
(other) ${{\color{Goldenrod}{ ████▍ }}}$ 17.6%, 172.90 KB

Meta file: packages/payload/meta_index.json, Out file: esbuild/index.js

Path Size
../../node_modules ${{\color{Goldenrod}{ █████████████████▏ }}}$ 68.7%, 951.98 KB
dist/fields/hooks ${{\color{Goldenrod}{ ▊ }}}$ 3.2%, 44.07 KB
dist/collections/operations ${{\color{Goldenrod}{ ▋ }}}$ 2.9%, 39.96 KB
dist/versions/migrations ${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 18.50 KB
dist/auth/operations ${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 15.63 KB
dist/fields/config ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 14.16 KB
dist/globals/operations ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 13.32 KB
dist/utilities/configToJSONSchema.js ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 13.13 KB
dist/queues/operations ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 12.71 KB
dist/fields/validations.js ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 10.54 KB
dist/bin/generateImportMap ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 9.08 KB
dist/collections/config ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 8.91 KB
dist/config/orderable ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 8.00 KB
dist/uploads/fetchAPI-multipart ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.80 KB
dist/index.js ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.79 KB
dist/database/migrations ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 7.54 KB
dist/config/sanitize.js ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 6.26 KB
dist/collections/endpoints ${{\color{Goldenrod}{ }}}$ 0.4%, 6.23 KB
dist/auth/strategies ${{\color{Goldenrod}{ }}}$ 0.4%, 5.50 KB
dist/queues/config ${{\color{Goldenrod}{ }}}$ 0.4%, 5.31 KB
(other) ${{\color{Goldenrod}{ ███████▊ }}}$ 31.3%, 433.14 KB

Meta file: packages/payload/meta_shared.json, Out file: esbuild/exports/shared.js

Path Size
../../node_modules ${{\color{Goldenrod}{ ███████████████████▊ }}}$ 79.4%, 148.89 KB
dist/fields/validations.js ${{\color{Goldenrod}{ █▍ }}}$ 5.6%, 10.54 KB
dist/config/orderable ${{\color{Goldenrod}{ ▍ }}}$ 1.7%, 3.13 KB
dist/fields/baseFields ${{\color{Goldenrod}{ ▍ }}}$ 1.5%, 2.79 KB
dist/utilities/deepCopyObject.js ${{\color{Goldenrod}{ ▎ }}}$ 1.4%, 2.54 KB
dist/auth/cookies.js ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 1.55 KB
dist/utilities/flattenTopLevelFields.js ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 1.42 KB
dist/fields/config ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 1.28 KB
dist/utilities/getVersionsConfig.js ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 1.04 KB
dist/utilities/flattenAllFields.js ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 943 B
dist/folders/utils ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 916 B
dist/utilities/unflatten.js ${{\color{Goldenrod}{ }}}$ 0.4%, 779 B
dist/utilities/sanitizeUserDataForEmail.js ${{\color{Goldenrod}{ }}}$ 0.4%, 713 B
dist/utilities/getFieldPermissions.js ${{\color{Goldenrod}{ }}}$ 0.3%, 651 B
dist/collections/config ${{\color{Goldenrod}{ }}}$ 0.3%, 570 B
dist/bin/generateImportMap ${{\color{Goldenrod}{ }}}$ 0.3%, 561 B
dist/auth/sessions.js ${{\color{Goldenrod}{ }}}$ 0.3%, 525 B
dist/fields/getFieldPaths.js ${{\color{Goldenrod}{ }}}$ 0.3%, 485 B
dist/utilities/getSafeRedirect.js ${{\color{Goldenrod}{ }}}$ 0.2%, 423 B
dist/utilities/deepMerge.js ${{\color{Goldenrod}{ }}}$ 0.2%, 413 B
(other) ${{\color{Goldenrod}{ █████▏ }}}$ 20.6%, 38.72 KB

Meta file: packages/richtext-lexical/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Path Size
dist/features/blocks ${{\color{Goldenrod}{ ███▏ }}}$ 12.7%, 36.34 KB
dist/lexical/plugins ${{\color{Goldenrod}{ ██▊ }}}$ 11.4%, 32.65 KB
dist/lexical/ui ${{\color{Goldenrod}{ ██▏ }}}$ 8.5%, 24.36 KB
dist/features/experimental_table ${{\color{Goldenrod}{ ██ }}}$ 8.3%, 23.70 KB
dist/packages/@lexical ${{\color{Goldenrod}{ █▋ }}}$ 6.6%, 18.99 KB
dist/features/link ${{\color{Goldenrod}{ █▋ }}}$ 6.5%, 18.53 KB
dist/features/toolbars ${{\color{Goldenrod}{ █▍ }}}$ 5.6%, 16.08 KB
dist/features/upload ${{\color{Goldenrod}{ █▏ }}}$ 4.8%, 13.77 KB
dist/features/textState ${{\color{Goldenrod}{ ▉ }}}$ 3.9%, 11.08 KB
dist/features/relationship ${{\color{Goldenrod}{ ▊ }}}$ 3.2%, 9.03 KB
dist/lexical/utils ${{\color{Goldenrod}{ ▊ }}}$ 3.1%, 8.79 KB
dist/features/converters ${{\color{Goldenrod}{ ▋ }}}$ 2.9%, 8.36 KB
dist/features/debug ${{\color{Goldenrod}{ ▋ }}}$ 2.6%, 7.40 KB
dist/utilities/fieldsDrawer ${{\color{Goldenrod}{ ▋ }}}$ 2.5%, 7.15 KB
dist/lexical/config ${{\color{Goldenrod}{ ▍ }}}$ 1.8%, 5.08 KB
dist/features/lists ${{\color{Goldenrod}{ ▍ }}}$ 1.7%, 5.00 KB
dist/features/format ${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 3.46 KB
dist/lexical/LexicalEditor.js ${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 3.23 KB
dist/field/Field.js ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 2.80 KB
dist/lexical/nodes ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 2.66 KB
(other) ${{\color{Goldenrod}{ █████████████████████▊ }}}$ 87.3%, 249.86 KB

Meta file: packages/ui/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Path Size
../../node_modules ${{\color{Goldenrod}{ ████████████▎ }}}$ 49.3%, 579.12 KB
dist/elements/FolderView ${{\color{Goldenrod}{ ▋ }}}$ 2.5%, 29.38 KB
dist/elements/BulkUpload ${{\color{Goldenrod}{ ▌ }}}$ 2.4%, 28.24 KB
dist/elements/WhereBuilder ${{\color{Goldenrod}{ ▍ }}}$ 1.5%, 17.38 KB
dist/views/Edit ${{\color{Goldenrod}{ ▍ }}}$ 1.5%, 17.30 KB
dist/forms/Form ${{\color{Goldenrod}{ ▎ }}}$ 1.4%, 15.91 KB
dist/fields/Relationship ${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 15.79 KB
dist/elements/Table ${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 15.77 KB
dist/fields/Upload ${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 14.22 KB
dist/fields/Blocks ${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 13.90 KB
dist/elements/QueryPresets ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 10.36 KB
dist/elements/PublishButton ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 9.11 KB
dist/providers/Folders ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 8.46 KB
dist/elements/HTMLDiff ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 8.38 KB
dist/elements/ListHeader ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 8.06 KB
dist/fields/Array ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 7.73 KB
dist/views/CollectionFolder ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.50 KB
dist/views/List ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.36 KB
dist/elements/ReactSelect ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.33 KB
dist/elements/LivePreview ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.03 KB
(other) ${{\color{Goldenrod}{ ████████████▋ }}}$ 50.7%, 595.21 KB

Meta file: packages/ui/meta_shared.json, Out file: esbuild/exports/shared_optimized/index.js

Path Size
dist/graphics/Logo ${{\color{Goldenrod}{ █████ }}}$ 20.0%, 3.12 KB
../../node_modules ${{\color{Goldenrod}{ ████▎ }}}$ 17.0%, 2.65 KB
dist/graphics/Icon ${{\color{Goldenrod}{ ██▍ }}}$ 9.8%, 1.52 KB
dist/utilities/formatDocTitle ${{\color{Goldenrod}{ ██▏ }}}$ 8.5%, 1.32 KB
dist/providers/TableColumns ${{\color{Goldenrod}{ █▍ }}}$ 5.5%, 862 B
dist/utilities/groupNavItems.js ${{\color{Goldenrod}{ █▎ }}}$ 5.2%, 814 B
dist/utilities/getGlobalData.js ${{\color{Goldenrod}{ █▏ }}}$ 4.9%, 762 B
dist/utilities/api.js ${{\color{Goldenrod}{ █▏ }}}$ 4.8%, 756 B
dist/elements/Translation ${{\color{Goldenrod}{ ▊ }}}$ 3.2%, 493 B
dist/utilities/handleTakeOver.js ${{\color{Goldenrod}{ ▋ }}}$ 2.8%, 440 B
dist/utilities/traverseForLocalizedFields.js ${{\color{Goldenrod}{ ▋ }}}$ 2.6%, 399 B
dist/elements/withMergedProps ${{\color{Goldenrod}{ ▌ }}}$ 2.2%, 339 B
dist/utilities/getVisibleEntities.js ${{\color{Goldenrod}{ ▌ }}}$ 2.1%, 329 B
dist/utilities/getNavGroups.js ${{\color{Goldenrod}{ ▍ }}}$ 1.9%, 301 B
dist/elements/WithServerSideProps ${{\color{Goldenrod}{ ▍ }}}$ 1.5%, 232 B
dist/utilities/handleGoBack.js ${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 180 B
dist/fields/mergeFieldStyles.js ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 159 B
dist/utilities/handleBackToDashboard.js ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 152 B
dist/forms/Form ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 147 B
dist/utilities/abortAndIgnore.js ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 146 B
(other) ${{\color{Goldenrod}{ ████████████████████ }}}$ 80.0%, 12.51 KB
Details

Next to the size is how much the size has increased or decreased compared with the base branch of this PR.

  • ‼️: Size increased by 20% or more. Special attention should be given to this.
  • ⚠️: Size increased in acceptable range (lower than 20%).
  • ✅: No change or even downsized.
  • 🗑️: The out file is deleted: not found in base branch.
  • 🆕: The out file is newly found: will be added to base branch.

Copy link
Copy Markdown
Contributor

@DanRibbens DanRibbens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need help understanding this. I know the API of toCSV is messy, but it is very contextually easy to work with when you have a field configured in code that gets shared across collections for example. Also writing hooks at the collection level for deeply nested fields is a PITA to work with.

Comment thread docs/plugins/import-export.mdx Outdated
@paulpopus paulpopus marked this pull request as ready for review April 16, 2026 12:39
@paulpopus paulpopus requested a review from denolfe as a code owner April 16, 2026 12:39
@paulpopus paulpopus changed the title feat(plugin-import-export): add support for hooks feat(plugin-import-export): add support for collection-level and field-level hooks Apr 16, 2026
@PatrikKozak PatrikKozak self-assigned this Apr 16, 2026
try {
const transformed = hook({
columnName: fieldKey,
data: doc,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data receives the current-scope doc, not the top-level document, right?

The type JSDoc says data is "The full flat row (CSV) or document (JSON) being imported", but for fields nested inside groups, arrays, or blocks, doc here is the narrowed sibling object at that level. A hook doing data[someTopLevelKey] will silently get undefined.

The export hook solves this by threading topDoc through every recursive call. Same fix needed here: add topDoc to ApplyArgs, initialize it as doc in applyFieldBeforeImportHooks, pass it down through all recursive calls, and change data: docdata: topDoc in the hook invocation.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

excerpt, hasManyNumber, hasOnePolymorphic_id seem like test collection field names, why are we logging these here?

field.custom?.['plugin-import-export']?.toCSV

if (typeof fieldExportHook === 'function') {
result[fullKey] = fieldExportHook as FieldBeforeExportHook
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Field hooks on block fields silently don't fire (CSV or JSON)

getExportFieldFunctions uses traverseFields to register user hooks under a static, index-free key — e.g. content_textBlock_text. At lookup time, flattenObject (CSV) constructs content_0_textBlock_text and falls back to text; applyFieldBeforeExportHooks (JSON) tries content_0_textBlock_text with no fallback at all. Neither path ever hits content_textBlock_text, so the hook is silently skipped.

Default handlers dodge this because they're also registered under baseKey = field.name (matching the text fallback), but user-defined hooks get no such alias.

Fix: strip runtime indices from the lookup key and try it as a secondary fallback in both flattenObject and applyFieldBeforeExportHooks.

PatrikKozak
PatrikKozak previously approved these changes Apr 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants