chore: Migrate bundle size telemetry upload to ff_pipeline_host by ChumpChief · Pull Request #27063 · microsoft/FluidFramework

ChumpChief · 2026-04-16T01:41:42Z

Description

Migrates bundle size telemetry upload from the old npm install @ff-internal/telemetry-generator@latest temp-directory approach to the ff_pipeline_host-based pattern established in #26205 for stage timing and test pass rate telemetry. The old path was fragile: it installed an unpinned version with unpinned transitive dependencies, and required manual package.json overrides to work around known vulnerabilities.

Changes:

Creates include-steps-upload-bundle-size.yml pluggable step template that downloads the bundleAnalysis pipeline artifact and runs telemetry-generator via pnpm exec from ff_pipeline_host
Wires the new template into the telemetry upload stage in both build-npm-client-package.yml and build-npm-package.yml, gated by taskBundleAnalysis
Widens the bundleAnalysis artifact publish condition to include all non-PR builds (previously required taskPublishBundleSizeArtifacts: true)
Removes the inline bundle size telemetry blocks from both build templates
Deletes include-telemetry-setup.yml (no remaining consumers)
Cleans up old-path variables (pathToTelemetryGenerator, pathToTelemetryGeneratorHandlers) from include-vars-telemetry-generator.yml
Adds missing upload-telemetry/ and include-vars-telemetry-generator.yml to build-client.yml trigger paths

Reviewer Guidance

The review process is outlined on this wiki page.

Validated via manual pipeline run #392747 — the telemetry upload stage ran successfully with the new bundle size steps
build-bundle-size-artifacts.yml was intentionally left unchanged — it already publishes the artifact but doesn't upload telemetry. Adding telemetry there can be a follow-up if desired

Remove the taskPublishBundleSizeArtifacts gate from the bundleAnalysis artifact publish condition. The step is already gated at template level by taskBundleAnalysis, so the extra parameter check was redundant and prevented CI builds from publishing the artifact. This is a prerequisite for moving bundle size telemetry to the dedicated telemetry upload stage, which needs to download the artifact from a prior stage. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

New pluggable step template for uploading bundle size telemetry to Kusto. Downloads the bundleAnalysis pipeline artifact and runs telemetry-generator with the bundleSizeHandler, using the ff_pipeline_host-based installation (pnpm exec) instead of the old npm install path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Add include-steps-upload-bundle-size.yml to the telemetry_upload_steps list in both build-npm-client-package.yml and build-npm-package.yml, gated by taskBundleAnalysis. This follows the same conditional pattern used for test pass rate telemetry in include-test-real-service.yml. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Remove the old inline bundle size telemetry upload from both build-npm-client-package.yml and build-npm-package.yml. This code installed telemetry-generator via npm into a temp directory and ran bundleSizeHandler.js from there. The functionality is now provided by the dedicated telemetry upload stage using ff_pipeline_host. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

This template installed telemetry-generator via npm into a temp directory with manual dependency overrides. It has no remaining consumers after migrating bundle size telemetry to the ff_pipeline_host approach. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Remove pathToTelemetryGenerator and pathToTelemetryGeneratorHandlers variables from include-vars-telemetry-generator.yml and their references in diagnostic print blocks. These were only used by the now-deleted inline bundle size telemetry upload. All telemetry now uses the ff_pipeline_host-based *New variants. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Override telemetry parameter to true so the telemetry upload stage runs during manual pipeline runs. This allows validating the bundle size telemetry migration end-to-end. TODO: Revert before merging. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Remove the Manual build reason exclusion from the telemetry upload stage condition so it runs during manual pipeline testing. TODO: Revert before merging. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Restore telemetry parameter to IndividualCI-only gating and re-add the Manual build reason exclusion on the telemetry upload stage. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

The upload-telemetry/ directory and include-vars-telemetry-generator.yml were missing from both the CI trigger and PR trigger path filters in build-client.yml. Changes to these templates would not have triggered a pipeline run. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Migrates bundle size telemetry upload in Azure Pipelines from an ad-hoc npm install @ff-internal/telemetry-generator@latest flow to the pinned ff_pipeline_host pattern already used for other telemetry uploads, improving reliability and reducing supply-chain/vulnerability exposure.

Changes:

Adds a pluggable step template to download the bundleAnalysis artifact and submit bundle-size telemetry via pnpm exec telemetry-generator from ff_pipeline_host.
Wires the new bundle-size upload steps into the existing upload-telemetry stage for both build-npm-client-package.yml and build-npm-package.yml.
Publishes the bundleAnalysis artifact for all non-PR builds when bundle analysis is enabled, and removes the old inline telemetry-generator install/setup path (including deleting include-telemetry-setup.yml).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tools/pipelines/templates/upload-telemetry/include-steps-upload-bundle-size.yml	New step template to download `bundleAnalysis` artifact and upload bundle-size telemetry using `ff_pipeline_host`.
tools/pipelines/templates/include-vars-telemetry-generator.yml	Removes old temp-install variables tied to the deprecated telemetry-generator installation approach.
tools/pipelines/templates/include-telemetry-setup.yml	Deleted; no remaining consumers after migrating to `ff_pipeline_host`.
tools/pipelines/templates/build-npm-package.yml	Removes inline bundle-size telemetry upload, always publishes bundleAnalysis artifact on non-PR builds, and plugs in the new bundle-size upload step template.
tools/pipelines/templates/build-npm-client-package.yml	Same as above, for the client build template.
tools/pipelines/build-client.yml	Updates trigger path filters to include the telemetry upload templates/vars so pipeline triggers on relevant changes.

Comments suppressed due to low confidence (1)

tools/pipelines/templates/build-npm-package.yml:645

The bundle-size upload steps are included whenever taskBundleAnalysis is true, but the bundleAnalysis artifact is only published on non-PR runs. If a consumer ever sets telemetry: true for PR builds, this will attempt to download a missing artifact and fail the telemetry stage. Consider additionally gating these steps (or the tasks inside the template) on ne(variables['Build.Reason'], 'PullRequest') to match the artifact publish condition.

                  pipelineIdentifierForTelemetry: BuildClient
              - ${{ if eq(parameters.taskBundleAnalysis, true) }}:
                - template: /tools/pipelines/templates/upload-telemetry/include-steps-upload-bundle-size.yml@self

Copilot · 2026-04-16T01:54:02Z

                    - output: pipelineArtifact
                      displayName: Publish Artifacts - bundle-analysis
-                      condition: and( succeeded(), ne(variables['Build.Reason'], 'PullRequest'), eq(${{ parameters.taskPublishBundleSizeArtifacts }}, true) )
+                      condition: and( succeeded(), ne(variables['Build.Reason'], 'PullRequest') )
                      targetPath: $(Build.ArtifactStagingDirectory)/bundleAnalysis
                      artifactName: bundleAnalysis


This artifact publish condition no longer uses taskPublishBundleSizeArtifacts, but the parameter still exists on the template. If publishing is now always required when taskBundleAnalysis is enabled, consider removing taskPublishBundleSizeArtifacts (and any callers) to avoid a misleading/unused knob; otherwise, keep the parameter in the condition so callers can still opt out of publishing the artifact.

Copilot · 2026-04-16T01:54:02Z

                  # Using 'all' signifies that we will upload all stages
                  stageIdFilter: 'all'
                  pipelineIdentifierForTelemetry: BuildClient
+              - ${{ if eq(parameters.taskBundleAnalysis, true) }}:


The bundle-size upload steps are included whenever taskBundleAnalysis is true, but the bundleAnalysis artifact is only published on non-PR runs. If a consumer ever sets telemetry: true for PR builds, this will attempt to download a missing artifact and fail the telemetry stage. Consider additionally gating these steps (or the tasks inside the template) on ne(variables['Build.Reason'], 'PullRequest') to match the artifact publish condition.

Suggested change

- ${{ if eq(parameters.taskBundleAnalysis, true) }}:

- ${{ if and(eq(parameters.taskBundleAnalysis, true), ne(variables['Build.Reason'], 'PullRequest')) }}:

Copilot · 2026-04-16T01:54:03Z

+      echo "Contents of $(Agent.TempDirectory)/bundleAnalysis:"
+      ls -laR $(Agent.TempDirectory)/bundleAnalysis
+


The recursive ls -laR of the entire downloaded bundleAnalysis artifact can generate extremely large logs (and extra time) if the artifact contains many files. Consider limiting the listing to the expected subfolder(s) (e.g. the bundle-size-tests output) or making the recursive listing conditional on a debug flag (like System.Debug).

Suggested change

echo "Contents of $(Agent.TempDirectory)/bundleAnalysis:"

ls -laR $(Agent.TempDirectory)/bundleAnalysis

ARTIFACT_ROOT="$(Agent.TempDirectory)/bundleAnalysis"

BUNDLE_SIZE_OUTPUT="$ARTIFACT_ROOT/@fluid-example/bundle-size-tests"

echo "Top-level contents of $ARTIFACT_ROOT:"

ls -la "$ARTIFACT_ROOT"

if [ "${SYSTEM_DEBUG:-false}" = "true" ] && [ -d "$BUNDLE_SIZE_OUTPUT" ]; then

echo "System.Debug=true; recursively listing expected bundle size output at $BUNDLE_SIZE_OUTPUT:"

ls -laR "$BUNDLE_SIZE_OUTPUT"

fi

Copilot · 2026-04-16T01:54:03Z

+      set -eu -o pipefail
+
+      echo "Listing files in '$WORK_FOLDER'"
+      ls -laR $WORK_FOLDER;


ls -laR $WORK_FOLDER is unquoted, which can break if the agent temp path ever contains spaces or glob characters. Quote $WORK_FOLDER consistently in the script commands.

Suggested change

ls -laR $WORK_FOLDER;

ls -laR "$WORK_FOLDER";

Copilot · 2026-04-16T01:54:03Z

                    - output: pipelineArtifact
                      displayName: Publish Artifacts - bundle-analysis
-                      condition: and( succeeded(), ne(variables['Build.Reason'], 'PullRequest'), eq(${{ parameters.taskPublishBundleSizeArtifacts }}, true) )
+                      condition: and( succeeded(), ne(variables['Build.Reason'], 'PullRequest') )
                      targetPath: $(Build.ArtifactStagingDirectory)/bundleAnalysis
                      artifactName: bundleAnalysis


This artifact publish condition no longer uses taskPublishBundleSizeArtifacts, but the parameter still exists on the template. If publishing is now always required when taskBundleAnalysis is enabled, consider removing taskPublishBundleSizeArtifacts (and any callers) to avoid a misleading/unused knob; otherwise, keep the parameter in the condition so callers can still opt out of publishing the artifact.

@ChumpChief I don't know if the build for anything other than the client release group (so the one where isBundleSizeArtifactsPipeline = true) will generate the correct bundle size reports to upload here. I'd say look at the other build pipelines triggered by the PR to see if the artifact they published makes sense. If not, I'd suggest we keep this condition as it was.

- Quote $WORK_FOLDER in the bundle size upload script for safety - Gate bundle size upload steps on non-PullRequest builds to match the artifact publish condition, preventing a missing-artifact failure if telemetry were ever enabled for PR builds - Rename taskPublishBundleSizeArtifacts to isBundleSizeArtifactsPipeline to better reflect its purpose: identifying the bundle-size-artifacts pipeline (which skips version setting), not controlling artifact publishing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…dle-size-telemetry

alexvy86

A few things that need to be addressed. Also just realized that build-bundle-size-artifacts still uses build-npm-package... not sure if it should actually use build-npm-client-package. But that's a separate topic.

alexvy86 · 2026-04-16T17:49:03Z

                    - output: pipelineArtifact
                      displayName: Publish Artifacts - bundle-analysis
-                      condition: and( succeeded(), ne(variables['Build.Reason'], 'PullRequest'), eq(${{ parameters.taskPublishBundleSizeArtifacts }}, true) )
+                      condition: and( succeeded(), ne(variables['Build.Reason'], 'PullRequest') )
                      targetPath: $(Build.ArtifactStagingDirectory)/bundleAnalysis
                      artifactName: bundleAnalysis


@ChumpChief I don't know if the build for anything other than the client release group (so the one where isBundleSizeArtifactsPipeline = true) will generate the correct bundle size reports to upload here. I'd say look at the other build pipelines triggered by the PR to see if the artifact they published makes sense. If not, I'd suggest we keep this condition as it was.

alexvy86 · 2026-04-16T17:56:27Z

+              - ${{ if and(eq(parameters.taskBundleAnalysis, true), ne(variables['Build.Reason'], 'PullRequest')) }}:
+                - template: /tools/pipelines/templates/upload-telemetry/include-steps-upload-bundle-size.yml@self


Not sure I'm convinced about moving the upload of this telemetry to this point, which requires publishing to feeds to have succeeded. The telemetry for the stage timing has to wait until this point because we want to report timing for all stages, but the bundle size one doesn't need to wait, and could now not get submitted when it would have before. If we split it into a separate stage, I think it needs to depend just on the build stage. Or if possible, make it just a separate job inside the build stage (that could run in parallel with the jobs that run tests).

Also, non-PR is not enough of a condition; the build-bundle-size-artifacts pipeline runs "as CI" (for every commit in main) in the public project and ff_pipeline_host doesn't exist there, so including the stage that uses it will fail. Bringing back the project=internal bit should be enough fix, I think.

- Move bundle size telemetry to a separate upload_telemetry_build stage in build-npm-client-package.yml that depends only on the build stage, so it isn't blocked by publish stage failures - Gate the bundle size telemetry stage on System.TeamProject=internal, since ff_pipeline_host is not available in the public project - Remove bundle size telemetry from build-npm-package.yml entirely, since only the client build produces the correct reports for upload - Restore the original artifact publish condition on build-npm-package.yml (gated on isBundleSizeArtifactsPipeline) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Replace stageIdFilter with telemetryDescription on the stage template to better describe what telemetry is being uploaded. Remove the implicit dependency behavior — callers now pass dependencies explicitly via dependsOn, which decouples the stage identity from its dependencies. The stageIdFilter parameter on the step templates (stage-timing and test-pass-rate) is unchanged — it still controls which stage's data to filter for in the telemetry query. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

ChumpChief and others added 10 commits April 14, 2026 18:03

chore: revert temporary testing overrides

7b38dd8

Restore telemetry parameter to IndividualCI-only gating and re-add the Manual build reason exclusion on the telemetry upload stage. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

ChumpChief marked this pull request as ready for review April 16, 2026 01:49

ChumpChief requested review from Abe27342, alexvy86 and Copilot April 16, 2026 01:49

Copilot started reviewing on behalf of ChumpChief April 16, 2026 01:49 View session

Copilot AI reviewed Apr 16, 2026

View reviewed changes

ChumpChief and others added 2 commits April 16, 2026 09:59

Merge remote-tracking branch 'upstream/main' into test/consistent-bun…

774a457

…dle-size-telemetry

alexvy86 reviewed Apr 16, 2026

View reviewed changes

ChumpChief and others added 2 commits April 16, 2026 14:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: Migrate bundle size telemetry upload to ff_pipeline_host#27063

chore: Migrate bundle size telemetry upload to ff_pipeline_host#27063
ChumpChief wants to merge 14 commits intomainfrom
test/consistent-bundle-size-telemetry

ChumpChief commented Apr 16, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 16, 2026

Uh oh!

Copilot AI Apr 16, 2026

Uh oh!

Copilot AI Apr 16, 2026

Uh oh!

Copilot AI Apr 16, 2026

Uh oh!

Copilot AI Apr 16, 2026

Uh oh!

alexvy86 Apr 16, 2026

Uh oh!

alexvy86 left a comment

Uh oh!

alexvy86 Apr 16, 2026

Uh oh!

alexvy86 Apr 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	- ${{ if eq(parameters.taskBundleAnalysis, true) }}:
	- ${{ if and(eq(parameters.taskBundleAnalysis, true), ne(variables['Build.Reason'], 'PullRequest')) }}:

		echo "Contents of $(Agent.TempDirectory)/bundleAnalysis:"
		ls -laR $(Agent.TempDirectory)/bundleAnalysis

-      echo "Contents of $(Agent.TempDirectory)/bundleAnalysis:"
-      ls -laR $(Agent.TempDirectory)/bundleAnalysis
+      ARTIFACT_ROOT="$(Agent.TempDirectory)/bundleAnalysis"
+      BUNDLE_SIZE_OUTPUT="$ARTIFACT_ROOT/@fluid-example/bundle-size-tests"
+      echo "Top-level contents of $ARTIFACT_ROOT:"
+      ls -la "$ARTIFACT_ROOT"
+      if [ "${SYSTEM_DEBUG:-false}" = "true" ] && [ -d "$BUNDLE_SIZE_OUTPUT" ]; then
+        echo "System.Debug=true; recursively listing expected bundle size output at $BUNDLE_SIZE_OUTPUT:"
+        ls -laR "$BUNDLE_SIZE_OUTPUT"
+      fi

		- ${{ if and(eq(parameters.taskBundleAnalysis, true), ne(variables['Build.Reason'], 'PullRequest')) }}:
		- template: /tools/pipelines/templates/upload-telemetry/include-steps-upload-bundle-size.yml@self

Conversation

ChumpChief commented Apr 16, 2026

Description

Reviewer Guidance

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

alexvy86 Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

alexvy86 left a comment

Choose a reason for hiding this comment

Uh oh!

alexvy86 Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

alexvy86 Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants