Conversation
…pple specs The WooCommerce TOTP test asserted only a URL match on /my-dashboard/, which passes even when the user is not actually authenticated (QAO-104). Add a Log Out link visibility check that proves the session is real. Delete authentication__apple.spec.ts: both tests inside it have been unconditionally skipped since Sept 2025 (account gets locked on Apple) and the WooCommerce case duplicates coverage already provided by the TOTP spec. Backend JWT validation coverage moves to a wpcom PHPUnit isolated test in a separate PR. Delete the legacy Jest-runner duplicate signup__woo-apple.ts (quarantined, scenario already covered by the Playwright Test specs).
…thentication-woocommerce-user
Jetpack Cloud Live (direct link)
Automattic for Agencies Live (direct link)
Dashboard Live (dotcom) (direct link)
|
DomainSearchComponent.selectSuggestion waits up to 30s for the Add-to-cart button to detach, but only 10s (the default) for the Continue button to appear on the same row. On slow CI runs the Continue button renders after 10s but well under 30s, causing setup-domain flow tests to fail with the Continue button actually present in the DOM snapshot captured at the moment of failure. Align the two waits at 30s.
…thentication-woocommerce-user
…-woocommerce-user' into fix/QAO-104-apple-authentication-woocommerce-user
|
I've added, in this PR, a fix for a test that would consistently fail in CI. |
|
This PR modifies the release build for the following Calypso Apps: For info about this notification, see here: PCYsg-OT6-p2
To test WordPress.com changes, run |
alopezari
left a comment
There was a problem hiding this comment.
Thanks for working on this @lucatume!
LGTM, just let's keep monitoring these tests in case extending the timeout doesn't solve the issues.
As a note, I've run the tests according to the testing instructions and there's one that failed:
[pixel] › specs/flows/setup-domain__flows.spec.ts:613:7 › Setup Domain Flows › As a new user, I can transfer an external domain to a new site @calypso-release › And I select the Personal plan
I ran it against staging, and it didn't fail because of the test, but because the site got stuck for more than 1 minute trying to register a domain:
video.webm
Could this be related to Calypso Live? Maybe it's a silly question, but still I'm not 100% sure about the scope of Calypso Live, it's an area where I haven't worked much yet.
Let me approve it because the PR adds good improvements anyway.
Part of QAO-104
Proposed Changes
authentication__totp.spec.ts:111-115) with a Log Out link visibility check that proves the session is real, not just a URL matchauthentication__apple.spec.ts: both tests inside it were unconditionallytest.skip(true, ...)since Sept 2025 (PR e2e tests: skip Apple authentication tests #106099), and the WooCommerce case duplicates coverage already provided by the TOTP specsignup__woo-apple.ts(quarantined, scenario covered by Playwright Test specs)DomainSearchComponent.selectSuggestionso theContinuebutton wait uses a 30s timeout, matching the precedingAdd to cartdetach waitWhy are these changes being made?
QAO-104 reported a false-negative in the Apple-for-WooCommerce E2E test: the final check only verified a redirect URL to
woocommerce.com, which passes even when the user is not actually authenticated. Investigation found that:This PR fixes the live test (TOTP) with a DOM-level logged-in check and removes the dead Apple specs. Backend JWT validation coverage is added in a companion wpcom PR: 211934-ghe-Automattic/wpcom
E2E stabilization:
DomainSearchComponent.selectSuggestionContinue-button waitWhile reviewing CI for this PR, the two
flows/setup-domain__flows.spec.tsspecs ("create a free site and then add a domain WITHOUT a plan upgrade" and "create a paid site, add a domain, then cancel the plan") failed as new failures on desktop Chrome and mobile Pixel with:The error-context DOM snapshot captured at the moment of failure showed the
Continuebutton was present in the first list item — it simply rendered after the default 10s timeout. The preceding step in the same method waits up to 30s for theAdd to cartbutton to detach, so the short 10s window for the replacementContinuebutton to appear was the tight spot on slow CI.Fix: pass
{ timeout: 30000 }to theContinuewaitFor, matching the upstream detach wait. No behavior change on fast paths.Testing Instructions
Log outlocator resolves onwoocommerce.com/my-dashboard/yarn playwright test specs/flows/setup-domain__flows.spec.ts --reporter=list --repeat-each=5against a local Calypso and confirm all runs pass on both desktop Chrome and mobile Pixelnpx tsc --noEmitintest/e2e/yarn eslint test/e2e/specs/authentication/authentication__totp.spec.ts packages/calypso-e2e/src/lib/components/domain-search-component.ts