Skip to content

Fix flaky Firefox E2E tests: async URL assertions and canvas animation wait#525

Merged
gkorland merged 2 commits into
stagingfrom
copilot/fix-navbar-external-navigation-tests
Mar 8, 2026
Merged

Fix flaky Firefox E2E tests: async URL assertions and canvas animation wait#525
gkorland merged 2 commits into
stagingfrom
copilot/fix-navbar-external-navigation-tests

Conversation

Copilot AI commented Mar 8, 2026

Copy link
Copy Markdown
Contributor

Two navbar tests timeout on Firefox due to synchronous URL assertions firing before external navigation completes. One canvas zoom test is flaky because scaling values are read before the zoom animation settles.

e2e/tests/navBar.spec.ts

  • Replace synchronous expect(page.url()).toBe(url) with Playwright's auto-retrying await expect(page).toHaveURL(url) for the logo redirect test and the navitems loop tests
- expect(page.url()).toBe(urls.falkorDBUrl)
+ await expect(page).toHaveURL(urls.falkorDBUrl)

e2e/tests/canvas.spec.ts

  • Add await codeGraph.waitForCanvasAnimationToEnd() after clickZoomIn() / clickZoomOut() calls in both zoom tests, before reading the updated canvas scaling values — consistent with how the same method is already used elsewhere in the file
Original prompt

Problem

Two Playwright E2E tests are failing consistently on Firefox in CI (shard 2), and one is flaky. These are pre-existing test reliability issues unrelated to application code changes.

Failure 1 & 2: Navbar external navigation tests timeout on Firefox

Files: e2e/tests/navBar.spec.ts (lines 17-21 and lines 28-34)

Tests:

  • "Verify clicking on falkordb logo redirects to specified URL" (line 17)
  • "Verify clicking on Main Website redirects to specified URL" (line 29, from the navitems.forEach loop)

Error: Test timeout of 30000ms exceeded followed by assertion mismatch:

  • Expected: "https://www.falkordb.com/"
  • Received: a different URL (the page didn't navigate in time)

Root cause: The tests call page.url() synchronously after clicking a link that navigates to an external URL (https://www.falkordb.com/). There is no waitForURL or equivalent wait before asserting. Firefox handles external navigation slower than Chromium in CI, causing the assertion to fire before the navigation completes.

Current code (line 17-21):

test("Verify clicking on falkordb logo redirects to specified URL", async () => {
    const navBar = await browser.createNewPage(CodeGraph, urls.baseUrl)
    const page = await navBar.clickOnFalkorDbLogo();
    expect(page.url()).toBe(urls.falkorDBUrl)
})

Current code (lines 28-34):

navitems.forEach(({navItem, expectedRes}) => {
    test(`Verify clicking on ${navItem} redirects to specified URL`, async () => {
        const navBar = await browser.createNewPage(CodeGraph, urls.baseUrl)
        const page = await navBar.getNavBarItem(navItem);
        expect(page.url()).toBe(expectedRes)
    })
})

Fix: Replace the synchronous expect(page.url()).toBe(...) assertions with Playwright's auto-waiting await expect(page).toHaveURL(expectedUrl) which will automatically wait for the URL to match before asserting. This is Playwright's recommended pattern for URL assertions.

For the logo test (line 20): change expect(page.url()).toBe(urls.falkorDBUrl) to await expect(page).toHaveURL(urls.falkorDBUrl)

For the navitems loop test (line 32): change expect(page.url()).toBe(expectedRes) to await expect(page).toHaveURL(expectedRes)

Failure 3: Canvas zoom-in test is flaky on Firefox

File: e2e/tests/canvas.spec.ts (lines 22-31)

Test: "Verify zoom in functionality on canvas"

Error: expect(received).toBeGreaterThan(expected) — the updatedGraph.scaleX was not greater than initialGraph.scaleX (value 0.19493818283081055).

Root cause: The test reads getCanvasScaling() immediately after two clickZoomIn() calls without waiting for the canvas zoom animation to finish. On Firefox the animation may not have completed when the scale values are read, causing a race condition.

Current code (lines 22-31):

test(`Verify zoom in functionality on canvas`, async () => {
    const codeGraph = await browser.createNewPage(CodeGraph, urls.baseUrl);
    await codeGraph.selectGraph(GRAPHRAG_SDK);
    const initialGraph = await codeGraph.getCanvasScaling();
    await codeGraph.clickZoomIn();
    await codeGraph.clickZoomIn();
    const updatedGraph = await codeGraph.getCanvasScaling();
    expect(updatedGraph.scaleX).toBeGreaterThan(initialGraph.scaleX)
    expect(updatedGraph.scaleY).toBeGreaterThan(initialGraph.scaleY)
})

Fix: Add a small wait after the zoom-in clicks to allow the canvas animation to settle before reading the updated scaling values. Use await page.waitForTimeout(500) or better yet, check if the CodeGraph POM has a waitForCanvasAnimationToEnd() method (it does — it's used elsewhere in this same file, e.g., line 64, 82, 147, 247). Add await codeGraph.waitForCanvasAnimationToEnd(); after the clickZoomIn() calls and before reading the updated scaling. Apply the same fix to the analogous "Verify zoom out functionality on canvas" test (lines 33-42) for consistency, since it has the same pattern and could also be flaky.

Important notes:

  • Only modify test files in e2e/tests/. Do not modify application source code.
  • The clickOnFalkorDbLogo() and getNavBarItem() methods return a Playwright Page object.
  • The expect(page).toHaveURL(url) assertion is already available from @playwright/test (it's an async assertion with auto-retry).
  • Look at how waitForCanvasAnimationToEnd() is already used in the same canvas.spec.ts file for reference.

This pull request was created from Copilot chat.


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

@vercel

vercel Bot commented Mar 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
code-graph Ready Ready Preview, Comment Mar 8, 2026 6:31am

Request Review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…tion wait

Co-authored-by: gkorland <753206+gkorland@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix timeout errors in navbar external navigation tests on Firefox Fix flaky Firefox E2E tests: async URL assertions and canvas animation wait Mar 8, 2026
@gkorland gkorland marked this pull request as ready for review March 8, 2026 06:45
@gkorland gkorland merged commit 73b6bf4 into staging Mar 8, 2026
8 checks passed
@gkorland gkorland deleted the copilot/fix-navbar-external-navigation-tests branch March 8, 2026 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants