4.x: The plugin "retryFailedStep" works in the 1st scenario only and not in other scenarios

[mirao]

Found after migration from 3.x where it worked well. Tested with 4.0.8 and 4.1.0-beta. Both versions are affected.

Important note: For testing, patch node_modules/codeceptjs/lib/plugin/retryFailedStep.js so that retryFailedStep works in --verbose mode. See #5128

Expected result:

• retries are performed in both scenarios

Actual result:

• retries are performed in the 1st scenario only

Feature('My');

Scenario('test something 1',  ({ I }) => {
    I.amOnPage("https://example.com");
    I.seeElement("blabla1");
});

Scenario('test something 2',  ({ I }) => {
    I.seeElement("blabla2");
});

codecept.conf.ts

export const config: CodeceptJS.MainConfig = {
  tests: "./tests/*_test.ts",
  output: "./output",
  helpers: {
    Playwright: {
      browser: "chromium",
      url: "http://localhost",
      show: true,
    },
  },
  include: {
    I: "./steps_file.ts",
  },
  noGlobals: true,
  plugins: {
    retryFailedStep: {
      enabled: true,
    },
  },
  name: "my",
  require: ["tsx/cjs"],
};

output:

mirao@rog:~/workspace/my$ codeceptjs run --verbose
***************************************
nodeInfo:  24.16.0
osInfo:  Linux 6.17 Ubuntu 24.04.4 LTS 24.04.4 LTS (Noble Numbat)
cpuInfo:  (16) x64 AMD Ryzen 7 9700X 8-Core Processor
chromeInfo:  149.0.7827.155
edgeInfo:  "N/A"
firefoxInfo:  152.0
safariInfo:  N/A
playwrightBrowsers:  "chromium: 149.0.7827.55, firefox: 151.0, webkit: 26.5"
If you need more detailed info, just run this: npx codeceptjs info
***************************************
CodeceptJS v4.0.8 #StandWithUkraine
Using test root "/home/mirao/workspace/my"
Helpers: Playwright
Plugins: screenshot, retryFailedStep

My --
/home/mirao/workspace/my/tests/My_test.ts
    [1]  Starting recording promises
    Timeouts: 
 › [Session] Starting singleton browser session
  test something 1
 › [New Session] {"ignoreHTTPSErrors":false,"acceptDownloads":true}
  Scenario()
    I am on page "https://example.com"
    I see element "blabla1"
    › No elements found by "blabla1"....
       [1] Retrying... Attempt #2
    › No elements found by "blabla1"....
       [1] Retrying... Attempt #3
    › No elements found by "blabla1"....
       [1] Retrying... Attempt #4
    › No elements found by "blabla1"....
       [1] Error | Error: Element "blabla1" is not visible on page. err => { step.status = 'failed' step.endTime = +Da...
    [1] Error | Error: Element "blabla1" is not visible on page. undefined...
    [1] <teardown>  Stopping recording promises
 › <screenshot> Saving screenshot test_something_1.failed.png
 › [Screenshot] output/test_something_1.failed.png
  ✖ FAILED in 1273ms

    [2]  Starting recording promises
  test something 2
 › [New Session] {"ignoreHTTPSErrors":false,"acceptDownloads":true}
  Scenario()
    I see element "blabla2"
    › No elements found by "blabla2"....
       [2] Error | Error: Element "blabla2" is not visible on page. err => { step.status = 'failed' step.endTime = +Da...
    [2] Error | Error: Element "blabla2" is not visible on page. undefined...
    [2] <teardown>  Stopping recording promises
 › <screenshot> Saving screenshot test_something_2.failed.png
 › [Screenshot] output/test_something_2.failed.png
  ✖ FAILED in 50ms

    [3]  Starting recording promises

-- FAILURES:

  1) My
       test something 1:
     
  Element "blabla1" is not visible on page.
      at dontSeeElementError (file:///home/mirao/workspace/my/node_modules/codeceptjs/lib/helper/errors/ElementAssertion.js:23:9)
      at Playwright.seeElement (file:///home/mirao/workspace/my/node_modules/codeceptjs/lib/helper/Playwright.js:1982:7)
  
  
  ◯ File: file:///home/mirao/workspace/my/tests/My_test.ts
  
  ◯ Scenario Steps:
  ✖ I.seeElement("blabla1") at Test.<anonymous> (./tests/My_test.ts:5:7)
  ✔ I.amOnPage("https://example.com") at Test.<anonymous> (./tests/My_test.ts:4:7)
  
  ◯ Artifacts:
  - screenshot: /home/mirao/workspace/my/output/test_something_1.failed.png
  
  ◯ Metadata:
  - browser: chromium
  - browserVersion: 149.0.7827.55
  - windowSize: 1280x720

  2) My
       test something 2:
     
  Element "blabla2" is not visible on page.
      at dontSeeElementError (file:///home/mirao/workspace/my/node_modules/codeceptjs/lib/helper/errors/ElementAssertion.js:23:9)
      at Playwright.seeElement (file:///home/mirao/workspace/my/node_modules/codeceptjs/lib/helper/Playwright.js:1982:7)
  
  
  ◯ File: file:///home/mirao/workspace/my/tests/My_test.ts
  
  ◯ Scenario Steps:
  ✖ I.seeElement("blabla2") at Test.<anonymous> (./tests/My_test.ts:9:7)
  
  ◯ Artifacts:
  - screenshot: /home/mirao/workspace/my/output/test_something_2.failed.png
  
  ◯ Metadata:
  - browser: chromium
  - browserVersion: 149.0.7827.55
  - windowSize: 1280x720


  FAIL  | 0 passed, 2 failed   // 2s

[kobenguyent]

Same #5598 I guess?

[mirao]

Exactly. It duplicates #5598. Thanks.