test: Refactor E2E test running - #8497
Merged
Merged
Conversation
Contributor
size-limit report 📦
|
mydea
force-pushed
the
fn/e2e-refactor
branch
8 times, most recently
from
July 13, 2023 09:52
a1ff4e0 to
60a6618
Compare
mydea
changed the title
mydea
force-pushed
the
fn/e2e-refactor
branch
2 times, most recently
from
July 13, 2023 10:53
2041d6a to
4ec760a
Compare
mydea
marked this pull request as ready for review
July 17, 2023 07:54
mydea
force-pushed
the
fn/e2e-refactor
branch
3 times, most recently
from
July 18, 2023 08:26
eed11a1 to
338a5ae
Compare
Lms24
approved these changes
Jul 19, 2023
Lms24
left a comment
Lms24
left a comment
Member
There was a problem hiding this comment.
Great work! Love the simplifications and that we now immediately see which test failed!
lforst
approved these changes
Jul 25, 2023
lforst
left a comment
lforst
left a comment
Contributor
There was a problem hiding this comment.
Now that we don't run in parallel on the same machine anymore we could switch back to yarn. But LGTM
Member
Author
|
FYI I also updated the canary issue creation to include which test failed, so we get distinct issues per E2E test failure. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This refactors how we run E2E tests, to a more simple approach.
CI
In CI, we now have a matrix job for the E2E tests, where each test runs in parallel & isolation.
Example: https://github.com/getsentry/sentry-javascript/actions/runs/5541658717
Canary
For the canary tests, we also have to specify each canary job we want to run in a matrix.
Example: https://github.com/getsentry/sentry-javascript/actions/runs/5541662790
Locally
You can either run all tests:
Or only run a single test suite: