Skip to content

internal: Add automated tests#858

Open
cristianoventura wants to merge 51 commits intomainfrom
internal/test-wdio
Open

internal: Add automated tests#858
cristianoventura wants to merge 51 commits intomainfrom
internal/test-wdio

Conversation

@cristianoventura
Copy link
Copy Markdown
Collaborator

Description

This PR attempts to implement E2E testing for k6 Studio using Webdriver.io, which is one of the recommendations on the ElectronJS website.

You can find a working CI action in https://github.com/grafana/k6-studio/actions/runs/18286347336/job/52062255939.

How to Test

Locally run npm install and run the test suite pointing to a build. This can be a Production build:

npm run wdio -- --app-path="/Applications/k6 Studio.app/Contents/MacOS/k6-studio"

Checklist

  • I have performed a self-review of my code.
  • I have added tests for my changes.
  • I have run linter locally (npm run lint) and all checks pass.
  • I have run tests locally (npm test) and all tests pass.
  • I have commented on my code, particularly in hard-to-understand areas.

Screenshots (if appropriate):

Related PR(s)/Issue(s)

Resolves #331

@cristianoventura
Copy link
Copy Markdown
Collaborator Author

@e-fisher Thanks for the review!

I've running locally using your provided command, but tests timeout. The studio window opens but nothing else happens.

I've fixed the window handler and it should work appropriately now. Please let me know how it goes. You can also see a complete run here. macos-13 takes a really long time to build, so I ended up cancelling that build.

Perhaps we could configure it as release-please PR check?

Great idea! I've included a release-please-e2e-tests.yml workflow that will run on branches starting with the release-please-- name. I've extracted most of the release-test-version.yml logic into a separate file, so we can reuse most of the workflow.

Are those copilot comments reasonable? Should we enable eslint on spec files to catch not awaited promises?

Those are wrong. These functions return a ChainablePromiseElement which is a normal WebdriverIO pattern. Only the element actions need to be awaited. I believe the warning is coming from @typescript-eslint/no-floating-promises, which is useful in app code but noisy in WebdriverIO specs. I ended up adding an override rule to .eslintrc.cjs, hoping that it helps with these Copilot comments going forward.

Copy link
Copy Markdown
Collaborator

@e-fisher e-fisher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works locally as well! 🙌 Let's get this working for us, if others don't have any comments 🚀

@e-fisher
Copy link
Copy Markdown
Collaborator

Just thought of this - is there a way we could run e2e against running dev app locally? This would help with debugging, creating a new build to verify if change fixed tests could be time-consuming.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add automatic smoke tests for the different os

3 participants