Selenium Timeout with Rails 6

[kuzmik]

# Gemfile

gem 'rails', '~> 6.0.2'

group :development, :test do
    gem 'selenium-webdriver'
    gem 'teaspoon', github: 'jejacks0n/teaspoon'
    gem 'teaspoon-jasmine', github: 'jejacks0n/teaspoon'
end

I am attempting to run teaspoon --driver=selenium --server-port=31337 --format=documentation spec/javascripts/vendoring/integration_form_spec.js and for some reason the Rails.env defaults to "development." This causes a bit of a problem with other gems we use, so I throw a RAILS_ENV=test on there, and the environment is set correctly, but I am still getting timeouts.

11:23 <kuzmik@ilus4:registry(kuzmik/REGISTRY-672/part-2 *) $ > RAILS_ENV=test rake teaspoon
Starting the Teaspoon server...
Teaspoon running default suite at http://127.0.0.1:56930/teaspoon/default
rake aborted!
Selenium::WebDriver::Error::TimeoutError: Timed out

Under rails 5.2 the exact same code worked fine, so I am kind of at a loss.

Anyone have any ideas? I saw there was another issue about rails 6, but I am not running into that issue or using any of those other libraries, just teaspoon and jasmine (and selenium I guess).

[kuzmik]

It looks like it's actually CSP related:

Refused to execute inline script because it violates the following Content Security Policy directive: "default-src 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-8fSyr/FANJTl286qJs3wNGU79q3WhHisHAwlSDlZf+c='), or a nonce ('nonce-...') is required to enable inline execution. Note also that 'script-src' was not explicitly set, so 'default-src' is used as a fallback.

I see there is a PR open for fixing CSP support, any chance that can be rebased and merged?

[mathieujobin]

I have rebased the PR #561 into a new branch

would you be willing to try it on your end?

group :development, :test do
    gem 'selenium-webdriver'
    gem 'teaspoon', github: 'jejacks0n/teaspoon', branch: 'pr-561'
    gem 'teaspoon-jasmine', github: 'jejacks0n/teaspoon', branch: 'pr-561'
end

[kuzmik]

Oh, unfortunately I've since changed jobs and no longer have access to that code base. I'll ping a former coworker and see if they can try it out.

Thank you, though!

[dparpyani]

Thanks for the ping @kuzmik 👋

@mathieujobin I'm seeing the issue on the new branch

$ RAILS_ENV=test rake teaspoon
Warning: the running version of Bundler (2.2.9) is older than the version that created the lockfile (2.2.11). We suggest you to upgrade to the version that created the lockfile by running `gem install bundler:2.2.11`.
Starting the Teaspoon server...
Teaspoon running default suite at http://127.0.0.1:65353/teaspoon/default
rake aborted!
Selenium::WebDriver::Error::TimeoutError: Timed out
/Users/deepak/.rvm/gems/ruby-2.6.6/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/wait.rb:73:in `until'
/Users/deepak/.rvm/gems/ruby-2.6.6/bundler/gems/teaspoon-6ccf2a51e449/lib/teaspoon/driver/selenium.rb:32:in `run_specs'
/Users/deepak/.rvm/gems/ruby-2.6.6/bundler/gems/teaspoon-6ccf2a51e449/lib/teaspoon/console.rb:55:in `run_specs'
/Users/deepak/.rvm/gems/ruby-2.6.6/bundler/gems/teaspoon-6ccf2a51e449/lib/teaspoon/console.rb:44:in `block in execute_without_handling'
/Users/deepak/.rvm/gems/ruby-2.6.6/bundler/gems/teaspoon-6ccf2a51e449/lib/teaspoon/console.rb:42:in `each'
/Users/deepak/.rvm/gems/ruby-2.6.6/bundler/gems/teaspoon-6ccf2a51e449/lib/teaspoon/console.rb:42:in `inject'
/Users/deepak/.rvm/gems/ruby-2.6.6/bundler/gems/teaspoon-6ccf2a51e449/lib/teaspoon/console.rb:42:in `execute_without_handling'
/Users/deepak/.rvm/gems/ruby-2.6.6/bundler/gems/teaspoon-6ccf2a51e449/lib/teaspoon/console.rb:27:in `execute'
/Users/deepak/.rvm/gems/ruby-2.6.6/bundler/gems/teaspoon-6ccf2a51e449/lib/teaspoon/console.rb:23:in `failures?'
/Users/deepak/.rvm/gems/ruby-2.6.6/bundler/gems/teaspoon-6ccf2a51e449/lib/tasks/teaspoon.rake:14:in `block in <top (required)>'
/Users/deepak/.rvm/gems/ruby-2.6.6/gems/rake-13.0.3/exe/rake:27:in `<top (required)>'
/Users/deepak/.rvm/gems/ruby-2.6.6/bin/ruby_executable_hooks:24:in `eval'
/Users/deepak/.rvm/gems/ruby-2.6.6/bin/ruby_executable_hooks:24:in `<main>'

[mathieujobin]

@dparpyani I'm afraid this can be a lot of things. let me know if you find out more details if you have time to debug this.

[mathieujobin]

Look what I did there 805523c

hopefully this helps with your problem?

[mathieujobin]

this seems to be a selenium config problem, not teaspoon related
reopen if I am wrong and you have more details

[dparpyani]

Our teaspoon-jasmine was pinned to e952dbc and had the following added to our CSP config for it to work (using the secure-headers gem):

SecureHeaders::Configuration.default do |config|
  ... <snipped> ... # default-src set to "'self'", script-src includes "'self'" and some other URLs, etc.

  if Rails.env.test?
    config.csp[:script_src] << "'unsafe-inline'"
    config.csp[:connect_src] = nil
  end
end

When using the pr-561 branch, it neither worked with nor without the script-src setting (and failed with the timeout error mentioned above). However, I unpinned the teaspoon-jasmine version today and let it pull the most recent versions (v2.9.1 with teaspoon v1.2.2). FWIW, now it works similarly to the previously pinned version i.e. works with the CSP config above, but not without it.

[mathieujobin]

I see, thanks for the extra troubleshooting and information. I'm not familiar with CSP or the secure-headers gem

what do you suggest we do so it works for people out of the box?

[dparpyani]

The secure-headers gem sets safe defaults for a bunch of important headers, CSP being one of them. In this case, we're modifying the CSP header during tests to allow 'unsafe-inline' for the script-src directive, which:

Allows the use of inline resources, such as inline <script> elements, javascript: URLs, inline event handlers, and inline <style> elements.

(from https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/script-src)

See here for more examples, but without diving into the teaspoon code, I think that it has inline elements that are being blocked by a strict CSP header. It looks like that is what #561 was trying to fix, but maybe there's a piece that was missed somewhere?