How to handle JavaScript/frontend errors with Selenium Webdriver

Martijn

Martijn Versluis – 24 April 2019
767 words in about 4 minutes

TL;DR The final (refactored) solution is at the bottom.

The modern Ruby/Rails web developer often has to deal with a complex frontend, which means that a passing Selenium test does not guarantee a 100% working web application. There can be JavaScript warnings or errors that popped up in a headless browser, but those are not visible in the test output.

Of course, it is possible to have a frontend (unit) test to catch those errors. But to make sure a feature really works end-to-end you need a feature spec (integration test) that covers the whole stack. There must be a way to handle frontend errors with Selenium…?

🎉 There is!

You have to dig a bit into the page object:

1
page.driver.browser.manage.logs.get(:browser)

It is a collection of Selenium::WebDriver::LogEntry objects. Let’s see what’s in there. Below is a pretty print of page.driver.browser.manage.logs.get(:browser). The message property is what we want to show up in our test output.

1
2
3
4
5
6
7
[#<Selenium::WebDriver::LogEntry:0x00007fab652d29f0 
  @level="INFO",
  @message=
   "http://127.0.0.1:3002/static/js/1.chunk.js 80889:20 \"%cDownload the React DevTools for a better development experience: https://fb.me/react-devtools\"",
  @timestamp=1554883156962>
  ....
  ]

So, by default the logs aren’t really useful. Let’s see what happens when there is a frontend error. We will randomly invoke function foobar, which we didn’t define. Afterwards, we inspect the logs again. It now contains an entry for our JavaScript error:

1
2
3
4
5
#<Selenium::WebDriver::LogEntry:0x00007fbfb4335b28
  @level="SEVERE",
  @message=
   "http://127.0.0.1:3002/static/js/1.chunk.js 82408:19 \"./src/components/ivr/IVRSettingsForm.jsx\\n  Line 71:  'foobar' is not defined  no-undef\\n\\nSearch for the keywords to learn more about each error.\"",
  @timestamp=1554883526785>

This entry is actual valuable information. If there exists such an entry, the Selenium test should fail. Note the @level="SEVERE" part. We can actually filter the logs and select the entries where level == 'SEVERE'. Let’s create our own SeleniumBrowserErrorReporter!

1
2
3
4
5
6
7
8
9
10
11
12
13
class SeleniumBrowserErrorReporter
  def initialize(page)
    self.page = page
  end

  def report!
    # The magic needs to happen here...
  end

  private

  attr_accessor :page
end

First, we select only severe errors, and return early when there are none:

1
2
3
logs = page.driver.browser.manage.logs.get(:browser)
severe_errors = logs.select { |log| log.level == 'SEVERE' }
return if severe_errors.none?

Then we format the entries by grabbing the message and join them with a blank line in between:

1
2
3
report = severe_errors.map { |error| error.message.gsub('\\n', "\n") }.join("\n\n")
raise "There #{severe_errors.count == 1 ? 'was' : 'were'} #{severe_errors.count} "\
      "JavaScript error#{severe_errors.count == 1 ? '' : 's'}:\n\n#{report}"

We attach the reporter in our RSpec configuration:

1
2
3
4
5
6
7
8
9
RSpec.configure do |config|
  ...
  
  config.after :each, type: :feature do
    SeleniumBrowserErrorReporter.new(page).report!
  end
  
  ...
end

The test failure looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
1) Failure/Error:
            raise "There #{severe_errors.count == 1 ? 'was' : 'were'} #{severe_errors.count} "\
                  "JavaScript error#{severe_errors.count == 1 ? '' : 's'}:\n\n#{report}"

          RuntimeError:
            There were 3 JavaScript errors:

            http://127.0.0.1:3002/static/js/0.chunk.js 82408:19 "./src/components/SettingsForm.jsx\n  Line 53:  'foobar' is not defined  no-undef\n\nSearch for the keywords to learn more about each error."

            http://127.0.0.1:3002/static/js/0.chunk.js 82408:19 "./src/components/SettingsForm.jsx\n  Line 53:  'foobar' is not defined  no-undef\n\nSearch for the keywords to learn more about each error."

            http://127.0.0.1:3002/static/js/main.chunk.js 4611:2 Uncaught ReferenceError: foobar is not defined
          # ./spec/support/selenium_browser_error_reporter.rb:12:in `report!'
          # ./spec/rails_helper.rb:120:in `block (2 levels) in <top (required)>'

There is one annoying thing: newline characters (\n) are escaped. Let’s replace them with actual newlines:

Now we have a nice report, just like our browser console would display it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
1) Failure/Error:
            raise "There #{severe_errors.count == 1 ? 'was' : 'were'} #{severe_errors.count} "\
                  "JavaScript error#{severe_errors.count == 1 ? '' : 's'}:\n\n#{report}"

          RuntimeError:
            There were 3 JavaScript errors:

            http://127.0.0.1:3002/static/js/0.chunk.js 82408:19 "./src/components/SettingsForm.jsx
              Line 53:  'foobar' is not defined  no-undef

            Search for the keywords to learn more about each error."

            http://127.0.0.1:3002/static/js/0.chunk.js 82408:19 "./src/components/SettingsForm.jsx
              Line 53:  'foobar' is not defined  no-undef

            Search for the keywords to learn more about each error."

            http://127.0.0.1:3002/static/js/main.chunk.js 4611:2 Uncaught ReferenceError: foobar is not defined
          # ./spec/support/selenium_browser_error_reporter.rb:12:in `report!'
          # ./spec/rails_helper.rb:120:in `block (2 levels) in <top (required)>'

Final solution

Finally, I’d like to refactor a bit, because our report! method grew large and got a bit unreadable.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
class SeleniumBrowserErrorReporter
  def initialize(page)
    self.page = page
  end

  def report!
    severe_errors = logs.select { |log| log.level == 'SEVERE' }
    return if severe_errors.none?

    report = error_report_for(severe_errors)
    raise "There #{severe_errors.count == 1 ? 'was' : 'were'} #{severe_errors.count} "\
          "JavaScript error#{severe_errors.count == 1 ? '' : 's'}:\n\n#{report}"
  end

  private

  attr_accessor :page

  def error_report_for(logs)
    logs
      .map(&:message)
      .map { |message| message.gsub('\\n', "\n") }
      .join("\n\n")
  end

  def logs
    page.driver.browser.manage.logs.get(:browser)
  end
end

Wrap up

Now we have a nice error reporter that makes us aware of frontend issues. Please let me know if you like this solution, or if you have questions or feedback.

At Kabisa we know a lot about writing good tests. Leave us a message if you would like to get in touch.

At Kabisa, privacy is of the greatest importance. We think it is important that the data our visitors leave behind is handled with care. For example, you will not find tracking cookies from third parties such as Facebook, Hotjar or Hubspot on our website. Only cookies from Google and Vimeo are used in order to improve the user experience of our visitors. These cookies also ensure that relevant advertisements are displayed. Read more about the use of cookies in our privacy statement.