Browser performance tests through selenium

Question

We are using protractor for testing internal AngularJS applications.

Besides functional tests, we check for performance regressions with the help of protractor-perfwhich is based on nodejs browser-perf library. Because "Performance is a feature".

With protractor-perf we can measure and assert different performance characteristics while making browser actions, for example:

browser.get('http://www.angularjs.org');

perf.start(); // Start measuring the metrics

element(by.model('todoText')).sendKeys('write a protractor test');

element(by.css('[value="add"]')).click();

perf.stop(); // Stop measuring the metrics 

if (perf.isEnabled) { // Is perf measuring enabled ?

    // Check for perf regressions, just like you check for functional regressions

    expect(perf.getStats('meanFrameTime')).toBeLessThan(60); 

};

---

Now, for another internal application we have a set of Selenium-based tests written in Python.

Is it possible to check for performance regressions with selenium-python, or should I rewrite the tests using protractor to be able to write browser performance tests?

Answer 1

It is possible to try and performance regression testing with Selenium.

However, as you might have already noted. The core essence of Selenium is that it mimics user behaviour. This means that selenium will solely perform the action (e.g. clicking on the button) if the user is able to perform the same action. Also taking into account certain code, workarounds (i.e. hard waits, numerous checks and custom code), required to even be able to run the Selenium script. This means that the "definition" of performance testing using selenium are slightly different compared to traditional performance testing.

What you'll wish to try to do is have a timer (start/stop) for every action selenium is performing. For example: Clicking on a button and log this to a file for later use.

Using selenium you'll be able to create a performance baseline and from there on onwards compare each consecutive result with the baseline. This will provide you with statistics that you can then use for more analysis. Selenium nor Webdriver (Selenium 2.0) come with this feature out of the box. So some custom coding has to happen for this to work.