webkit  2cdf99a9e3038c7e01b3c37e8ad903ecbe5eecf1
https://github.com/WebKit/webkit
WebKit Performance Dashboard

The WebKit performane dashboard is a website to track various performance metrics of WebKit: https://perf.webkit.org/

Checking Out the Code and Installing Required Applications

The instructions assume you're using macOS as the host server and installing this application at /Volumes/Data/perf.webkit.org.

You can choose between using Server.app or install the required tools separately.

  1. Install Server.app (if you don't want to use Server.app, install PostgreSQL: http://www.postgresql.org/download/macosx/)
  2. Install node.
  3. Install Xcode with command line tools (only needed for svn)
  4. svn co https://svn.webkit.org/repository/webkit/trunk/Websites/perf.webkit.org /Volumes/Data/perf.webkit.org
  5. Inside /Volumes/Data/perf.webkit.org, run npm install pg and mkdir -m 755 public/data/

Testing Local UI Changes with Production Data

The front end has the capability to pull data from a production server without replicating the database locally on OS X Yosemite and later. To use this feature, modify config.json's remoteServer entry so that "remoteServer.url" points to your production server, and "remoteServer.basicAuth" specifies the username and the password that is used by the production sever.

Remove "basicAuth" entry for production servers that doesn't require a basic authentication (e.g. perf.webkit.org).

{
"url": "http://perf.webkit.org",
"basicAuth": {
"username": "webkitten",
"password": "webkitten's secret password"
}
}

Then run tools/remote-cache-server.py start. This launches a httpd server on port 8080.

The script caches remote server's responses under public/data/remote-cache and never revalidates them (to allow offline work). If you needed the latest content, delete caches stored in this directory by running tools/remote-cache-server.py reset.

Configuring Apache

Instructions if you're using Server.app

Instructions if you're not using Server.app

Starting Apache

You can use apachectl to start/stop/restart apache server from the command line:

The apache logs are located at /private/var/log/apache2.

Production Configurations

  1. Update ServerAdmin to your email address
  2. Add the following directives to enable gzip: 
    <IfModule mod_deflate.c>
    AddOutputFilterByType DEFLATE text/html text/xml text/javascript application/javascript text/plain application/json application/xml application/xhtml+xml
</IfModule>
  3. Add the following directives to enable zlib compression and MultiViews on DocumentRoot (perf.webkit.org/public): 
    Options Indexes MultiViews
php_flag zlib.output_compression on

Protecting the Administrative Pages to Prevent Execution of Arbitrary Code

By default, the application gives the administrative privilege to everyone. Anyone can add, remove, or edit tests, builders, and other entities in the database.

We recommend protecting the administrative pages via Digest Auth on https connection.

Generate a password file via htdigest -c <path> <realm> <username>, and then create admin/.htaccess with the following directives where <Realm> is replaced with the realm of your choice, which will be displayed on the username/password input box:

AuthType Digest
AuthName "<Realm>"
AuthDigestProvider file
AuthUserFile "<Realm>"
Require valid-user

Configuring PostgreSQL

Run the following command to setup a Postgres server at /Volumes/Data/perf.webkit.org/PostgresSQL (or wherever you'd prefer): python ./tools/setup-database.py /Volumes/Data/perf.webkit.org/PostgresSQL

It automatically retrieves the database name, the username, and the password from config.json.

Starting PostgreSQL

The setup script automatically starts the database but you may need to run the following command to manually start the database after reboot.

Initializing the Database

Run database/init-database.sql in psql as webkit-perf-db-user: /Applications/Server.app/Contents/ServerRoot/usr/bin/psql webkit-perf-db -h localhost --username webkit-perf-db-user -f init-database.sql

Making a Backup and Restoring

Concepts

See init-database.sql for the complete database schema.

Data Models for Test Results

In the performance dashboard, each test can have a parent test or arbitrary many child tests or subtests. Each test can then have multiple metrics to measure a specific unit'ed value. For example, Speedometer benchmark has the top level score, which is computed by the total time of running subtests. As such, the top level test has two metrics: Score and Time which is the aggregated total sum of Time metrics of the subtests:

Speedometer (A test)

Since each test metric can be measured on arbitrarily platforms (e.g. MacBookAir7,1 on macOS Sierra), the dashboard supports showing the baseline results (e.g. benchmark scores on Safari 10) in addition to the results from the current sofware (the trunk WebKit build), we use a triple (test metric, platform, type) called a test configuration to group a collection of data points reported to the dashboard.

Then each test configuration has arbitrary test runs. A test run represents the score or more broadly the result of a single test metric on a specific platform at a particular time. For example, running Speedometer once and reporting the results to the performance dashboard results in the creation of a test run for each of Score and Time : Total metrics as well as Time metrics of all subtests (e.g. AngularJS-TodoMVC). Each test run can then have arbitrarily many iteration values which are indivisual measurement of some test metric within the benchmark. For example, Speedometer runs the same test twenty times and uses the average time and the score of those twenty iterations to compute the final score. Each one of twenty iterations constitutes a single iteration value.

Each test run are related to one another via builds which is uniquely identified by its builder and a build number. For example in the following sample data points for Speedometer, two test runs 789 and 791 are reported by a build 1001, and test runs 876 and 878 are reported by build 1002. Because Speedometer's total score is computed using the time took to run subtests, a builder reported both values in a single build as expected.

(Speedometer's Score, Sierra MacBookAir7,1, "current")

(AngularJS-TodoMVC's Time, Sierra MacBookAir7,1, "current")

Reporting Results

To submit a new build, or a set of data points to an instance of the performance dashboard, you need to do the following:

Format of Results JSON

The JSON submitted to /api/report should be an array of dictionaries, and each dictionary should must contain the following key-value pairs representing a single run of tests and its subtests on a single build:

In the example below, we have the top-level test named "PageLoadTime". It measures two metrics: Time and FrameRate. Time metric is the arithmetic mean of each subtest's Time metric (webkit.org and www.w3.org). The computed arithmetic means are [965.6, 981.35, 947.15] in this case. The test also reports FrameRate but this metric is measured only for the entire suite not per each subtest.

[{
"buildNumber": "651",
"buildTime": "2013-01-31T22:22:12.121051",
"builderName": "Trunk Mountain Lion Performance Tests",
"slaveName": "bot-111",
"slavePassword": "somePassword",
"platform": "Mountain Lion",
"revisions": {
"OS X": {
"revision": "10.8.2"
},
"WebKit": {
"revision": "141469",
"timestamp": "2013-01-31T20:55:15.452267Z"
}
},
"tests": {
"PageLoadTime": {
"metrics": {
"Time": ["Arithmetic"],
"FrameRate": {
"current": [31, 24, 29]
}
}
"tests": {
"webkit.org": {
"metrics": {
"Time": {
"current": [629.1, 654.8, 598.9]
}
},
"url": "https://webkit.org/"
},
"www.w3.org": {
"metrics": {
"Time": {
"current": [1302.1, 1307.9, 1295.4]
}
},
"url": "https://www.w3.org/"
},
},
},
}
}]
Generated by   doxygen 1.8.13