Skip to content

Home

Jump to bottom
Gregory Mierzwinski edited this page Jul 17, 2020 · 4 revisions

Purpose of this repository

The data as contained in this repository is used for Raptor YouTube playback performance tests. Given that some additional modifications of the code haven't been landed in the upstream repository yet, local patches are getting applied.

This page will explain how to check for updates on the original test page, and update the code for using it with Raptor.

Check for upstream changes

To check if there are any upstream changes available go to: https://ytlr-cert.appspot.com/2019/main.html. The version of the page is displayed at the very top like 20190523164643. If that version is more recent than the last landed change on the master branch, pick it for downloading.

Note: The page is served through a sub path which is equal to the current year. As such make sure to update the above URL accordingly for the update check. Former years won't get updates anymore.

Don't miss to also check the list of commits on https://github.com/youtube/js_mse_eme/commits/master for further details of included changes, whereby code is not always immediately synced between the repository and the live site.

To be sure that the updated source code works correctly with Raptor test, run the raptor-youtube-playback benchmark through a try build. Therefore just modify the test_url setting in raptor-youtube-playback to match the above hostname.

Only if the tests on try pass, follow-on with the next steps.

Download and apply changes

If there were changes to the test harness itself, which provide additional features or bug fixes, it might be worth getting those. As such find the Download-Source link on that page. Click it to download the latest source.

Note: If you need the media files locally, click Download-Media-files to download them. This is a 14Gb download.

Important: If additional videos have been added for testing, the existing set of videos on our mirror host has to be updated first! Otherwise tests would start failing due to missing media files. To inform IT for updating the files crate a bug similar to bug 1548062.

Once the source code has been downloaded, ensure that you are on the master branch, and then replace all the files under the eg. 2019 sub folder with the archive's content. Run a git status to check if new files have to be added, or no longer existing files to be removed. Once fixed, run git commit -a to commit all the changes.

Don't forget to include the source code version into the commit message! It will drastically help later in figuring out if new updates are available.

Update the Raptor branch for deployment

The code on the raptor branch contains all the modifications, which are necessary to run the playback performance tests in Raptor. As such the branch contains a couple of additional commits.

Before continuing read and understand the following warning:

Important: Be sure of what you land on the raptor branch, because it is picked-up by the mirror in scheduled intervals. If there are wrongly solved conflicts it can cause all the Raptor tests to fail. As such it is advised to first create your own fork of the repository, and always submit a pull request instead of directly pushing the code changes.

As next switch to the raptor branch, and rebase its content against the master branch by running git rebase master. Usually it should just pass, but if there are merge conflicts make sure to correctly resolve those.

Finally push the changes and as best create a pull request for merging those into the original repository.

Running Tests Locally

If you need to run these tests locally, perform the following - you might want to start with (4):

  1. Clone this repo.
  2. Start a local web server, i.e. apache2 for linux.
  3. Place the cloned code into a location that is accessible to your local server.
  4. Download videos with the Download-Media-files button from the main page (a 14 Gb download).
  5. Extract videos and add them to a folder called test-materials under perf-youtube-playback/2019/ (they need to be in the same directory as main.html). The path to the video files should be something like perf-youtube-playback/2019/test-materials/media/*.
  6. Navigate to http://localhost/perf-youtube-playback/2019/main.html (or where ever you placed the code).
  7. Click [raptor] Playback Performance Tests, then click one of the tests to start the playback.

Upgrading to a New Year

For updating to a new year follow these steps, in this case we use 2100 as the example year:

  1. First, download the 2100 year's source code from https://ytlr-cert.appspot.com/2100/main.html Click on Download-Source when you are at the page.
  2. Clone the mozilla/perf-youtube-playback repo locally from https://github.com/mozilla/perf-youtube-playback and run these commands:
git clone --branch master https://github.com/mozilla/perf-youtube-playback
cd perf-youtube-playback
mkdir 2100
cp -R <2100-source-files-location> 2100
git commit -a
# Make a commit message
git remote add myfork <YOUR-GIT-FORK>
git push yourfork master

  1. Make a PR for the master branch changes and land them (you can continue onto the next steps while waiting).

  2. Setup the raptor-2100 branch now - at the end of these commands, your folder should no longer contain the 2*** folders, and contain the contents of the 2100 folder:

git checkout master
git branch raptor-2100
git checkout raptor-2100
cp -R 2100/. .
# Remove all 2*** folders
rm -rf 2*
git commit -a
# Make a commit message
# You could push to your fork's raptor-2100 branch now as well if you want (not required)
  1. Port all the patches from the 2099 branch into the 2100 branch:
# Checkout the previous year's branch
git checkout raptor-2099

# Find the first and last commits to rebase onto the new branch. Each branch will have some commits which shouldn't be rebased because they create the
# folder strucutre from the master branch. Pick the first commit which is unrelated to structuring.
# The last commit should be the newest commit in that branch.
git rebase --onto raptor-2100 <COMMIT-START> <COMMIT-END>
# Fix conflicts as you go along (`git add CONFLICTED-CHANGES`, then `git rebase --continue` as you move through them).

# After completing this, push to your fork and make a PR to add a new branch (or to the mozilla repo in the raptor-2100 branch)
git push myfork raptor-2100 # OR git push origin raptor-2100

  1. File an issue and mention @bqbn in there so he can setup the new branch in production. See this issue for instructions on how to set it up: https://github.com/mozilla/perf-youtube-playback/issues/19

  2. Test the page locally within Raptor to make sure it still works - push fixes to the mozilla/raptor-2100 branch. Once it works, tests can be setup for the new year.

Clone this wiki locally