Mozregression 2.3.0 and GUI 0.9.0 has been released!

Changes for both GUI and command line:

• Add support for firefox and jsshells pgo builds (bug 1234030)
• Add support for firefox and jsshell asan builds (bug 1233900)
• Add debug build type for fennec (bug 1233900)
• Ability to change the nightlies archive server (bug 1232874)
• Really old builds no longer break mozregression (bug 1241724)
• use android-api-15 for recent fennec builds (bug 1245335)
• Fixed a hang on windows when firefox was restarted in safe mode (bug 1245332)

GUI changes only:

• It is possible to filter the displayed logs by level (bug 1173984)
• Stopping a bisection is faster in some cases (bug 1241830)
• Added a calendar to choose dates (bug 1231388 and bug 1246339)
• Good and bad options are now available directly for an easy flow (bug 1235003)
• Auto-completion is provided for the branch field (bug 1231332)
• Fixed a crash when updating preferences for the first time. Also sensitive defaults are now defined to use a persistent dir
• Fixed a bug that prevented to retry the right build (bug 1245333)
• Fixed the invisible hovered right click selection in the info panel (bug 1245506)

Command line changes only:

• CLI output has been improved - removed useless log parts, and added coloring on windows (bug 1245433)
• Log option group from the CLI has been removed, in favor of a simple --debug flag (bug 1245433)

Thanks to Wasif Hyder, Mike Ling and Saurabh Singhal for their contributions!

New mozregression releases, coming with great new features!

Changes for both GUI and command line:

• Avoid unnecessary downloads by reusing persistent builds (bug 1160078). Builds that are “near” the ones we want in bisection will be used by default now, instead of downloading the accurate build. In case you want to deactivate this behavior, you can use the command line flag “–approx-policy=none”, or see in the “Global preferences” view in the GUI. You can also add “approx-policy = none” in your config file.
• Print the url of the bug that contain the changes that probably caused the regression (bug 1239699).
• add support for running builds from the try branch (bug 1240033).
• Fixed a bug that prevented empty profile dirs to be used (bug 1233686).

GUI changes only:

• Fixed a regression that prevented to run a build after a user cancel (bug 1240393).
• Fixed a crash when the builds were too old to contain metadata (bug 1239993).
• Remove modal dialogs in bisection (bug 1235380).
• Fixed the issue that minimized the log view on Windows (bug 1219887).

Command line changes only:

• Check good and bad builds prior to start the bisection (bug 1233896). This is an important change, now mozregression will by default make you test the first and last builds of a range before starting the bisection. The idea is to be sure that what you think is good and bad really is. Note that you can deactivate it by using the command line flag “–mode=no-first-check”, or by adding “mode = no-first-check” in your config file.
• Allow to run a single b2g build with --launch (no associated bug).

This is a minor release of the command-line and GUI mozregression tools.

On the command line side:

• Added basic support for jsshell integration (bug 1232036)
• Reduced build launch time (bug 1216232)
• Added certificate-based TaskCluster authentication (bug 1227780)

And other minor features and fixes: bug 1195390, bug 1231745, bug 1232879, bug 1233649 and bug 1233905.

On the GUI side:

• allow to run a single build - equivalent to --launch argument (bug 1231399)
• fixed a regression from 0.6.0 that prevented to show the last step of the bisection (bug 1232261)
• added profile persistence support (bug 1220553)

And other fixes: bug 1232660 and bug 1233657

Thanks to Saurabh Singhal and GopianiS! They are new mozregression contributors who helped me with this release.

After mozregression 2.0, it is time for the GUI to follow!

0.6.0 GUI release is based on the changes from mozregression 2.0: the bisection flow is now updated, and starting a bisection should be a lot easier since it does not ask anymore for a nightly or inbound bisection kind - simply choose an application, possibly a branch and some options (build type, bits) then choose the regression range based on dates, release numbers, build ids or raw changesets.

That’s all. :)

All in all, a great simplification of the interface and more power. Give it a try!

2.0.0 is a major release of mozregression, as we changed the bisection flow based on ideas in the post I wrote a couple of weeks ago.

Now mozregression will automatically detect a merge commit, and switch to bisect in the branch where the merged commits comes from. So mozilla-inbound is no longer the default for Firefox when bisecting by date is done, since there is no default now.

Based on that, we have been able to simplify the overall usage of mozregression:

• removed the --inbound-branch option. Just use --repo now when you want to specify a branch, no matter if it is an integration or a release branch.

• allowed to bisect using dates on an integration branch

• removed the --good-rev, --bad-rev, --good-release and --bad-release options. Just use --good and --bad for everything now, being a date, a changeset, a release number or a build id.

• added some aliases for the branch names: you can use m-i, inbound or mozilla-inbound to describe the same branch - another example, m-c, central and mozilla-central are equivalent.

• mozregression is smarter to let you test builds with a specific flavor, e.g. debug builds.

Those changes adds some new possibilities to bisect which were not available before, like bisecting using changesets on mozilla-central, only specify a good changeset (the bad changeset will be implied, and will be the most recent one).

Some examples:

# bisect using dates
mozregression -g 2015-11-20 -b 2015-11-25  # implied branch is m-c
mozregression -g 2015-11-20 -b 2015-11-25 --repo inbound
# bisect using changesets
mozregression -g dcd5230c4ce1 -b 931721112d8e  # implied branch is m-i
mozregression -g 1b2e15608f34 -b abbd213422a5 --repo m-c
# use debug builds
mozregression -g 2015-11-20 -b 2015-11-25 -B debug
mozregression -g dcd5230c4ce1 -b 931721112d8e -B debug
# of course, --launch works the same way now
mozregression --launch abbd213422a5 --repo m-c
mozregression --launch 2015-11-25 --repo m-i -B debug

Just keep in mind that when you use a changeset, the default branch will be the default integration branch for the application instead of the release branch. For firefox, mozilla-inbound will be the default when you use a changeset, and mozilla-central will be used otherwise. This is historical and we may change that in the future - for now just keep that in mind, or always specify a branch to be sure.

See Bugs 1095058, 1210013, 1225544, 1228951, 1225541 and 1228857 for a description of technical implementation.

Minor releases of the command line and GUI mozregression tools.

On the command line side:

• new feature, you can now use a buildid instead of a date when doing the bisection or running a single build (bug 1217906):

mozregression --launch 20151102030241

• added a new parameter to control the tested binary output, -P/--process-output (bug 1214386)
• a few improvements for the general output (bug 1225755, bug 1225548 and bug 1227134)
• fixed a bug in the back command, when it was used after a skip (bug 1203855)

On the GUI side:

• allow to choose a buildid or a release number instead of a date (bug 1225437)
• fixed a bug that prevented to use a profile directory containing non-ascii characters (bug 1224083)

This new release of mozregression includes some new features:

• allow to use debug builds. Note that it will only work for inbounds (for example when using --good-rev --bad-rev) - Bug 1211138
• print the pushlog regression range every step - Bug 1216270
• allow to launch a specific release (e.g., --launch 41) - Bug 1214192
• able to run under cygwin python - bug 1185602
• able to work with preview releases of Android - bug 1185787

And a few bugfixes also:

• results of Nightly-bisection not showing the dates anymore - Bug 1176708
• not all options are shown in the resume info - Bug 1176100

Thanks to Mikeling for being really active on some bugs here!

There is also a basic support for firefox os builds (flame, aries, simulator). Lots of work still needs to be done to make it really useful, (see bug 1205560) but it is now possible to bisect between dates or changesets on a given branch: mozregression will download the builds and ask you to flash it on the device.

# Regression finding by date range with aries-opt builds (defaults to b2g-inbound)
mozregression --app b2g-aries --bad 2015-09-10 --good 2015-09-07
# Regression finding on mozilla-inbound for debug builds
mozregression --app b2g-aries --build-type debug --bad 2015-09-10 --good 2015-09-07 \
              --inbound-branch mozilla-inbound
# Flame builds with a good and bad revision
mozregression --app b2g-flame --bad-rev c4bf8c0c2044 --good-rev b93dd434b3cd
# find more information
mozregression --help
mozregression --list-build-types

Thanks to Michael Shal, Naoki Hirata and others for helping me on this.

Sorry for the delay on GUI release and posts, but we were quite busy with mozregression command line these days!

Still we don’t forget about the GUI. And now there are a bunch of great new cool things with this 0.4.0 release!

• quite a lot of bugfixes, I won’t enumerate them; :)
• this is now possible to choose the next build to be tested after a skip, thus allowing to be in the good-bad range again faster (bug 1203852)
• automatic check if new releases are available and inform the user (bug 1177001)
• we added a crash report dialog, that will show up on crashes, so you will be able to help us fix the bug by reporting the internal error shown. I hope you won’t see this often though! (bug 1202242)

As usual now, big thanks for MikeLing and Jonathan Pigree for their work and time! They both have a really great impact on the software, from decisions to implementations.

We made a lot of great changes in mozregression recently, from refactoring to really cool new features. So I thought it was time to switch to a newer major version!

So here we are, the 1.0.0 release! Some great features:

• Offer a way to launch a specific build (bug 1194941, --launch)
• Allow to choose the next build to test after a skip (bug 1195294)
• Add a command line option to not copy the profile each time a build is started (bug 1200862, --profile-persistence)
• Print a warning when the browser crashes while evaluating (bug 1201527)
• Keyboard interruption (CTRL+C) are more responsive (bug 1200893)

This release also fixes an intermittent bug introduced with 0.42, see bug 1200270.

Last but not least, mozregression is now prepared for mach integration! Look at bug 1200293 to follow that.

0.42 is an early release because inbound bisection for firefox was broken, see bug 1199618.

Still there is some great improvements in there!

• Add an option to limit the persist folder size (bug 1195394)
• Help the user to use a config file (bug 1198866)
• Error message is not clear when looking for inbounds using changesets that are in the same push (bug 1195622)
• Remove the --http-cache-dir option (bug 1195085), prefer --persist and --persist-size-limit
• Changed the default good date of firefox 64 builds (bug 1194937)
• Disable the warning message about InsecurePlatformWarning for python <= 2.7.9 (bug 1199020)

Also we did a really nice work of making the code source easier and cleaner. Big thanks to MikeLing and Jonathan Pigree for their patches, reviews and testing contributions!

mozregression 0.41 fixes bug 1185756, which caused the following warning on Windows:

WindowsError: [Error 32] The process cannot access the file
because it is being used by another process

Also, mozregression is now more tolerant when facing failures like a build not installable or a crash from the tested application (see bug 1192485).

And finally, when using mozregression with a custom profile it won’t be modified anymore (see bug 999009). This was a longstanding bug - thanks to Jonathan Pigree for his help in fixing it!

This is it, now mozregression uses fully the new build storage!

mozregression now downloads builds from archive.mozilla.org. In many cases this should be faster than before. Good news!

Also this 0.40 release fixes an intermittent windows bug introduced with 0.37, see bug 1185756.

The 0.38 release fixes a minor regression introduced with the 0.37 release (bug 1184915).

Just after that mozlog 3.0 was released, breaking compatibility with older mozbase packages, so we decided to ensure fully working dependencies and released mozregression 0.39! :)

I’m excited to announce that mozregression now supports using taskcluster for inbound bisection with this release!

This considerably speeds up the process of downloading builds from integration branches (like mozilla-inbound) and also supports work by Mozilla Release Engineering to migrate build storage from ftp to s3.

Bugfixes:

• correctly print the resume information (quoting args so the command can be copy/pasted) (bug 1176033)
• temporary files uses a different name while downloading - this fix an issue if the mozregression python process crash while using the persist option, causing next runs to use an incomplete build file to try (bug 1176699)
• Retrying a build while background-downloads do not cancel the next background downloads anymore (bug 1182761)

New features:

• use taskcluster for inbound bisection (search and download builds) (bug 1177923)
• added a warning on windows when using 64 bits non explicitly ( bug 1129899)
• added a new command line flag --background-dl-policy that allow to keep every background download when using persist and background modes (bug 1178028)

I am proud to announce the first release of mozregression-gui, a graphical interface for mozregression!

This is a pre-release alpha version, provided so you can help us to find bugs and give us some feedback.

Give it a try!

This release includes a bugfix for the download in backgound feature introduced in 0.35 (see bug 1153801).

Also, on Firefox for Android we now use a custom profile, meaning:

• the profile on the device is no more overriden by mozregression
• the profile sent by mozregression can be customized

See bug 1147576 for more information.

This release contains the implementation of a new algorithm to download the next builds while you are currently evaluating one (bug 999019).

On a fast enough connection, this can considerably speed up the bisection process by reducing the amount of time spent waiting for downloads to complete.

Hello!

There is a bunch of new things in this new release of mozregression (0.34)!

• Fixed a bug for fennec where sometimes a wrong package name was used.
• Thanks to Jalpreet Singh Nanda, mozregression is more tolerant to network issues.
• the pushlog url and some help to resume a mozregression run are now printed in case of network error or keyboard interrupt (bug 1048858).
• Thanks to Jonathan Pigree, mozregression can now be configured with a config file! Some documentation is coming soon for that.
• Mozregression can now define preferences for firefox (Jalpreet Singh Nanda).
• Mozregression inbound bisection no longer need the [--inbound flag], thanks to Rahul Ranjan.
• Mozregression will automatically warn users that a new version is available thanks to the work of Hrudaya Rn. Sahoo.

Hey,

Mozregression 0.33 is released!

There is one new cool feature, a “back” command is available when doing a manual bisection! See the relevant bug for more information. mozregression can also now download recent windows 64 nightlies.