- contribute
- release process
- Manual test suite
Changes
Packages
This test can only be done by a Tails developer (Foundations Team).
Compare the list of bundled packages and versions with the one shipped last
time. The new .packages
file should have been attached to the
call for testing email sent by the Release Manager after the images
have been built.
/usr/bin/diff -u \
${MASTER_CHECKOUT:?}/wiki/src/torrents/files/tails-amd64-${PREVIOUS_VERSION:?}.packages \
${ISOS:?}/tails-amd64-${VERSION:?}/tails-amd64-${VERSION:?}.packages \
| wdiff --diff-input --terminal
Check the output for:
new packages that may cause harm or make the images unnecessarily big
packages that were erroneously removed
new versions of software we might not have audited yet
Image size
Check the images size has not changed much since the last release.
In a directory with many Tails ISO and USB images (archive):
(
cd "${ISOS:?}"
find -L \( -iname "tails*.iso" -o -iname "tails*.img" \) \
-type f -ls | sort -n -k 7
)
Automated test suite
Point
--old-iso
to the ISO image of the previous stable release.We want the test suite run to be as "vanilla" and reproducible as possible, so the only allowed options for
run_test_suite
are:--artifacts-base-uri
--capture
and--capture-all
--tmpdir
--view
--vnc-server-only
--iso
and--old-iso
(in fact, both of these are mandatory)
For
cucumber
itself we only allow the--format
option.
See:
Tor Browser
Miscellaneous functionality
Test if uBlock works:
The uBlock icon must be visible.
Visit a website that normally displays ads, such as https://www.nytimes.com/. The ads should not be displayed and the uBlock icon should display a strictly positive number of blocked elements.
Security and fingerprinting
Download the version of Tor Browser included in the Tails release you are testing.
Start that Tor Browser on a non-Tails Linux.
Compare the fingerprint of that Tor Browser with the one of Tails' Tor Browser, using Test your browser on https://coveryourtracks.eff.org/.
To see the details we're interested in, scroll down to check Detailed results and click on Detailed view.
The exposed
User-Agent
should match the latest Tor Browser's one.Ignore the result of the "blocking tracking ads" and "blocking invisible trackers" tests, which seem unreliable (we've seen different results for the very same version of Tor Browser).
Ignore the "bits of identifying information" in the "Your results" summary, it is not clear what this number is (it is not the sum of the results for each section).
Instead compare results section-by-section. The only section that is allowed to differ is the Screen Size and Color Depth in case you couldn't test the browsers with the exact same window dimensions.
If the fingerprints differ, ask the Release Manager to update the Browser fingerprint section of the known issues page.
Thunderbird
Only perform this test if one of these conditions is met:
We're testing a release candidate, such as Tails 7.42~rc1.
The release under testing upgrades to a major Thunderbird version. For example, the last Tails release shipped Thunderbird 87.9, and the one you're testing includes Thunderbird 88.1.
To check that the EHLO/HELO SMTP message is not leaking anything at the application level:
In the Welcome Screen, set an administration password.
Start Thunderbird using the GNOME Applications menu.
Configure the outgoing SMTP server for an email account so it uses STARTTLS on port 587 (Thunderbird will send two EHLO/HELO: one before TLS is initiated; one after; the assumption here is that Thunderbird will send the same both times).
Run
sudo tcpdump -n -i lo -w dump.pcap
while sending an email to capture the packets before Tor encrypts it, then close tcpdump. Note that the packet containing EHLO/HELO will be sent really early, so even if the email failed (e.g. because the mail server doesn't support plaintext SMTP on port 587) we are ok.Check the dump for the HELO/EHLO message and verify that it only contains
127.0.0.1
:sudo tcpdump -A -r dump.pcap | grep EHLO
WhisperBack
I should be able to send a bug report with WhisperBack.
When we receive this bug report on the tails-bugs mailing list, Schleuder tells us that it was sent encrypted.
Incremental upgrades
⚠ Wait until the Release Manager removes this warning.
(automate: #18004)
Follow the instructions below for each of these versions of Tails:
⚠ PLACEHOLDER
Install this version of Tails from scratch to a USB stick. A system that was upgraded to that version will not do.
Start from that USB stick.
Set an administration password in the Welcome Screen.
Prepare the system so it can upgrade to the version we're testing:
sudo sh -c 'sed -i /^TAILS_CHANNEL=/d /etc/os-release && echo TAILS_CHANNEL=\"test\" >> /etc/os-release' && \ systemctl --user restart tails-upgrade-frontend.service
Don't wait for the last command to complete: this will happen later on.
Connect to Local Network and to Tor
Accept the suggestion to upgrade to the version we're testing
Once the upgrade has been applied and you're suggested to restart Tails, do that.
Verify that the resulting, upgraded system "works fine":
it boots
The About Tails app reports the expected version: the one we are testing
You can connect to Tor using Tor Connection
Tor Browser can browse websites
the Unsafe Browser starts
Torrents
⚠ Wait until the Release Manager removes this warning.
Download the images with the Torrent files provided by the Release Manager:
ISO
USB image
Wait until the downloads complete before calling this done.
Real (non-VM) hardware
UEFI boot
Note that for emergency releases, we do not always optimize the ordering of files in the SquashFS, which might make them boot somewhat slower.
On a 64-bit computer that has Secure Boot enabled and that is configured to prefer UEFI:
Freshly install the version of Tails being tested to a USB stick.
Boot this USB stick on bare-metal a first time to trigger re-partitioning.
You should see the "GNU GRUB" title, as opposed to a syslinux menu.
Wait until you see the Welcome Screen.
Shutdown Tails
Boot this USB stick a second time, add the
login
option to GRUB, and measure the boot time (from the GRUB menu until the GNOME desktop is ready). Take note of the boot time you measured.Go through steps 1-4 again, but this time using the previous Tails version. Use the same computer and the same USB stick.
Compare the boot times you measured at steps 4 and 5. Expected result: the new Tails should not be significantly slower to start than the old one.
BIOS boot
On a computer configured to prefer legacy BIOS boot:
Freshly install the version of Tails being tested to a USB stick.
Boot this USB stick on bare-metal a first time to trigger re-partitioning.
You should see the "SYSLINUX" title in the bootloader, as opposed to GRUB.
Wait until you see the Welcome Screen.
Shutdown Tails
Boot this USB stick a second time.
You should see the "SYSLINUX" title in the bootloader.
You should see the Welcome Screen appear.