Changes

Packages

This test can only be done by a Tails developer (Foundations Team).

Compare the list of bundled packages and versions with the one shipped last time. The new .packages file should have been attached to the call for testing email sent by the Release Manager after the images have been built.

/usr/bin/diff -u \
        ${MASTER_CHECKOUT:?}/wiki/src/torrents/files/tails-amd64-${PREVIOUS_VERSION:?}.packages \
        ${ISOS:?}/tails-amd64-${VERSION:?}/tails-amd64-${VERSION:?}.packages \
        | wdiff --diff-input  --terminal

Check the output for:

• new packages that may cause harm or make the images unnecessarily big
• packages that were erroneously removed
• new versions of software we might not have audited yet

Image size

Check the images size has not changed much since the last release.

In a directory with many Tails ISO and USB images (archive):

(
cd "${ISOS:?}"
find -L \( -iname "tails*.iso" -o -iname "tails*.img" \) \
       -type f -ls | sort -k 7
)

Reproducibility

⚠ Wait until the Release Manager removes this warning.

This section can not be done by the RM.

1. Check that the https://tails.net/contribute/calendar/ documents who is the Trusted Reproducer for this release. If this is not the case, ask the RM (this is the only exception to "do not trust anything said by the RM about this process").

2. Download the ISO and USB images.

3. Clear-sign the hashes of all products using your OpenPGP key and gzip the output (otherwise the signed text could be mangled at some point in the email chain):

    DEST_DIR=$(mktemp -d)
    sha512sum *.iso *.img \
      | gpg --clear-sign \
      | gzip \
      > "$DEST_DIR/TR-bits.gz"
   

4. Locate the file generated by the command above. To display its location, execute the following command:

    echo "$DEST_DIR/TR-bits.gz"
   

5. Send the aforementioned generated file as an attachment to the Trusted Reproducer.

6. If the Trusted Reproducer is around, ask them:

   • if they've received your email
   • if they could successfully decompress the attachment
   • if they could check the inline signature of the attachment once decompressed
   • if the signed text contains all the information they need

Automated test suite

• Revert 4d9b13e6b8c055237aab403945c6bb9d03ac6d85.

• Point --old-iso to the ISO image of the previous stable release.

• We want the test suite run to be as "vanilla" and reproducible as possible, so the only allowed options for run_test_suite are:

  • --artifacts-base-uri
  • --capture and --capture-all
  • --tmpdir
  • --view
  • --vnc-server-only
  • --iso and --old-iso (in fact, both of these are mandatory)

  For cucumber itself we only allow the --format option.

See:

• automated test suite documentation
• setup
• usage

Tor Browser

Miscellaneous functionality

• Test if uBlock works:
  • The uBlock icon must be visible.
  • Visit a website that normally displays ads, such as https://www.nytimes.com/. The ads should not be displayed and the uBlock icon should display a strictly positive number of blocked elements.

Security and fingerprinting

1. In Tails' Tor Browser, visit about:config. Check that media.peerconnection.enabled is set to false.

2. Download the version of Tor Browser included in the Tails release you are testing.

3. Start that Tor Browser on a non-Tails Linux.

4. Compare the fingerprint of that Tor Browser with the one of Tails' Tor Browser, using Test your browser on https://coveryourtracks.eff.org/.

   To see the details we're interested in, scroll down to check Detailed results and click on Detailed view.

   • The exposed User-Agent should match the latest Tor Browser's one.
   • Ignore the result of the "blocking tracking ads" and "blocking invisible trackers" tests, which seem unreliable (we've seen different results for the very same version of Tor Browser).

5. If the fingerprints differ, ask the Release Manager to update the Browser fingerprint section of the known issues page.

Thunderbird

Only perform this test if one of these conditions is met:

• We're testing a release candidate, such as Tails 7.42~rc1.
• The release under testing upgrades to a major Thunderbird version. For example, the last Tails release shipped Thunderbird 87.9, and the one you're testing includes Thunderbird 88.1.

To check that the EHLO/HELO SMTP message is not leaking anything at the application level:

1. Start Thunderbird using the GNOME Applications menu.
2. Configure the outgoing SMTP server for an email account so it uses STARTTLS on port 587 (Thunderbird will send two EHLO/HELO: one before TLS is initiated; one after; the assumption here is that Thunderbird will send the same both times).
3. Run sudo tcpdump -n -i lo -w dump.pcap while sending an email to capture the packets before Tor encrypts it, then close tcpdump. Note that the packet containing EHLO/HELO will be sent really early, so even if the email failed (e.g. because the mail server doesn't support plaintext SMTP on port 587) we are ok.
4. Check the dump for the HELO/EHLO message and verify that it only contains 127.0.0.1: sudo tcpdump -A -r dump.pcap | grep EHLO

WhisperBack

• I should be able to send a bug report with WhisperBack.
• When we receive this bug report on the tails-bugs mailing list, Schleuder tells us that it was sent encrypted.

Virtualization support

• Test that Tails starts and the browser launches in VirtualBox.

  Hint: VirtualBox works inside a KVM virtual machine. So if you can't easily setup VirtualBox on your host, you can run it inside a guest VM.

Incremental upgrades

⚠ Wait until the Release Manager removes this warning.

(automate: #18004)

Follow the instructions below for each of these versions of Tails:

⚠ PLACEHOLDER

1. Install this version of Tails from scratch to a USB stick. A system that was upgraded to that version will not do.

2. Start from that USB stick.

3. Set an administration password in the Welcome Screen.

4. Prepare the system so it can upgrade to the version we're testing:

    sudo sh -c 'sed -i /^TAILS_CHANNEL=/d /etc/os-release &&
                echo TAILS_CHANNEL=\"test\" >> /etc/os-release' && \
    systemctl --user restart tails-upgrade-frontend.service
   

   Don't wait for the last command to complete: this will happen later on.

5. Connect to Local Network and to Tor

6. Accept the suggestion to upgrade to the version we're testing

7. Once the upgrade has been applied and you're suggested to restart Tails, do that.

8. Verify that the resulting, upgraded system "works fine":

   • it boots
   • The About Tails app reports the expected version: the one we are testing
   • You can connect to Tor using Tor Connection
   • Tor Browser can browse websites
   • the Unsafe Browser starts

Torrents

Download the images with the Torrent files provided by the Release Manager:

• ISO

• USB image

Wait until the downloads complete before calling this done.

Real (non-VM) hardware

UEFI boot

Note that for emergency releases, we do not always optimize the ordering of files in the SquashFS, which might make them boot somewhat slower.

On a 64-bit computer that has Secure Boot enabled and that is configured to prefer UEFI:

1. Freshly install the version of Tails being tested to a USB stick.
2. Boot this USB stick on bare-metal a first time to trigger re-partitioning.
   • You should see the "GNU GRUB" title, as opposed to a syslinux menu.
   • Wait until you see the Welcome Screen.
3. Shutdown Tails
4. Boot this USB stick a second time, add the login option to GRUB, and measure the boot time (from the GRUB menu until the GNOME desktop is ready). Take note of the boot time you measured.
5. Go through steps 1-4 again, but this time using the previous Tails version. Use the same computer and the same USB stick.
6. Compare the boot times you measured at steps 4 and 5. Expected result: the new Tails should not be significantly slower to start than the old one.

BIOS boot

On a computer configured to prefer legacy BIOS boot:

1. Freshly install the version of Tails being tested to a USB stick.
2. Boot this USB stick on bare-metal a first time to trigger re-partitioning.
   • You should see the "SYSLINUX" title in the bootloader, as opposed to GRUB.
   • Wait until you see the Welcome Screen.
3. Shutdown Tails
4. Boot this USB stick a second time.
5. You should see the Welcome Screen appear.

Journal

This test can only be done by a Tails developer (Foundations Team).

Check that the output of sudo journalctl looks OK.

For a quick overview, something like sudo journalctl -p 0..3 can be helpful.