The TPM releases the key to the OS at boot time. Without that, there would be no way for the OS to load (assuming the root FS is encrypted).
The key is bound to PCRs in the TPM, which control under what conditions the key can be released. For example, it can be tied to secure boot, bios settings, etc.
Immutable/offline backups. If you backup to local physical media (HDD/tape), physically disconnect/eject it and store it somewhere safe. If you back up to cloud storage (S3, etc), many of them have immutability options. If configured properly nobody (not even you) can delete or modify the backups (within the specified time period).