Let’s say you have access to a remote machine and use it to copy backups occasionally, eg with rsync. Your local machine has credentials stored that allow write access on the remote machine, however if the local account was compromised that could also allow access to the remote machine and the data stored there.
How can you grant access to an account to write remotely, but also protect the data from this account? One possibility could be to change the permissions on the data after it is copied to prevent deletion/interference, although I’m just making this up. Is there a standard practise for this?
Do pull backups instead of push backups: Backup server connects to local machine.
What’s the rationale for this? Genuinely curious.
The reasoning is that your backup server should be more secure than production. Production has to have a bunch of stuff open in order to be useful and convenient. The backup server does not. It can be basically fully locked down.
If the main site gets compromised the credentials there must be considered lost and known to che attackers.
with a pull backup that’s not an issue because the main site has no access to the remote system; it is a process on the remote site that has credentials to access the main site and not the other way around.
the remote system may
receiveretrieve a compromised copy of the data, but the attacker cannot tamper with previous backups so recovery is still possible.That makes sense. I use NFS, so there are other controls for security because “offsite” is another building on my property, but still in the same pool of subnets…
This is the main reason I had in my head about pull backups. Thanks for the explanation.
Why downvote this?
That’s an interesting idea
Restic has quite a good solution for this: https://restic.readthedocs.io/en/stable/060_forget.html#security-considerations-in-append-only-mode
Funny thing is that blockchains are actually good for this type of thing. Too bad crypto bros got a hold of it.
Append only mode.
The suggestion I have heard is to have the remote machine connect to the machine on a schedule and pull the backups onto itself. Then your local machine doesn’t have direct access to the backups, making it harder to compromise the backups if hacked. But this also assumes the backup machine is locked down and isolated so it is lower risk than the local machine.
A system like proxmox backup server can do this scurely. There you can create a user that can only add new backups and read the existing ones, but cannot delete any or read anything else on the remote host.
Otherwise if you only care to protect the remote machine, then something like an ssh chroot jail would also work.
Just a small sidenote: If you do not trust your local machine you should think about why and how to change that.
Well I think she amount of caution is still appropriate no matter how much you manage to secure the local machine
Of course, of course :)
I have solved that by giving the distant machine the credentials to connect to the local machine. And the distant machine can’t be accessed from the outside.
Yes that sounds like pull backups, which is worth exploring
What you are talking about is Immutability and an append only backup. The s3 file system and some others has Immutability built in. Not all backups can do append only.
Scheduled snapshots (btrfs or zfs). If the compromised account deletes or modifies files, they’re still there in the past snapshots
Filesystem-level snapshots are quite space-efficient because they don’t make copies of all the files or even whole files; just the blocks that changed.
You’ve got the right idea with the permission change… the key is that you can have code executing on the remote side with different permissions. So the writer process has permission to write in one directory, and the turnsyle procees (often the root super-user) rotates the files or directories at a different time (or on a signal, sometimes).
I mean depends on the solution you are using, but you can have multiple accounts on the remote backup. IE so upon completion of the backup. The remote machine moves the backup to an offline or read only share (depending if you need those credentials to access the data again later),
Obviously most important thing is your credentials that make the backup… should be very limited in scope to just doing those backups.
WORM: write once, read many. Any good backup software supports this.
You could also keep offline backups. You can’t compromise what you can’t reach.
Append only permissions.
Encrypted deltas.
Basically the time of the connection is the name of the only that folder that you have access to.
You can also setup a yubikey (or nitrokey) that requires you to physically process and would be immune to the host being compromised.
Yep offline backups are useful, although it does require remembering & making the effort to do it each time
For doing snapshots did that means the local system identifies the changes? Or it all gets copied each time?