Originally published December 28, 2019 @ 12:42 am

The first well-known case of ransomware was documented in 1989. The so-called AIDS Trojan was delivered on a floppy disc; encrypted data; demanded $189.00 (nearly four hundred bucks in today’s money) as a “license fee.” The trojan was quickly defused due to its reliance on weak symmetric cryptography. In contrast, data encrypted by modern ransomware is virtually irrecoverable without the decryption key.

Ransomware attacks can be very damaging and carry a stigma, forcing the victims to keep quiet. If you get hit by ransomware today and don’t have a usable backup, your only realistic hope to recover your data is to pay the ransom. The FBI disagrees, but in such situations, the agency’s main objective is to discourage ransomware authors – not to recover your data.

No tool or method guarantees a ransomware-free computer. Sooner or later, you’ll find it, install it somehow, and then spend days wondering how you could’ve been so stupid. The key here is to have timely backups stored at a location ransomware cannot access.

And this is not a trivial technical challenge. Any practical backup scheme needs to be automated. Any automated process running on your computer needs unattended access to backup storage. If your computer gets infected, ransomware will also access your storage. It will encrypt data on your computer and everywhere else it can get to, leaving you without a usable backup.

Bare Bones

So, it would help if you had a mechanism that moves your data from the primary backup location to somewhere “offline,” where the computer cannot get to it without your manual intervention. The most straightforward illustration of such a setup would be a dual-disk USB3 replication station.

When the station contains two disks and is plugged into your computer, you can only access the first disk. This would be your primary backup destination. Once the backup is done, you push the disk copy button, and the primary disk is cloned to the secondary disk. Your computer cannot access this secondary disk unless you physically swap the two disks.

blank
I have a more advanced backup system, but I still use this simple setup occasionally.

A setup like this is cheap, simple, secure. There are manual steps: you need to connect your computer to the disk station, kick off the backup, and, once the backup completes, push the disk replication button. This is not something you would do every day, but if you’re hit by ransomware, having a usable backup a week or even a month old is better than having nothing at all.

Imagine a similar setup where you don’t need to push a button to copy data between the disks. Also, instead of cloning the entire hard drive, only your latest backup is copied to the “offline” disk. Finally, it would be more convenient if you could access the backup destination via the network. Consider the following diagram:

The key here is the “Replication” device. Ideally, it should have no network connection. It would be directly connected via USB to the NAS (the box on the right) and the “offline” storage (the box on the left).

A tricky part is a connection between the NAS and the replication device. Most NAS servers have USB3 interfaces for connecting expansion storage. Here we need a NAS that can be connected to a computer as a USB storage device in addition to NAS functionality. We’re talking about something like this network-enabled hard drive case, for example:

blank
KuWfi U35WF USB3.0 Wireless Hard Drive Case

The replication device can be any basic computer equipped with dual USB3 ports.

Raspberry Pi 4 comes with two USB3 ports, a gigabit network, and 4GB of RAM.

And the “offline” storage device can be any USB3 disk enclosure offering disk redundancy and sufficient capacity.

blank
Yottamaster Aluminum Alloy 2 Bay 3.5 inch USB3.0 Hard Drive RAID External Array

A setup like this would be sufficient for a developer but not for a sysadmin. I felt I needed something a bit more sophisticated. But before you follow me down that rabbit hole, I think a few words need to be said about hindering ransomware attacks.

Honeypot

Just a few paragraphs above, I said that nothing could prevent a ransomware attack, and I stand by that statement. However, there is a way to intercept it and, hopefully, do so quickly enough to avoid significant damage. There are commercial ransomware protection tools that monitor for processes modifying files at a high rate.

When a process is spotted, it can be paused, killed, or otherwise rendered inactive or incapable of performing write I/O. The key here is time. Ransomware seeks out data it considers most valuable and encrypts it first. This happens very fast. When your anti-ransomware utility detects something suspicious, it may already be too late.

One approach (used by the FBI) is to present the suspected ransomware with an extensive collection of seemingly valuable user data. This delay may give your antivirus scanner just enough time to catch the ransomware process and terminate it, hopefully, before it gets to the real stuff.

I don’t know exactly how this method works, but I would imagine your local and network file storage is peppered with data honeypots of substantial depth and realism to both slow down ransomware and shine a spotlight on it. Just off the top of my head, here’s a simple example:

We create ten folders and fill them with a thousand empty files (in a more realistic scenario, these would be actual or sufficiently realistic MS Office documents and somesuch):

for d in `seq 1 10`
do 
  mkdir -p /var/tmp/honeypot/dir_${d}
  for j in `seq 1 100`
  do
    touch /var/tmp/honeypot/dir_${d}/file_${j}
  done
done

Now we set up an inotifywait process that will spring into action as soon as one of those one thousand files is modified. The inotifywait process will detect the PID of the process that is accessing the folder containing the files, and it will record detailed information about this process:

inotifywait -mr /var/tmp/honeypot -e modify | while read path action file
do
  pid="$(lsof "${path}/" | tail -1 | awk '{print $2}')"; lsof -p "$({ echo "$pid"; pgrep -P "$pid"; } | paste -sd , -)"
done

Sample output when I echo 1 > /var/tmp/honeypot/dir_1/file_83:

COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
bash    19675 root  cwd    DIR  253,2     4096  526058 /var/tmp/honeypot/dir_1
bash    19675 root  rtd    DIR  253,0     4096       2 /
bash    19675 root  txt    REG  253,0   942200 1181178 /bin/bash
bash    19675 root  mem    REG  253,0   161776  393259 /lib64/ld-2.12.so
bash    19675 root  mem    REG  253,0  1930416  393274 /lib64/libc-2.12.so
bash    19675 root  mem    REG  253,0    23088  393412 /lib64/libdl-2.12.so
bash    19675 root  mem    REG  253,0   134792  398348 /lib64/libtinfo.so.5.7
bash    19675 root  mem    REG  253,0 99174448 1097834 /usr/lib/locale/locale-archive
bash    19675 root  mem    REG  253,0    66432  411184 /lib64/libnss_files-2.12.so
bash    19675 root  mem    REG  253,0    26060 1052122 /usr/lib64/gconv/gconv-modules.cache
bash    19675 root    0u   CHR  136,1      0t0       4 /dev/pts/1
bash    19675 root    1u   CHR  136,1      0t0       4 /dev/pts/1
bash    19675 root    2u   CHR  136,1      0t0       4 /dev/pts/1
bash    19675 root    6u   CHR  136,1      0t0       4 /dev/pts/1
bash    19675 root  255u   CHR  136,1      0t0       4 /dev/pts/1

If simultaneously you’re also running atop, you will get a lot more detailed information about the PID responsible for modifying the files in this folder, when it was created and how. Instead of recording the details of what happened, you can tell the inotifywait process to just pause (or kill) the suspicious PID:

inotifywait -mr /var/tmp/honeypot -e modify | while read path action file
do
  pid="$(lsof "${path}/" | tail -1 | awk '{print $2}')"
  echo ${pid}
  kill -STOP ${pid}
done

You can also identify which PIDs have received the STOP signal

ps -e -o stat,command,pid | grep '^Ts+ '

And resuming them is easy:

kill -CONT ${PID}

Putting all of this together, the final result may look something like this:

Sample output

Any process that modifies even one file inside the honeypot folder will be put on ice, and its details will be recorded. This will only affect processes that change files – not just access them, so your malware scanner and backup application will be fine.

It is also possible to combine this process with notification; rsync, to immediately restore any modified or deleted files from a remote location; nmap to scan the suspicious source of the connection, the firewall to block that connection, and just about anything else you can think of:

A Thousand-Dollar Solution

With this out of the way, let’s get back to a better ransomware-resistant backup setup. I had a couple of older Synology DS1813+ NASes that I wanted to dedicate to my “business continuity” purposes.

LAN2 is a private network allowing data replication from NAS1 to NAS2. LAN1 is connected to your home network. E2.1 connection is unconfigured and reserved for disaster recovery scenario

Each NAS has four gigabit NICs allowing me to connect one of them to my home network using LACP aggregation offering 3Gbps bandwidth, while the remaining NIC on the “online” NAS was used to connect it to the “offline” NAS via a small managed TP-Link gigabit switch running a private network.

The remaining step was to configure periodic snapshots of my backup volume and set up a snapshot replication service to copy the snapshots to the “offline” NAS.

If you’re looking for a DIY solution, you need something with at least a couple of USB3 ports and a dual gigabit NIC.  The options here are minimal and expensive, like the $230 Grapeboard. And you would still need two USB3 RAID-enabled disk enclosures. At this price, you’re better off with something like QNAP TS-253Be or Synology DS718+.

Here’s a rough breakdown of your costs for a setup like this offering 4TB 4x-redundant storage capacity:

Item Quantity Unit Price (USD) Line Cost (USD)
WD Red 4TB NAS Hard Drive 4 $100 $400
QNAP TS-253Be-2G-US 2 $330 $660
TP-Link 8-Port TL-SG108E switch 1 $28 $28
Cat 7 Shielded Ethernet Cable 5 ft 4 $5 $20
TOTAL $1,108

blank

You should also spend about $35 per license for some decent backup software for each PC you need to protect. On Windows, I’ve been using Acronis True Image for many years now and had no reason to complain. The added bonus is that True Image comes with its own type of ransomware protection—the more, the merrier.

The inotifywait utility is available via CLI on both Synology and QNAP, so it’s no problem to set up some honeypots and have the system watch for any changes. I have a honeypot process running on the 5-bay QNAP TS-563 that I use as the default dumping ground for my laptops, making it more susceptible to a ransomware attack. There are no idiot-proof solutions, but, so far so good.