UFS Recover Files Removed with RM

Amzo

Amzo

So I made a blunder today. I was working on the tensorflow 2.x port and I was clearing up getting ready to commit it to git. During the clearing up, I went to rm some test files I had, but during my impatience, I tab completed the name which complete to files/ and deleted all my patches for tensorflow to work.

I was wondering if there is anyway to recover these with ffs2recov being broken in FreeBSD 13. Don't really want to dig through all the files and recreate 30+ patches.
 
If you happen to have an usb key laying around, just install FreeBSD 12.2 on it and try to use ffs2recov.
Before the clearing up did you create some commit on git(or at least a git add) or none ? In case you did, you could simply use
git fsck --lost-found
 
Nah, I didn't add the files. I had jsut finished the work and was cleaning up before I did a git add, and git commit.
 
I once managed to recover a few files using something like this: dd if=/dev/ada0p2 of=/tmp/tmpfile bs=1m skip=72 count=100. Works by copying parts of the disk to a tmp file and then searching that file. However, you need to know exactly what you are looking for and this probably won't work on a ZFS system.
 
I'm guessing with the disk being active, recovery will be impossible. I'll just recreate them all. It wouldn't have been so bad if I didn't clean my work directory first, as I could've just used

Code: 
make makepatch

Tempted to make a shell aliases for rm -i.
 
Better answer: Do snapshots regularly. Depending on your disk space, performance needs, and workload pattern, you might be able to do them frequently. If you run a good file system (ZFS!), you can even decimate them: Take a snapshot every minute. After 10 minutes, delete all of them, except keep one every 10 minutes. After an hour, delete all of them, except keep one every hour. After a day, delete all of them, except keep one a day. You get the idea.

Best answer: Get into the habit of not deleting files. There is a reason many people have rm and mv aliased to the -i versions of the command.
 
I use UFS as the file system for my main drive that has the OS installed, and I have a secondary drive running ZFS which I use for poudriere building. Makes my system usable without heavy I/O while compiling. I was working on patches on the UFS drive instead of the ZFS drive unfortunately.
 
UFS also has snapshots; I don't know whether they would be usable for intense cycling (like one every minute and then decimating them).
 
Ufs allows a limited number of snapshots.
You could take snapshot 1 on day 1, ... , snapshot 5 on day 5 , and recycle and take snapshot 1 on day 6 , etc ...
Though I never used it. Currently i take incremental zfs snapshots every 30 minutes.
 
Tieks said:
I once managed to recover a few files using something like this: dd if=/dev/ada0p2 of=/tmp/tmpfile bs=1m skip=72 count=100. Works by copying parts of the disk to a tmp file and then searching that file.
Click to expand...
As this is most likely source code, in other words text, strings(1) is extremely useful to run that image through. The data is going to be partial and not be in the correct order but it's better than nothing. It's how I restored a large part of code for a client, saved them 4 weeks worth of lost work.
 
  • Like
Reactions: mtu
I removed a few Perl sources. I got 'em back using grep(1), but strings(1) could have worked too. To whom it concerns, here is the script I used with some added comments. Could be useful.
Code: 
#!/bin/sh
#
# Search a disk for a (part of) removed file.
#
# Will write chunks of HDD to a tmp file, then search that tmp file. Stops when hit found,
# then have a look at $tmpfile to see what was found.
# Needs a long enough search string to grep for without giving too many false positives.
# Beware that parts of a file may be found at different places on the HDD.
# In case of a false positive add +1 to skipcount and restart.
# This script won't work on compressed and/or encrypted disks.
#

tmpfile='/tmp/tmpfile'

skipcount=0
while dd if=/dev/ada0p2 of=$tmpfile skip=$skipcount bs=10m count=1 > /dev/null 2>&1 && \
      ! grep '$StuderPrompt' $tmpfile ; do

    skipcount=$(expr $skipcount + 1)
    echo -n ".$skipcount "

done
exit 0
 
  • Like
Reactions: mtu
I appreciate the answers above, they may help in OPs specific scenario. However, I ran into a similar situation right now, except that my files are about 300GB of photographs. I accidentally executed the equivalent of mv -f file dir instead of mv file dir/file. I unplugged the system within a few seconds once I realized the mistake -- so I bet that most of the data was not overwritten (it's an old-school HDD). I have backups of some of them, but not all. I could write my own utility to scan the disk for some known headers (JPGs, TIFFs and MP4s), but since big files could be fragmented it could be trickier than that.

Are there any existing tools for that? It's a FreeBSD UFS partition, but I can access it from Windows/Linux if needed.
 
If the "dir" portion of your move is on the same physical device then the files are still there, however the 'directory' information is now lost because each file has been essentially renamed to "dir". This is the inode.

Performing something like Tieks mentioned here, is your best chance. Mount the disk read-only and copy chunks of data to somewhere and then using a hex editor seek the file tag for the images. But like you say, they may not be contiguous, so it's likely a futile struggle. It can depend on circumstances, though. If you were to have done a dump/restore, they would be contiguous.

The un-plugging may have actually saved some images due to system caching of inodes.

Someone more knowledgeable in UFS/FFS might be able to help but as I understand it, once those inodes are changed, then there's nothing to get them back, OTHER than to scan the file system looking for file tags indicating JPEG etc. That's what most disk recovery software would do, I imagine.

I do recall there was something called ufsfinder or ufsexplore (yes, vague) that purported to recover files.
 
It had happened to me and all my recovery actions where in vain. The most i could recover was a list of the filenames.
Delete without backup can be unforgiving.
 
Thanks for all the suggestions. I'll use the different tools and see what works. That's not urgent (though is valuable) -- and I won't try any closed source tools or tools that modify the filesystem (like fsck) without creating a full clone first.

mark_j said:
That port's marked broken, so use at your own risk (if it even compiles).
Click to expand...
sysutils/ffs2recov doesn't compile. Also seems to just scan the inodes -- so if they are zeroed out it won't help.

mark_j said:
The un-plugging may have actually saved some images due to system caching of inodes.

Someone more knowledgeable in UFS/FFS might be able to help but as I understand it, once those inodes are changed, then there's nothing to get them back, OTHER than to scan the file system looking for file tags indicating JPEG etc. That's what most disk recovery software would do, I imagine.
Click to expand...
I've studies a bit how UFS works since that happened. I want to note that there's more tricks that can be done there. For example the inodes could be zeroed out, but the indirect blocks would not be -- so a smart utility could use those in combination with the free block bitmap to find deleted files even if they were fragmented. I don't know if commercial tools do that, but ffs2recov does not.

I'm pretty confident that I could recover the data myself if I had enough time on hand.
 
Via <http://wiki.kmu.edu.tw/index.php/FreeBSD_Data_Rescue> (I'm finding things such as these, but have no experience with them):
Code: 
root@mowa219-gjp4-8570p:~ # pkg info --list sleuthkit | grep bin
        /usr/local/bin/blkcalc
        /usr/local/bin/blkcat
        /usr/local/bin/blkls
        /usr/local/bin/blkstat
        /usr/local/bin/fcat
        /usr/local/bin/ffind
        /usr/local/bin/fiwalk
        /usr/local/bin/fls
        /usr/local/bin/fsstat
        /usr/local/bin/hfind
        /usr/local/bin/icat
        /usr/local/bin/ifind
        /usr/local/bin/ils
        /usr/local/bin/img_cat
        /usr/local/bin/img_stat
        /usr/local/bin/istat
        /usr/local/bin/jcat
        /usr/local/bin/jls
        /usr/local/bin/jpeg_extract
        /usr/local/bin/mactime
        /usr/local/bin/mmcat
        /usr/local/bin/mmls
        /usr/local/bin/mmstat
        /usr/local/bin/sigfind
        /usr/local/bin/sorter
        /usr/local/bin/srch_strings
        /usr/local/bin/tsk_comparedir
        /usr/local/bin/tsk_gettimes
        /usr/local/bin/tsk_loaddb
        /usr/local/bin/tsk_recover
        /usr/local/bin/usnjls
root@mowa219-gjp4-8570p:~ #
 
I just wanted to post an update: I managed to recover what seems like over 99% of the data.

I couldn't figure out how to use those tools out-there, and most of them don't even seem to be applicable. Instead I loaded the UFS free-bitmaps and scanned for free blocks that look like indirect blocks (point into other free blocks and divisible by 8). Back-off by 12 blocks and assume that the file starts there. Check headers. Load the image and the EXIF thumbnail. Compare them to check if they agree. This has an incredible success rate.

Alain De Vos said:
It had happened to me and all my recovery actions where in vain. The most i could recover was a list of the filenames.
Delete without backup can be unforgiving.
Click to expand...
I feel your pain. For me it's the opposite -- Even if I could recover the filenames, there's no way to assign them to the recovered data because the inodes are already wiped clear. :(
 
You must log in or register to reply here.
Back
Top