Recovering huge EXT4 partition crashes R-Studio every time

[tbessie]

It fills them with the pattern virtually, during recovery, correct? It doesn't actually write to the disc being recovered?

Sure! R-Studio never ever tries to write anything on the disk.

In any case, R-Studio has been running all night, and says it's 95% finished (which is a lot further than it got before), and is only using 9gb of RAM at the moment, so I am crossing my fingers that this scan will finish properly.

- Tim

Good luck in data recovery!

Good to know it doesn't write to the source disk.

Well, the scan finished on one of the 3tb drives; now I'm having it run on the other one. Total time is about 10 hours for 2tb of data on a 3tb drive, scan file ended up being around 25gb, most memory used appeared to be around 9gb. So now you folks have some stats that might help others in similar circumstances. I'm glad upgrading to 16gb of ram (and possibly enlarging the paging file) solved this issue.

Now I just have to see how well scan file analysis works - in itself it looks like that takes up even more memory than just the scanning. :-/

I shall check that and report back later.

- Tim

[tbessie]

Well, both scans completed. However, in attempting to load the scan in the "Extra Found Files" section, all available memory is being taken up, and the pagefile is growing to what looks like the max size of the partition (which is 45gb). I predict a crash soon.

If so, that means I'll have to move the pagefile to another - this time very large - partition and allow it to grow to some huge size, or build myself a server-class machine and put 64gb or 128gb on it... that might be cost-prohibitive.

BOY R-Studio takes a lot of RAM and swap to to its work!

EDIT: Yes, Windows ran out of all available memory and swap and stopped functioning well, and wanted to kill R-Studio.
Are there ANY documents about how much memory and swap you need for a given number of files/gb/etc?
Any stats from R-Studio would be helpful. I know they can't predict every situation, but a general formula
for know this info would be helpful.

EDIT 2: Even better, you could provide a calculator (online or not) wherein you enter the size of a partition or disk, an estimated amount of data (in mb or gb) you believe exists (perhaps divided by existing files and deleted files), and the number of these files, and it would calculate how much RAM and SWAP you would need to do both the scan operation and the scan-examination/recovery operation. Since it's taking me a week or two to figure this out through trial and error, it'd be nice if this kind of information were provided the user. If it already exists and I've missed it, please let me know.

- Tim

[tbessie]

Well, I set my paging file to grow up to 512gb, and at about 1/3 finished it's already grown to 100gb or more when asking to look at files it's found.

Once again - I get the feeling that R-Studio is bad at memory management when doing it's job - either that, or it somehow thinks that every block is a separate file. The file signatures I'm searching for are only 4 types, and very distinctive, so I'd find that hard to believe.

I hope your support guys get back to me with some word from your developers - this is looking rather ridiculous.

By the way, did you know that a lot of your web pages redirect to nonexistent pages now? Why is that?

- Tim

EDIT: Actually, that's a good question - the file signatures I set up, I did using your file signature editor. The XML generated contains empty "END (OR)" clauses; as well, I created a file signature that contains NO signature, but only an extension (.ISO) in the assumption that R-Studio will look for files by extension as well, if there is a match, and not match by data in the file (because, as in this example, .ISOs don't have a signature that's easily recognizable, at least from what I could discover).

Would either of the above custom file signature situations make R-Studio match ANY block as a potential file match? The documentation didn't seem very clear (or at least not obviously so) on this point.

[Guest]

Hi Tim,

I just recently recovered a large ext4 partition on a Centos server, I had two degraded 1.5TB HDDs on a RAID5 consisting of four HDDs.
I used R-STUDIO for windows and two Windows7 pro workstations with each PC running with i7 + 8GB RAM. I grabbed an image of each HDD four in total,
each image was copied to a single HDD and were added as single drives on one workstation, then on the second machine I made a 5TB JBOD raid to dump the recovered data and it was from this workstation that I executed R-STUDIO, once I worked out the correct parity rotation it took 27 hours to do the scan and another 25 hours to do the recovery I had a very small negligent loss, I rebuilded the RAID5 on the Centos server and restored the data.

I know that my scenario was different to your current situation but as far as r-studio goes it can deal with very large EXT4 partitions.