Data Recovery and Disk Utilities Forum @R-TT

Hi Tim,

I just recently recovered a large ext4 partition on a Centos server, I had [b]two[/b] degraded 1.5TB HDDs on a RAID5 consisting of four HDDs.
I used R-STUDIO for windows and two Windows7 pro workstations with each PC running with i7 + 8GB RAM. I grabbed an image of each HDD four in total,
each image was copied to a single HDD and were added as single drives on one workstation, then on the second machine I made a 5TB JBOD raid to dump the recovered data and it was from this workstation that I executed R-STUDIO, once I worked out the correct parity rotation it took 27 hours to do the scan and another 25 hours to do the recovery I had a very small negligent loss, I rebuilded the RAID5 on the Centos server and restored the data.

I know that my scenario was different to your current situation but as far as r-studio goes it can deal with very large EXT4 partitions.

Well, I set my paging file to grow up to 512gb, and at about 1/3 finished it's already grown to 100gb or more when asking to look at files it's found.

Once again - I get the feeling that R-Studio is bad at memory management when doing it's job - either that, or it somehow thinks that every block is a separate file. The file signatures I'm searching for are only 4 types, and very distinctive, so I'd find that hard to believe.

I hope your support guys get back to me with some word from your developers - this is looking rather ridiculous.

By the way, did you know that a lot of your web pages redirect to nonexistent pages now? Why is that?

- Tim

EDIT: Actually, that's a good question - the file signatures I set up, I did using your file signature editor. The XML generated contains empty "END (OR)" clauses; as well, I created a file signature that contains NO signature, but only an extension (.ISO) in the assumption that R-Studio will look for files by extension as well, if there is a match, and not match by data in the file (because, as in this example, .ISOs don't have a signature that's easily recognizable, at least from what I could discover).

Would either of the above custom file signature situations make R-Studio match ANY block as a potential file match? The documentation didn't seem very clear (or at least not obviously so) on this point.

Well, both scans completed. However, in attempting to load the scan in the "Extra Found Files" section, all available memory is being taken up, and the pagefile is growing to what looks like the max size of the partition (which is 45gb). I predict a crash soon.

If so, that means I'll have to move the pagefile to another - this time very large - partition and allow it to grow to some huge size, or build myself a server-class machine and put 64gb or 128gb on it... that might be cost-prohibitive.

BOY R-Studio takes a lot of RAM and swap to to its work!

EDIT: Yes, Windows ran out of all available memory and swap and stopped functioning well, and wanted to kill R-Studio.
Are there ANY documents about how much memory and swap you need for a given number of files/gb/etc?
Any stats from R-Studio would be helpful. I know they can't predict every situation, but a general formula
for know this info would be helpful.

EDIT 2: Even better, you could provide a calculator (online or not) wherein you enter the size of a partition or disk, an estimated amount of data (in mb or gb) you believe exists (perhaps divided by existing files and deleted files), and the number of these files, and it would calculate how much RAM and SWAP you would need to do both the scan operation and the scan-examination/recovery operation. Since it's taking me a week or two to figure this out through trial and error, it'd be nice if this kind of information were provided the user. If it already exists and I've missed it, please let me know.

- Tim

[quote="Alt"][quote="tbessie"]
It fills them with the pattern virtually, during recovery, correct? It doesn't actually write to the disc being recovered?[/quote]
Sure! R-Studio never ever tries to write anything on the disk.
[quote="tbessie"]
In any case, R-Studio has been running all night, and says it's 95% finished (which is a lot further than it got before), and is only using 9gb of RAM at the moment, so I am crossing my fingers that this scan will finish properly. :-)

- Tim[/quote]
Good luck in data recovery![/quote]

Good to know it doesn't write to the source disk. :-)

Well, the scan finished on one of the 3tb drives; now I'm having it run on the other one. Total time is about 10 hours for 2tb of data on a 3tb drive, scan file ended up being around 25gb, most memory used appeared to be around 9gb. So now you folks have some stats that might help others in similar circumstances. I'm glad upgrading to 16gb of ram (and possibly enlarging the paging file) solved this issue.

Now I just have to see how well scan file analysis works - in itself it looks like that takes up even more memory than just the scanning. :-/

I shall check that and report back later.

- Tim

[quote="tbessie"]
It fills them with the pattern virtually, during recovery, correct? It doesn't actually write to the disc being recovered?[/quote]
Sure! R-Studio never ever tries to write anything on the disk.
[quote="tbessie"]
In any case, R-Studio has been running all night, and says it's 95% finished (which is a lot further than it got before), and is only using 9gb of RAM at the moment, so I am crossing my fingers that this scan will finish properly. :-)

- Tim[/quote]
Good luck in data recovery!

[quote="Alt"]Well, the area where R-Studio can potentially crash may be quite small, so one or two files would be missed.
Excluded areas are areas that R-Studio fills with a pattern for bad sectors. That means the files spanning across those areas would have that pattern in them. Maybe, some of such files could be recovered.[/quote]

It fills them with the pattern virtually, during recovery, correct? It doesn't actually write to the disc being recovered?

Incidentally, I made 48gb available for swap and added 8gb more to my motherboard, maxing out the memory to 16gb (this is a previous generation i7 chip, so I can't bump it up to 32gb unfortunately).

In any case, R-Studio has been running all night, and says it's 95% finished (which is a lot further than it got before), and is only using 9gb of RAM at the moment, so I am crossing my fingers that this scan will finish properly. :-)

- Tim

Well, the area where R-Studio can potentially crash may be quite small, so one or two files would be missed.
Excluded areas are areas that R-Studio fills with a pattern for bad sectors. That means the files spanning across those areas would have that pattern in them. Maybe, some of such files could be recovered.

[quote="Alt"]I mean a little bit different thing: start scanning from the point where R-Studio crashed without loading the scan info. Does R-Studio still crash?[/quote]

Ah - that's one I haven't yet tried, as I'm concerned about the issue of files spanning the sections scanned. Probably not a HUGE issue, but enough that it makes me wonder if I'd miss some files due to that.

R-Studio doesn't have any way to associate files that span multiple sections scanned separately, does it?

- Tim

I mean a little bit different thing: start scanning from the point where R-Studio crashed without loading the scan info. Does R-Studio still crash?

[quote="Alt"]I assume that you saves the scan info to a file. You may load it later and resume the scan. Moreover, even if R-Studio crashes, the file remains valid.
You can scan the disk in parts, read more about it on the R-Studio on-line help:[url=http://www.unformat-unerase.com/Unformat_Help/discscan.html]Advanced scan[/url].
You need to select only the Ext file system for the scan to reduce memory load. Most likely, you'll have to remain Extra Search for Known File Types option selected.

But there is a possibility that R-Studio crashes at a certain point of the disk rather that because it's eaten up all the available memory. To check that, specify the area on the disk (quite approximately) on the Advanced scan dialog box and run the scan. If R-Studio crashes, you may create [url=http://www.unformat-unerase.com/Unformat_Help/exclusive_regions.html]an exclusive region[/url] on the disk, scan it, and recover files from it.

I understand that all that requires a lot of time, but that the way I see how to get around the crash.[/quote]

Oh, if I resume the scan from where it crashed, it crashes again, and very soon after. That is, if I load the precious scan, and resume, the R-Studio process quickly grows to 8gb, the new scan file it creates quickly grows to the same size as the original, and it crashes again, so "resuming" doesn't work. I *could* do it piece by piece, but then there'd be no connection between the two sets of scan results - what if some files span the two sets of results? Their connection would be lost if I did that.

I already am following all the suggestions for reducing memory (selecting only EXT filesystems, limiting search patterns to just the 5 types I'm looking for, etc.), to no avail.

I could use the Exclusive Regions method, but, again, would that not miss cases where files span 2 different regions?

Wouldn't it be smarter for R-Studio to not attempt to keep so much scan info in RAM while running, but write more to the scan file (or other metadata storage file)? It might've been more convenient for the programmers that way, but it limits R-Studio's usefulness for huge partitions. A well-designed scanner doesn't NEED to keep everything in memory at once, even if that simplifies the program's design (speaking as a programmer here :-) ).

- Tim