Preventing R-Studio from extracting several files simultaneously
Posted: Thu Jul 18, 2019 8:06 pm
I was extracting large folders of AVI files from a video-surveillance system with R-Studio, and the rate of extraction seemed rather slow for such large files (about 100MB each, normally the reading / writing speed is significantly reduced only for small or very small files, <1MB). Then the system crashed (Windows 7 on a NVMe SSD, I get many random crashes, with different kinds of errors, this time it was “MEMORY_MANAGEMENT”, and still couldn't figure out the root cause(s), but that's beside the point, R-Studio is not the culprit, this is just for context). So I re-extracted the last extracted folder, which was incomplete, and proceeded to compare it with the one extracted before the crash, using WinMerge : 11 files were partially or totally corrupted. Strangely, those files were not the last ones in sequential order, either by name sorting or by date sorting, or even by size sorting, so apparently R-Studio relies on another criterion to determine the order in which files are extracted (and folders too : for instance a folder named “0813” was completely extracted before “0812” which is the one incomplete and containing corrupted files). I examined the MFT on the recovery partition : the corrupted files do correspond to the last 11 MFT records [*], so at least that part is consistent, and it's unlikely that other files have been corrupted in other folders.
Then I compared those files with their re-extracted counterparts, using HexWorkshop (much faster than WinMerge for the comparison of large files) : some are completely empty, but several are partially empty, with only the last 10MB or so filled with 00s. I figured that whatever caused this issue should have affected the last files extracted in exactly the same way, i.e. making all of them totally empty, except the one which was extracted when the problem began, which would be correct at the begining and then corrupted at some point. If several files are partially corrupted at the end, it means that several files were being extracted simultaneously, which would explain the poor rate of extraction. To verify this, I analyzed the recovery partition, which was created just yesterday and only contains the files recovered by R-Studio from that video-surveillance drive, with a defragmentation software (Defraggler) : many of them are indeed fragmented, with up to 10 fragments each (and a global fragmentation rate of 43%), which would not have happened if the files had been extracted one by one. But I found no way of controling this behaviour in R-Studio's settings. This is a major problem because, as I said, it significantly slows down the extraction, especially on mechanical drives. Did I miss something ?
TL;DR : As the title says, is there a way of preventing R-Studio from extracting several files simultaneously, so as to optimize the speed of extraction ? If not, there definitely should be.
[*] Well, the last ones corresponding to AVI files, as the “$Recycle.bin” and “System Volume Information” folders were created when the system rebooted.
Then I compared those files with their re-extracted counterparts, using HexWorkshop (much faster than WinMerge for the comparison of large files) : some are completely empty, but several are partially empty, with only the last 10MB or so filled with 00s. I figured that whatever caused this issue should have affected the last files extracted in exactly the same way, i.e. making all of them totally empty, except the one which was extracted when the problem began, which would be correct at the begining and then corrupted at some point. If several files are partially corrupted at the end, it means that several files were being extracted simultaneously, which would explain the poor rate of extraction. To verify this, I analyzed the recovery partition, which was created just yesterday and only contains the files recovered by R-Studio from that video-surveillance drive, with a defragmentation software (Defraggler) : many of them are indeed fragmented, with up to 10 fragments each (and a global fragmentation rate of 43%), which would not have happened if the files had been extracted one by one. But I found no way of controling this behaviour in R-Studio's settings. This is a major problem because, as I said, it significantly slows down the extraction, especially on mechanical drives. Did I miss something ?
TL;DR : As the title says, is there a way of preventing R-Studio from extracting several files simultaneously, so as to optimize the speed of extraction ? If not, there definitely should be.
[*] Well, the last ones corresponding to AVI files, as the “$Recycle.bin” and “System Volume Information” folders were created when the system rebooted.