Bulat_Ziganshin
Silver Member | Редактировать | Профиль | Сообщение | Цитировать | Сообщить модератору
SREP 2.95 alpha:
* -mem option limits amount of RAM used for future-LZ decompression
* -vmfile and -vmblock options fine-tunes VM file used in -mem mode
Tests with the same 22gb->7gb file:
Код: : Cpu 211.560 mb/sec, real 165.110 mb/sec. Matches 0 174390 1449482, I/Os 0, RAM 0/1919, VM 0/0, R/W 0/0
-mem1g : Cpu 203.116 mb/sec, real 151.955 mb/sec. Matches 0 174390 1482625, I/Os 0, RAM 0/983, VM 0/1000, R/W 1560/1560
-mem500: Cpu 193.556 mb/sec, real 136.138 mb/sec. Matches 0 165382 1621776, I/Os 0, RAM 0/460, VM 0/1656, R/W 3768/3768
-mem200: Cpu 146.510 mb/sec, real 99.487 mb/sec. Matches 0 70888 2376056, I/Os 0, RAM 0/160, VM 0/2136, R/W 12120/12120
-mem100: Cpu 76.815 mb/sec, real 54.448 mb/sec. Matches 0 29368 4950862, I/Os 0, RAM 0/60, VM 0/2360, R/W 36760/36760
|
So, we are able to decompress 22gb file with 2gb of RAM, or even with 200 mb RAM and 2gb VM file, and speed remains very high, at 100 mb/s! I think, it's outstanding result
Info about interpreting new decompression stats. Let's consider the following line:
RAM 346/1024, VM 664/984, R/W 824/1488
It means that there are 1024 MiB of RAM allocated, of those 346 MiB is currently used (memory is never returned to OS)
VM file is 984 MiB long, of those 664 MiB is in use now
1488 MiB were written to VM file, of those 824 MiB was already read back
VM.current = VM.W - VM.R equation is always true, here it's 664=1488-824
At the end of decompression, we have a sort of that:
RAM 0/983, VM 0/1000, R/W 1560/1560
i.e. two zeros and R=W, while the rest of nu,bers shows how much memory/disk was used and how much data were written to VM file
Also, sum of RAM and VM size required for given file decompression should be constant. Actually, it slightly grows as -mem decreased, due to inefficiencies in the memory management
Please also note that -mem limits total amount of RAM used, that includes 40 mb for I/O buffers, while RAM value in stats shows only memory used for match data |
