Disk Space Compression Using Compact LZX On Windows 10


#1

So I’m getting low on SSD space and probably just need to buy some more (although buying things got me in this predicament in the first place :slight_smile:), but I did see the new Windows 10 LZW compact option mentioned on reddit the other day and thought I could share here.

Essentially what this is is that Windows has an in-built way to compress / uncompress files on the fly. You right-mouse-button on a folder or file in Windows Explorer and choose the ‘General’ tab, hit the ‘Advanced…’ button and then tick the ‘Compress contents to save disk space’ option. This then works completely transparently and for things not already compressed (some image and video formats etc) it works really well. So why not do it all the time? Well, the downsides are:

  • As it uncompresses on the fly when you access the file it will use (a) a bit of CPU (say, about 5% across all cores while accessing the file) and (b) a bit of memory. So it’s a trade off, but on most modern games it would be hard to even notice, especially as a lot of them preload into memory where they can (DCS etc) and don’t fully utilize all cores…

  • Old versions of windows on your network might not be able to read the files. The default compression is fine, but you can tweak the algorithm for some dramatic disk savings, with the cost being old-windows compatibility.

The most intense compression is to be found with the new Windows 10 only LZX compression. To use it you have to manually compress the folders/files yourself, because the option isn’t a default one that compresses things in folders on the fly. The uncompression is automatic though, as in transparent to you using the files.

To compress my entire ‘E:\Games’ folder hierarchy using LZX I would do this:

compact /c /s /a /i /exe:lzx "E:\Games\*"

(the /c stands for ‘compress’ as you can use /u to ‘uncompress’ manually if you want, the /s means ‘here and sub-directories downwards’, the /a means do hidden files as well and the /i means don’t stop on a single file error)

So the special ‘/exe:lzx’ is specifying the stronger compression method. Here’s a fuller overview of the options for those that want to dig in more:

Example time:

Here’s my X-Plane Ortho for California folder. Chunky at 108GB, note the ‘size of disk’ second figure.

image

So I run this and then go away for about an hour (this stuff is ideally done when you aren’t sat looking at the computer and/or need a toasty room) or so that it applies to the 23,783 files in there(!):

> compact /c /s /a /i /exe:lzx "F:\Custom Scenery\zOrtho4XP_California\*"

6592_2928.png           31781 :      8192 = 3.9 to 1 [OK]
6592_2944.png           25484 :      4096 = 6.2 to 1 [OK]
6608_2784.png           16886 :      4096 = 4.1 to 1 [OK]
(snip, lots of files)

23793 files within 7 directories were compressed.
116,619,267,746 total bytes of data are stored in 88,231,050,163 bytes.
The compression ratio is 1.3 to 1.

The new size is 82.1 GB, a saving of about 26 GB:

image

X-Plane sort of reads these files at a fairly slow rate as you fly along, so a performance impact on a SSD is not really detectable.

So to recap, the Win 10 Compact LZX really compresses things really well, takes a small amount of CPU/memory to uncompress on the fly transparently, and needs to be applied via the manual command line in Windows 10 only. If you copy the files elsewhere they won’t be compressed still, so to basically ‘undo’ any of this you can just copy to another location.

If there is any interest I’ll try with a game that has an in-built benchmark, like Dirt Rally or something, as that will provide a decent ‘game’ example of this in use.


#2

Would it be possible to do this to DCS terrain folders and would that cause it to fail the integrity checks ya’ think?


#3

To the game or app it’s transparent that it’s happening, so it can’t fail IC.

I’m just doing a quick benchmark on Project Cars 2 (something that is pretty big but never survives on my SSD for long as I change my mind about keeping around). I’ll report back in a minute, just to see what sort of impacts it has in a real-world example of a complete game folder compressed LZX, i.e. load times and FPS hits (or not).

EDIT: Hmm, project cars 2 is actually custom compressed already with their own storage, so not a good app to try. I’ll pick another one.


#4

According to this they got DCS from 46 GB instead of 86 GB using LZX. The claim is no perf impact at all, but personally I’d like to try it and see I guess.


#5

Sounds like useful tool as I have also ssd almost full now with Orthos :slight_smile:

Will wait for your test results.