On-demand part and model textures via unity mipmap streaming

[Phantomical]

This PR uses unity's mipmap streaming feature to stream in part and IVA textures on demand. This means that you get to set a memory budget and unity will take care of automatically streaming mipmaps from disk to vram depending on what parts are currently visible.

Here's how it works behind the scenes:

• With Optimize and rewrite texture loading #369 we now load all dds and cached png textures by creating an asset bundle that points to their files on disk, then having unity load that asset bundle.
• With Load mesh assets using temporary asset bundles #411 we do something similar for mu files, though without the streaming.

With a few edits to these two we can:

• enable mipmap streaming on all textures we load, and,
• calculate the appropriate UV distribution metrics for all meshes loaded via mu files.

As it turns out, that's basically all you need to get mipmap streaming working. We can set QualitySettings.streamingMipmapsAddAllCameras, then toggle whether it is enabled using QualitySettings.streamingMipmapsActive.

I don't have a profile in front of me right now, but the overhead for this looks to be about 0.5ms/frame. On the other hand, for an install with many part mods it looks to be able to save up to ~5GB of VRAM.

A testing build is available here for those who want to try it out
KSPCommunityFixes_1.41.1.zip

[xCuri0]

Some IVAs have blurry textures on displays, besides that can't find any obvious issues.

Still it's way better than before I couldn't install more than one IVA pack without the game crashing from out of memory now can use multiple.

[Phantomical]

Some IVAs have blurry textures on displays, besides that can't find any obvious issues.

Still it's way better than before I couldn't install more than one IVA pack without the game crashing from out of memory now can use multiple.

What IVA packs were those? This is something I'd ideally like to chase down before I merge this.

Though I may merge it anyway and ship it as being disabled by default for now so it can get more testing.

[xCuri0]

@Phantomical Any that have a screen are affected, you can try DE, DE+MAS or ASET.

After using it for around a month the only other bug I found is the game rarely crashes while loading during "ModuleManager: Running post-patch callbacks". During this period the memory usage is 97-99% even when successful so I guess it's caused by Kopernicus not being able to allocate memory.

The log ends like this in those cases

[LOG 23:58:38.347] [ModuleManager] Calling Kopernicus.ConfigLoader.ModuleManagerPostLoad()
[LOG 23:58:38.356] [Kopernicus Configurations] Using: 
[LOG 23:58:38.356] HomeWorldName: Kerbin
[LOG 23:58:38.356] EnforceShaders: False
[LOG 23:58:38.356] WarnShaders: False
[LOG 23:58:38.356] EnforcedShaderLevel: 2
[LOG 23:58:38.356] UseKopernicusAsteroidSystem: True
[LOG 23:58:38.356] SolarRefreshRate: 1
[LOG 23:58:38.356] EnableKopernicusShadowManager: True
[LOG 23:58:38.356] ShadowRangeCap: 50000
[LOG 23:58:38.356] DisableMainMenuMunScene: True
[LOG 23:58:38.356] KSCLightsAlwaysOn: False
[LOG 23:58:38.356] UseOriginalKS2: False
[LOG 23:58:38.356] HandleHomeworldAtmosphericUnitDisplay: True
[LOG 23:58:38.356] UseIncorrectScatterDensityLogic: False
[LOG 23:58:38.356] DisableFarAwayColliders: True
[LOG 23:58:38.356] EnableAtmosphericExtinction: False
[LOG 23:58:38.356] UseStockMohoTemplate: True
[LOG 23:58:38.356] UseOnDemandLoader: True
[LOG 23:58:38.356] UseRealWorldDensity: False
[LOG 23:58:38.356] ApplyDensityMultToMinorObjects: 1
[LOG 23:58:38.356] RecomputeSOIAndHillSpheres: False
[LOG 23:58:38.356] PrincipiaFriendlySOIComputation: False
[LOG 23:58:38.356] LimitRWDensityToStockBodies: True
[LOG 23:58:38.356] UseOlderRWScalingLogic: False
[LOG 23:58:38.356] RescaleFactor: 1
[LOG 23:58:38.356] RealWorldSizeFactor: 10.625
[LOG 23:58:38.356] SettingsWindowXcoord: 0
[LOG 23:58:38.357] SettingsWindowYcoord: 0

[Phantomical]

After using it for around a month the only other bug I found is the game rarely crashes while loading during "ModuleManager: Running post-patch callbacks". During this period the memory usage is 97-99% even when successful so I guess it's caused by Kopernicus not being able to allocate memory.

That's not too surprising, unfortunately. The way this works is that textures all get loaded up front and once we start loading models then we tell unity it is allowed to stream mipmaps for textures used by those models. This does mean that memory usage will max out right about when MM finishes patching. Not sure there's too much I can do about that. We don't have any way of knowing which textures are safe to make on-demand until they actually get used by a model.

The usual advice of bumping up your swap space will probably help you here as well. Once you get beyond that and into part compilation then unity will be able to unload textures as you go so you should be safe after that.