DevBlog #91 | Foundry Fridays: Save File Size Optimizations
Greetings Founders!
On today’s FOUNDRY Friday I wanted to share some of the behind-the-scenes work that’s being done to support the announced features for update 3.0 that we’ve covered in Foundry Friday #89. When adding trains for example, we also wanted players to utilize more of the infinite world that Foundry has to offer.
I did an initial analysis to see what issues crop up when playing in a much larger world with trains, and most things thankfully seem to hold up well, with the exception of some visual bugs that we’ll have to mitigate.
But a larger problem is that due to all the world exploration, the save file that I used for testing had already surpassed 100mb. This save file growth needs to be addressed with some significant optimizations. Generally the saves that we receive from players reporting their feedback (thank you!) range from 10mb to 50mb depending on how far they’ve gotten in the game. Of course there’s some more dedicated fans with even larger saves, but those will benefit even more from these save file optimizations. The reason why a smaller save file is beneficial is that it makes it way quicker when a friend joins your world, syncing your saves with Steam Cloud and it also reduces the time it takes to (auto-)save your game.
With some profiling, in the save files that are shared (but especially in my save file) a lot of it is just the terrain data from the chunks that was travelled through to get to their destination, with a singular belt or train (soon™!) connecting those points. But every chunk that is viewed at some point, unmodified or not, is still stored in the save file. This causes roughly half of your save file to consist of terrain data.
To tackle this, we tried a few options, starting with the easier ones:
As an additional security measure to make sure that the game never deletes anything important, I’ve made it so that it’ll always fully store any of the chunks near your factory, and that it also won't guess which chunks might have been modified in any of your existing saves.
Here’s a visualization of a base surrounded by a large number of unmodified chunks that will now no longer have their terrain data saved.
We’ll wait until we get positive feedback from our playtesters there’s any unforeseen issues with it, but so far it has shown great results. There’s quite a lot more stored inside the chunk data that is able to be regenerated, so it's definitely possible to be reduced even further.
- Milan
Follow us on socials:
Stay tuned for more news!
https://store.steampowered.com/app/983870/FOUNDRY/
On today’s FOUNDRY Friday I wanted to share some of the behind-the-scenes work that’s being done to support the announced features for update 3.0 that we’ve covered in Foundry Friday #89. When adding trains for example, we also wanted players to utilize more of the infinite world that Foundry has to offer.
I did an initial analysis to see what issues crop up when playing in a much larger world with trains, and most things thankfully seem to hold up well, with the exception of some visual bugs that we’ll have to mitigate.
But a larger problem is that due to all the world exploration, the save file that I used for testing had already surpassed 100mb. This save file growth needs to be addressed with some significant optimizations. Generally the saves that we receive from players reporting their feedback (thank you!) range from 10mb to 50mb depending on how far they’ve gotten in the game. Of course there’s some more dedicated fans with even larger saves, but those will benefit even more from these save file optimizations. The reason why a smaller save file is beneficial is that it makes it way quicker when a friend joins your world, syncing your saves with Steam Cloud and it also reduces the time it takes to (auto-)save your game.
With some profiling, in the save files that are shared (but especially in my save file) a lot of it is just the terrain data from the chunks that was travelled through to get to their destination, with a singular belt or train (soon™!) connecting those points. But every chunk that is viewed at some point, unmodified or not, is still stored in the save file. This causes roughly half of your save file to consist of terrain data.
To tackle this, we tried a few options, starting with the easier ones:
- Increasing the level of compression used (Foundry internally uses Zstandard to compress the 3D chunk data with great results) but this didn’t really net any significant gains and came with a large speed penalty.
- Compressing multiple chunks in groups. Due to a big overlap and continuity between the chunks in a group, this seemed like a great idea, but it unfortunately only netted a 15% improvement for the scene I tested.
- Don’t store the chunks that aren’t modified, and simply regenerate them as needed. This allows us to save (pun not intended) a lot of data, but it comes with the big downside of having to make sure that
- all* places in Foundry’s code make sure to tell the saving system that a chunk is modified if there’s anything that a player does there.
As an additional security measure to make sure that the game never deletes anything important, I’ve made it so that it’ll always fully store any of the chunks near your factory, and that it also won't guess which chunks might have been modified in any of your existing saves.
Here’s a visualization of a base surrounded by a large number of unmodified chunks that will now no longer have their terrain data saved.
We’ll wait until we get positive feedback from our playtesters there’s any unforeseen issues with it, but so far it has shown great results. There’s quite a lot more stored inside the chunk data that is able to be regenerated, so it's definitely possible to be reduced even further.
- Milan
Follow us on socials:
Stay tuned for more news!
https://store.steampowered.com/app/983870/FOUNDRY/