Asset Variant System with major performance and memory improvements
Hello again everyone!
Welcome back to our Friday update. We have an exciting milestone to talk about today! About a week ago, we’ve unveiled the first working implementation of our Asset Variant system. We’ll be going over what this means for Neos and its users.
We’re going to be live streaming on Twitch about when this post is 15 minutes old. If you would like to know more, please join us! We’ll be taking suggestions and answering questions about this optimization update as well as showcasing how it works! Also come try out our new interactive Twitch commands !bobblehead and !coffee.
This major feature helps to optimize the Neos experience by taking original assets and computing “variants” of them. These variants are versions of said assets that are much better optimized for your device, connection, and settings. Currently, this version of the system focuses on generating optimized textures.
This will lead to significantly less RAM and VRAM consumption, which will help Neos run much more smoothly. Complex worlds with a lot of large textures not only run better, but they load in a fraction of time they used to before too!
The textures are compressed using native Block Compression algorithms. Opaque textures are compressed with BC1, which allows the texture to be loaded using 8 times less memory, and alpha textures are compressed with BC3 which uses 4 times less memory. Additionally the textures are crunch compressed for smaller download and storage size.
Texture heavy worlds such as Club Fur used to take a long time to load, in some cases upwards of 5+ minutes. After the Asset Variant system implementation, we’ve seen load times as low as a few seconds! This is thanks to the reduction in computations needed to load assets and a much lower volume of data uploaded to the GPU.
These textures use a lot less memory bandwidth when rendering as well. For example Viridian Island would cut a lot of users FPS to an uncomfortably low level. Now, many users are reporting much higher frame rates while visiting!
Avatars with multiple 4K textures do not lag anymore when they load. Photogrammetry scans also tremendously benefit from this system as they often use dozens of high resolution textures.
The latency of the loading process has been improved as well by pipelining the mipmap levels. Textures are downloaded and loaded into the GPU piece by piece, with the lower resolution version shown as soon as possible, which will keep getting sharper as more of the texture’s data is downloaded. That way you don’t have to wait as long for everything to fully load.
This system works for locally generated assets as well (such as photos taken in-game or freshly imported 3D models and images), uploading parts to the GPU and other users as soon as they’re ready, while the rest is still being computed. The peer-to-peer asset transfer protocol has been extended to support this, improving the loading times and latency, especially for users with slower connections.
We have also implemented lots of tweaks, bugfixes and improvements since the first release of the asset variant system based on your feedback. The system should be much more reliable and stable now and the quality of the compressed textures should be improved.
Opaque textures use ETC2 RGB8, which compresses textures down to 4 bits per pixel and alpha textures use ETC2 RGBA8 which compresses down to 8 bits per pixel. Same as on desktop, the textures are crunch compressed and load from the smallest mipmap, getting progressively sharper.
The mobile build now bypasses Unity’s texture pipeline, using native OpenGL to upload textures to the GPU directly. This allows Neos on mobile to spread the upload over several frames rather than all at once, which stops Neos from hanging when loading textures and reducing overall CPU and memory usage.
This was the major part of the porting process, which took a lot of time, effort, and native code. The desktop version was already using DirectX 11 to upload textures to the GPU directly, but it has received some improvements as well, fixing GPU driver crashes reported by some users.
This is a major step towards Neos being available publicly on mobile and Quest. Worlds that previously crashed Quest now load and run reasonably well, even when they’re originally made for desktop. For example Viridian Island would just crash the Quest build before, but now it loads and runs at upwards of 40 FPS.
One major benefit that the asset variant system brings over traditional in-engine compression is future compatibility. All of these optimizations are fully automatic for all submitted worlds and items, regardless of when they were first uploaded. As a creator you don’t have to do a thing!
We take compatibility very seriously and design our systems so they do not permanently break any user submitted content and don’t require reupload anytime we publish a major update like this. If you have already submitted content before, the textures have been automatically optimized for both desktop and mobile, with more to come.
The asset variant system is capable of gradually reprocessing all assets in the cloud whenever we implement new improvements, better compression and optimization methods or fix reported issues and ensure that even content uploaded years ago can receive the benefits of the latest technology.
Besides the release of the Asset Variant system, we’ve pushed out a few upgrades and bugfixes. Frooxius has fixed a startup crash due to SteamVR Input bugs, such as having more input devices plugged such as gamepads and joysticks, running Oculus through SteamVR, or using less common headsets like Pimax.
We’ve upgraded to Unity 2019.3 beta for its engine improvements and bugfixes and will be taking advantage of new high-performance mesh API’s soon. Both SteamVR and Oculus SDK’s have been updated to their latest versions as well.
There were dozens of other bugfixes, tweaks and additions as well, check out the #neos-updates channel on our official Discord for full changelog!
One user made a wall-detecting Roomba (featured in our Twitch streaming house!) while another user made a follower robot NPC! Feeling creative? Feel free to join! They’re hosted every week on Sunday.
Medra is currently making a trailer for the Creator Jams, and needs members of the community to be a part of it. He’s in need of 10+ actors, so give him a message, he more than likely needs a few more to make a full scene.
As he’s stated in our #meetups channel, this will be a fun and playful type of environment, but some shots may take 4 or more takes. He’s done a lot to bring wonderful content to Neos, so if you’re feeling inclined to help him out, please do!
The Sydney Human Factor Research (Known as SHFR on our Discord and in Neos), who have been our long time academic users, are preparing for a big presentation at the Sydney Festival of Inclusion at the Centre for Disability Studies.
If you’ve been facing challenges like agoraphobia, anxiety, immobility, deafness or others and have found Neos or VR in general to help you in some way, you can get in touch with prof. Hamish MacDougall on our Discord (@SHFR) and share your story for the event. All stories will be anonymized.
You can find more about the event here: https://cds.org.au/event/festival-of-inclusion/
We are beginning work on phase one of the major UI Overhaul. There are two major phases to this. The first phase is a technical backend phase which will be replacing the old hybrid Unity/Neos frameworks with our completely custom high-performance ones and give us the full control for the next phase.
Phase two is the visual and UX overhaul phase, where we will see a major improvement to user friendliness, customization possibilities, and way more!
First on the list is our own font rendering system which will be a replacement to Unity’s. Loading large amounts of text will not lag anymore as everything will be generated asynchronously on background threads.
This upgrade will also allow you to import your own fonts in-game, improve the text editing ability (for example by being able to click anywhere in the text to place the cursor) and allow 3D mesh fonts in the future too. It will support dynamic symbol generation for languages such as Japanese, Chinese and others that have lots of symbols.
The asset variant system that we just implemented will actually help us with this upgrade as well, allowing to store pre-generated font atlases as a variants of given font access. If we add new rendering method for new devices (e.g. using on GPU vector rendering) in the future, it will mean simply adding another variant and re-generating it for all already uploaded fonts.
We’ll keep you updated as we progress on this. Check out our official Discord for daily updates and peek behind the scenes and don’t forget to join our livestream as well, where you can chat with us about these updates and ask us questions.
See you next time!
Welcome back to our Friday update. We have an exciting milestone to talk about today! About a week ago, we’ve unveiled the first working implementation of our Asset Variant system. We’ll be going over what this means for Neos and its users.
We’re going to be live streaming on Twitch about when this post is 15 minutes old. If you would like to know more, please join us! We’ll be taking suggestions and answering questions about this optimization update as well as showcasing how it works! Also come try out our new interactive Twitch commands !bobblehead and !coffee.
Asset Variant System
This major feature helps to optimize the Neos experience by taking original assets and computing “variants” of them. These variants are versions of said assets that are much better optimized for your device, connection, and settings. Currently, this version of the system focuses on generating optimized textures.
This will lead to significantly less RAM and VRAM consumption, which will help Neos run much more smoothly. Complex worlds with a lot of large textures not only run better, but they load in a fraction of time they used to before too!
The textures are compressed using native Block Compression algorithms. Opaque textures are compressed with BC1, which allows the texture to be loaded using 8 times less memory, and alpha textures are compressed with BC3 which uses 4 times less memory. Additionally the textures are crunch compressed for smaller download and storage size.
Texture heavy worlds such as Club Fur used to take a long time to load, in some cases upwards of 5+ minutes. After the Asset Variant system implementation, we’ve seen load times as low as a few seconds! This is thanks to the reduction in computations needed to load assets and a much lower volume of data uploaded to the GPU.
These textures use a lot less memory bandwidth when rendering as well. For example Viridian Island would cut a lot of users FPS to an uncomfortably low level. Now, many users are reporting much higher frame rates while visiting!
Avatars with multiple 4K textures do not lag anymore when they load. Photogrammetry scans also tremendously benefit from this system as they often use dozens of high resolution textures.
The latency of the loading process has been improved as well by pipelining the mipmap levels. Textures are downloaded and loaded into the GPU piece by piece, with the lower resolution version shown as soon as possible, which will keep getting sharper as more of the texture’s data is downloaded. That way you don’t have to wait as long for everything to fully load.
This system works for locally generated assets as well (such as photos taken in-game or freshly imported 3D models and images), uploading parts to the GPU and other users as soon as they’re ready, while the rest is still being computed. The peer-to-peer asset transfer protocol has been extended to support this, improving the loading times and latency, especially for users with slower connections.
We have also implemented lots of tweaks, bugfixes and improvements since the first release of the asset variant system based on your feedback. The system should be much more reliable and stable now and the quality of the compressed textures should be improved.
Android & Quest port of the asset variant system
The Asset Variant system has now been ported to our Android and Quest builds. This is a crucial step for mobile devices to be able to use Neos, as these devices have both low memory and low memory bandwidth.Opaque textures use ETC2 RGB8, which compresses textures down to 4 bits per pixel and alpha textures use ETC2 RGBA8 which compresses down to 8 bits per pixel. Same as on desktop, the textures are crunch compressed and load from the smallest mipmap, getting progressively sharper.
The mobile build now bypasses Unity’s texture pipeline, using native OpenGL to upload textures to the GPU directly. This allows Neos on mobile to spread the upload over several frames rather than all at once, which stops Neos from hanging when loading textures and reducing overall CPU and memory usage.
This was the major part of the porting process, which took a lot of time, effort, and native code. The desktop version was already using DirectX 11 to upload textures to the GPU directly, but it has received some improvements as well, fixing GPU driver crashes reported by some users.
This is a major step towards Neos being available publicly on mobile and Quest. Worlds that previously crashed Quest now load and run reasonably well, even when they’re originally made for desktop. For example Viridian Island would just crash the Quest build before, but now it loads and runs at upwards of 40 FPS.
Your content is future proofed
One major benefit that the asset variant system brings over traditional in-engine compression is future compatibility. All of these optimizations are fully automatic for all submitted worlds and items, regardless of when they were first uploaded. As a creator you don’t have to do a thing!
We take compatibility very seriously and design our systems so they do not permanently break any user submitted content and don’t require reupload anytime we publish a major update like this. If you have already submitted content before, the textures have been automatically optimized for both desktop and mobile, with more to come.
The asset variant system is capable of gradually reprocessing all assets in the cloud whenever we implement new improvements, better compression and optimization methods or fix reported issues and ensure that even content uploaded years ago can receive the benefits of the latest technology.
Other Updates
Besides the release of the Asset Variant system, we’ve pushed out a few upgrades and bugfixes. Frooxius has fixed a startup crash due to SteamVR Input bugs, such as having more input devices plugged such as gamepads and joysticks, running Oculus through SteamVR, or using less common headsets like Pimax.
We’ve upgraded to Unity 2019.3 beta for its engine improvements and bugfixes and will be taking advantage of new high-performance mesh API’s soon. Both SteamVR and Oculus SDK’s have been updated to their latest versions as well.
There were dozens of other bugfixes, tweaks and additions as well, check out the #neos-updates channel on our official Discord for full changelog!
Community Highlights
Medra has been hosting weekly Creator Jams, where the community comes together and, based on the weekly theme, creates content for the rest of the community. They’ve hit the 6 month mark after 24 jams, the most recent theme centering around NPC’s and Robots.One user made a wall-detecting Roomba (featured in our Twitch streaming house!) while another user made a follower robot NPC! Feeling creative? Feel free to join! They’re hosted every week on Sunday.
Medra is currently making a trailer for the Creator Jams, and needs members of the community to be a part of it. He’s in need of 10+ actors, so give him a message, he more than likely needs a few more to make a full scene.
As he’s stated in our #meetups channel, this will be a fun and playful type of environment, but some shots may take 4 or more takes. He’s done a lot to bring wonderful content to Neos, so if you’re feeling inclined to help him out, please do!
The Sydney Human Factor Research (Known as SHFR on our Discord and in Neos), who have been our long time academic users, are preparing for a big presentation at the Sydney Festival of Inclusion at the Centre for Disability Studies.
If you’ve been facing challenges like agoraphobia, anxiety, immobility, deafness or others and have found Neos or VR in general to help you in some way, you can get in touch with prof. Hamish MacDougall on our Discord (@SHFR) and share your story for the event. All stories will be anonymized.
You can find more about the event here: https://cds.org.au/event/festival-of-inclusion/
Coming next: The UI Overhaul
We are beginning work on phase one of the major UI Overhaul. There are two major phases to this. The first phase is a technical backend phase which will be replacing the old hybrid Unity/Neos frameworks with our completely custom high-performance ones and give us the full control for the next phase.
Phase two is the visual and UX overhaul phase, where we will see a major improvement to user friendliness, customization possibilities, and way more!
First on the list is our own font rendering system which will be a replacement to Unity’s. Loading large amounts of text will not lag anymore as everything will be generated asynchronously on background threads.
This upgrade will also allow you to import your own fonts in-game, improve the text editing ability (for example by being able to click anywhere in the text to place the cursor) and allow 3D mesh fonts in the future too. It will support dynamic symbol generation for languages such as Japanese, Chinese and others that have lots of symbols.
The asset variant system that we just implemented will actually help us with this upgrade as well, allowing to store pre-generated font atlases as a variants of given font access. If we add new rendering method for new devices (e.g. using on GPU vector rendering) in the future, it will mean simply adding another variant and re-generating it for all already uploaded fonts.
We’ll keep you updated as we progress on this. Check out our official Discord for daily updates and peek behind the scenes and don’t forget to join our livestream as well, where you can chat with us about these updates and ask us questions.
See you next time!