Baking rigged meshes to static, custom world thumbnails, security improvements
Hello everyone and welcome to another of our weekly updates!
While we are internally finalizing one of the educational experiences for public release, there's a number of general features and improvements that have been sprouted from this work, that should benefit anyone using Neos.
You can now bake rigged (posed) meshes into a static one! They keep their current pose and blendshapes when baked, essentially becoming a mesh snapshot. There's already an easy to use tool made by Orange that lets you easily strike a pose and generate a figurine of your avatar. It behaves as any other mesh in Neos, so it can not only be used for fun, but also to optimize your worlds or export your avatar in a pose for 3D printing or rendering!
Related to this, SkinnedMeshRenderer has a new Explicit bounds mode. This can be used to fix cases where animated meshes would randomly get culled when they shouldn't, without the performance penalty of dynamically computing its bounds every frame.
We also added ability to customize the world thumbnails! This makes it easier to ensure a good thumbnail for any worlds you publish to help entice players to visit them. Not only it helps our own official worlds, but any others as well, including the incoming MMC submissions as it's nearing its end.
Based on community reports, we also improved security of the initial session connection when joining users get authenticated and fixed issue where importing malicious .blend files could run Python scripts in the embedded file. As a reminder, you can report any security issues at moderation.neos.com according to our Security Reporting Policy.
[h2]Explicit Skinned Mesh Renderer bounds[/h2]
During our work on the educational demos, we have noticed an issue with a number of animated rigged meshes - they would de-render at random times and simply not be visible, even though they were there.
This was due to them using a static bounding box of the mesh in its default (binding) pose for culling, which didn’t correspond to where the mesh actually was positioned by its bones as part of the animation.
One approach for this is to simply recompute the bounding box dynamically every frame. However this has a noticeable performance cost, which adds up with more meshes in the scene.
To deal with this better, we implemented the capability to perform skinned mesh transformations in our own data structures, which enables a whole bunch of applications and features.
Notably it now lets you pre-compute an explicit bounding box on the current pose of the mesh, to ensure that it’s always rendered properly without an extra per-frame performance cost. You can even expand the current bounding box to incorporate a new pose with a press of the button.
The explicit bounds can be manually edited with hand too if you know how and where the mesh should be rendered, though it is not recommended to simply make it cover a huge area unless absolutely necessary, otherwise the mesh will need to be calculated even when you cannot see it.
[h2]Baking Skinned Meshes into a static pose[/h2]
Perhaps a much more fun feature to sprout from this new internal addition is the ability to bake skinned meshes into a static mesh in its current pose and with its current blendshapes. It’s something that’s been requested by a number of community members as well.
For example this lets you take your avatar and simply bake yourself into a static mesh in whatever pose and expression you strike at the moment. The resulting mesh is like any other. You can even give it a collider afterwards!
Our Japanese community has already built a tool on top of this feature - simply step on a pedestal, press a button and in a few seconds a small figurine of your avatar pops out! Check out the video below if you’d like to see it in action.
https://twitter.com/mikan3134/status/1438804932858023938
Another practical use case comes from another of the projects that our team has been involved in - the Metamovie. There are a number of corpses lying around the world, all of them skinned meshes, fully posed directly in Neos.
However previously we had to export those out and bake them outside of Neos and reimport, which is a tedious process, but improved the world’s performance. Now with this tool baking them is a one-click operation directly in Neos.
To actually bake a mesh, you can click a button on the SkinnedMeshRenderer. Alternatively, to bake multiple meshes (both static and skinned) you can use the Glue tool in the “Bake Meshes” mode or the LogiX node for baking.
[h2]Customizable World & Session Thumbnails[/h2]
Yet another long-requested feature has been prodded into implementation with our educational project - being able to customize the thumbnail of your saved world and locations of session thumbnails.
Thumbnails can be important when publishing a world, as they can be the deciding factor whether someone visits your world or not, but previously they would always be captured at whatever point you were in the world when hitting the Save button.
There are now new sets of components that let you override this. If you’d still like to have the most recent thumbnail captured whenever you save, you can use the WorldCaptureThumbnailSource component.
This will capture a fresh thumbnail from wherever this component is located. You can even add an optional overlay texture on top of it. Alternatively you can use WorldTextureThumbnailSource to simply provide a static 360 texture for the thumbnail if you made a polished one.
We also added SessionCaptureThumbnailSource. You can scatter these components around popular locations in your world, so whenever a thumbnail is captured for the session, it will be taken from one of those points. The system will pick a random one every time, but prioritize locations with more people nearby than others.
[h2]Improved session connection handshake and other security improvements[/h2]
Based on security reports through our moderation ticket system, we implemented more security improvements. Notably we reworked a portion of the protocol that establishes the initial connection to a session.
This process performs a cryptographic authentication of any user joining the session, to ensure that they are who they say they are. However, part of this process was susceptible to a man-in-the-middle attack, which while difficult to execute, would allow someone to impersonate another user in a different session.
This mechanism has now been expanded to significantly mitigate the possibility of such attack as well as number of others, providing greater security guarantees on user identities.
Additionally we also improved security of importing .blend files. Blender files can contain Python scripts that run automatically when they are opened. When the file is imported directly to Neos, those scripts no longer run automatically, preventing attack where importing a malicious .blend file could run arbitrary code on your computer.
If you find any security issues, please report them responsibly at moderation.neos.com! Reports are eligible for CDFT rewards, depending on combination of their severity, quality of the report and other factors. You can learn more about our Security reporting policy on our GitHub here.
[h2]Bug Fixes, small optimizations and tweaks[/h2]
Among those larger additions and improvements, we released a number of other bug fixes, tweaks and additions. For example during the work on the skinned mesh renderer, we noticed unnecessary data being stored on each bone binding and removed it, reducing some memory usage and GC workload.
We improved the avatar IK feet behavior, to avoid the legs going up when near another player, but still keep reacting to other parts of their body - for example their hands if they’re pushing another player’s feet up.
We fixed a number of crashes too and DynamicBonePlayerColliders components were ignored after recent rework of the system. We updated the Assimp importer library to the latest as well, bringing about 150 commits of bug fixes and additions that should improve compatibility with various 3D model formats.
If you’d like to know more, you can check out our release notes on Steam or Discord! We hope that everyone had a great week and that the next one will be awesome too and as always - huge thanks to you for helping to keep this project going. Neos is a living, breathing metaverse and we're amazed at the creativity we see every day!
While we are internally finalizing one of the educational experiences for public release, there's a number of general features and improvements that have been sprouted from this work, that should benefit anyone using Neos.
You can now bake rigged (posed) meshes into a static one! They keep their current pose and blendshapes when baked, essentially becoming a mesh snapshot. There's already an easy to use tool made by Orange that lets you easily strike a pose and generate a figurine of your avatar. It behaves as any other mesh in Neos, so it can not only be used for fun, but also to optimize your worlds or export your avatar in a pose for 3D printing or rendering!
Related to this, SkinnedMeshRenderer has a new Explicit bounds mode. This can be used to fix cases where animated meshes would randomly get culled when they shouldn't, without the performance penalty of dynamically computing its bounds every frame.
We also added ability to customize the world thumbnails! This makes it easier to ensure a good thumbnail for any worlds you publish to help entice players to visit them. Not only it helps our own official worlds, but any others as well, including the incoming MMC submissions as it's nearing its end.
Based on community reports, we also improved security of the initial session connection when joining users get authenticated and fixed issue where importing malicious .blend files could run Python scripts in the embedded file. As a reminder, you can report any security issues at moderation.neos.com according to our Security Reporting Policy.
[h2]Explicit Skinned Mesh Renderer bounds[/h2]
During our work on the educational demos, we have noticed an issue with a number of animated rigged meshes - they would de-render at random times and simply not be visible, even though they were there.
This was due to them using a static bounding box of the mesh in its default (binding) pose for culling, which didn’t correspond to where the mesh actually was positioned by its bones as part of the animation.
One approach for this is to simply recompute the bounding box dynamically every frame. However this has a noticeable performance cost, which adds up with more meshes in the scene.
To deal with this better, we implemented the capability to perform skinned mesh transformations in our own data structures, which enables a whole bunch of applications and features.
Notably it now lets you pre-compute an explicit bounding box on the current pose of the mesh, to ensure that it’s always rendered properly without an extra per-frame performance cost. You can even expand the current bounding box to incorporate a new pose with a press of the button.
The explicit bounds can be manually edited with hand too if you know how and where the mesh should be rendered, though it is not recommended to simply make it cover a huge area unless absolutely necessary, otherwise the mesh will need to be calculated even when you cannot see it.
[h2]Baking Skinned Meshes into a static pose[/h2]
Perhaps a much more fun feature to sprout from this new internal addition is the ability to bake skinned meshes into a static mesh in its current pose and with its current blendshapes. It’s something that’s been requested by a number of community members as well.
For example this lets you take your avatar and simply bake yourself into a static mesh in whatever pose and expression you strike at the moment. The resulting mesh is like any other. You can even give it a collider afterwards!
Our Japanese community has already built a tool on top of this feature - simply step on a pedestal, press a button and in a few seconds a small figurine of your avatar pops out! Check out the video below if you’d like to see it in action.
https://twitter.com/mikan3134/status/1438804932858023938
Another practical use case comes from another of the projects that our team has been involved in - the Metamovie. There are a number of corpses lying around the world, all of them skinned meshes, fully posed directly in Neos.
However previously we had to export those out and bake them outside of Neos and reimport, which is a tedious process, but improved the world’s performance. Now with this tool baking them is a one-click operation directly in Neos.
To actually bake a mesh, you can click a button on the SkinnedMeshRenderer. Alternatively, to bake multiple meshes (both static and skinned) you can use the Glue tool in the “Bake Meshes” mode or the LogiX node for baking.
[h2]Customizable World & Session Thumbnails[/h2]
Yet another long-requested feature has been prodded into implementation with our educational project - being able to customize the thumbnail of your saved world and locations of session thumbnails.
Thumbnails can be important when publishing a world, as they can be the deciding factor whether someone visits your world or not, but previously they would always be captured at whatever point you were in the world when hitting the Save button.
There are now new sets of components that let you override this. If you’d still like to have the most recent thumbnail captured whenever you save, you can use the WorldCaptureThumbnailSource component.
This will capture a fresh thumbnail from wherever this component is located. You can even add an optional overlay texture on top of it. Alternatively you can use WorldTextureThumbnailSource to simply provide a static 360 texture for the thumbnail if you made a polished one.
We also added SessionCaptureThumbnailSource. You can scatter these components around popular locations in your world, so whenever a thumbnail is captured for the session, it will be taken from one of those points. The system will pick a random one every time, but prioritize locations with more people nearby than others.
[h2]Improved session connection handshake and other security improvements[/h2]
Based on security reports through our moderation ticket system, we implemented more security improvements. Notably we reworked a portion of the protocol that establishes the initial connection to a session.
This process performs a cryptographic authentication of any user joining the session, to ensure that they are who they say they are. However, part of this process was susceptible to a man-in-the-middle attack, which while difficult to execute, would allow someone to impersonate another user in a different session.
This mechanism has now been expanded to significantly mitigate the possibility of such attack as well as number of others, providing greater security guarantees on user identities.
Additionally we also improved security of importing .blend files. Blender files can contain Python scripts that run automatically when they are opened. When the file is imported directly to Neos, those scripts no longer run automatically, preventing attack where importing a malicious .blend file could run arbitrary code on your computer.
If you find any security issues, please report them responsibly at moderation.neos.com! Reports are eligible for CDFT rewards, depending on combination of their severity, quality of the report and other factors. You can learn more about our Security reporting policy on our GitHub here.
[h2]Bug Fixes, small optimizations and tweaks[/h2]
Among those larger additions and improvements, we released a number of other bug fixes, tweaks and additions. For example during the work on the skinned mesh renderer, we noticed unnecessary data being stored on each bone binding and removed it, reducing some memory usage and GC workload.
We improved the avatar IK feet behavior, to avoid the legs going up when near another player, but still keep reacting to other parts of their body - for example their hands if they’re pushing another player’s feet up.
We fixed a number of crashes too and DynamicBonePlayerColliders components were ignored after recent rework of the system. We updated the Assimp importer library to the latest as well, bringing about 150 commits of bug fixes and additions that should improve compatibility with various 3D model formats.
If you’d like to know more, you can check out our release notes on Steam or Discord! We hope that everyone had a great week and that the next one will be awesome too and as always - huge thanks to you for helping to keep this project going. Neos is a living, breathing metaverse and we're amazed at the creativity we see every day!