Neos VR News

Some more optimizations and related functionality! Skinned Meshes now have new modes for updating bounds, which should help fix the culling issues with some avatars without taking too much performance. If you're experiencing issues with parts of your avatar disappearing at certain angles, do following:

- If your avatar is in multiple pieces, try baking the meshes with the Glue tooltip first if possible. This is optimal for performance and allow the avatar to share a single big bounding box.
- If you can't bake it (e.g. you swap pieces in realtime), switch the affected meshes to "FastDisjointRootApproximate" method. This has an extra performance impact, but is much lower than the SlowRealtimeAccurate method (the previous default)

I've also made changes to Viseme analysis, which is surpringly CPU costly (~1 ms per user per frame) and the internal OVRLipSync library locking was source of major background thread starvation (blocking other computations) with multiple people in the world. Visemes are now computed on each user and streamed to others. There is no explicit mechanism to sync it with voice, but both voice and viseme data are sent as soon as possible and from internal testing this achieves really good latency.

There are some smaller optimizations, like MultiValueStream which significantly reduces overhead of multiple ValueStream objects (network impact is unchanged right now) and more things, I'm going after the most impactful low hanging fruits now and working way to larger reworks and refactoring. More to come!

Also Neos is now available in Dutch thanks to @jeana and @Beaned! I've got the dash and other things to work on Android again, Android/Quest builds incoming!

[h2]New Features:[/h2]
- SkinnedMeshRenderer now has 4 methods to compute bounds and handle culling (big thanks to @DeliriousJax for providing avatar sample for experimentation!):
-- Static (default) - fastest, based on the mesh alone, no realtime computations
-- FastDisjointRootApproximate - merges all bones into disjoint groups (any overlapping bones are merged into a single one) to reduce overall number of bones and uses those to approximate bounds in realtime. Fastest, recommended if part of mesh gets culled
-- MediumPerBoneApproximate - computes mesh bounds from bounds of every single bone. More accurate, but also much slower
-- SlowRealtimeAccurate - uses actual transformed geometry, requiring the skinned mesh to be processed all the time. Very heavy, but will respect things like blendshapes in addition to bones
-- NOTE: When the renderer is in user's hierarchy and set to Static, it'll use FastDisjointRootApproximate method internally, so existing avatars work without changes and don't get culled
- Added "Strip Empty Bones" on SkinnedMeshRenderer, which will remove all empty bones from the mesh
-- You can use this to optimize your meshes/rigs, reducing their CPU, memory and some GPU overhead
-- This only removes them from the mesh, the actual objects representing the bones are still kept in the hierarchy, so it should be generally safe to run this optimization
- Added "Visualize bone bounding boxes" to SkinnedMeshRenderer, which will visualize boudning boxes for each bone affecting the mesh (this includes all vertices that given bone affects)
-- Also added "Remove bone bounding box visuals" to quickly clear them
-- The visuals are non-persitent so won't save
- Added "Visualize approximate merged bone sphere bounds" which will visualize merged bone bounding spheres, representing a fast approximation of disjoint areas of influence of the rig
- Static Mesh "Get Bone List" now prints how many vertices are affected by each bone and shows bones with no bound vertices as red

- Added Dutch locale by @jeana and @Beaned
- Added pluralizer for Polish (requested by @art0007i)

[h2]Optimizations:[/h2]
- Added MultiValueStream, which handles multiple values in a single stream rather than using stream-per-value, significantly reducing associated CPU and memory overhead
-- Switched finger pose estimation to use a single stream for all fingers, rather than individual ValueStream for each bone
- VisemeAnalyzer now streams results of analysis to other users for voice input, rather than having each user compute it on their end independently
-- This should greatly improve CPU load in heavily populated worlds, as analysis takes roughly 1 ms per user per frame (it is done on background thread though, so this will free up lots of background processing resources)
-- As a bonus, Linux users will now see visemes of all other non-Linux users. Windows users will see visemes of all Linux users, because the system will compute them locally
-- Other audio sources (e.g. using it for visualizing music) are still computed locally
-- Currently there's no explicit synchronization mechanism with voice, but the streams are sent out as soon as possible, so they should generally match up close enough (possibly even improving visual latency of the analysis). Let me know if there are issues though
- Fixed VisemeAnalysis causing severe background thread starvation with multiple scheduled analysis due to internal locking mechanism of the OVRlipsync library (this should help some cases of things getting stuck and taking a while to load)
- Reduced CPU and memory usage for LogiX display nodes when they're packed
- Fixed dash projection geometry still being rendered even if it's closed
- Reduced duplicate work with explicit stream encoding (also fixing bug where forced explicit update points wouldn't encode correctly)
- Small low level optimizations when passing in LocaleString values

[h2]Tweaks:[/h2]
- Implemented system to embed the UI locale files in the Android builds for Quest build
- SkinnedMeshRenderer inspector actions are now localizable
- BipedRig inspector actions are now localizable
- Upgraded to Unity 2019.4.11f1 (form 2019.4.10f1)
- Merged French locale new strings by @FreakyWaves Music
- Merged Finnish locale updates by @Nammi and @AdmiralSoap
- Merged "Friends -> Contacts" replacements in English locale by @Alex from Alaska
- Merged "Silence" string localization to Japanese, Esperanto and Chinese by @Melnus
- Merged German locale updates by @Elektrospy

[h2]Bugfixes:[/h2]
- Fixed UserPermissions being misconfigured in some cases, allowing Guest roles to Silence users (reported by @Zane)
- Fixed DynamicReferenceVariableReset having invalid type check, not allowing any proper types (reported by @Earthmark)
- Fixed "Strip Empty Blendshapes" no SkinnedMeshRenderer not releasing read lock on the mesh data
- Fixed "Switch Lomotion Module" not using the locale key when "Exact Match" is enabled (reported by @Shifty | Quality Control Lead)
- Fixed internal UVSphereCapsule procedural generator not setting up triangle correctly when there's pre-existing geometry in the mesh
- Fixed bounding sphere encapsulation producing incorrect result when a smaller sphere is fully contained within another one
- Fixed sending items through messaging system not respecting the item save permissions (reported by @Dante)

Starting to work on the optimizations, here are a few initial bits! Added ColliderUserTracker component which I wanted to implement for a while, it tracks all the users in a collider (trigger) for you, which can be used to easily setup things like doors, culling zones and more!

Also made change for SkinnedMeshRenderer, which now avoids constantly recomputing bounds (it's pretty much only done during import auto-sizing process unless you re-enable it manually), which provided a pretty nice boost in testing.

This build also has a bunch of changes to Neos Classroom, a spin-off version used for education. I'm working on making new Quest build for that as well, so once that's done there will be a new Quest build as well.

[h2]New Features:[/h2]
- Added ColliderUserTracker (under Physics/Utility), which automatically tracks if any users are inside given trigger volume
-- It outputs "IsLocalUserInside" and "IsAnyUserInside" booleans for convenient use as well
-- It's based on physics system events and highly efficient as result, can be used to setup events, animations (e.g. doors opening), culling volumes and so on
- Added ability to Silence users in the session (muting them globally for everyone) (based on request by @Karel | CEO for Neos Classroom and others before)
-- Also added "AllowSilence" permission to UserPermissions. By default it inherits permission from "AllowKick"
- Added ability for sessions to be assigned a "Universe ID", which effectively separates all public (and private) lists from all others (based on request by @Karel | CEO for Neos Classroom)
-- The sessions with assigned UniverseID will show up in listing only if explicitly requested. This can be used to separate various business/educational sessions
- Added simple automated benchmarking mode through -Bench commandline argument
-- This will load the world, wait 30 seconds, then capture performance data for 30 seconds and shutdown
-- Captured performance data is stored in a "Bench" folder, currently as CSV file
- Added EnumToInt node, which converts enum value to its corresponding integer value
-- For HttpStatusCode enum this is the numeric value of the actual code (requested by @ProbablePrime and @Bitman | PolyLogiX Studio)

[h2]Neos Classroom Additions/Tweaks:[/h2] (requested by @Karel | CEO)
- Added ability to use a remote session key with Neos Classroom (add remoteSessionKey entry to the config), allowing the sessions to be joiend over the internet rather than just on LAN
- Added ability to toggle user voice on individual users
- Added ability for students to "Raise their hand"
- Added display number of joined students
- Enabled usernames above heads
- Fixed loading of the device name from the Neos Classroom config and not remembering the one saved

[h2]Optimizations:[/h2]
- Added "Compute Realtime Bounds" option to SkinnedMeshRenderer, which allows toggling whether bounding box is being recomputed every frame from the deformed mesh
-- This is now off by default, which resulted 25% performance boost in a test scenario with empty world and a dozen copies of my avatar
-- As a side effect, SkinnedMeshRenderers are now not sources of bounds (I might add simpler and less accurate on-demand method)
-- If you're heavily deforming or distorting the rig, you might see it popping invisible, you might need to re-enable this in such cases, but it's heavily not recommended to do so unless absolutely necessary
-- The option is temporary enabled during model import for accurate auto-sizing
- Added memory pooling when requesting session information to avoid some memory allocations
Tweaks:
- World browser will now link active session information locally for non-public worlds if they're owned by the user hosting it, rather than create a separate entry for it
- Dynamic variable node labels now contain the type of the variable, making it easier to tell which type they operate on (based on feedback by @Hayden (PolyLogiX - ZyroDesign))
- Lowered default Max User limit to 16 (requested by @Karel | CEO)
- Marked gizmos as not being locomotion grippable, preventing users from grabbing onto them by accident when gripping on anything is enabled (based on feedback by @GearBell, @Kal and @Turk )
- Slot gizmo options will no longer accept physical touch (based on feedback by @GearBell, @Kal and @Turk )
- Added "Slot" and "User" dynamic variable generic presets to the LogiX node menu (based on feedback by @Turk)
- Some small security improvements for inventory
- Merged French locale fixes by @FreakyWaves Music and @Archer
- Merged new/updated British English locale strings and English fixes by @Enverex
- Merged new/updated Finnish locale strings by @Napo
- Merged Korean locale fixes by @MirPASEC

[h2]Bugfixes:[/h2]
- Fixed AlwaysOnFacetModule breaking with multiple children (based on report by @Alex from Alaska)
- Fixed MultiSlerp node showing as "Multi Lerp" (reported by @3x1t_5tyl3)
- Fixed component attacher "Back" button not returning to the actual previous folder when generic component type selector is opened (based on report by @ProbablePrime)

The dynamic variable system is now maturing and ready for proper use! Added LogiX bindings to make it easy to integrate it into your interactive setups as well as versions of the components for reference dynamic variables.

Also adding initial Finnish and French translations from community contributions! The user search has been fixed as well as some shaders (like fog box volume rendering through).

[h2]New Features:[/h2]
- Added LogiX nodes to work with the dynamic variable spaces (under Variables). Same syntax as with components applies.
-- DynamicVariableInput - will link up with dynamic variable in the hierarchy the node is in (meaning you need to pack first to work). Will notify when the variable value changes, RECOMMENDED to use when possible!
-- ReadDynamicVariable - reads dynamic variable from a target point in hierarchy, use for dynamic lookups, won't notify when variable changes
-- WriteDynamicVariable - write a value to existing dynamic variable at some target point in hierarchy
-- CreateDynamicVariable - will create backing storage for a dynamic variable if it does not exist
-- WriteOrCreateDynamicVariable - will create backing storage if it doesn't exist or just write a new value
-- DeleteDynamicVariable - will remove all backing storage components for given dynamic variable

- Added DynamicReferenceVariable - has storage for a dynamic variable that's a reference to an object
- Added DynamicReferenceVariableReset - analogous to the value reset, but for references
- Added DynamicReference - uses external reference as backing storage for reference dynamic variable (analogous to DynamicField)
- Added DynamicReferenceVariableDriver - drives a target reference from a dynamic variable (no backing storage)
- Added support for generic node presets in the node browser for common types (this allows creating nodes like DynamicImpulseReceiver from common types directly from the menu)
- Added initial Finnish (fi) translation strings by @AdmiralSoap
- Added initial French (fr) translation strings by @Archer

[h2]Tweaks:[/h2]
- Merged German locale improvements by Schwefelhexafluorid (GitHub handle, not sure what the Discord one is, sorry!)
- Merged Turkish MMC strings by @Blaze
- Merged Esperanto, Japanese and Chinese locale fixes from @Melnus
- LogiX variable nodes were moved to "Variables/Storage"
- Added French pluralizer (requested by @Archer)
- Renamed DynamicVariable to DynamicValueVariable
- Renamed DynamicVariableReset to DynamicValueVariableReset
- Renamed DynamicVariableDriver to DynamicValueVariableDriver

[h2]Bugfixes:[/h2]
- Fixed DynamicVariableReset potentially resetting dynamic variable with no backing storage, causing inconsistent behavior
- Fixed user search on the Contacts screen not showing any newly registered users after recent changes to API sanitization (reported by @Kulza#8623 and @MattyK through @Shifty | Quality Control Lead)
- Fixed FogBoxVolume in Worldspace mode rendering through everything, even if the boundaries of the volume are obscured (discovered with @Rue Shejn | Artist 3D)
- Fixed PBS Displace Metallic/Transparent in AlphaBlend mode not casing properly displaced shadows (discovered with @Aegis_Wolf | Art Director)

Just a small patch, a few additions and tweaks. Compatible with current build, no need to update immediatelly if you're not affected by the issues.

New Features:
- Added British English locale by @Nammi (this only overrides certain strings with British English equivalents)
- Added MMC translation strings by @ProbablePrime (note that these are temporary for the MMC and will be removed after the contest ends)
- Added Japanese, Chinese and Esperanto translations for the MMC strings by @Melnus

Tweaks:
- Userspace now has "World" dynamic variable space as well (based on feedback from @Alex from Alaska)
- Categorized NoDestroyUndo under Transform/Tagging (requested by @Alex from Alaska)

Bugfixes:
- Fixed Lip tracking not correctly initializing in latest build (reported by @umbran)

Hello and welcome back to the Neos Weekly Update!

This week we thank all of our wonderful translators who’ve been working hard this week! Thanks to them Neos is now available in 9 different languages! We’ve got some more technical stuff in this update so check it out below as well.



[h2]Thank you, Translators![/h2]
The Neos UI is now available in multiple languages! It’s been a long-requested feature to add translations to the UI and with it now being implemented, we’ve seen a large number of community members contributing translations of Neos into other languages.

We haven’t expected to see this many so soon, but we’re really excited by your enthusiasm in helping to make Neos more comfortable for non-native English speakers. As a small thank you, we’ll be giving everyone who contributed a 25 GB bump in storage.

On our official Discord we also have a new Translators role, so we can notify whenever there are new strings to be translated. If you’d like the role, just ask a team member to assign it to you! We’d also like to shout out to all of the individuals involved in helping with this project!

A special thanks goes out to Orange for Japanese, Melnus for Chinese and Esperanto, Nammi for Icelandic and British English, MR Alex and Avunia for German, BlazeVR for Turkish, MirPasec, LUA and R3C0D3r for Korean and of course, a thanks to Frooxius for providing Czech! You can see a full list of contributors on the official GitHub repository.

We have also approved temporary addition of Metaverse Maker Contest strings by ProbablePrime, which will make the voting UI of this community-held contest available in different languages. However those will be removed once the competition is over.

[previewyoutube][/previewyoutube]

[h2]Stabilizing the dynamic variable system[/h2]
The dynamic variable system was originally introduced for the new World UI, but it has been designed for general use. However since we weren’t fully settled on some parts of its design, we have recommended to be careful with its usage due to potential breaking changes.

This week we have finalized those parts and the system should now be ready! One of the major additions is the ability to directly bind variable spaces by their name. This allows variables to link up with a particular space in their parent hierarchy, even if there’s a closer DynamicVariableSpace to them.

Each world is now also setup with a space named “World” at the root and each user with one called “User”, which allows for convenient scoping of variables within each user or within the whole world.

To achieve this linking, we have reserved all Unicode symbols and punctuation, except spaces, periods and underscores for a custom syntax. If you used those symbols in your variable names then you’ll need to change them, but otherwise your existing setups will be unaffected.

If you want a variable to bind to a particular space, simply prefix it with the name of the space and a forward slash, like “User/Health” for example. We’ll keep extending the syntax in the future to allow for things like optional bindings, binding multiple names and potentially more complex pattern matching.

[previewyoutube][/previewyoutube]

We have also added LogiX nodes to work with the new system, to greatly simplify and modularize setups. You can implicitly use dynamic variables as inputs. The system will automatically link up with the variable and trigger changes whenever the variable changes. This process is designed to be highly efficient and avoid expensive look-ups.

Other nodes allow to dynamically read variables from a target hierarchy and write them back. For the variables to be readable or writable, there needs to be a backing storage in the target hierarchy, for example the DynamicVariable component. However you can also use WriteOrCreateDynamicVariable node which will create this backing storage for you if it doesn’t exist.

We hope that this new system will open up a lot more efficient and flexible approaches to building interactive content in Neos and we can’t wait to see what you’ll make with it! If you run into any issues or problems, please let us know!

[h2]Sampling Color in LogiX, improving localization system and more[/h2]
As usual we have made a variety of small additions, tweaks and bugfixes to Neos over the week. Notably there is now a new LogiX node “Sample Color”, which lets you sample a visible color from a point and direction in the world, like a small color probe. Note that this is a relatively expensive operation as it renders a very tiny image of the scene and should be used only sparingly.

As part of the improvements to the new localization system we have also made contributions to the MessageFormat.NET library which implements the ICU MessageFormat syntax. Its simplified plural formatter has been replaced with a proper implementation of the ICU standard.

This ensures that the printed numbers can be grammatically correct depending on the language and allows one to one porting of the proper pluralization algorithms defined in Unicode CLDR. For example in English when decimal numbers are visibly printed, the nouns will be pluralized correctly: “1.0 items” instead of “1.0 item”.

We have also added several more pluralizers to increase the coverage of the languages and made some other additions and improvements. If you’d like to use the library with those improvements for your own project, you can get our fork here at GitHub.

[h2]Community Highlights[/h2]

[h3]Updated Kinect Tracking Driver by Groxxy[/h3]
Groxxy has brought us an updated version of the Kinect tracking drivers which includes support for the 11 point tracking in Neos! It does so by emulating 8 tracking points, which is an improvement from the usual 3 reserved for hips and feet. Now the Kinect software can capture elbows, chest, knees as well as hips and feet for a true full-body tracking experience and much more realistic avatars! Thank you Groxxy!

[previewyoutube][/previewyoutube]

[h3]Everyone working on MMC creations![/h3]
You are all working so hard, that you all deserve a huge shoutout from the Neos team! Neos team members who are not judges have been able to visit some of your worlds while you’re working, and we know you’re giving it your all! Only 10 days left until the end of the MMC! Keep going, everyone!



[h2]What’s Next?[/h2]
Our main goal right now are optimizations and related features for Neos. We have started investigation into upgrading to BEPUv2 and other optimizations, but ran into some initial snags (check out the #devlog channel on Discord for details).

However there are dozens of different optimizations planned, some major and minor, with more to be discovered as we keep profiling and finding bottlenecks, so Neos should get progressively smoother as time goes!

Thank you again for reading the Neos Weekly Update! We want to thank our Patrons for bringing Neos up to $9000 a month this past week. This project would not be able to survive without you!