Neos VR News

Hello everyone! Sorry for relatively small build, I'm still focusing a lot into designing and underlying bits for the new desktop mode, but here's a bunch of accumulated additions, tweaks and bugfixes on the side, as well as new bits of the desktop mode (not quite in practically usable state yet though).

One important thing added in this build is ability for Patrons to set a particular GitHub issue as priority one through a bot command, which will help us better tell what to prioritize. It's not the best UX, but it's quick to add in this form and we'll be able to build more UI and features into it in the future, so hopefully it'll serve well in the meanwhile!

I've also been investigating some cases of record syncing taking long, particularly the upload finalizations and found the underlying issue, so uploads should now be faster, particularly when there's a lot of them happening at the same time from many users!

There are also updated Mentor/Moderator badge designs by @Coffee | Programmer and first custom notification sounds by @Lewis Snow | Lead Audio Engineer! And lots of other tweaks and crucial bugfixes too.

[h2]New Features:[/h2]
- Added /priorityIssue Neos bot command, which lets Patreon supporters give a priority vote to GitHub issues to let us know what issues are most improtant to our supporters when deciding what to prioritize (suggested by @Karel | CEO, based on inquiries by @Epsilion, @Rukio and others)
-- E.g. send /priorityIssue #149 to the "Neos" account in Neos to give your Patreon vote to GitHub issue with a specific number
-- The command will echo back the title of the issue on success
-- You can only have 1 priority issue at the time. You can change your vote at any time
-- Current votes can be seen at https://www.neosvr-api.com/api/stats/priorityIssues, with vote count for each and vote points (points are proportional to level of support on Patreon, updated automatically)
-- Note that the priority issue list is updated asynchronously and can take a few minutes to reflect your vote in some cases
-- Later on we'll provide UI for this mechanism as more fine-grained options (e.g. giving specific amount points to multiple issues), but that requires more development time

- Added TouchButton (under Transform/Interaction) which provides IButton implementation based on pure touch events (based on request by @Epsilion)
-- This component acts as any touchable (e.g. can be touched from any angle unlike PhysicalButton, supports filtering Physical and Remote touches)
-- It will trigger proper IButton events, either to components on the same slot or through LogiX event listener
- Added "Master Volume" to settings, which allows controlling global volume of everything in Neos
- Added "Disable LAN" setting, which disabled announcing the session on LAN as well as receiving announced session information from LAN
- Added MasterVolume, NormalizationThreshold, DisableNormalization and NoiseSupression to AudioSettingSync
- Added new custom Receive message & Receive invite sound effects (sound effects created by @Lewis Snow | Lead Audio Engineer)

- Added new "Assemble" quiz type to Neos Classroom, which randomly scatters pieces of the model and requires the student to put it back together, measuring their time (requested by @Karel | CEO)

[h2]Work in progress:[/h2]
- Implemented neck model for the desktop mode, which uses the neck as the pivot for head rotation, rather than the eyes, resulting in a more natural head movement
-- For VR IK Avatars the neck offset is taken from the actual avatar, otherwise a hardcoded value is used
- Added AvatarNeckOffset (under Users/Common Avatar System) which allows providing custom neck model offset for the desktop mode (e.g. you can set it to 0 to get the old behavior with pivot being around the eyes)
-- Offset is in UserRoot coordinates and should match physical units (e.g. [0; -0.1; -0.12] is 10 cm down, 12 cm back from the eyes)
- Added a mechanism for separate settings for Desktop Mode and VR
- Audio settings (Master Volume, Noise Gate/Filtering, Normalization and audio input device) are now unique whether Desktop Mode or VR mode is active and will instantly switch as the modes are toggled
- Added mechanism to position simulated hands based on hips position, rather than eyes, providing basis for more natural positioning (currently full positioning isn't implemented yet)
- Added partial implementation of internal input binding system (not in any active use yet)

[h2]Tweaks:[/h2]
- Updated designs of Moderator and Mentor badges to provide better contrast at small sizes and unify the style (badge designs created and updated by @Coffee | Programmer)
- When the hinted user's root coordinate space is child of the user (e.g. a held item) it is now ignored
-- This fixes other users not seeing the user moving when they're prop-surfing on a held item
-- This should also fix user flying in random direction when trying to enter an anchor on user they are climbing (reported by @Robyn (QueenHidi))
- Audio Device List now uses fixed size for device buttons and will scroll if there's more devices than fit on the screen (based on report by @Gourry, @Fleur24 and @Turk)
- Flat positioning of objects (e.g. used by the GrabbableReceiverSurface) now ignores Trigger colliders when determining the placement boundaries (based on feedback by @Lewis Snow | Lead Audio Engineer and @Coffee | Programmer)
- Settings screen panes will now scroll if there's too many settings to be displayed (based on report by @orange)
- Upgraded SteamVR Unity Plugin to 2.7.2

- Merged English & German locale corrections by @dfgHiatus
- Merged Korean locale fixes by @MirPASEC
- Merged French locale fixes by @brodokk
- Merged Russian locale addition by @Shadow Panthe1r [RU/EN, UTC+3]

[h2]Optimizations:[/h2]
- Added hierarchy exclusion caching to DroneCamera and removed object size measuring for exclusion, to avoid heavy performance loss in certain worlds (based on feedback from @Nexulan | Community Manager)
- Added caching to internal default audio device index, to prevent it from being constantly searched on every access
- Optimized long unecessary polling waits when finalizing asset uploads, resulting in blocking of the queue and uploads taking long to finish as a result with a large surge of requests
- Increased number of active cloud upload workers, to reduce waits when there are many active uploads

[h2]Bugfixes:[/h2]
- Fixed rigged model import breaking on hand rig detection when the thumb has only 2 actual bones (reported and sample models provided by @Shifty | Quality Control Lead, @gallusfox, @Ian Corvid#8921 )
- Fixed Audio Stream muting when the host is recording a voice message (reported by @Tatsu Kimiero)
-- Note that if you're streaming the same device as your normal voice input, it'll still get muted
- Fixed driving of recordingVoiceMessage on User causing the user to not get muted when recording a voice message (reported by @DJNightmares, @xXx_swiggity_xXx#2124 and @Frank, PHD)
- Fixed incorrect caching of nested external references when destroying objects while preserving assets, resulting in some assets to be inconsistently preserved when they shouldn't be
-- This fixes undoing deletion of a photo resulting in it being blank (reported by @Tatsu Kimiero, @H3BO3, @Turk and @Shifty | Quality Control Lead)
- Added invalid value (NaN/Infinity) filtering to InteractionLaser to prevent the laser from breaking due to NaN's/Infinities being fed into the system (based on report by @Ardes, @CyberZott#5567 and @EvawdnousCimota#2730)
- Added invalid value filtering to BentTubeMesh to prevent it from getting corrupted (based on issue above)
- Fixed AudioSettingSync providing incorrect values in NoiseGateAttack, Hold and Release properties

Small hotfix for the new parent streaming sometimes producing invalid offsets (e.g. some user anchors making the user too big/small or offseting them incorrectly). Sorry it took so long, it took a bit to trace it down and I ended up having to rework the math for it!

Compatible with previous build, but if you or other users are on the old one, you still might get incorrect behavior until everyone updates.

[h2]Tweaks:[/h2]
- Merged Japanese, Esperanto and Chinese locale additions by @Melnus
- Merged French locale addition by @brodokk
- Merged Korean locale additions by @MirPASEC
- Merged Czech locale addition by @rampa_3 (UTC +1, DST UTC +2)

[h2]Bugfixes:[/h2]
- Reworked transform offset math for TransformStreamDriver so it always produces correct offsets with the new space streaming system
-- This fixes avatar anchors and other behaviors that translate/rotate/scale user after parenting them resulting in invalid offsets or scale (reported by @Furf, @chemicalcrux, tested by @Shifty | Quality Control Lead, @chemicalcrux and @Tatsu Kimiero)
- Added guards against invalid/degenerate rotation and scale offsets to TransformStreamDriver to prevent potential misbehavior

Still focusing more on the desktop implementation, so the build is relatively small, but it has some crucial bugfixes and improvements. I've traced down source of lag when duplicating text renderers and UIX elements that use unique materials, which was particularly noticeable with some context menu injectors and such. The system will no longer degrade in performance with repeated duplications and deletions and parts of it are now faster too!

There's also another change I wanted to do for a while, the stream of user's root position, rotation and scale now also streams which space are the values in, which fixes the random flickering of the user when they get parented/unparented. As a nice bonus, it now also makes the physical locomotion grabbing position the user properly relative to a moving gripped object/other user for other users, rather than having them lag behind it due to network!

There's a bunch of other tweaks and additions, some more bits of the desktop mode internally, but most things don't affect anything visible yet.

[h2]New Features:[/h2]
- Added ReferenceStream which allows using unreliable streams for references
- TransformStreamDriver now supports synchronizing the current transform space, ensuring that the transform values are properly interpreted when the object is reparented
-- This is now used by default for user's root transform. This should fix random brief flickering of the users seen by others when they transition under a different parent or the user being in incorrect location for a short while when there are network delays for reliable data model changes (note that if the user is parented under a newly created slot, the system will still have to wait until that is synchronized first before they can be properly positioned) (pre-released tested and debugged with help by @Shifty | Quality Control Lead, @ohzee, @Zane and @Turk)
- Added a mechanism to hint at user's root transform space, which allows using a different space to synchronize user's root transform
-- This is now used for physical locomotion gripping, ensuring that when gripping onto a moving object/player, the user's relative position to the object is the same for other users, rather than lagging behind due to network latency
- Added mechanism to initialize default value for Value streams
-- This is used to fix some odd behaviors and better implement value streams used to synchronize value streams that need to start with non-default value (e.g. scale streams starting with [1;1;1] instead of [0;0;0]), regardless of who constructs the stream first (based discoveries from the testing session for feature above)

[h2]Optimizations:[/h2]
- Heavily optimized CPU and memory usage when destroying objects with asset preservation by reducing redundant walking of the data model for each element and optimizing other operations
-- This will significantly reduce hitches when destroying some complex objects (e.g. through the context menu)
- Added more memory pooling to the saving process when processing dependencies, reducing memory allocations and GC churn
- Fixed Value and Reference drivers not being cleaned up when a font material is removed
-- This fixes progressively worse performance as UIX graphic and TextRenderer instances with unique materials are repeatedly added and destroyed (reported and replication steps provided by @3x1t_5tyl3, @chemicalcrux, @かず (kazu / GitHub: kazu0617), @あむ（Rabbuttz） and @Hayden)
- Skipped unecessary Value/ReferenceCopy deduplication search for local TextRenderer and UIX materials (discovered thanks to the issue above)

[h2]Work in progress:[/h2]
- Added a mechanism for controlling cursor lock/visibility state
-- Switching to desktop mode while running in VR now locks and hides the cursor
- Added mechanism for notifying screen moduels when user switches to screen mode, initializing certain values (e.g. first person module now updates the head root offset based on current VR headset position to avoid moving head back to playspace origin on XZ axes)
Tweaks:
- IsPresent user status is now true when the user is switched to the desktop mode
- Destroy Selected on developer tooltip now has a confirmation (requested by @Robyn (QueenHidi))
- Changed how TransformStreamDriver synchronizes scale offset, using multiplication instead of shift
-- This fixes some oddities and bugs with transforming the scale offset, resulting in invalid scale in some scenarios (discovered thanks to testing by @Shifty | Quality Control Lead, @ohzee, @Zane and @Turk)
- Upgraded to Unity 2019.4.19f1 (from 2019.4.18f1)

- Merged Dutch locale additions by @AnotherFoxGuy
- Merged German locale additions by @Elektrospy
- Merged French locale additions by @brodokk
- Merged Japanese locale fix by @Aesc

[h2]Bugfixes:[/h2]
- Improved hand rig detection, which will now check if finger bone candidates are actually weighted bones on the rig
-- This fixes some models having their fingers bend incorrectly due to detecting proximal bones as metacarpals when there are two finger tips (reported and sample model provided by @Medra)
- Fixed Patreon data stopping to update due to the list of patrons for PatreonSkyController becoming too large
- Added exception guard to Collision event nodes, to prevent LogiX errors from breaking other collision events
-- This should fix avatar anchors breaking due to laggy collisions when LogiX is involved (reported by @Earthmark and @kleinlowe#2600) and many similar scenarios
- Fixed some cases where invalid rotation data get sent in a rotation stream, resulting in some transform operations producing invalid rotational offset

Hello everyone and welcome to another weekly update!

This week we have some awesome community highlights. We have went through many awesome worlds and gadgets (including some games) that you have built inside of Neos, if you missed the stream, check out the recording and some extra highlights below!

Due to development focus shifting more to desktop things are a bit quieter in terms of major features. Among many smaller (but important) bugfixes, tweaks and additions, we have a greatly improved GLTF 2.0 support, which should now import properly with Blendshapes and their names. We have also done some smaller optimizations, including moving the mesh metadata generation to the asset variant system.



[h2]Mentors Network: Week One[/h2]
Many of you in the community might be noticing people running around with green nametags/question Marks above their head! These people are Mentors in our community, and if you have a question or need help while your new, contact one of these people and hopefully we can set you in the right direction!

Mentors are members of the community who don the role of the green to show or inform users old or new alike. It’s only been one week so far of on-boarding everyone and so far some of the reports have been coming in about some of their efforts! Thanks everyone! -Turk



The new designs for mentor, moderator and Neos Team badges

[h2]Updated guidelines[/h2]
Over past weeks, we have also updated and expanded our usage Guidelines. If you haven't checked them out yet, please take a moment to review them here.

[h2]Community Highlights[/h2]

[h3]January Community Showcase livestream[/h3]
This Friday In our regular end of the month livestream we have taken a look at some of the community creations and worlds from January. We checked out some new gadgets and minigames, a fishing game world, beautiful hangout worlds, curving bullets shooter game and more! If you missed the stream you can watch the recording here:
[previewyoutube][/previewyoutube]

[h3]Vending Machine by GhostZ[/h3]
Every felt thirsty in VR? Need a weapon to protect yourself? Grab it from a vending machine! GhostZ is well known for his various drinks, grenades, and his various modeling endeavors, but now we are at the point where he’s made them dispensable now! The only thing you need to know is to buy anything. it costs some KFC. So make sure you have some around to spend.



[h3]Time Machine Recreation by H3B03[/h3]
One of our Moderators H3B03 who’s been heavy at work in the discord answering users questions, assisting with LogiX, or keeping the peace has made a lovely recreation of a time machine from a certain movie! During his free time he’s been churning away at this bad boy make it look spiffy as can be.



[h3]Wavefront Windfarm Circuit: V A P O R W A V E D: by Aegis Wolf[/h3]
Some of you might recognize this map! Yet, did little did we know Aegis added a secret “V A P O R W A V E” button to the map! It makes everything all trippy and wavey, as you hop on a bike and travel down the track. It’s an experience once should experience in VR.


https://twitter.com/Teh_Turk/status/1355991498664370183

[h3]Weekend at Jayes: A Wonderful moment in Neos by KungFuMonkey, Spexfox, Baxter, Jaye[/h3]
Neos sometimes brings something special out of others, and this is one to note. Watching the video you can see that “Jaye” friend of SpexFox had some computer troubles recently. He sadly felt left out or unable to join in the fun and joy that is VR. Sooo, one of our newer users in the community KungFuMonkey helped out in this special moment! Thanks guys! You make Neos awesome :)

[previewyoutube][/previewyoutube]

[h2]Improved GLTF 2.0 model import support and more[/h2]
Assimp is an open source library capable of reading over 40 different 3D model files and the backbone of Neos' 3D model import support. Due to regressions causing some FBX animations to import incorrectly we have avoided upgrading the library to latest version for at least half a year, as it was in a good enough state.

With recent surge in interest in the GLTF 2.0 format and some issues with its import (primarily blendshapes either not importing at all or missing names) we were prompted to look at it again in detail and managed to find a cause in the library itself for the FBX animation import regression and fix it.

Thanks to this, we were able to upgrade to the latest version straight from the source, featuring over a 1000 commits to the library by its maintainers and contributors. This has significantly improved GLTF 2.0 support, making blendshapes import properly with full names, making it much easier to use this modern format in Neos.

There are many more improvements as well for other formats, including bugfixes for certain model files failing to import, or even outright crashing whole Neos. Thanks to the fix, we can keep updating the library from source more frequently now, bringing its latest improvements and additions to Neos.

[h2]Optimizing mesh metadata computation with the asset variant system[/h2]
Another 3D model related change involves Neos' own mesh asset format. Whenever meshes are loaded in Neos, various metadata is computed on the background threads as the model loads, particularly bounding box and bone metadata.

To conserve some CPU resources and speed up the upload process, we have moved this system to Neos' powerful asset variant systems. Thanks to this, the metadata is automatically fetched from cloud when available or computed locally if not and always cached. This way the overall CPU and memory load when loading meshes is decreased, freeing up resources for other tasks.

We plan to build more on top of this functionality in the future as well, computing different variants of the meshes - decimated versions for better performance (and LODs), pre-generated mesh collider data and more.

[h2]Progress on Desktop mode[/h2]
While the upcoming desktop mode is still heavily in design phase, the latest builds already have parts of it implemented. We have laid out the basic skeletons of some of the core systems, particularly for positioning user's head and hands and providing first and third person rendering and interactions.

The systems are very barebones as of now, but technically already allow a switch from VR to desktop rendering with mouse look and back. They're not in a state where they would be usable in practice, but they lay an important foundation for what's the come.

[previewyoutube][/previewyoutube]

As we'll start "fattening" those mechanisms up and adding more, the desktop mode will become more usable over time. Once we have basic interactions implemented, we can start focusing on various stretch goals, like ability to pin inspectors and other windows to the UI and customize the layout.

An Input Binding system is also becoming big part of this work, as it'll help to abstract and simplify a lot of the systems, without having to hard-code a lot of the behavior into individual interaction systems.

If you'd like to keep with the latest developments, check out the #devlog channel on our official Discord! You can also check the latest design notes here:

Latest desktop mode design notes (click to enlarge)

Anyway that's all for this week! We can't wait to see you at the next one. Thank you for reading and thank you for supporting Neos! We've been hitting record numbers of users over past week and that's in big thanks to your passion and creativity that makes this platform such a fun place for many people to enjoy and make their home.

Also a mango

Sorry for smaller build, I'm putting more focus on the desktop mode now, there's a skeleton of the first core systems for it now! Not really in any usable state yet, but if you want to toy with it a bit, you can press F8 while in VR (note it'll mess up your VR view when active at the moment) and you'll be able to... look around xD More to come!

But in the meanwhile here's a bunch of tweaks and bugfixes so they're not waiting too long before being released, hopefully they'll help out! Thanks to a fix lbry.tv links now work for video playback! There's some more polish for the issues caused by the changes to the shader system too, hopefully that'll patch up some of the remaining problems.

[h2]Work in progress features:[/h2]
- Added ScreenController, which manages and update different screen mode components
- Added HeadSimulator, which is responsible for simulating user's head in screen mode
- Added HandSimulator, which is responsible for simulating user's hands in screen mode (or when no controller is available)
- Added FirstPersonTargettingController, which implements a first person view controls and behaviors

[h2]New Features:[/h2]
- Added Mute badge (badge designed by @Coffee | Programmer)
-- You can request a team member to add it

[h2]Tweaks:[/h2]
- Updated the mentor badge to a proper design (badge designed by @Coffee | Programmer)
- Added small delay before the next button on the Welcome Guide screen can be clicked, to prevent accidental double clicking and skipping a screen
-- This might potentially fix welcome tutorial screens sometimes becoming blank (reported by @Earthmark, @Furf and @Shifty | Quality Control Lead)
- Improved permission system security to avoid the host setting a role to higher level than they're allowed to (based on report by @Hodufan)
- Welcome Screen & tutorial will now show on Linux as well
- "Copy Link" context menu option won't appear if multiple InventoryLink items are held to avoid generating too many items and ambiguity (based on report by @3x1t_5tyl3)
- Userspace user will now have the proper UserID and Username of the currently signed user (based on feedback by @ProbablePrime)
- Added Mango
-- It is advised to not wait too long before eating it, otherwise it'll get mushy
- Upgraded to youtube-dl 2021.01.24.1 from 2021.01.08
- Set default doppler level to 0 on the AudioStream dialog (suggested by @Robyn (QueenHidi))
- Button, PhysicalButton and NeosButton no longer send press events to disabled button event listeners on the Slot
-- This fixes VirtualMultiKey and others working when disabled (reported by @Psychpsyo)



- Merged Spanish locale update from @Ruzert
- Merged Czech locale additions from @rampa_3 (UTC +1, DST UTC +2)

[h2]Bugfixes:[/h2]
- Old shader providers with deprecated neoshader:// URL will now be cleared on load
- Shader references on materials are now non-persistent and never copied/collected, instead being freshly re-initialized every time
-- Combined with the clearing above this should help fix increasing lag due to duplicated shader providers when they're saved together with the object (based on report by @あむ（Rabbuttz）, @3x1t_5tyl3, @Hayden and @chemicalcrux)
-- As a side effect, this now also excludes the shader assets from being pinned on user's account in the cloud and being included in the sum size of all assets showed in the inventory (reported by @LeDrascol and others)
- Fixed Mentor badge not showing up in the Session dialog (based on report by @Ryuvi | Technical Artist)
- Fixed Color blind badge not showing up in the Session dialog
- Fixed Refract shader not activating the right variant when using normal map, causing it to not take effect (reported by @Vigilabo, @Hayden and @Ultranique)
- Fixed youtube-dl stream extraction not working for services where extracting format metadata is not possible, as they offer only a single stream URL
-- This fixes lbry.tv videos not loading and potentially some other services as well (based on report by @H3BO3 and @Shifty | Quality Control Lead)
- Fixed exceptions in record saved events when saving items to the root folder in the inventory
- Fixed deleted lasers on user (e.g. by deleting their armature) breaking InteractiveCamera, Finger camera gesture and other behaviors that try to get roots of lasers of user(s) in the world (reported by @ItsDusty )
- New points of ValueGradientDriver are now initialized to a valid default value for all datatypes
-- This fixes ValueGradientDriver misbehaving when the default value of any of the points is not changed (reported by @Zyzyl)
- Added detailed logging for Welcome Guide screen to help diagnose cases of the screen becoming randomly blank for some users (reported by @Earthmark and @Shifty | Quality Control Lead)