Neos VR News

Hello everyone, I'm back! :smile: Here's a bunch of smaller additions, tweaks and bugfixes to clear up a bunch of the GitHub issues that have popped up while I was on the break. This includes numerous improvements and fixes for the Context Menu Injection, LogiX, avatar legs randomly popping up (and other random glitches, as it was caused by an underlying math function that's used in a lot of places), glitchy audio on Unity Native video playback and more!

I'll be working on some larger things next, I've already started poking around BEPUv2, so that will be a major upgrade, but not certain how long will that quite take yet, as well as on some cloud and builds things to improve some internal processes. Hopefully this build will help get rid of some of the pain points in the meanwhile!

[h2]New Features:[/h2]
- Added "Keep Position" to ContextMenuSubmenu which will not reposition the context menu when it's opened through this option (requested by @3x1t_5tyl3)
-- Note that you should make sure that the context menu is in a good position when it's opened first before using this, otherwise it'll end up in whatever random last position it was last spawned in, resulting in usability issues
- Added "OnlyForSide" field to RootContextMenuItem, which allows the context menu item to be optionally filtered based on the hand that opens the context menu (requested by @Shadow Panther [RU/EN, UTC+3])
- Added TooltipAvatarPoseFilter (under Transform/Interaction) which allows installing avatar pose filters when equipping tooltips (requested by @3x1t_5tyl3)
-- This needs to be placed on the same slot as the tooltip to work
- Texture Asset Variant system now supports RawRGBAHalf texture compression for HDR and high precision textures
-- This fixes HDR textures set as "Uncompressed" loading as LDR RGBA32 textures, resulting in loss of all the HDR data (reported by @SmolCookie | VRtualis CEO)

[h2]Tweaks:[/h2]
- A dummy character controller is now activated when no locomotion module is active (e.g. when sitting in an avatar anchor or when all are disabled by permissions)
-- This fixes users in avatar anchors not being tracked by ColliderUserTracker and being excluded from culling systems and events
-- If you run into issues where you need the user to be excluded from events and such with your avatar anchor, let us know and we'll add a mechanism to provide this. Currently I'm not sure if there are any use-cases for this.
- SessionOrbsController now skips updates in background worlds with no users present to avoid unecessarily loading and unloading thumbnails
- Default User Scale is now respected when user is reset because of falling outside of the map or when their scale gets corrupted (reported by @Gawdl3y)
- Categorized Float2/3/4Driver components under Transform/Drivers (requested by @Epsilion)
- Importing all asset types (Audio, Fonts, import dialogs...) will now place them in the user's current space rather than root (reported by @Kyuubi)
- Pressing secondary when dragging a wire to a Type input on a LogiX node will now spawn the Type input node (requested by @3x1t_5tyl3)
- Increased the maximum value of "Cleanup Interval" in the session menu to 3600 (based on feedback by @3x1t_5tyl3)
- FindCharacterControllerFromSlot now ignores any disabled character colliders
- All avatar records that aren't in public folders have been marked as non-public
-- Note that you won't see this reflected in your inventory right away, as the records will still be cached. I recommend waiting a day and clearing your database if you want the UI to reflect the change, but it's not necessary for the security improvement to work

- Merged Japanese locale additions and tweaks by @Aesc
- Merged Russian locale additions and fixes by @Shadow Panther [RU/EN, UTC+3]
- Merged Korean locale additions by @Guide
- Merged Czech locale additions by @rampa_3 (UTC +1, DST UTC +2)
- Merged Spanish locale additions by @Ruz

[h2]Bugfixes:[/h2]
- Fixed custom context menu not responding to changes of the Sprite on ContextMenuItemSource when the menu is opened (reported by @seif1 and @Axius)
- Context Menu open/close speed is now sanitized, preventing invalid values (0, negative, infinity, NaN...) from breaking it (reported by Asakleiman)
- Neos will now retry API requests on 502, 503 and 504 responses, which usually indicate transcient issues
-- This should improve robustness of cloud functionality and syncing (based on report by @Shadow Panther [RU/EN, UTC+3])
- Fixed Context Menu breaking when the Label is null (reported by @Hayden)
- Custom context menu will now preserve the CommonTool summoner and pointer when already opened or try to use the one from the hand that has opened it
-- This fixes flick misbehaving when opened from the main context menu (reported by @Gawdl3y)
-- This also fixes custom context menus not being usable when the laser is disabled (reported by @Electronus, @Hayden, @Ukilop and @Psychpsyo)
-- This also fixes the context menu not preserving its position when the item is clicked with other hand (reported by @Gawdl3y and @Psychpsyo)
-- This also fixes not being able to interact with the context menu when the laser is occluded by a collider (@Ian Corvid and @Epsilion)
- Fixed Unity Native video playback engine producing glitchy audio on videos that aren't stereo (e.g. Mono, Quad or 5.1) (reported by @Gawdl3y and @Enverex)
- Fixed Slerp operation producing NaN quaternions in some rare cases due to floating point imprecision, resulting in random corruptions and glitchy behavior
-- This fixes avatar legs randomly jumping up on some avatars (reported by @Earthmark, @Cael Thunderwing, @H3BO3 and @HaruMaki)
-- This will also likely fix other random cases of glitchy behavior across the board (including when you use the Slerp function in LogiX) as the function is used widely internally
- Fixed LookAtUser always looking at the local user when the target node is set to View (reported by @AshtonSparx and @Sox)
- Fixed "Prevent reticle from going below the horizon" option no longer working after desktop update (reported by @ooBLANKAoo)

Another build sorry! This is only tweaks to the Minecraft importer I made though, I wanted to push them out so everyone gets the improved handling for things.

Build compatible with previous one, no need to update unless you're playing with the Minecraft imports.

[h2]Tweaks:[/h2]
- Culling colliders are now Spheres instead of Boxes
- Saplings now use alpha clip and have no collisions
- Rails and Powered rails now use alpha clip and have collisions (to avoid easily running on them like ramps)
- Light sources are now default off and controlable by World/Minecraft.Lights dynamic variable
- Torches now use alphaclip, fixing their look
- Redstone torches are now classified as torches as well (meaning no collisions and use proper alphaclip)
- Redstone wires now have no collision and use alpha clip

Hello everyone! I know I said no builds, but I ended up working on a silly thing that was just easier to integrate directly for now - Neos can now directly import Minecraft Worlds! :smile: It even sets them up with culling system (both rendering and assets), so you can import quite large maps.

I'm still playing with this now, so there will be some more updates to it, but it's not a feature that "official" right now, it's just there for you to toy with as well if you'd like, but it'll just be left as is at some point for a while. The source of it is available too though if you'd like to have a look or make contributions!

Also a few small addditions and tweaks (mostly related to this) thrown in!

[previewyoutube][/previewyoutube]

[h2]Silly Features:[/h2]
- Added support for directly importing Minecraft Worlds into Neos
-- In order to import, you need to install Mineways (https://www.realtimerendering.com/erich/minecraft/public/mineways/) either into C:\Program Files\Mineways or under Tools\Mineways in your Neos installation
-- To import, simply import the folder containing your Minecraft world (import the folder containing the level.dat file)
-- Alternatively import on the headless using importMinecraft command
-- Neos will import all chunks with actual block data. For large maps import can take pretty long (dozens of minutes), but you'll be able to interact with the map as chunks of it import
-- Neos will setup and customize all the materials, collisions as well as a culling system which will only load meshes and render chunks within distance around the player to support large maps efficiently
-- For older maps, Neos will setup point light sources for torches (might turn this off if it ends up too heavy). For newer ones this isn't currently supported
-- Importer source code available for reference (and contributions) here: https://github.com/Neos-Metaverse/Minecraft2Neos
-- Please note that this isn't a fully officially supported feature, but something done for fun. Feature requests and bug reports won't likely be fullfilled unless we want to work on them. We might separate this into a separate optional importer plugin in the future.

[h2]New Features:[/h2]
- ContextMenuItemSource now proxies button events as IButton, allowing it to be used with the Button Events LogiX node (based on feedback by @ProbablePrime | Docs, @3x1t_5tyl3, @Kal, @Cyro and others)
- Added "CloseMenuOnPress" to ContextMenuItemSource which will close the context menu when the item is pressed
- Added CloseContextMenu LogiX node (under LogiX/Interaction)
-- You can provide summoner, which will only close the menu if the current summoner is the same one as you provided. This is typically done so you avoid closing context menu when it was opened by something else in the meanwhile

- Added "Force Point Filtering" advanced model import setting, which will use point filtering for all textures on the imported model
-- Note this is mostly for stylistic purposes, do not use this as optimization technique
- Added "No MipMaps" advanced import setting for models, which will disable mipmaps on all textures
-- It's recommended to only use this when really needed for visual style, do not use this as "optimization" unless you really know what you're doing, otherwise you might cause visual artifacts and cause worse performance
- Added "Force Uncompressed" advanced import setting for models, which will set preferred texture format to RawRGBA
-- It's strongly recommended to use this only when really needed, typically for very small textures. Switch textures to alternate compression formats first like LZMA or BC7 variants first before using this option, as it will use significantly more memory (4-8 times more)

[h2]Tweaks:[/h2]
- Neos and CloudX can now detect Matroska (MKV) file types from the header

- Merged Icelanding locale updates and fixes by @Nammi
- Merged German locale additions by @3x1t_5tyl3

[h2]Bugfixes:[/h2]
- Batch importing models now imports them one by one, rather than importing everything at once in one go, causing the background processing to get overloaded and everything to freeze (reported by @Bitman (Neos.js Developer), @Elektrospy, @3x1t_5tyl3, @marsmaantje, @Ian Corvid and others)

I've got one last build for you before the week break, I just wanted to get a few of the quicker to do priority issues done and out of the way. Most notably proper context menu injection! This provides a proper, efficient and stable (it won't break due to random changes) mechanism to add custom items to your context menus or even open fully custom context menus.

It's designed to be pretty flexible, allowing you to show custom items on grabbed objects, tools (might add some more things specific to this if needed later), avatars and so on, check below for details.

Also a few bugfixes and tweaks as well!

[h2]New Features:[/h2]
Added Context Menu Injection system for adding custom items (components under Radiant UI / Context Menu) (requested by @Earthmark, @Turk | Mentor Manager, @Electronus and GitHub / Patreon Priority Voting)

- Added ContextMenuItemSource, which provides desceription of a context menu item - label, color and sprite
-- Disabling this component will prevent the item from being injected
-- All the visuals are driven in realtime and will be reflected even when the menu is opened
-- Any button press and hover events are relayed to the Slot that this component is on, allowing you to use all the standard button components and events (under Common UI / Button Iteractions)
-- ButtonEnabled property will still generate the item, but it will be disabled in the context menu

- Added RootContextMenuItem, which will inject target ContextMenuItemSource into the root menu whenever opened
-- In order to inject the item, it needs to be anywhere within the user's hierarchy. It uses efficient registration mechanism, so it can be placed anywhere in the hierarchy
-- This works for held items too - e.g. you can add custom options that will appear for grabbed objects
- Added ContextMenuSubmenu for invoking fully custom context menu
-- This receives standard button press events and can be invoked either from another context menu item or any other button
-- By default it will generate items from children of a Slot (they need to have ContextMenuItemSource on them), you can optionally enable it to search whole hierarchy using SearchWholeHierarchy, but it's not recommended for performance and does not guarantee order of items
-- You can disable flick, override opening speed and also generate the items in counter clockwise order

--------------------

- Added ButtonPressEventRelay, which can relay button press events to another slot
- Added ButtonHoverEventRelay, which can relay button hover events to another slot

[h2]Tweaks:[/h2]
- Removed the Patreon badge gap for now, until we find a better solution (requested by @Karel | CEO based on feedback)
-- Currently this technically breaks some of the guidelines, but should look visually better. We're looking into some alternate solutions, but those will take a bit longer

- Merged Russian locale fixes by @Shadow Panther [RU/EN, UTC+3]
- Merged Japanese locale tweaks by @Aesc

[h2]Bugfixes:[/h2]
- Fixed internal check for value and reference types when setting up drives checking against the type of the field, rather than the value/target
- Fixed implicit color->float4 and float4->color casts not being generated in some cases
- Fixed audio clips imported from URL throwing an exception and not importing properly (found in a log from @ProbablePrime | Docs)
- Increased maximum nested depth when loading saved items and worlds, to fix cases of certain items with very deep hierarchies not being able to be loaded (reported by @Shadow Panther [RU/EN, UTC+3])
- Added extra logging for gather jobs to help diagnose gather jobs being stuck for some users
- Fixed LogiX tip spawning a node when holding an item with laser and double pressing trigger to switch to the freeform rotation mode (reported by @Snooper and Patreon Priority Voting)
- Removed collider from temporary LogiX wire when connecting nodes to prevent it from interfering with the laser (@Ukilop, @H3BO3 and @3x1t_5tyl3)

Hello everyone and welcome to another of our weekly updates!

With the first phase of desktop complete, we have focused on clearing up a bunch of smaller additions, tweaks and bugfixes. The desktop Freeform camera now automatically outputs your voice when applicable and we added more notes for accessing the user's viewpoint and camera state.

We added support for the HP Reverb G2 Omnicept Edition eye tracking, if you have this headset, your gaze, eye openness and pupils should now be mapped to your avatar out of the box! More of our native dependencies have been moved to Azure Pipelines and updated to latest version too, notably Opus, improving voice and music compression quality and Freetype, fixing bugs and adding WOFF font format support.

For some quality of life improvements, LogiX itself received new nodes, particularly for bitwise masks working with vectors and the Material tooltip can now batch convert materials to another type. We have also fixed numerous bugs, crashes and made security improvements, details are in the post below!

And last, but not least, we did some significant optimizations to the session thumbnail system, reducing CPU, memory and network bandwidth usage, as well as making more related optimizations for UIX panels, which overall improve smoothness of the experience and reduce a lot of cases of microstutters.



[h2]Games of Neos livestream[/h2]
On our last regular Friday livestream, we went back to a bunch of the fun worlds created by our community and played a bunch of games! If you missed it, check out the footage below, as we drive around some race tracks, figure out who the murderer is in MurderX, talk to ghosts to figure out meme pictures and more!

[previewyoutube][/previewyoutube]

[h2]HP Omnicept Eye Tracking Support[/h2]
With the release of the HP Reverb G2 Omnicept Edition headset, we have integrated the SDK to provide full eye tracking support for anyone owning this headset. This includes the eye direction, eye openness (closing your eyelids) and pupil size.

Thanks to our generalized input system, there’s nothing extra that you need to do on your end! If you own the headset and have the Omnicept runtime installed, simply starting Neos and using any avatars already setup with eyes will work out of the box, just like with the Vive Pro Eye headset.

At the moment there doesn’t seem to be an API available for the lip tracking functionality, once that is made available to developers we’ll be happy to integrate it as well, as providing more options for our users for expression is an important aspect of Neos.

[h2]Voice output for Freeform Camera and more desktop polish[/h2]
We’ve also made some last additions and polish to our new desktop mode. Most notably, the Freeform Camera will now automatically output your voice for other users if they’re sufficiently far away from your actual avatar and the camera is closer, making it easier to communicate.

New LogiX nodes for accessing various desktop information were added as well, allowing you to find out where the actual user’s viewpoint is, whether their Freecam is active and whether it’s voice is currently active too.

If you have a customized view visual, you will need to set up the AudioOutput and AvatarAudioOutputManager as you’d on the rest of your avatar, but check the “IsViewVoice” field on the latter component, so it gets activated properly.

Also note that Whisper mode won’t work in the Freecam mode to avoid some weirdness (both your avatar and camera can be potentially heard), you’ll need to switch back to first or third person for this mode.

[h2]Updated Opus (audio encoding) and Freetype (WOFF font support)[/h2]
Thanks to our ongoing transition to Azure Pipelines, we were able to upgrade more of our native dependencies to their latest versions easily. Among the libraries that we moved were Opus and Freetype, responsible for audio encoding and font decoding respectively.

We have upgraded the Opus library to latest 1.3.1 from 1.1.3, which includes several years worth of improvements and bug fixes and should help improve the audio quality, especially at lower bitrates. This library is used to encode both your voice over network, as well as any desktop audio with our audio streaming feature.

Freetype library is used internally for decoding font files for the text rendering system and has been updated to 2.10.4 from 2.10.0. This mainly includes bug fixes, including an important security patch. However we also now added support for the WOFF font format support, alongside TTF and OTF.

Currently WOFF2 isn’t supported yet, despite the Freetype library supporting it, because it requires additional dependencies, but we’d like to eventually add support for this as well.

[h2]New LogiX operator nodes[/h2]
Based on a few requests, we have expanded the LogiX nodes to include new operators and overloads as well to make it easier to work with vectors and bitmasks in particular. Boolean vectors (bool2, bool3 and bool4) now work with the bitwise logical operators as well as bit-shifting and bit-rotating nodes.

Integer vectors like int2, int3, int4, long3, long4 and so on now support bitwise operations as well. Any vectors can also be used with comparison operators, producing boolean vectors. E.g. comparing two float3 values will result in bool3 with each element of the vector being compared individually.

[h2]Batch Material Conversion[/h2]
As a quality of life improvement, we added the ability to batch convert materials in the hierarchy to another type. Previously you could use the Material Tooltip to convert a single material, but oftentimes users would need to convert multiple materials in the scene or object/avatar.

In the latest version of Neos, you can simply grab a slot reference in the inspector with the Material Tooltip and you’ll see a new option in the context menu “Convert All To…”. Running this will convert all the materials within that hierarchy to a particular type.

As another quality of life, we also made the extraction process for all the materials in the hierarchy undoable, so you don’t have to manually delete all the orbs if you do it by mistake.

[previewyoutube][/previewyoutube]

[h2]Mipmap Generation Control[/h2]
Another quality of life improvement is now the ability to control mipmap generation for 2D textures. You can turn the mipmaps completely off for textures that you know won’t require them, such as skyboxes, saving some VRAM usage in the process.

For textures that do need mipmaps, you can now choose which rescaling algorithm will be used to generate them. You can currently pick between Bilinear, Box (default) and Lanczos3. The last filter in particular can be useful for certain textures and images, as it will produce sharper visuals at a distance, particularly with contrasting detail.

[previewyoutube][/previewyoutube]

[h2]Updated Account / OAuth 2.0 website[/h2]
The work in progress account and OAuth 2.0 website has received a small lick of paint courtesy of our new team member ProbablePrime and has been moved to a nicer domain https://auth.neos.com by Karel, in order to give it a more official look.

Alongside this we have also moved the main API endpoint to api.neos.com to help unify things a bit more, so if you have any 3rd party applications, make sure to update! The old endpoint will continue to function for a while, but will drop at some point in the future.



[h2]Optimized UIX and Thumbnail System[/h2]
While investigating random hitches and stutters for the upcoming MTC Streamer room, we found a few underlying issues with the session thumbnail system and UIX and some related subsystems.

In the latest build we have implemented several different optimizations which overall should help reduce CPU, memory and network usage and provide smoother experience. Parts of the thumbnail system were removed to avoid overloading the system resources, particularly when just starting the session.

Headless servers will now also avoid compressing and reuploading thumbnails taken from other users or the default world thumbnail and any thumbnails are also registered with local cache, to avoid cases of having to redownload different instances of the same image or even own uploaded thumbnail in some cases.

We discovered that even disabled UIX canvases were being computed in the background, resulting in unnecessary CPU and memory usage. In the latest build, the UIX Canvas will defer any computations until it’s activated again. This way any inactive dash screens, closed dash, context menu or locally hidden/culled interfaces in the world will have minimized performance impact.

A few other smaller optimizations include directly loaded textures (thumbnails being a prime example) skipping metadata computation since it’s not necessary and any thumbnail updates in the UI being deferred until it’s activated as well.

We’re already getting reports of reduced microstutters and higher framerates on the latest build, so we’re happy that those changes are being felt despite being relatively small!

[h2]Fixed text layout glitches and flickering[/h2]
Among many smaller bug fixes, one of the notable ones (if only for the few hours it took to narrow down) is the auto-size layout for the text rendering system. This has been actually caused by two major underlying issues, which have both been corrected.

Those combined caused auto-sized text to randomly spaz out and glitch when being changed and some words to get forcefully wrapped to multiple lines. With the bugfix in place, the text rendering system should now be more robust and produce more reliable results.

[previewyoutube][/previewyoutube]

[h2]Favorite avatar security improvements[/h2]
To improve general security of your inventory and particularly favorite avatars, we have reworked the mechanism by which your favorite avatar is loaded in worlds. Instead of the record being marked public and potentially accessible by anyone who knows its exact ID, a temporary one-time key is generated instead just prior to spawning.

While this doesn’t completely eliminate the possibility of ripping (that’s unfortunately not technically possible to prevent completely), it should significantly mitigate the possibility of one type of attack and provide better security hygiene by not marking avatar records as public.

We’ll be automatically unflagging those avatars as public soon as the new system proliferates, unless they are present in a public folder.

[h2]Various crash bugfixes & better Unicode handling[/h2]
Other notable bugfixes from last week include numerous security improvements and some sources of crashes and freezes. A regression in network encoding for a few less commonly used types was patched, which resulted in worlds crashing on use.

Another notable fix is also for pasting invalid Unicode strings from the clipboard, for example when deleting half of Unicode character surrogate for certain symbols like 🔆. Previously doing this would freeze Neos until restart, although it would only impact the user executing this action and couldn’t be used for an attack.

In tandem with this, we improved the handling of Unicode strings for the text editing in Neos, which is now aware of those surrogate pairs and will treat them as a single symbol, rather than two independent ones.

[h2]bobool3ol[/h2]

No two bobool3ol's are the same. They're always 7.

[h2]What’s next[/h2]
With a bunch of smaller tasks, improvements and bug fixes out of the way and the first phase of desktop support nearly complete, I’ll be now taking a roughly week long break as mentioned in the last weekly update, to reset and refresh before tackling next major development tasks.

This unfortunately means that for the following week there will be no Neos updates, responses to questions and issues (from me anyway) and no weekly update next week, unless there’s an emergency. You’ll still be able to reach other team members in case you need assistance.

I hope that everyone will have a great week regardless. And as usual, big thanks to everyone for your support! Without you we wouldn’t be able to keep improving Neos every day! We’ll see you soon, with more big tasks and features to tackle!

You can find what's planned at our GitHub roadmaps and watch the latest developments on our official Discord server.