Neos VR News

Hello, sorry for a smaller build this time, I just want to push out some fixes and tweaks rather than waiting on some bigger things to be finished first, particularly some regressions for the facial tracking blendshape auto-detection.

There's a few other smaller additions too, for example you can adjust the volume of people outside of your whisper bubble now.

Some smaller improvements and progress to the new desktop mode as well, with key bindings and legacy dialogs now working properly. I'll have more soon!

[h2]New Features:[/h2]
- Added "Whisper Voice Volume (outside bubble)" setting, which allows configuring the voice volume of users outside the whisper bubble (requested by @Robyn (QueenHidi))
- Added "StrengthMultiplier" to the AvatarRawMouthData, allowing scaling the strength of all the properties

- Added TongueRaise and TongueExtend expressions to the AvatarExpressionManager
-- Those are useful when there are two separate blendshapes for raising the tongue and then extending it out
-- The heuristics was updated to map the TongueLongStep1 and TongueLongStep2 to these two, resulting in better behavior for models following the HTC Facial Tracker naming convention

- AvatarExpressionManager heuristics now checks for the exact expression naming of the blendshapes first, before using heuristics (based on feedback by @GearBell)
-- Note that for this to work, you must name the blendshapes exactly the same as the expression names you see in the inspector, without any extra spaces, periods or underscores. It is however case insensitive

[h2]Tweaks:[/h2]
- Keyboard bindings that use Ctrl, Shift and Alt keys will now also accept the right Ctrl/Shift/Alt keys (based on requested by @Psychpsyo)
- Legacy dialogs (e.g. Hyperlink warning, Create New Directory, Credits Transfer and so on) will now open in the overlay in the new desktop mode so they can be easily interacted with
-- Note that you cannot grab and move them around, only click to interact. This might not be implemented before they're replaced with more modern variants
- Any focused elements (e.g. text fields) will now block keyboard interactions in userspace
-- This fixes the dash opening/closing when using keyboard to type (based on report by @Shadow Panther [RU/EN, UTC+3])
- Text fields will now unfocus when the Escape key is pressed

- Merged Russian locale additions (including MTC Avatar Lobby) by @Shadow Panther [RU/EN, UTC+3]
- Merged Japanese locale addition by @Aesc
- Merged Czech locale addition by @rampa_3 (UTC +1, DST UTC +2)
- Merged Korean locale addition by @MirPASEC

[h2]Bugfixes:[/h2]
- Fixed regressions in AvatarExpressionDriver, BipedRig and other naming heuristics, causing some blendshapes/bones and/or their chirality to not be detected properly
-- This fixes the HTC blendshape naming convention not automatically setting up all the names anymore
-- If you setup your avatar recently, it's recommended to rerun the auto-assignment
- Fixed headless not measuring auto restart and auto save intervals properly, using a mix of UTC and local time, resulting in incorrect intervals (reported by @Shadow Panther [RU/EN, UTC+3])
- Fixed regression where it wasn't possible to interact with touchables with the same hand that's holding an object with the laser
-- This fixes not being able to invoke the Overwrite menu for world orbs
-- It also fixes not being able to assign references/values to the inspector fields by clicking (reported by @Turk)
- Fixed being able to use worlds that you don't have write permissions for to overwrite your own worlds with (reported by @sirkitree)
- Fixed Unity Native video playback returning audio data when paused
-- This should fix VolumeMeter reporting volume when the source video is paused (reported by @Toxic_Cookie)
-- It should also fix AudioSourceWaveformMesh showing undesirable visuals (reported by @AshtonSparx)

A whole bunch of new goodies for the new desktop mode! You can now rotate items (GMod style!), scale them, scale yourself, take in-game photos (even with timer), save items to inventory and quick switch common tooltips. I've also changed a bunch of the existing bindings too (e.g. the Dash is now opened/closed simply by pressing Esc) and there's a whole bunch of other tweaks, improvements and bugfixes to make the new desktop mode more usable.

Currently it is still usable only by pressing F8 after launching in VR, but I'll be swapping it out within next few days, after adding a few more crucial things. It's coming along!

Also added a bunch of bugfixes to this one (including broken Reflection Probes on Linux, which turned out to be just a stupid typo in a flag when uploading HDR textures to GPU, sorry!) and some regressions (sorry for the inconvenience!). More to come soon!

[h2]New Features:[/h2]
- Added "Save To Inventory" context menu option, which will save the held object to the current inventory folder
-- Note that you currently need to open inventory folder where you have write permissions first. This will be simplified with the upcoming inventory UI redesign
-- This also allows easily saving items in the new desktop mode

- Added an internal mechanism to load-equip and stash tooltips from an URL
-- Tooltips and other equippable items are now internally tagged as equippable when saved to inventory, which will allow future UI to allow one-click equip directly from inventory
-- When switching tooltips, if the current one is quick loaded, it will be "stashed" - disabled and hidden, preserving its state on the next equip in the same session
- You can now quick switch tooltips in the new desktop mode via tha alphanumeric keys. Bindings are following (the loadout will be customizable in the future):
-- 1 - Dequip
-- 2 - Developer Tip
-- 3 - LogiX Tip
-- 4 - Material Tip
-- 5 - Shape Tip
-- 6 - Light Tip
-- 7 - Grabbable Setter Tip
-- 8 - Character Collider Setter Tip
-- 9 - Microphone
-- 0 - Glue Tip

- You can now capture in-game photos in the new desktop mode
-- To capture instant photo, press Shift+P or Shift+PrintScreen
-- To start timer photo, press Ctrl+P or Ctrl+PrintScreen
-- The capture won't start until you release the combination, allowing you to hold it to see a preview

- You can now rotate held items in the new desktop mode
-- Hold the E key while holding an item and move the mouse around to rotate object around X and Y axes
-- Hold the Shift+E key to rotate only around the vertical axis
- You can now scale held items in the new desktop mode
-- Hold the Shift key while holding an item and use the mouse scroll wheel to scale it

- Added a mechanism to block inputs within InputGroup by specifying a priority level for each binding, simplifying setup of common control schemes (e.g. Ctrl+, Shift+ and as modifiers)
- Reworked FingerPhotoGesture into a more general PhotoCaptureManager

[h2]Tweaks:[/h2]
- Dash is opened/closed by the Escape key now in the new desktop mode
- The locomotion switch and scaling options are now shown on the context menu even with equipped tooltip
- Keyboard & Mouse input is now fully supressed while the dash is open in the desktop mode
- Lowered the limit of how close you can bring grabbed items to your viewpoint from 10 cm to 5 cm in the new desktop mode
- Increased vertical mouse look angle limit to 89 degrees from 80 to allow looking near straight up and straight down
- You can now use PageUp and PageDown to quickly cycle between active locomotion modules
-- This will have a visual at some later point, not too usable at this point just yet, but it's been added as part of larger work
- You can now self-scale in new desktop mode by holding Ctrl and using the Mouse Scroll button
-- Scaling still needs to be enabled first in the context menu
- Tweaked finger photo gesture supression level, to be more tolerant of bent fingers
- Developer Tooltip laser now considers Gizmo controls as preferred targets and will pass through other objects and stick to them (based on report by @H3BO3)
- Undo & Redo context menu options are not generated when holding items with the active hand

- Merged Japanese locale additions for the new MTC Avatar Lobby by @Aesc
- Merged Czech locale additions for the new MTC Avatar Lobby by @rampa_3 (UTC +1, DST UTC +2)
- Merged Korean locale tweaks by @MirPASEC

[h2]Bugfixes:[/h2]
- Context menu positioning in the new desktop mode now properly handles user's scale
- Interaction Laser now consider user's current scale when ignoring near raycast hits
-- This fixes the context menu not being usable when the user scale is too small in the new desktop mode
- Fixed SyncType inspector field not accepting any type strings after recent changes (reported by @3x1t_5tyl3)
- Fixed incorrect flags being set for BC6H textures when uploading them to the GPU through OpenGL, resulting in the texture data to be interpreted incorrectly on Linux and Android
-- This fixes ReflectionProbes being overly bright on Linux (@Shifty | Quality Control Lead, @happysmash27, @Polychrome, Silvea12, Enverex and @Develon)
- Fixed favorite avatar, keyboard, camera and home being unfavorited on manual log out (reported by @Flame Soulis and @H3BO3)
- Fixed gamepad mouse look being framerate dependent (reported by @Lewis Snow | Lead Audio Engineer)
- Fixed regression breaking the context menu flick functionality (reported by @Flame Soulis, @DeliriousJax, @ohzee, @H3BO3 and @Turk)
- Fixed regression when aligning laser-held objects to the vertical axis having a small offset from the aligned position, depending on the hand movement speed (reported by @orange)

An extra hotfix to fix some more issues that were found. Compatible with previous build, but update recommended for headless, since it mainly concerns bugs manifesting on those.

[h2]Tweaks:[/h2]
- Merged Korean locale translations by @MirPASEC (including new MTC Avatar Lobby)

[h2]Bugfixes:[/h2]
- Fixed Neos trying to use VR bindings against VirtualController instead of VR one in some cases, causing the user's session to crash
-- This should fix some session crashes when joining worlds or switching locomotion in desktop, particularly with Oculus runtime (reported @epicEaston197, @Shifty | Quality Control Lead, @/home/JellyOsaurusPC)
- Fixed exceptions due to updating InputGroups for owners that have been removed in the input update cycle
- Fixed grabber transform computations using local user's space, instead of the user actually owning the grabber
-- This also fixes exceptions on the headless for the CommonTool, due to not having a user root
- Fixed VRIKAvatar trying to calculate distance to user's head when the head position is not available (e.g. during respawn or on headless)
-- This fixes IK avatars stopping to update when the user exits them on a headless session

Hello everyone, sorry it took a while to get this build ready, but it presents a major milestone in the develpment of the new desktop mode! You can now fully use Context Menus and Equippable items (Tools, Weapons, Gadgets...), making the mode fully usable for core interactions! With those in place, I'll start progress on making this the default desktop mode now. I'll keep the old one as legacy for a bit while it still gets more polish. Right now you still need to startup in VR and press F8 to activate it, but hopefully it'll prove very useful already!

There are some semi-major things that changed/expanded in this internally, the input handling has been restructures quite a bit, which reduced latency of some input handling by a frame (making things feel more responsive), but watch out for potential side effects. Thanks to the following danger testers for catching some game breaking bugs before this went live: @/home/JellyOsaurusPC, @Kulza, @Raith (CytraX), @Alex the pet peeve avali 🐦, @Epsilion, @Gawdl3y and @Shadow Panther [RU/EN, UTC+3]

This build has a whole bunch of tweaks, security improvements and fixes too, so hopefully that'll solve some of the pains and problems that you've been dealing with, sorry that those fixes got held up a while! I hope to have more ready soon, probably tomorrow. I'll probably have to release the weekly update tomorrow too instead of today, it took a bit longer than expected to get this ready and not sure how much longer I'll be able to stay up and focus.

But in the meanwhile, have fun! There's a lot more to come to desktop, but with this we should have all the necessary basics.

[h2]New Features:[/h2]
- Context Menus are now usable in the new desktop mode
-- Press the Middle mouse button or the "Y" button on gamepad to open/close them
-- They will also open when clicking on certain objects (e.g. Avatars or Tooltips/Equippable items)
-- When Context menu is open, the cursor will switch to a freeform one, allowing to easily select and click on the items without having to aim head at them

- Tools, Gadges and other equippable items are now usable in the new desktop mode
-- When equipped, the avatar's hand is positioned relative to the head so the item is visible in the first person mode
--- The positioning is based on the grip pose. If it's not setup properly, the item will be misaligned
-- The tool/item/gadget is automatically aimed at the point in world currently under the cursor reticle
--- For this to work correctly, the item needs to have a properly setup TipReference, that corresponds to the exact tip and direction that the tool should aim at (using Z axis as forward)
--- Existing RawDataTooltip items are auto-upgraded using heuristics, using sources of raycast or particles. You might need to adjust the TipReference on some manually if the detection doesn't work properly
-- Current bindings are (these are tentative and will likely change)
--- Primary - Left Mouse Button / Right Trigger (gamepad)
--- Secondary - R key / Mouse Button 4 (is present) / X button (gamepad)

- RawDataTooltip & StandardController now provide proper inputs in the desktop mode, making items that rely on those work
-- The values are fed through a VirtualController type from the current input bindings of corresponding actions

- Pointer interaction system now routes the first and second pointer (mouse cursor and/or touch) through the hand interaction system
-- This results in the intereractions posing the avatar's primary and secondary hand and being able to grab and scale with toushcreen
- Neos now renders its own cursor even in freeform cursor mode to provide visual consistency and contextual behavior and avoid the system cursor to be overlaid on top of the in-game reticle
- InputDeviceStreamDriver now disables streams for input devices that are currently inactive, saving network bandwidth
-- E.g. VR controllers when Screen is active and VirtualControllers when VR is active

- You can now self-assign the hearing impaired, color blind, mute and potato badges
-- To assign or remove, simply send following commands to the Neos bot account in your Contact list:
-- /addHearingImpaired, /removeHearingImpaired, /addColorBlind, /removeColorBlind, /addMute, /removeMute, /addPotato, /removePotato

[h2]Tweaks:[/h2]
- Restructured update order of the input handling components, avatar posers, VRIK and CommonTool
-- This reduces latency of input response by a frame (e.g. making the laser/cursor respond in the same frame that the input happened, rather than lagging a frame later)
-- It also allows for automatic interactions based on a laser hit (e.g. based on a screen touch that happens instantly)
- Desktop mode grabbing now uses user's head forward direction as rotation reference instead of the hand
-- This reduces slight rotation of the grabbed object upon grabbing due to the hand being procedurally moved
- Cleaned up and tweaked several user related components for driving the root objects based on inputs, tracking compensation and so on
-- Many of the components now implicitly work based on the user they're initialized on, rather than explicitly assigned
-- Where possible, the components only register events for the user actually using them, rather than for everyone
- Improved parsing of URL's when classifying asset types to search the URL query for a file extension in addition to the main data path
-- This makes certain types of URL to get detected properly and imported directly from the clipboard (e.g. IPFS file URL's, as reported by @Komdog)
-- E.g. https://example.com/somepath?file=MyFile.mp4&someparam=42 will now properly import as a video
- Near head hand gestures/shortcuts are no longer triggered in the desktop mode
- Saving world with Ctrl+S is now supressed when the new desktop mode is active (based on report by @Cyro)
- Crouch is now on the Left Bumper on gamepad

- Added new English MTC Avatar Lobby locale strings by @Ryuvi | Technical Artist and @Aegis_Wolf | Art Director

- Merged Estonian locale additions and fixes by @Meron
- Merged Russian locale addition & tweak by @Shadow Panther [RU/EN, UTC+3]
- Merged Korean locale addition by @MirPASEC
- Merged Japanese locale addition by @Aesc
- Merged Esperanto and Chinese locale additions and tweaks by @Melnus
- Merged Czech locale addition by @rampa_3 (UTC +1, DST UTC +2)
- Merged English store page fixes by @rampa_3 (UTC +1, DST UTC +2) and Enverex (will take a bit before it's pushed to store)
- Merged British English addition by Enverex

[h2]Security:[/h2]
- Fixed SimpleAvatarProtection throwing an exception when checked against user who has been destroyed (based on report by @3x1t_5tyl3 and @marsmaantje)
- Fixed CommonAvatarBuilder not cleaning up user's default avatar if it fails to be equipped for any reason (based on report by @3x1t_5tyl3 and @marsmaantje)
- Improved security of touch sources (based on report & sample by @Psychpsyo and @Epsilion)
- Improved sanitization of held items being transferred through the userspace (based on report & sample by @Psychpsyo and @Epsilion)
- Reworked mechanism for cleaning up Userspace-only components, so placing them on protected Slots (e.g. Root slot) doesn't cause exceptions and failure to clean them up (based on report by @Psychpsyo)
- Fixed usernames not being sanitized in the World UI and on the world orbs, causing any RTF tags in the username to be parsed (based on report by @Kodey and @Psychpsyo)

[h2]Bugfixes:[/h2]
- Fixed Neos locking the cursor when running in the VR mode (reported by @Hayden)
- Fixed headless running some actions that should only be ran by user under who is given behavior parented
- Fixed BeforeInputUpdate event being potentially executed twice on some components
- Fixed SmirkLeft and SmirkRight expressions breaking the AvatarExpressionDriver when activated (reported by @Alex the pet peeve avali 🐦)
- Fixed ContextMenu collider being enabled until it's first used
- Fixed some odd interactions when using the new desktop mode with the old one
- Fixed ScreenController update order being same as TrackedDevicePositioner, causing the simulated hand to freak out in some cases (where it ends up initialized after the TrackedDevicePositioner)
- Fixed ValueField and other generic components that handle value types allowing certain non-value types (e.g. System.Type) and causing the session to crash (reported by @dfgHiatus, @Epsilion, @Psychpsyo and @Zyzyl)
- Fixed component exception handling misbehaving when placed on a destroy-protected slot (based on feedback by @Psychpsyo)
- Fixed Neos crashing when running on devices without a mouse present (e.g. Android) (based on report by @かず (kazu / GitHub: kazu0617))
- Fixed reliable user positioning breaking and throwing exception when the user is deleted in the middle of the process, causing other processes to break (based on report by @3x1t_5tyl3 and @marsmaantje)
- Added generic type validation to more components to prevent crashes when creating them with invalid argument (based on findings by @Zyzyl and @Banane9)
- Fixed some World event behaviors (e.g. focus change, on save, user join or leave) breaking due to not setting up proper update scope (e.g. when fields are driven)
- Added Type validation for setting SyncType and related operations, to avoid setting/processing of invalid types (e.g. type with invalid generic argument) and causing the session to crash (reported by @Psychpsyo)
- Fixed LipMoveLeftLower expression on AvatarExpressionDriver driving the target with movement to right, instead of left (reported by @umbran)
- Fixed nameplates using the IsPresentInHeadset state instead of IsPresent, causing them to become semi-transparent when the user switches to the desktop mode after existing their headset (reported by @Shifty | Quality Control Lead)
- Fixed Userspace having two instances of PointerInteractionController, resulting in cursor and touch interactions in userspace being processed twice
Known Issues:
- When interacting with context menu on gamepad, the camera view will move if you don't let go before the menu disappears
- Some tooltips are hard to use without additional behaviors (e.g. LogiX tooltip)
- Currently key bindings are tenative and will likely be tweaked as more behaviors are added

Hello everyone! I wanted to do a bunch more work on the new desktop before releasing this build yet, but there's an important bugfix that heavily improved the cloud API reliability (this should fix sync errors and other problems due to interminnent network problems) as well as some important improvements for the face tracking!

The heuristics got expanded and improved, adding support for more avatars. Autodesk Character Creator avatars should now get setup with lip tracking (not 100% support either, but a bunch of blendshapes will be auto-mapped). Also there's now a mechanism where the voice data supresses certain blendshapes to avoid overdriving the mouth when talking with the Vive Facial Tracker.

There's a bunch of underlying work on the interaction system for the new desktop mode as well. Not that much visible yet, but touchscreen now has official initial support! Still more work to do, but you can play with it a bit if you'd like! I've got more coming soon!

[h2]New Features:[/h2]
- Added Timestamp to AvatarRawEyeData, which is relative time in seconds of the current snapshot of the eye tracking data (requested by @SHFR_H)
-- This allows calculating things like the current angular velocity of the eye movement with much better accuracy
- Added missing "Squeeze" and "Frown" properties to AvatarRawEyeData (based on report by @Ryuvi | Technical Artist)
-- Note that it's strongly recommended to use EyeManager, rather than this component unless you're building specialized application, as EyeManager is significantly more efficient and modular

- Added "Tongue Up Left", "Tongue Up Right", "Tongue Down Left" and "Tongue Down Right" expressions to the AvatarExpressionDriver (based on request by @umbran)
-- These drive based on the combination of both directions. Typically used for corrective deformation blendshapes
-- The heuristics has been updated as well to auto-assign those blendshapes (you will need to re-run the heuristics to assign missing blendshapes on already setup models)
- Added SmileClosed(Left/Right) expression to AvatarExpression driver
-- This will drive a smile that's only active when the lips are covering the teeth and will go to 0 when the teeth are visible
-- This can be used when there are separate smiling blendshapes for when the teeth are visible and when not
- Added VolumeSource and SilenceSource to AvatarExpressionDriver, which allow supressing the face expressions when the user is talking
-- VolumeSource is checked first, then SilenceSource (inverse of the first) when calculating supression
-- Each Expression now has "VolumeSupressionStrength", indicating how much is given expression supressed by volume (1.0 full supression, 0.0 no supression)
-- Existing instances are auto-initialized using VisemeAnalyzer on the avatar default supression weights are setup
-- By default, only jaw open and lips raising and pouting expressions are supressed. This should prevent the mouth opening too much when talking

- Added initial multi-touch support - the new desktop mode now properly handles touch screen inputs and multiple pointers for interacting with the UI
-- While touch inputs are active, the mouse input is supressed (this is to avoid double interactions due to Windows simulating mouse inputs from the primary touch)
-- Note that it doesn't go through the laser interaction system yet, there's more work that needs to be done on this
- You can now adjust the FOV (field of view) of the new desktop mode in the settings

[h2]Tweaks:[/h2]
- The FOV increases slightly in new desktop mode when running with the physical locomotion
- Extended the heuristics of AvatarExpressionDriver to provide better support for automatically assigning Autodesk Character Creator models
-- Note that you might need to tweak strengh of some of the blendshapes, like smiling
- Improved blendshape/bone name splitting to support naming conventions that start with a chirality letter (e.g. "RlipDown", which would previously split to "Rlip" and "Down")
-- This heaivly improves auto-detection of blendshapes on the Autodesk Character Creator models
- AvatarExpressionDriver no longer needs "shape" name for the "Ape Shape" (Jaw Down) name heuristic (based on model by @GearBell)
-- E.g. "mouth_ape" is now sufficient for the blendshape to be mapped, instead of needing "mouth_ape_shape"
- Preset names when starting worlds on the headless are not case-insensitive (based on report by @Epsilion)

- Merged Korean locale additions by @MirPASEC
- Merged Russian locale additions and tweaks by @Shadow Panther [RU/EN, UTC+3]
- Merged Japanese, Esperanto and Chinese locale additions by @Melnus
- Merged Czech locale additions by @rampa_3 (UTC +1, DST UTC +2)

[h2]Bugfixes:[/h2]
- Fixed Neos not retrying Cloud API requests when the server returns 429 or 500 codes
-- This should greatly increase the robustness of any cloud based functionality under load or interminnent network errors, fixing cases where different processes would get stuck
-- This should also significantly reduce number of sync errors due to network issues (e.g. connection briefly dropping) as reported by @PlasmaRaven, @Elektrospy, @Honeypotbutt and @Shifty | Quality Control Lead
- Added a mechanism to manually correct Patreon support for accounts where the amount of support stopped reporting correctly from Patreon API
-- If your Patreon account got messed up and you're no longer receiving rewards, even though the charges are going through, let us know and we'll update your account manually
- Fixed GrinLeft using the right side of the upper lip to modulate
- Fixed invite and restart headless commands throwing exceptions when no world is focused (reported by @Glitch)
- Fixed non-host users showing "Hidden Contacts Only World" as "Contacts Only World" (reported by @Shifty | Quality Control Lead and @Cyro)
- Fixed Neos adding invalid entry to the configuration file when a world is started with invalid template and then configuration is saved (reported by @Epsilion)