2021.1.17.998 - View/listener transform override, cloud optimizations and more
With the work starting to ramp up on the desktop mode, here are some first related fruits implemented thanks to underlying architectural changes and additions, particularly ability to override rendering root, view and ears (listener) transforms!
There are two main features built on top of this right now - one for avatars, giving you full control to create custom setups, override where the user "sees" and "hears" from or putting them out of the actual avatar body and easy ability to render all audio from viewpoint of the streamer camera, making the footage audio make more sense for viewers for any spatialized sources.
Another major thing that's more internal are some cloud infrastructure reorganizations and optimizations, mainly to reduce some inefficiencies to save costs and improve performance and scalability. There's some more to come as well as I continue to monitor the impact of the changes.
A bunch of other smaller additions, tweaks and bugfixes as well. There's bits of the proper desktop mode too, nothing usable yet though, but it's starting to take form. More to come soon!
[h2]New Features:[/h2]
- Added a mechanism to override the transform of user's root, view and "ears" for audio-visual output, while preserving their actual UserRoot in the world by assigning Slot references on the UserRoot component
-- Note that view should be used sparingly for VR users (if at all) as it will negate any actual head movement which will be dizzying to a lot of users. If you want to create "out of body" experience, override the root instead.
-- The view override can be put on the avatar's head so the view still moves with the head, but using the avatar's motions, but be careful with this as well, as the motion not being 1:1 can cause nausea as well. I STRONGLY discourage using this as standard feature of avatars for adjusting the view (even if subtle) and only recommend it for gimmick avatars and personal use
- Added AvatarUserRootOverrideAssigner (under Users/Common Avatar System) which allows overriding the root, view and ears from equipped avatar (requested by @GearBell, @Abysmal, @ProbablePrime, @Shifty | Quality Control Lead and @Hayden)
-- The component updates in realtime. You can set the Override target to null to disable or change it to a target object to dynamically assign a new target
- Added "Audio from camera viewpoint" setting to Interactive Camera dialog, which will render the audio from the camera's viewpoint (requested by @Nexulan | Community Manager and many users before (sorry I didn't keep a list! ;_;))
-- This can be useful for streams for spatialized audio to make it less confusing for viewers, at cost of making it more confusing for the streamer
- Added LookLeft, LookUp, LookRight and LookDown to EyeLinearDriver, which allows driving 0...1 directional values (e.g. blendshapes) based on where the eye is currently looking (based on request by @Groxxy the Eye-Puppeteer)
-- This can be used to drive extra blendshapes that morph the eye when it looks in particular directions
-- The response of the values can be controlled using the LookMultiply and LookPower fields
- Added "Extract Sides" audio processing option to the inspector, which removes the center channel from stereo/quad tracks
-- This can be used to remove vocals from music tracks which are typically centered in mono
- Added "Prefer Specular" to advanced model import settings, which will prefer the Specular variants of PBS materials when importing (requested by @Robyn (QueenHidi))
- Added Rect text input node (x, y, width, height)
-- This now also makes it possible to extract driver node or direct inputs for Rect values (based on report by @chemicalcrux)
[h2]Work-in-progress features:[/h2]
- Expanded Neos' input/output system to support two toggleable rendering outputs - VR and Screen, using "VR_Active" state that can be changed at any time (for testing purposes it's currently bound to the F8 key, but it's not in usable state yet)
- Unified Neos' head output system to remove duplicates and remove any deprecated versions to simplify the code and management
- Added "VR Active" state to user and corresponding node, which determines if the user is currently in VR mode or screen mode
-- Note that the legacy screen mode is treated as "VR" mode. This info will be useful as the proper desktop mode is fully implemented
[h2]Tweaks:[/h2]
- Redesigned part of cloud storage system architecture to utilize multiple differently configured storage services to reduce cost and optimize performance
-- Any operational data (file upload chunks, asset variant computations, thumbnails, status updates and related queue/table objects and their work items, generally anything that's ok if it's lost) is now using low-redundancy, low-latency and low-operation cost (backed by SSD's) storage systems to prevent extra costs with geo-redundant storage
-- Any permanently saved data (assets, records) are still using geo-redundant storage, to ensure durability even in case of data center disasters
- Reworked asset restore mechanism to run only at a single worker at the time and less frequently to avoid unecessary load on the cloud resources
- Removed obsolete session thumbnail extension mechanism, reducing some networking / Cloud API load
- When "Setup IK" in advanced import is disabled, the biped detection report will no longer be generated (based on report by @H3BO3 and @chemicalcrux)
- Upgraded CloudX dependencies and related libraries
- Upgraded to Unity 2019.4.18f1 (from 2019.4.17f1)
- Merged Esperanto, Japanese and Chinese locale updates by @Melnus
- Merged Korean locale update by @LUA
- Merged French locale update by @brodokk
- Merged Czech locale update by @rampa_3 (UTC +1, DST UTC +2)
- Merged Dutch locale update by @AnotherFoxGuy
- Merged Russian locale update by @Shadow Panther [RU/EN, UTC+3]
[h2]Bugfixes:[/h2]
- Improved Inventory and File Browser interaction security (based on report by @Komdog)
- Fixed cloud asset variant processing not deleting associated queue data blobs, resulting in increased costs
- Fixed errors in cloud asset dependency processing, preventing asset dependencies from being removed in cases where even after the removal, the user would still be over their quota limit, causing the storage usage to get stuck and resulting in delays in cloud processing (based on report by @DJNightmares)
- Added generic argument validation to ValueTag and ValueFieldProxy components (based on discovery by @Cyro)
- Added guard to exceptions in OnWorldSaved and OnFocusChanged world events, to prevent errors in those from crashing the entire world (based on report by @Epsilion)
- Fixed world object transfer mechanism trying to transfer already destroyed items, causing exceptions (based on report by @Epsilion)
- Grabbable configured with DestroyOnRelease will now clean itself up when pasted as well, to prevent from getting stuck (based on report by @Epsilion)
- Fixed LogiX tooltip throwing an exception when trying to extract driver node for unsupported datatypes (based on report by @chemicalcrux)
- Potentially fixed desktop view being incorrectly rotated when using Portait mode on a screen that's connected to integrated GPU (based on report by @Freyar)
- Fixed FocusWorld node being able to keep the user in the current world when impulsed every frame (reported by @Cyro)
- Added extra logging for headset presence status changes, to help diagnose issues when the headset disconnects (based on report by @Rue Shejn | Artist 3D)
There are two main features built on top of this right now - one for avatars, giving you full control to create custom setups, override where the user "sees" and "hears" from or putting them out of the actual avatar body and easy ability to render all audio from viewpoint of the streamer camera, making the footage audio make more sense for viewers for any spatialized sources.
Another major thing that's more internal are some cloud infrastructure reorganizations and optimizations, mainly to reduce some inefficiencies to save costs and improve performance and scalability. There's some more to come as well as I continue to monitor the impact of the changes.
A bunch of other smaller additions, tweaks and bugfixes as well. There's bits of the proper desktop mode too, nothing usable yet though, but it's starting to take form. More to come soon!
[h2]New Features:[/h2]
- Added a mechanism to override the transform of user's root, view and "ears" for audio-visual output, while preserving their actual UserRoot in the world by assigning Slot references on the UserRoot component
-- Note that view should be used sparingly for VR users (if at all) as it will negate any actual head movement which will be dizzying to a lot of users. If you want to create "out of body" experience, override the root instead.
-- The view override can be put on the avatar's head so the view still moves with the head, but using the avatar's motions, but be careful with this as well, as the motion not being 1:1 can cause nausea as well. I STRONGLY discourage using this as standard feature of avatars for adjusting the view (even if subtle) and only recommend it for gimmick avatars and personal use
- Added AvatarUserRootOverrideAssigner (under Users/Common Avatar System) which allows overriding the root, view and ears from equipped avatar (requested by @GearBell, @Abysmal, @ProbablePrime, @Shifty | Quality Control Lead and @Hayden)
-- The component updates in realtime. You can set the Override target to null to disable or change it to a target object to dynamically assign a new target
- Added "Audio from camera viewpoint" setting to Interactive Camera dialog, which will render the audio from the camera's viewpoint (requested by @Nexulan | Community Manager and many users before (sorry I didn't keep a list! ;_;))
-- This can be useful for streams for spatialized audio to make it less confusing for viewers, at cost of making it more confusing for the streamer
- Added LookLeft, LookUp, LookRight and LookDown to EyeLinearDriver, which allows driving 0...1 directional values (e.g. blendshapes) based on where the eye is currently looking (based on request by @Groxxy the Eye-Puppeteer)
-- This can be used to drive extra blendshapes that morph the eye when it looks in particular directions
-- The response of the values can be controlled using the LookMultiply and LookPower fields
- Added "Extract Sides" audio processing option to the inspector, which removes the center channel from stereo/quad tracks
-- This can be used to remove vocals from music tracks which are typically centered in mono
- Added "Prefer Specular" to advanced model import settings, which will prefer the Specular variants of PBS materials when importing (requested by @Robyn (QueenHidi))
- Added Rect text input node (x, y, width, height)
-- This now also makes it possible to extract driver node or direct inputs for Rect values (based on report by @chemicalcrux)
[h2]Work-in-progress features:[/h2]
- Expanded Neos' input/output system to support two toggleable rendering outputs - VR and Screen, using "VR_Active" state that can be changed at any time (for testing purposes it's currently bound to the F8 key, but it's not in usable state yet)
- Unified Neos' head output system to remove duplicates and remove any deprecated versions to simplify the code and management
- Added "VR Active" state to user and corresponding node, which determines if the user is currently in VR mode or screen mode
-- Note that the legacy screen mode is treated as "VR" mode. This info will be useful as the proper desktop mode is fully implemented
[h2]Tweaks:[/h2]
- Redesigned part of cloud storage system architecture to utilize multiple differently configured storage services to reduce cost and optimize performance
-- Any operational data (file upload chunks, asset variant computations, thumbnails, status updates and related queue/table objects and their work items, generally anything that's ok if it's lost) is now using low-redundancy, low-latency and low-operation cost (backed by SSD's) storage systems to prevent extra costs with geo-redundant storage
-- Any permanently saved data (assets, records) are still using geo-redundant storage, to ensure durability even in case of data center disasters
- Reworked asset restore mechanism to run only at a single worker at the time and less frequently to avoid unecessary load on the cloud resources
- Removed obsolete session thumbnail extension mechanism, reducing some networking / Cloud API load
- When "Setup IK" in advanced import is disabled, the biped detection report will no longer be generated (based on report by @H3BO3 and @chemicalcrux)
- Upgraded CloudX dependencies and related libraries
- Upgraded to Unity 2019.4.18f1 (from 2019.4.17f1)
- Merged Esperanto, Japanese and Chinese locale updates by @Melnus
- Merged Korean locale update by @LUA
- Merged French locale update by @brodokk
- Merged Czech locale update by @rampa_3 (UTC +1, DST UTC +2)
- Merged Dutch locale update by @AnotherFoxGuy
- Merged Russian locale update by @Shadow Panther [RU/EN, UTC+3]
[h2]Bugfixes:[/h2]
- Improved Inventory and File Browser interaction security (based on report by @Komdog)
- Fixed cloud asset variant processing not deleting associated queue data blobs, resulting in increased costs
- Fixed errors in cloud asset dependency processing, preventing asset dependencies from being removed in cases where even after the removal, the user would still be over their quota limit, causing the storage usage to get stuck and resulting in delays in cloud processing (based on report by @DJNightmares)
- Added generic argument validation to ValueTag and ValueFieldProxy components (based on discovery by @Cyro)
- Added guard to exceptions in OnWorldSaved and OnFocusChanged world events, to prevent errors in those from crashing the entire world (based on report by @Epsilion)
- Fixed world object transfer mechanism trying to transfer already destroyed items, causing exceptions (based on report by @Epsilion)
- Grabbable configured with DestroyOnRelease will now clean itself up when pasted as well, to prevent from getting stuck (based on report by @Epsilion)
- Fixed LogiX tooltip throwing an exception when trying to extract driver node for unsupported datatypes (based on report by @chemicalcrux)
- Potentially fixed desktop view being incorrectly rotated when using Portait mode on a screen that's connected to integrated GPU (based on report by @Freyar)
- Fixed FocusWorld node being able to keep the user in the current world when impulsed every frame (reported by @Cyro)
- Added extra logging for headset presence status changes, to help diagnose issues when the headset disconnects (based on report by @Rue Shejn | Artist 3D)