Desktop audio streaming, Matcap for XiexeToon, Mentor Network, MTC Avatar Room
Hello and welcome to another weekly update!
This week we have a few exciting things for you. Neos now has the ability to easily share your desktop audio to other users in full stereo with high quality, letting you stream live music to your hangout worlds, run live DJ events and do lots of creative things that we haven’t even thought of!
We made some improvements to the shader asset system as well, making it much easier for us to update shaders. Thanks to those changes, we integrated the Matcap support in Xiexe Toon shader, as well as ColorMasking and using secondary UV’s, added PBS stencil and made all the filter shaders usable in UIX with full masking support.
We’re also introducing a new Mentor Network, a way for community members to get better visibility and communication tools to help out new users. If you’re interested and would like to apply, check out details below.
The avatar lobby for the new version of Metaverse Training Center has made significant progress. There’s still work to do, but you can check out the work in progress on the new environment below. We have some more details on progress on the desktop mode as well.
[h3]Friday Livestream - showcasing MTC avatar lobby and announcing the mentor network[/h3]
[previewyoutube][/previewyoutube]
[h2]Audio Streaming - stream your desktop audio in high quality[/h2]
One of the major new features added this week is the new ability to easily stream your desktop audio into your worlds in full stereo and with higher bitrate than used for the voice. Simply click the new “Stream Audio” on the Home screen in your dash, select the audio device you want to stream, adjust the target bitrate (goes from 2.5 kbps all the way 500 kbps using the Opus codec) and hit “Start Streaming”.
Using this feature you can easily stream music from your desktop in your social hangout worlds or do live DJ events without need for any complex streaming setup or external software. The system supports both audio inputs as well as capturing audio playback from your system, giving you great flexibility for the setup.
The streamed audio is independent of your voice data as well, so you can still talk with others as usual, switch to whisper or mute yourself, without affecting the audio stream. The only caveat is that you should use a different audio device than the one that Neos sends it sound to, otherwise you’ll be streaming everyone’s voices and in-game audio as well.
Streamed audio data is also treated the same as any other source, so you can easily plug it into various visualizers and build audio reactive worlds. To help this along, the streaming control dialog also serves as a proxy to the audio data, allowing you to drop it directly into the inspector. To stop the streaming, simply close the dialog spawned in the game.
We hope that this new feature brings a lot more utility and some exciting use cases, not only to the WaveXR Resident DJ community that has been exploring Neos recently and has requested this feature for their events, but to everyone else who was previously looking to bring their music to Neos or even perform live.
[previewyoutube][/previewyoutube]
It's a real world devlog! Note that this shows an earlier work in progress version of the feature, with some bugs and without the UI fully finished.
[h2]Matcap support for Xiexe Toon, Filter shaders in UIX, PBS Stencil[/h2]
Since we have accumulated numerous requests for additions, tweaks and fixes to various shaders in Neos, we have decided to fully transition our shader asset system to the asset variant system, used by other types of assets like textures or cubemaps.
Previously the shader metadata and individual variants were uploaded to a special location and used special keyword URL’s, rather than regular neosdb:/// ones. This made any updates to shaders a tedious process, as we had to manually update those files and URL’s in the code.
By treating the shaders more like normal assets, with the cloud system extracting and managing the metadata and variants as it does with other assets, not only has this process been greatly simplified, but it fixed some long standing loading issues and improved performance, allowing metadata to be cached, rather than fetched from the cloud at every startup.
Thanks to these changes we were able to fulfill several of the requests. The Xiexe Toon material in Neos now has MatCap support, as well as ColorMasking and ability to switch to secondary UV’s on the mesh for each individual texture map.
An example of the use of Xiexe with Matcap in Neos, screenshot courtesy by DAWKY
Another example of the use of Xiexe with Matcap in Neos, screenshot courtesy by DAWKY
The filter materials, like HSV, Invert, Blur, Grayscale, LUT and many others were also extended with stencil support, which now makes them fully usable in UIX with proper mask support. This will allow for more flexibility and customization when building both your own custom UI and making official ones.
Other changes include also new PBS Stencil shader, which is a vanilla PBS material with added stencil support for masking effects and some bug fixes for other shaders ignoring texture scaling options.
[h2]Introducing the Neos Mentor Network[/h2]
The Neos Mentor Network is here! The Mentor Network is an initiative for the Neos Community to have representatives in the community new or old, who like to show new users around and help them get started, inform fellow community members and also point out problems that Neos has as a whole so that we can make the platform better for all.
The Mentor Network will act as a place where other mentors can connect and also improve their craft. Neos is a really big platform and we all understand as things scale up, managing it and making sure things go smoothly can be problematic. It’s no lie that sometimes Neos can be really overwhelming, but once you’re past that initial barrier, things can be so much fun and oh so chaotic in a way that makes VR awesome!
Do you like to contribute to Neos? Do you like to help out the community? Want to see it grow or learn some better tools of the trade to help out and coordinate with others? Then send in an application and join today!
Of course if you prefer helping other users independently of the network, that’s perfectly fine as well, there are lots of different ways to contribute and help out the community! We’re proud that helping each other is one of the core values of our community and we hope it continues to be as it continues to grow, with this program making it easier to make some of the members easier to approach for new users and providing tools for better organization and team communication.
The mentor network is managed by Turk, so if you have any questions about the program or any concerns, please direct them to him. Here’s a sneak peak at what the badge will look like. It’s not fully final yet, so expect the design to change!
[h2]Progress on the MTC Avatar Lobby[/h2]
We have made significant progress on another part of the reworked MTC 2.0, our tutorial world to help new users - the avatar & social room. The old MTC avatar room is one of the most visited parts, so it was an easy decision to rework it next.
The new version provides the user a comfortable environment, where they can learn how to set up their own avatar, but also hang out with other users. We often see users hanging out in the avatar room with the community members helping them out, so we wanted to make it into a comfortable environment.
There is still work to do, but the environment has been coming along nicely. If you’d like to see it in action, check out the recording from our Friday’s livestream where we showcased the progress so far. We hope to have a finished version soon and set it live.
[h2]Fixing choppy voice, particle system freezes, cloud and more[/h2]
Thanks to working on the audio streaming feature, we’ve been able to find a long-standing bug in the audio streaming code, which would cause the voice to randomly become very choppy for some users. This issue would affect some users more than others and was quite difficult to diagnose, but as of the latest build it should be completely gone.
We traced down another of the elusive bugs, causing a random freeze on particle bursts for any particles with collisions enabled, after long periods without any particles with collision emitted. We have thrown in a few CPU and memory optimizations for the multithreading system handling those collisions as well.
Another of the notable fixes and optimizations this week has been to the cloud infrastructure. Earlier this week the cloud service started degrading significantly. We have investigated this issue and traced down the culprit to a combination of a poorly coded bot and an edge case in the database SDK. We have corrected both of the problems, while also adding new internal diagnostic tools to help analyze any future cases like this.
We also finally have some data on the cloud cost optimizations that we implemented last week. So far it seems really promising, with the cost being significantly reduced thanks to more efficient structure and elimination of unnecessarily wasted bandwidth.
Apart from these notable fixes and improvements, there were dozens of other bug fixes, tweaks and additions. If you’d like to learn more, check out the #neos-update channel on our Discord or the patch notes on stream.
[h2]Progress on the desktop mode[/h2]
The desktop mode is still heavily in the design phase, but we have made some good progress on that end as well. As we went through all the different subsystems and how to abstract them for desktop use, it became obvious that implementing at least a partial version of an input binding system is the best way forward.
This will help separate concerns and reduce desktop-specific code in a lot of the codebase. For example a locomotion module can only be concerned with the direction the user wants to move in, axes around which they want to turn and whether they want to jump, but not which combination of inputs achieve this.
By having this computed in the input binding system and only passing the resulting directional and boolean inputs, the logic can be greatly simplified. Given the previous example, the movement direction can be computed using a controller joystick and head direction in VR, while using the mouse and keyboard on desktop.
Once implemented, we can also expand this system to allow customization, allowing both desktop and VR users to customize how Neos is controlled. This is something that would come later though, as our initial goal is to first do this abstraction internally and nail down the details of the system.
[h2]Community Highlights by Turk[/h2]
Hello Neos Community!I hope your week has been fantastic. People this week have been going crazy for visuals. So, I’d thought I’d use this weekly update to show off some of the crazy visuals people have been making and doing in Neos lately :)
[h3]Curving Bullets by orange[/h3]
First up is Orange’s Curving Bullet world! Ever watch the movie Wanted? Wanted to feel like a super train assassin who can Curve the bullet? Want to flex on your friends and get the high score? Then come on down to the Curving Bullet Range and show off some guns! Thank you Orange-san for your efforts!
https://twitter.com/mikan3134/status/1351125817615654914
[h3]Kaliedoscopic Playground by Sirkitree:[/h3]
Long time Neos veteran and All-around Mandala-Master/Enthusiast sirkitree thoroughly entrances us with his craft. In His World KaliedoScopic Playground he goes into showcasing how he makes some of his visuals in Neos, and how easily you can too if you’d like to add to the wonderful entropy that these visuals bestow on you as you witness them. Thanks sirkitree for the world!
[h3]Club VibeZ by QueenHidi[/h3]
A exciting update from QueenHidi, she’s updated her Club VibeZ world to include more visual effects for those who want to party and those who want some sights to their DJ sessions! Watch as the winding tunnels send particles all over around you, and ebb and flow to the beat entrances you. Many of you might not be aware, but the WaveXR Community are setting up shop over in Neos, and this was made to have them feel more at home and teach them some of the ropes! So Thanks QueenHidi
------------------------------------------------------------------
Anyway this is all for this week! We'll see you at the next one with more cool stuff and developments and as usual, big thanks to everyone supporting this project, whether it's through Patreon, Twitch, GitHub, Discord, helping the community in-game or just simply being part of the platform!
See you next week!
This week we have a few exciting things for you. Neos now has the ability to easily share your desktop audio to other users in full stereo with high quality, letting you stream live music to your hangout worlds, run live DJ events and do lots of creative things that we haven’t even thought of!
We made some improvements to the shader asset system as well, making it much easier for us to update shaders. Thanks to those changes, we integrated the Matcap support in Xiexe Toon shader, as well as ColorMasking and using secondary UV’s, added PBS stencil and made all the filter shaders usable in UIX with full masking support.
We’re also introducing a new Mentor Network, a way for community members to get better visibility and communication tools to help out new users. If you’re interested and would like to apply, check out details below.
The avatar lobby for the new version of Metaverse Training Center has made significant progress. There’s still work to do, but you can check out the work in progress on the new environment below. We have some more details on progress on the desktop mode as well.
[h3]Friday Livestream - showcasing MTC avatar lobby and announcing the mentor network[/h3]
[previewyoutube][/previewyoutube]
[h2]Audio Streaming - stream your desktop audio in high quality[/h2]
One of the major new features added this week is the new ability to easily stream your desktop audio into your worlds in full stereo and with higher bitrate than used for the voice. Simply click the new “Stream Audio” on the Home screen in your dash, select the audio device you want to stream, adjust the target bitrate (goes from 2.5 kbps all the way 500 kbps using the Opus codec) and hit “Start Streaming”.
Using this feature you can easily stream music from your desktop in your social hangout worlds or do live DJ events without need for any complex streaming setup or external software. The system supports both audio inputs as well as capturing audio playback from your system, giving you great flexibility for the setup.
The streamed audio is independent of your voice data as well, so you can still talk with others as usual, switch to whisper or mute yourself, without affecting the audio stream. The only caveat is that you should use a different audio device than the one that Neos sends it sound to, otherwise you’ll be streaming everyone’s voices and in-game audio as well.
Streamed audio data is also treated the same as any other source, so you can easily plug it into various visualizers and build audio reactive worlds. To help this along, the streaming control dialog also serves as a proxy to the audio data, allowing you to drop it directly into the inspector. To stop the streaming, simply close the dialog spawned in the game.
We hope that this new feature brings a lot more utility and some exciting use cases, not only to the WaveXR Resident DJ community that has been exploring Neos recently and has requested this feature for their events, but to everyone else who was previously looking to bring their music to Neos or even perform live.
[previewyoutube][/previewyoutube]
It's a real world devlog! Note that this shows an earlier work in progress version of the feature, with some bugs and without the UI fully finished.
[h2]Matcap support for Xiexe Toon, Filter shaders in UIX, PBS Stencil[/h2]
Since we have accumulated numerous requests for additions, tweaks and fixes to various shaders in Neos, we have decided to fully transition our shader asset system to the asset variant system, used by other types of assets like textures or cubemaps.
Previously the shader metadata and individual variants were uploaded to a special location and used special keyword URL’s, rather than regular neosdb:/// ones. This made any updates to shaders a tedious process, as we had to manually update those files and URL’s in the code.
By treating the shaders more like normal assets, with the cloud system extracting and managing the metadata and variants as it does with other assets, not only has this process been greatly simplified, but it fixed some long standing loading issues and improved performance, allowing metadata to be cached, rather than fetched from the cloud at every startup.
Thanks to these changes we were able to fulfill several of the requests. The Xiexe Toon material in Neos now has MatCap support, as well as ColorMasking and ability to switch to secondary UV’s on the mesh for each individual texture map.
An example of the use of Xiexe with Matcap in Neos, screenshot courtesy by DAWKY
Another example of the use of Xiexe with Matcap in Neos, screenshot courtesy by DAWKYThe filter materials, like HSV, Invert, Blur, Grayscale, LUT and many others were also extended with stencil support, which now makes them fully usable in UIX with proper mask support. This will allow for more flexibility and customization when building both your own custom UI and making official ones.
Other changes include also new PBS Stencil shader, which is a vanilla PBS material with added stencil support for masking effects and some bug fixes for other shaders ignoring texture scaling options.
[h2]Introducing the Neos Mentor Network[/h2]
The Neos Mentor Network is here! The Mentor Network is an initiative for the Neos Community to have representatives in the community new or old, who like to show new users around and help them get started, inform fellow community members and also point out problems that Neos has as a whole so that we can make the platform better for all.
The Mentor Network will act as a place where other mentors can connect and also improve their craft. Neos is a really big platform and we all understand as things scale up, managing it and making sure things go smoothly can be problematic. It’s no lie that sometimes Neos can be really overwhelming, but once you’re past that initial barrier, things can be so much fun and oh so chaotic in a way that makes VR awesome!
Do you like to contribute to Neos? Do you like to help out the community? Want to see it grow or learn some better tools of the trade to help out and coordinate with others? Then send in an application and join today!
Of course if you prefer helping other users independently of the network, that’s perfectly fine as well, there are lots of different ways to contribute and help out the community! We’re proud that helping each other is one of the core values of our community and we hope it continues to be as it continues to grow, with this program making it easier to make some of the members easier to approach for new users and providing tools for better organization and team communication.
The mentor network is managed by Turk, so if you have any questions about the program or any concerns, please direct them to him. Here’s a sneak peak at what the badge will look like. It’s not fully final yet, so expect the design to change!
[h2]Progress on the MTC Avatar Lobby[/h2]
We have made significant progress on another part of the reworked MTC 2.0, our tutorial world to help new users - the avatar & social room. The old MTC avatar room is one of the most visited parts, so it was an easy decision to rework it next.
The new version provides the user a comfortable environment, where they can learn how to set up their own avatar, but also hang out with other users. We often see users hanging out in the avatar room with the community members helping them out, so we wanted to make it into a comfortable environment.
There is still work to do, but the environment has been coming along nicely. If you’d like to see it in action, check out the recording from our Friday’s livestream where we showcased the progress so far. We hope to have a finished version soon and set it live.
[h2]Fixing choppy voice, particle system freezes, cloud and more[/h2]
Thanks to working on the audio streaming feature, we’ve been able to find a long-standing bug in the audio streaming code, which would cause the voice to randomly become very choppy for some users. This issue would affect some users more than others and was quite difficult to diagnose, but as of the latest build it should be completely gone.
We traced down another of the elusive bugs, causing a random freeze on particle bursts for any particles with collisions enabled, after long periods without any particles with collision emitted. We have thrown in a few CPU and memory optimizations for the multithreading system handling those collisions as well.
Another of the notable fixes and optimizations this week has been to the cloud infrastructure. Earlier this week the cloud service started degrading significantly. We have investigated this issue and traced down the culprit to a combination of a poorly coded bot and an edge case in the database SDK. We have corrected both of the problems, while also adding new internal diagnostic tools to help analyze any future cases like this.
We also finally have some data on the cloud cost optimizations that we implemented last week. So far it seems really promising, with the cost being significantly reduced thanks to more efficient structure and elimination of unnecessarily wasted bandwidth.
Apart from these notable fixes and improvements, there were dozens of other bug fixes, tweaks and additions. If you’d like to learn more, check out the #neos-update channel on our Discord or the patch notes on stream.
[h2]Progress on the desktop mode[/h2]
The desktop mode is still heavily in the design phase, but we have made some good progress on that end as well. As we went through all the different subsystems and how to abstract them for desktop use, it became obvious that implementing at least a partial version of an input binding system is the best way forward.
This will help separate concerns and reduce desktop-specific code in a lot of the codebase. For example a locomotion module can only be concerned with the direction the user wants to move in, axes around which they want to turn and whether they want to jump, but not which combination of inputs achieve this.
By having this computed in the input binding system and only passing the resulting directional and boolean inputs, the logic can be greatly simplified. Given the previous example, the movement direction can be computed using a controller joystick and head direction in VR, while using the mouse and keyboard on desktop.
Once implemented, we can also expand this system to allow customization, allowing both desktop and VR users to customize how Neos is controlled. This is something that would come later though, as our initial goal is to first do this abstraction internally and nail down the details of the system.
[h2]Community Highlights by Turk[/h2]
Hello Neos Community!I hope your week has been fantastic. People this week have been going crazy for visuals. So, I’d thought I’d use this weekly update to show off some of the crazy visuals people have been making and doing in Neos lately :)
[h3]Curving Bullets by orange[/h3]
First up is Orange’s Curving Bullet world! Ever watch the movie Wanted? Wanted to feel like a super train assassin who can Curve the bullet? Want to flex on your friends and get the high score? Then come on down to the Curving Bullet Range and show off some guns! Thank you Orange-san for your efforts!
https://twitter.com/mikan3134/status/1351125817615654914
[h3]Kaliedoscopic Playground by Sirkitree:[/h3]
Long time Neos veteran and All-around Mandala-Master/Enthusiast sirkitree thoroughly entrances us with his craft. In His World KaliedoScopic Playground he goes into showcasing how he makes some of his visuals in Neos, and how easily you can too if you’d like to add to the wonderful entropy that these visuals bestow on you as you witness them. Thanks sirkitree for the world!
[h3]Club VibeZ by QueenHidi[/h3]
A exciting update from QueenHidi, she’s updated her Club VibeZ world to include more visual effects for those who want to party and those who want some sights to their DJ sessions! Watch as the winding tunnels send particles all over around you, and ebb and flow to the beat entrances you. Many of you might not be aware, but the WaveXR Community are setting up shop over in Neos, and this was made to have them feel more at home and teach them some of the ropes! So Thanks QueenHidi
------------------------------------------------------------------
Anyway this is all for this week! We'll see you at the next one with more cool stuff and developments and as usual, big thanks to everyone supporting this project, whether it's through Patreon, Twitch, GitHub, Discord, helping the community in-game or just simply being part of the platform!
See you next week!