Neos VR News

Just a small build with some bugfixes and security improvements. Compatible with the previous one, so you don't need to update urgently.

[h2]New Features:[/h2]
- Added support for some Japanese MMD naming conventions (suggsted and chart provided by @AshtonSparx)
-- DirectVisemeDriver now detects あ, い, う, え, お and ん
-- Blink bendshape detection now detects まばたき, ウィンク, ウィンク右 and ウィンク２
-- AvatarExpressionDriver now detects 笑い, 敵意, にっこり, ∧, □ and ぺろっ
-- Please let me know if there are any issues with these, I'm relying on a visual chart to map these and I'm not 100 % sure I got the mapping correct

[h2]Security:[/h2]
- Fixed editor components being able to indirectly modify protected fields (based on report by @xLinka and others)
- Improved security of world session configuration
- Added extra diagnostics to changing session configuration
- FileBrowser component will automatically clean itself up in Worldspace (based on report by @xLinka)
-- Note that even when spawned in Worldspace it can't be used as it has internal userspace checks for its methods
- Added logging for permission components being removed to help diagnose cases of them being

[h2]Bugfixes:[/h2]
- Fixed headless prompt not returning after running the restart command (reported by @Glitch)
- Fixed insecure websocket connections not working on headless (reported by @Gawdl3y, @Kulza and @Epsilion)
- Fixed unread messages not being properly marked as read when there's more than 512 (reported by @Lucaedr, @Kodey and @Sox)
- Fixed developer tooltip destroying last selected object with "Destroy Selected" even after it was deselected (reported by @amber)
-- Note that if it's deselected by another user, it will still be destroyed. Dealing with this properly will require the implementation of proper selection management system

Hello everyone and welcome to another weekly update!

This week we have briefly shifted focus from BEPUv2 integration to integrating the SignalR push notification service for our messaging system and other cloud optimizations to improve responsiveness and fix a number of issues, such as message history not loading for certain users.

Thanks to this new system, the load on the cloud infrastructure has significantly dropped. This should keep the cloud responsive even during busy days with lots of people on and keep scaling as the community keeps growing.

It also opens up doors for new upcoming features and abilities, like full real time sync for cloud variables, cloud impulses and optimizing other parts of the cloud infrastructure. We also used it to improve security of the moderation system, ensuring that any account restrictions take immediate effect.

Security and moderation have received some more improvements as well, preventing some cases of ban evasion. You can now also restrict the in-game File Browser to only specific locations, or none at all.

And also in big news, the VBLFC furry con is taking place this weekend! This is the first con held officially in Neos VR and we are proud to host it on our platform. You can find more information at their Discord and Twitter or register at the website here: They have a lot of exciting events planned. We hope everyone will enjoy their time there!



[h2]VBLFC[/h2]
The Virtual BLFC con is taking place this week! This is the first con held officially in Neos VR and the biggest event to date on our platform. The organizers have prepared an awesome set of worlds and other cool things for the attendees. We are honored to host this event and we hope that everyone will have an awesome time!

[previewyoutube][/previewyoutube]

You can find more information at their official website and register for attendance. They have also made this amazing trailer for the event.



[h2]Integrating SignalR for messaging system[/h2]
As our community keeps growing, we have started noticing the cloud infrastructure being heavily taxed during the busy days, causing the cloud service to become less responsive. With some big events coming up, we decided to prioritize integration of SignalR push notification service, which was on the roadmap for a while, to improve how well the system scales and introduce some new capabilities.

The in-game messaging system has been redesigned to use this service. Instead of repeated REST (HTTPs) requests, all the communication is done via WebSockets. This is significantly more efficient for both your connection, as well as on the cloud service - the previous system was responsible for using the majority of the system’s throughput!

Thanks to the new system, the messaging system should be significantly more reliable and robust in the long term (after fixing some initial bugs and issues), decreasing the latency of the messages and cases where they wouldn’t be delivered at all or be slow to respond.

Decreased database usage after optimizations. Shows last 7 days.

The system also enables some new features, like getting message read notifications (can be disabled in settings based on community request) and in the future message writing or voice message recording notifications.

The service is also planned to be used for more as well in the future, notably updating the session status, current profile status and synchronizing cloud variables fully in real time across different clients. You can check out more at our roadmap on GitHub.

[h2]More cloud optimizations[/h2]
When working on the cloud, we found out more underlying causes of issues that were resulting in some slowness, particular slow message loading, due to the database queries being highly suboptimal. Those have now been optimized, resulting in significantly more efficient queries and much quicker loading of message history as a result.

Server usage split by endpoint. The orange line is fetching new messages. You can see the significant decrease after SignalR release, but still some large spikes. Those were addressed near the end of the grpah.

Some other parts of the cloud were tweaked and improved as well to be more robust, improving the rate limiting system with a new solution based on Redis and adding fallbacks in case of transient connectivity issues.

Overall we hope those changes will help resolve a number of issues that you might’ve been experiencing in the past, but we’ll keep monitoring the system and making more improvements as the community keeps growing.


If you run into any problems, let us know either on Discord or our GitHub!

[h2]Security and moderation improvements[/h2]
Our moderation tools have gotten some improvements as well based on some community reports and incidents. Notably we have expanded the ban evasion detection system to cover more loopholes in the system and used the SignalR service so any global bans take effect immediately.

We have also added a configuration option that allows restricting the functionality of the built-in File Browser so it can only access certain paths on your system or none at all. Simply setup an array of paths under pathWhitelist in Config.json file in the Neos installation. Providing an empty array (“pathWhitelist: [ ]”) will disable the File Browser completely.

This can be useful for additional security or when running Neos at public events, schools and other places where you don’t want the visitors/users to access all the files from within VR.

--------------------

Anyway that's all for this week! Also apologies for posting so late, this week has been a bit crazy. We've had some storms and lost power (and internet) for a while, which delayed a few things, but better late than never!

We'll see you next one, keep an eye on our official Discord for info. And as always, huge thanks for everyone for supporting the platform, without your help we wouldn't be here!

Hello everyone!
I've got a new build for you! This is mainly cloud changes, that should greatly improve the responsivness and make the new services more robust! The switch to SignalR ended up a bit bumpier than I hoped for (thanks everyone for understanding despite the inconveniences ^^;), but it should be really worth it in the long run and scale much better as we get more users.

Notably, I found root cause of message history taking really long time to load. In some cases the database queries were very suboptimal, taking several seconds of database time for a single fetch. For example loading message history I used to test consumed around 14000 RUs (Request Units, pretty much database throughput units). After adding extra field to simplify the query and adding composite index (you probably felt that yesterday, as all existing messages had to be re-indexed!), it now takes just 8 RUs!

As a result, the message history should now load super fast across the board and stop hammering the cloud, causing other things to be slow! But please let us know if you still run into some issues.

There are a bunch of improvements as well. The cloud also ran into another issue, where due to Redis server restarting, the rate limiting library wouldn't reinitialize and everything would break! This should be fixed now. The system now also has fallback in case the Redis fail temporarily and will restart itself automatically if it keeps failing.

Anyway, hopefully things should run pretty smoohtly right now. I'll be monitoring them and see if we run into some more issues. There some more things that we need to optimize as well (e.g. world searching/browsing, initial loading of contacts and bunch of others), but this should be a pretty important step forward!

Oh also this build is compatible with the last since most of the changes were in the cloud (but there should be some neat stuff in the build too)!

[h2]Tweaks:[/h2]
- Added optimized index values to messages in the database and optimized message history queries to significantly reduce (orders of magnitude) required database throughput when fetching message history
-- Message history should now load significantly faster and cause less load on the cloud backend (to give example, one fetch would previously consume ~14000 RUs, the same exact fetch now consumes 8 RUs)
-- This should also fix cases of message history not loading at all in somem cases due to the request timing out
-- Additionaly this should help with overall cloud responsiveness, thanks to message history loading not hammering the database like crazy anymore
-- All existing messages have been processed and re-indexed (this was one of the major factors for the slowness yesterday). If you notice any messages not loading or missing, please let us know!
- Improved health check for the cloud API server, to ensure the server gets rebooted in case of CosmosDB or Redis issues
-- This will auto-recover the cloud in case of unexpected connectivity issues that require reinitializing connections
- Tweaked database query batch size when fetching records, to prevent certain operations (e.g. generating record usage report) from resulting in large load on the database, causing cloud slowness
- Record usage report/JSON is now pipelined and rate limited to avoid it from potentially disrupting cloud services and to improve robustness
-- You can only run the command once every 10 minutes and max 8 times a day. This is subject to change, but it's generally not recommended to run this too often
-- The requests are processed in sequence in case of multiple requests to avoid overloading database with too many expensive tasks at once. You will be notified when the generation finishes
- Cloud messaging system now locally tracks sent status of a message
-- This ensures that message is properly colored as sending or failed to sent when the message history is refreshed
- Added in-process rate limiting fallback during Redis connectivity issues
-- This should improve robustness of the cloud services in case of interminnent connectivity issues to the Redis server
- VBLFC badge now shows up in the Session list as well @Delta and @Kulza
- Changed deadzone algorithm for axis movement to be based on magnitude, rather than each individual axis, which results in better diagonal behavior (based on report by @H3BO3)
- If you send a message to user before the history loads, the newly sent message shouldn't disappear anymore

- Merged Russian locale addition by @Shadow Panther [RU/EN, UTC+3]
- Merged Japanese locale additions and tweaks by @Aesc
- Merged Korean locale additions by Holy_Water
- Merged Chinese locale additions and tweaks by Holy_Water
- Merged Czech locale addition by @rampa_3 (UTC +1, DST UTC +2)
- Merged German locale additions and tweaks by @InnocentThief

[h2]Security:[/h2]
- Added extra obfuscation to host user's ID's (based on report by @runtime)

[h2]Bugfixes:[/h2]
- Fixed bug in FireflySoft.RateLimit library that would cause all rate limit checks to fail when the Redis server is unexpectedly rebooted, due to Lua Redis script being cleared
-- This was the primary cause of the cloud issues yesterday
-- Pull request for our fix is available here: https://github.com/bosima/FireflySoft.RateLimit/pull/7
- Fixed full message fetch rate limit not working properly, allowing the full message fetches to be executed too often
- Fixed wrong debug message on SignalR (reported by @Bitman (Neos.js Developer))
- Fixed incorrect URL when fetching records at path with access key (reported by @Bitman (Neos.js Developer))
- Smooth Turn exclusive mode will now also prevent movement when already turning, instead of just preventing turning while moving (reported by @H3BO3 and @Kaptain Krunch)

Do you guys ever get a day where you start working on a thing, then someone needs you to do another thing so you switch focus on that, you find you need to do another thing to do that, then in middle of that someone needs you to do another thing so you focus on that one and as you do another thing comes up. And you finish that one, go back to other things and more things come up and you never even get to the original set of things?

Long story short, no weekly update today, but at least here's a build with a bunch of additions and improvements for the new SignalR message delivery system to solve some initial issues and add a few options based on community feedback. Also some security improvements and bunch of other tweaks and additions! :smile:

[h2]New Features:[/h2]
- Added "Don't send realtime message read status" setting which allows disabling sending message read status to the other user in realtime (requested by @Yellow, @Electronus and @H3BO3)
-- Note that they will still be able to eventually tell that you read them at some point since it's updated in the database, but the status won't change for them the moment you open them
-- When enabled, you don't see any read status on your end either (meaning all messages stay yellow)
- Added SetDateTimeKindNode (implemented by @ProbablePrime | Docs)
-- This allows setting the Kind property on a DateTime instance
- Added VBLFC badge (requested by @Shifty | Quality Control Lead on behalf of @Kulza)

[h2]Security:[/h2]
- Improved ban evasion detection system, preventing users from bypassing public bans by logging out of their accounts (based on reports by the moderation team)
- Added salts to various ID's, to increase difficulty of matching them through a rainbow table (based on report by @runtime)
- Improved the global moderation system to apply account restrictions immediatelly, rather than having to wait for the user to leave and/or rejoin (based on request from @CanadianGit | Moderation Team, @Veer | CMO and rest of the moderation team)
- Added pathWhitelist setting to Config.json which allows restricting (or completely disabling) locations accessible via the built-in File Browser (requested by @Earthmark)
-- This is an array of root paths that can be browsed. E.g. C:\Data\Neos will allow browsing everything in and under that folder, but nowhere else
-- Providing empty array ([ ]) will disable the File Browser functionality completely
-- This can be useful also for events and other situations where you don't want other people browsing everything on your computer

[h2]Tweaks:[/h2]
- When set to Invisible, you automatically don't send realtime message read status to the other user
- Tweaked CosmosDB timeouts and other settings to potentially reduce number of failed requests (based on recent partial outage and community reports)
- Added better REST message API rate limiting mechanism to prevent bad requests from 3rd party applications from causing cloud unresponsiveness
-- If you're making such application, please note that we currently do not officially support this. At very least we ask you to be considerate of how you use the API (e.g. avoid frequently fetching large amounts of messages), otherwise we'll have to tighten the restrictions more or even add a lockout mechanism if too many problematic requests come from your app
- Registration button on auth.neos.com now links to account.neos.com, since the registration on the new website isn't fully implemented yet (based on report by @Levi)
- Removed developer notification message form the Error page on auth.neos.com (based on report by @Levi)
- UserTime node now outputs DateTime with Unspecified kind, rather than UTC (based on report by @I'm Erin., implemented by @ProbablePrime | Docs)

- Merged Japanese locale update by @Aesc
- Merged Russian locale update by @Shadow Panther [RU/EN, UTC+3]
- Merged Czech locale additions and update by @rampa_3 (UTC +1, DST UTC +2)

[h2]Bugfixes:[/h2]
- Fixed Twitch dialog buttons not working after recent change (reported by @LemonCement and @Fuzzy, fixed by @ProbablePrime | Docs)
- Fixed some messages not being properly mark as read on the next login despite reading them, due to the marking as read being processed before the message is fully persisted in the database
- Fixed AdminX not working due to missing some SignalR related libraries

Hello everyone! I got another build for you! As I mentioned in #devlog, I paused work on BEPUv2 for a bit (just about two days) to integrate the SignalR service, given some issues that cloud has been having. As of this build it now powers the messaging system, which should greatly improve the overall responsiveness and robustness of the cloud, since the old system was the major source of load, especially with lots of people on.

It's also good foundation for lots of other features that are to come later as well (see the cloud roadmap on GitHub for more)! I might do a few of them soon too, I'll see how it goes. Since this is a major change to how the messaging system works too, keep an eye out on some weirdness and bugs as usual, we'll clean them up as they come and make adjustments like usual whenever something new comes out!

Also there's a whole bunch of goodies and quality of life improvements from @ProbablePrime | Docs, including user timezone nodes and blocking gizmo interactions from guests!

[h2]New Features:[/h2]
- Implemented new message delivery system based on the SignalR service
-- This is significantly more efficient and scaleable system (push notification vs polling)
-- This reduces the load on your connection and on the cloud servers, resulting in much better overall responsiveness (more cloud resources are available for other things like inventory, world searching and so on)
-- This also reduces the message delivery latency to near instant (ignoring networking lag)
-- The robustness is also improved, decreasing likelihood of failed delivery
-- The system is compatible with the old message API, but not that it will likely disappear or be severely restricted at some point!
-- Messages that have been viewed by the user are now marked green (note that it doesn't necessarily mean the other user has actually read them)
--- Note that this won't work for users on the old build
- Added nodes for getting user's time offset from UTC (requested by @Earthmark, @Alex from Alaska, implemented by @ProbablePrime | Docs)
-- User Time Offset - gives current TimeSpan offset of given user relative to UTC
-- User Time - gives given user's current time in their timezone
-- Local User Time Offset - gives time offset of local user

- Added DatetimeKindNode (implemented by @ProbablePrime | Docs)
-- This provides the type of DateTime structure - UTC, Local or Unknown

[h2]Tweaks:[/h2]
- Non-builders can no longer grab or click the LogiX node browser (reported by @Shifty | Quality Control Lead, Maliceowo, implemented by @ProbablePrime | Docs)
- Non-builders can no longer click gizmo buttons (reported by @Shifty | Quality Control Lead, Maliceowo, implemented by @ProbablePrime | Docs)
- DateTime LogiX display now shows (UTC) at the end when it's UTC. Nothing otherwise (based on request by @Shadow Panther [RU/EN, UTC+3], implemented by @ProbablePrime | Docs)
- Increased timeout for getting message history (this can help resolve cases of the history failing to load in some cases, reported by multiple users)
- Added warning about wallet needint to support ERC-20 tokens when making NCR withdrawal (suggested by @pammematth)
- Upgraded Cloud libraries to latest versions

- Merged Korean locale addition by @MirPASEC
- Merged English locale typo fix by @InnocentThief

[h2]Bugfixes:[/h2]
- Fixed CDN not being used for all accounts
- Fixed API accepting credit transactions addressed to the same account they're sent from