Hello everyone!
I’m Frooxius, the creator of Neos VR. I haven’t posted a development update in a while, so I’m here to remedy that. To be honest, I always find it difficult to create these posts, because after I push out a new build I just gravitate towards tackling the next set of features and bugs, instead of writing about what has been done over past several builds.
However
in the past three months, we’ve hit many important milestones with Neos so I’d really like to get the word out about those. This will only cover the major additions, if you’d like to see all the new features, tweaks and bugfixes, I recommend joining our Discord, where I post daily updates as they’re released to the public. There’s too many to list here, but if you like reading
here's the changelog for the past 3 months.
Also thanks so much to our community and our Patreon supporters! Without you I wouldn’t be able to work on these features and keep improving Neos every day. Anyway, without much ado, here we go!
Blendshapes / Visemes
Thanks to the great work of the Assimp library contributors (and to members of our community that rushed to support the project) that we’re using for model import,
we can now import blendshapes from FBX files!This has opened a whole new level of avatar realism, allowing for intricate expressions and visemes for voice visualization. If you’re using a common naming scheme for those (such as the one used in VRChat)
those are setup completely automatically!
Upgrading your existing avatar is super easy too - simply import the mesh again and use the rig transfer tool to upgrade your existing avatar with the new data - all materials, customizations and scripts will be preserved!
But as with everything in Neos,
you can customize or build any custom setup you want using the inspector. The DynamicVisemeDriver component in Rendering category in particular offers great flexibility and if you want even more, you can always use LogiX, our visual scripting system.
This combination
brings the avatars truly to the next level, as they can now react to various cues - closing eyes when you put your hand near their head, sticking out tongue when you boop their nose or making a horrified face when you pull their ears.
Blendshapes aren’t limited for avatars either! Our community has utilized them to make lots of fun stuff too, like soda cans that you can crush against your head, speakers that pulse and deform in response to music or an adorable pet parrot that you can feed and pet.
[previewyoutube][/previewyoutube]
Eye Tracking
Neos’ goal has always been to provide wide hardware support and the latest cutting edge VR tech is no exception.
Vive Pro Eye has been fully integrated on request from the Sydney Human Factors Research, exposing all eye tracking data (eye position, look direction, pupil dilation, eye openness) to our visual scripting system allowed for rapidly building new research and educational applications.
However this technology has great applications in social as well! If you’re using the common eye system on your avatar, it will pick up the eye tracking tech fully automatically!
Your avatar will look where you are looking and it will blink when you blink. It might seem like a detail, but the level of realism and immersion this adds in a social setting can be surprising.
[previewyoutube][/previewyoutube]
Camera / Streaming
With all cool features coming in, it’s often been tricky sharing them. Our in-game virtual DSLR could technically be used for recording and streaming, but the setup was too tricky and involved too many steps.
No more!
Now you can record and stream from Neos with just a few clicks! Simply go to Tools -> Camera / Streaming on the Neos dash (the belt menu), click “Mirror To Display” and you’re set.
The UI gives you full control of the camera - you can change mode (smooth first person, third person, automatic group, world point or fully manual), framing, distance, elevation, zoom and other settings from within one place and the camera will automatically follow you from world to world.
There are a lot of advanced options as well that we’re already seeing used for production of some really cool videos.
You can create camera anchors in the world easily switch between them or script them with LogiX,
make the camera follow other users (or ignore them), adjust their local voice/volume or even give them camera operator permission so they can handle it for you.
As a cherry on top there’s
OBS integration as well, so you can
start/stop recording and streaming without removing your VR headset.
Twitch Integration
To make the new camera even sweeter for streamers,
Neos now has Twitch integration! You can spawn chat for any channel and
see the feed in real time without having to rely on 3rd party software - it even
shows custom emotes, follows, subscriptions, cheers and more!
Having the readable chat is just the beginning though.
The Twitch data is exposed to LogiX, so you can receive impulse every time someone sends a message, follows, subscribes or cheers and perform arbitrary action in the world or on the avatar!
This brings the chat interaction to the next level, because
you can rapidly build interactive streams right within VR and let your viewers interact with the world in realtime. We’ve been using this functionality for our official live streams and have had great reception so far.
As expected,
people really love flipping the in world gravity in particular, exploding us or spawning hats and pizza on our heads! The less uhm… “trolly” commands seem quite fun too, like sending us some love, changing the skybox or the lights in the scene.
If you haven’t already,
follow us on Twitch (or subscribe if you want to use some of the more fun commands ;) ) , we’re doing at least one live stream a week!
And of course
we’d love to see what interactive commands you’ll make yourself if you’re a streamer, the possibilities are virtually endless!
[previewyoutube][/previewyoutube]
Interactive Dynamic Bones
What blendshapes have done for avatar expressivity,
the new interactive dynamic bones do for avatar interaction! A long requested feature has been implemented with all the bells and whistles!
Dynamic bone chains make the avatar’s hair, clothes, pendants, ears, tails and other parts come alive and react not only to your motion, but also to touch and other avatars. They
will automatically collide with yours and other people’s hands, heads, whole bodies and whatever custom colliders you and others define.
To push this feature even further, we’ve implemented
support for grabbing and optionally stretching, which adds level of interaction unseen in other platforms. As with everything, you can also interact with them using visual scripting and build extra expressivity on top of them.
And like with everything in Neos, our community has started using them outside of avatars in lots of creative ways! We’ve seen
retractable chain weapons, wobbly pens (that actually draw) or grassy field which reacts to the players running through it.The system is also our
own custom implementation. We have made sure that
it’s designed in a very performant way and its behavior remains the same with varying framerate - it’s going to simulate silky smooth when you’re running fast, but it’s not going to bog you down when you’re already running heavy.
[previewyoutube][/previewyoutube]
200 Patreon supporters and 100 reviews!
We’ve also hit another of our milestones on Patreon! The level of support our community has shown for this project is incredible and
it always makes me feel happy that people believe in our work enough to spare some of their money to keep it going.We don’t have big VC or corporate support as some other projects, but
thanks to you, our supporters and our community, we can keep delivering these new features that surpass competition and keep us at the cutting edge of what’s possible in VR. Thank you so much!
And a huge thanks to
our new $500 level Patreon Rush N Cat! We haven’t even had the chance to meet you yet, but we’re very humbled by your generous support!
Coincidentally we have also got 100 reviews on Steam and stayed at Very Positive status! I’m grateful that people are enjoying this project, even though it’s rough in certain parts. Even with early access status it can be difficult to communicate that certain parts are still unfinished or even mostly missing, but
having so many people get what the project is about and look out for its future is an amazing feeling.
Speaking of its future...
What’s coming next?
There’s several other major features that are largely implemented and can be used, but are still not fully finished.
For example we now have a
headless client! It’s essentially a version of Neos without graphical interface, which lets you host persistent worlds. It is significantly cheaper and more efficient to run and
works on Linux as well. No more kicking everyone out from your world when you want to close Neos! Currently it’s only available in early beta for our Patreon supporters, but it will be public very soon.
Some work has been done on the Linux support for the graphical client as well, but currently it’s held up by a Unity bug, so we’re unfortunately stuck for the moment.
The full body support has also received a lot of work. There’s a
new calibrator with immediate feedback and much easier to use (and will get even better with some of the additions) and lots of the issues have been fixed already - knees not following correctly, hips twisting, head folding into the body and some other minor issues.
We’re also in the process of developing a brand new tutorial and learning experience, dubbed “
Metaverse Training Center”. This will provide a good starting place for newcomers, who currently get thrown into Neos mostly unguided and get overwhelmed by all the tools and options at their disposal.
Avatar anchor system is also currently in development, allowing things like
seats, vehicles, beds, slow down effects and a lot more ways to interact with the user or put constraints and filters on their movement.
The
asset variant system has taken a bit of a back seat with all these new features, since we needed to focus on getting more support for this project, but it’s going to be finished relatively soon, significantly improving loading times, memory usage and general performance.
Some setbacks
However there are some bad news as well. Originally we planned to release the “
Apollo Reconstructed” project this month and have the metaverse training center ready. But due to some unfortunate events, I’ve had to spend the last two weeks with my family, without access to all my development hardware.
As a result some things got done that otherwise wouldn’t (like the headless client), but some have gotten delayed.
This means that Apollo Reconstructed has been pushed to the next month. I’m sorry for the delay, I’m really excited about finally releasing the 3D reconstructions to the public, but they’ll have to wait a little longer.
Next Major Features
Following those semi-major features, our
next two major goals will be new UI system and full physics. We’ll start with writing a custom high-performance UI framework, since it’s one of the major sources of performance issues is very limiting in terms of possible UI designs and UX interactions.
Physics will receive an upgrade to the second version of the BEPU physics engine first (which should boost performance too), after which rigidbody physics, constraints and other parts of the simulation will be integrated.
Going forward
Starting with this update
I’d like to post a bit more often! Since the new Steam Events beta now has a framework for posting small updates, I’ll try posting those there too instead of just our Discord channel - previously I was afraid that daily patch notes would be too spamming when posted there (or on Patreon).
Anytime a new feature is finished I’ll try to get a post out summing it up too, that way you won’t miss the latest developments!
That said, please
join our Discord if you haven’t already! It’s the most active place for the community, great place to share your creations, tutorials and experiences in Neos, to chat with us and it also shows all the active public sessions.
We’re also streaming regularly on Twitch now, which is another great way to see what’s going on in Neos and talk with us in real time. Plus we’ve got some
fun interactive chat commands for you ;)
For now this is all, this post has already gotten longer than I planned it to be, but I hope it was enjoyable nevertheless. And as always, let me know if you have any questions, suggestions or comments, I’m happy to answer!
See you in the metaverse!
Froox
