1. Neos VR
  2. News

Neos VR News

Thank you all for fueling the flame of the metaverse in 2019; What's coming 2020

Hello everyone!

A year ago I wrote a post about how you, the community and our early supporters lit the spark of the metaverse in 2018. You saw the potential in the pile of code and dreams I called Neos and supported it in its earliest stages with your time, creativity and money.

Now a year later, that spark has grown into a full blown flame. Our community grew tremendously, with many more passionate and creative people joining and spending their time (almost) every day in Neos, building, creating and having fun together.

I’m very thankful that the core of our community keeps this kind of spirit. My work on Neos is just one part of the equation, but your creativity, helpfulness and kindness is what really makes the place and makes it inviting for new members.

[h2]Growing both Neos and the Neos Team[/h2]

Our team has grown as well, pretty much exclusively from community members, helping accelerate overall Neos’ development as a platform and filling in wherever it was needed. We started doing regular Twitch streams, which turned out to be a lot more fun than we expected, releasing weekly update posts to keep you informed and the support on Patreon, ICO and Twitch has exceeded our expectations and allowed us to keep going full steam ahead.

Neos itself has grown and transformed significantly as well. At the beginning of the year we’ve just added physical locomotion. We launched NCR, got rid of constantly present lasers, got rid of garbage collector stutters, switched to more efficient single-pass rendering and GPU instancing, implemented asset variant system which slashed loading times and memory usage to a fraction of what they were, reworked and upgraded our cloud infrastructure and done other major optimizations.

https://www.youtube.com/watch?v=qqBmQ6Qlx48

[h2]Bringing avatar system to the cutting edge[/h2]

A lot of the work was also focused on bringing Neos up to standard of other social platforms and even beyond. We were one of the first platforms to introduce full finger tracking for Index Controllers with custom avatars, using a hardware agnostic system with full skeletal remapping, which allowed us to easily support Leap Motion and other controller skeletal models as well. Gone were the days of pancake hands!

https://www.youtube.com/watch?v=oEG2-Yc-0Wc

We’ve finally gotten blendshape/viseme support, with fully automatic setup on import for most avatars (including eyes and blinking), making avatars significantly more expressive out of the box. This was largely possible because of you as well, as you sprung to support the Assimp project, a library that Neos uses for model import.

The full body support was overhauled as well, making the setup much easier and more automated, fixing tons of IK issues and extending support to up to 8 trackers. And to top it off, we’ve added our custom dynamic bones, which not only support collisions with any players in the world, but even grabbing and stretching as well!

But even more amazingly, I’ve seen the community build some impressive tools and scripts, taking their avatars even further. You’ve made variety of gesture systems, personal HUD’s and custom UI’s completely with the tools provided in-game, that could easily pass on as official in any other platforms.

https://www.youtube.com/watch?v=z5a_Npq1ttY

[h2]Creative Resonance, Streaming and Sharing[/h2]

With more community members we saw explosion in general creativity as well. The worlds, items, gadgets, weapons, vehicles and avatars have gotten more interactive and complex, some first NPC’s were built, we’ve gotten lots of new community made tools and utilities and we’ve gotten some quite comprehensive game worlds.

One of my favorite effects that I’ve noticed is something I like to call “creative resonance”. Often times when somebody builds something, I’d see other community members join in and start remixing the idea or building things around it. A wasp avatar? Suddenly a wasp nest map is made. A hook shot? A spiderman remix and big city to swing around!

Seeing your creativity resonate with each other is one of the things that makes me most proud and happy on Neos. My goal was to always build a shared space where people can seamlessly play and collaborate with each other, but seeing it happen so naturally and effectively went beyond what I imagined would be possible.

For over half a year now, Medra and the rest of the community have organized a weekly Creator Jam, where both old and new users come together and build based on a theme every Sunday. This has led to some impressive creations.

https://www.youtube.com/watch?v=JLPdZX-UvEU

Another major addition this year was a new streamer camera with lots of fancy features, control panels and tools and Twitch integration. Our goal was to make it easier for you and streamers to share your Neos experiences and creations with others.

We’ve seen many of you starting to stream regularly, share what you made and make amazing tutorials, all of which has helped the community grow even more and I believe it will even more in the upcoming months and years.

[h2]Headless server, Quest, eye tracking, science and more[/h2]

One of the things I didn’t quite expect to add this year, but ended up being the only thing I could work on in the midst of some unexpected family issues was a headless client/server, a version of Neos without any graphical interface that allows hosting worlds on dedicated machines.

To my relief it proved to be quite a welcome addition, as it helped hosting more stable and reliable worlds. Many of you started playing and experimenting with the headless and hosting your own worlds for the community.

We’ve also added support for lots of new hardware this year, making sure Neos can utilize the latest cutting edge hardware and we want to keep on that path.

Eye tracking is one of the new additions with the Vive Pro Eye headset, originally added for research purposes on request from the Sydney Human Research Factor (who have now built an amazing world to study arachnophobia), but also found great use for social, making the avatar’s facial expressions significantly more realistic.

https://www.youtube.com/watch?v=bQ887HX8Exc

Neos now also runs on the Oculus Quest with preview builds available to Patrons, but is currently held back by some missing optimizations and more focus on PCVR and general features. Despite that, we’ve seen some interesting work and experiments done with those builds, particularly at the SHFR labs.

We saw more utilization of Neos at schools, universities and other projects as well. Two semesters were now taught at the Czech Technical University in Prague using Neos. The Metamovie project has moved on to Neos as well and is currently working on their upcoming Alien Rescue film on our platform.

https://www.youtube.com/watch?v=dHKj29_L8Js

[h2]UI: The last frontier[/h2]

A lot of the work this year has brought Neos up to standards and make it into an experience where both existing and new users like to spend their time every day. With many of these major additions and improvements and countless smaller ones we saw the number of daily concurrent users steadily grow, usually hovering around 30-40 nowadays. Just barely a year ago we were celebrating when we’ve peaked at 30 during one of our big events.

However one of the major obstacles still remains, perhaps the most important one: UI/UX. Because of its importance, dependency on other features and the amount of work involved I’ve left it for last.

Fortunately the biggest technical part of the UI overhaul has been already done at the end of 2019: Text Rendering. Neos now has a completely custom high-performance system for rendering text, with support for custom TrueType/OpenType fonts and many more languages, with more features to come.

https://www.youtube.com/watch?v=n0erarwt5No

The remaining part is now design and implementation of new UI framework around this new text rendering, which brings me to...

What’s coming in 2020?


While the theme of 2019 has been bringing Neos up to standard of social VR, in 2020 we want to make its unique features come to the front and shine. I’m very excited for this year, because I will finally work on a lot of the “unicorn” features I wanted to work on for so long.

Here are some of the things that I plan on working this year. Take the list a bit of a grain of salt though, development and life itself can be quite unpredictable and priorities can change, especially if you, the community, will have new and unexpected needs.

The order of most of these is also arbitrary, I’m not sure yet which ones will get worked on first (except UI, physics and license system), we’ll see what you need the most as we go!

Also this list by no means complete, you’ll likely see new major features and additions that aren’t listed here.

[h2]The UI and UX[/h2]

Continuing from last year, the UI overhaul is the current priority. The work is already underway on the new UI framework, which should appear relatively soon, at which point I can start redesigning how the actual UI looks and how you interact with it.

I have originally hoped to have this part already finished by the last year, but the complexity of the text rendering and other unexpected problems and events have thrown a wrench into that and I am sorry for that.

I believe the time put into the UI will be worth it. The new framework is designed so it can last us years ahead and be built upon and the look and feel of the new UI and UX will dictate how will everyone interact with Neos for long time ahead.

This update has the potential to remove a major entry barrier for new users, both casual and creative and allow them to discover all the unique and rich functionality Neos offers and that makes it stand out from other platforms.

The current janky UI is the result of accumulated hacks, ad hoc tweaks and technical limitations all the way from early alpha versions, so with this, we get to define what the “proper” Neos UI is.

Once I am (mostly) finished with the technical part, I plan on doing a special livestream and discussion on how the new UI and UX should work, because I want you to be part of defining this, because you use Neos every day.

[h2]Rigidbody Physics[/h2]

This is another feature I hoped to finish by the end of 2019, so I’d like to apologize for missing that deadline again, especially since I promised it would be coming before 2020 and I don’t like breaking my promises.

However after the UI update physics will be the major priority and will happen soon after. The goal is to first upgrade to a version 2.0 of the BEPU physics engine (currently using 1.0), which alone offers major performance improvements and then follow by integrating components for rigidbody simulation and various constraints (joints, hinges, force fields).

Having full physics simulation will bring lots of new fun into Neos, allowing to build physics based games, environments, vehicles, gravity guns and just general goofing around!

[h2]Object ID / License system and Neos Store[/h2]

Seeing all the incredible creations you have made over the past year, I feel like this feature is overdue now. Neos is built for creators first and foremost, so we want to help not only protect your content, but also make a living doing what you love.

This system will allow to tie ownership information tightly to objects and corresponding assets and define what other users can do, based on which license they hold. Using this, you’ll have a flexible and powerful system to manage your assets.

You’ll be able to restrict who can visit your worlds, save your items or equip your avatars. Any object or asset could be unique, restricted to just a few users, or only allowed to be used if the user purchases it.

This is where Neos Credits (NCR) will come into play and turn into a full blown in-game currency, enabling an economy to naturally arise from your creativity and help support both you and Neos’ development.

[h2]Terrain System[/h2]

This is one of the features I’m itching to work on. For years I have been thinking about a terrain system for Neos, which would allow for building large and dynamic worlds right within VR. My plan is to have this system split into several modular parts, to provide large flexibility.

First major component would be the data provider - a heightmap, voxel map, procedural generation or combination of those, describing what the terrain should look like. The second one would be “mesher/rendender”, which will determine how is the terrain actually displayed. That way you can have a variety of realistic and stylized terrain (e.g. Minecraft-style maps).

As with everything, all will be accessible with scripting and fully dynamic, allowing to build everything from within VR and letting you build your own tools and interactions.

[h2]Screen Share[/h2]

Being able to see and share your screen is surprisingly commonly requested feature, so it is something I’d like to add this year.

Interacting with your desktop from within Neos should alone improve usability by leaps and bounds, as many of our users often use tools like Blender to prepare content for Neos or just want to check the Discord.

However you’ll be able to bring your screen into the shared world so other users can see what you’re doing as well. You could use this for collaboration, presentations, but also entertainment, being able to watch a video or have your friends watch a game you play.

[h2]Desktop client[/h2]

Neos’ major focus is on VR and it will always stay that way. I’ve intentionally chosen this because designing tools and interactions to work well both on screen requires too many compromises, sacrificing usability of both.

However with a lot o the major features out of the way and lots of users without VR headsets who wish to participate, I will begin work on a proper desktop mode and tools for Neos, making it much easier for non-VR users to participate and build.

This will benefit VR users as well, because my goal is to have Neos dynamically switch between desktop and VR, rather than requiring restart, so if you want to jump out of VR and but stay in the world with the friends you’ll be able to.

[h2]Mesh Editing[/h2]

Currently Neos largely relies on 3rd party software to build and import complex meshes and environments. One of my long term goals for Neos is bringing more and more tools natively into VR, so you can do more from within.

Being able to manipulate vertices and faces and create completely new ones will make Neos significantly more independent on external tools for many common tasks and bring collaboration to building models as well.

The importing is not going away, though, we’ll never quite fully match the rich functionality that other software has built over the years or decades. I think that having both options is incredibly powerful combination. See a small issue with a model from Blender? Now you can fix it right inside of Neos! (not “now” now of course, once this feature is built! ;) )

[h2]Builder Tool Improvements[/h2]

In its current state, many of the builder tools in Neos are only half-way done. I want to add more gizmos for visual editing, improve their behaviors and usability and also introduce a more physical way of working with abstract concepts, like vectors, rotations and various transformations.

One of the major editing additions will also be a proper selection system working across different tools. This would allow you to select multiple objects, either manually or by search and perform an operation on them as a group.

[h2]Custom Shader Support[/h2]

Another very commonly requested feature, but also one of the trickiest. Some of the major work to enable this has already been done in 2019, introducing custom shader package format and pipeline for compiling and handling them, but it’s still only used internally.

On the technical side, there isn’t too much preventing the support from being added, but shaders are highly tied to Unity Engine itself, with little control on our side, which makes it much harder to ensure long term backwards compatibility, something that I take very seriously for Neos.

Regardless, I want to investigate the options more and find the best solution for the time being. And who knows, maybe by the time other features are finished some new options will open up!

[h2]More Optimizations[/h2]

And as always, there are many more optimizations to come to Neos to make it run smoother and faster! I plan on significantly improving multi-threading and bringing it to more parts of the pipeline, adding tools for LOD groups and occlusion culling, adding more tools for baking and atlasing in-game and more.

I also want to make a switch to a more modern rendering pipeline, likely the new Universal Rendering Pipeline in Unity and if possible to Vulkan in place of DirectX 11. This should offer not only performance benefits, but also better growth and feature set in the future.

Generally optimizations are always an ongoing process, as new bottlenecks are being identified or more efficient implementations are found.

[h2]Procedural Generation?[/h2]

Now this one I’ve wanted to work on for a long while, but I’m not sure if it will get prioritized this year or not, but I’m adding it here regardless! When I was prototyping ideas for Neos, one of my favorite demos that I built was Neos: The Origin.

https://www.youtube.com/watch?v=nLXQXFMaMyU

A creative app, which allows you to build a simple environment with your hands, particularly shoot various plants and watch them grow in front of your eyes. This is a very fun and effective way of creating, as you can very quickly get natural and unique looking environments.

My goal is to expand this into a full framework called GeneX, which will allow for building a virtual “genome” using a set of operators, that describes how a plant (or any object) grows and then make instances of it in the environment.

Like I said, this one is definitely up in the air. If possible I’d love to get it this year, but we’ll have to wait and see!

[h2]And other things…[/h2]

As always, those are the major features that I plan on working on, but there will be many more minor ones sprinkled all around, depending on what’s needed at the time. I believe 2020 will be a very fun year for Neos, that will build upon all the hard work done in 2019 and allow us to focus on lots of cool and novel features as the platform matures.

Thank you all again for making this possible


Once again, thank you everyone for all your support, hard work and even friendship. I never quite imagined community growing like this and coming up with so many great ideas.

You are really what gives Neos its life and spirit and what makes me happy to spend the majority of my every day working on it to make it better. It wouldn’t be possible to build all those things without you.

Sometimes I would worry that big companies, with budgets magnitudes of order bigger than ours and teams of dozens of people would stomp us into the ground. It can be a daunting task at times knowing that we’re up against some giants in the industry.

But while many of those companies rise and fall, let go more employees than we ever had on our team and dissolve their communities, our community and numbers keep slowly, but steadily growing.

Our combined passion drives this project forward, it turns that little spark from last year into a full flame and I feel like it might be burning on the wick of a dynamite.

Knowing that passion and hard work is worth way more than a big venture capital gives me a lot of strength and hope for this project and I can’t wait to see where it takes us in 2020.

Thank you all!
Your Froox

0.8.19.34421 - Bugfixes for new issues due to rigorous resource cleanup and more

A bunch of bugfixes for the things broken after the introduction of more rigorous cleanups to fix memory/resource leaks and improve memory usage. Should hopefully fix all the major problems reported, let me know if you find some more. A few unrelated bugfixes as well.

[h2]Bugfixes:[/h2]
- Removed expensive (and unecessary) Unity resource cleanup when closing a world, causing Neos to freeze up for a second or two (based on report by @Alex from Alaska)
- Fixed exceptions when trying to register change of component update order when it was already disposed
- Fixed user respawning not working correctly (based on report by @Turk )
- Fixed invalid encoding of string primitves starting with '@' symbol, preventing worlds or items with those from being saved (reported and sample model provided by @EroSensei)

Following were discovered in log from @Coffee (should fix issues reported by @Groxxy and @Turk):
- Fixed AudioStreams throwing exceptions if read during/after world disposal
- Fixed static asset providers continuing load process after they have been disposed, potentially leading to asset resource leak
- Added guard to brushes, which prevents from trying to add new points if the currently drawn mesh has been destroyed in the meanwhile
- Disposed components that didn't go through the destroy process are now handled as removed as well (will evaluate to null in references)
-- This fixes certain bugs where they're accessed when they shouldn't be
-- This should fix dynamic bones randomly breaking due to invalid accesses when evaluating colliders
- Fixed logix driver nodes being evaluated after they were already destroyed
- Fixed FocusManager throwing exceptions on events after it has been disposed

0.8.19.16069 - Numerous memory/resource leak fixes, 2019 user badge and more

After two days of staring and poking through raw memory, losing sanity, summoning Cthulhu and plugging a bunch of memory leaks this build is finally ready! :smile: Has significant improvements in memory/resource usage, things should be garbage collected properly in most cases. There are some relatively drastic changes, be on a lookout for new bugs/crashes. Also a bunch of other improvements and fixes!

[h2]New Features:[/h2]
- Everyone who registered in 2019 now gets a special unique badge!
- Neos Dash now shows the specific Patreon perk as the account type
- Added extra storage for higher Patreon levels
- Added "Force Full Garbage Collection" to the debug dialog
- Added gc command to the Neos Headless to invoke immediate full garbage collection

[h2]Tweaks:[/h2]
- Built missing libmsdfgen.so for the Linux headless
- Built FreeType 2.10.0 from source for Linux for the Linux headless
- Neos will now skip attempts to initialize and connect to Emotiv Cortex service if it's not installed in the system
- Added protection against setting SimpleAvatarProtection to non-persistent (based on report by @Lewi-bean)
- Tooltip Multiplexer now properly emulates active tooltip, which fixes tooltip actions that depend on it
-- This fixes developer tooltip not being able to open inspector (reported by @CodeF53 and others (sorry I forgot who originally reported it ;_;))
- Upgraded to Unity 2019.3.0f4 (from 2019.3.0f3)

[h2]Optimizations:[/h2]
- Neos will now break up references to world on disposal of many classes, components and primitives, to allow for better/sooner garbage collection and reuse of memory
- Fixed variety of different memory leaks (thanks to @Shifty and @alex derpy avali 🐦 for providing some leads)
-- Fixed Local Slots not being properly disposed
-- Fixed Root components not being properly disposed
-- Fixed session network listener not being properly disposed
-- Fixed memory leak in background world processor, retaining references to the last background task, causing objects to not be deallocated as soon as they should or at all (if there's no more background work)
-- Fixed streams and users not being properly disposed (e.g. when user leaves or world is closed)
--- This also fixes native Opus Encoder/Decoder resource leak due to not being freed up
-- Fixed Unity connectors not always getting fully cleaned up
-- Fixed session thumbnail management from preventing world memory from being freed up
-- Fixed SyncElementBags not disposing their elements when disposed
- Fixed UserComponents not being properly disposed
- Fixed World sync members not being disposed when the world is closed
- Fixed encoding delta data of sync elements that were disposed in the meanwhile
- Fixed UserAudioStream not always unregistering audio source events
- Fixed TrackingOffsetCompensator not always unregistering input system events
- Fixed ClipboardImporter not always unregistering drag and drop event handler
- Fixed links propagating to the disposed parts of the hiearchy
- A bunch of others that I forgot to write down...

[h2]Bugfixes:[/h2]
- GET_String and POST_Strings now catch request exceptions and trigger OnError impulse, rather than logging exceptions (found in log from @Medra)
- Fixed resource leak in GET_String and POST_String LogiX nodes
- Fixed exception in updating MultiLineMesh with smooth normals on caps when there's currently just one segment (found in log from @Medra)
- Fixed error when saving worlds with nested worlds that require further processing (found in log from @Medra)
- Fixed audio events not running for all headless worlds

0.8.19.39448 - Assimp importer library update, up to 8 UV channel support

Another smallish build. Updated Assimp native importer library to latest, added arbitrary UV channel and 3D/4D UV support to internal mesh format.

[h2]Tweaks:[/h2]
- Added support for arbitrary number of UV channels for the MeshX (Neos' internal mesh format)
-- This is futureproofing the system. Currently up to 8 channels will be uploaded to the GPU
-- Model importer will import all channels that it finds
- Added support for 3D and 4D UV channels, in addition to standard 2D
-- This allows packing extra data into the meshes. This is necessary for certain TiltBrush brushes that @0utsider is working on
- Updated Assimp importer library to latest version from source (over 300 commits)
-- This has numerous improvements and fixes, including improved FBX compatibility (e.g. some more PBR textures should now import)
- Added protections against deleting or moving some crucial slots in the world that would cause issues (based on feedback by @Shifty and @RueShejn)

Happy Holidays from the Neos team and polish for the new text rendering system

Welcome again to our weekly update!

We all hope that you’re having great holidays so far and spending time with your friends and family. You may have thought we were taking a break for Christmas, but Frooxius has been hard at work getting the bugs out and polishing the new text rendering system!

Nevertheless this week has been quieter, as we all spent time with our family. If you’re reading this right at the release and have some time to spare, join us for a livestream right now on Twitch!



[h2]Text rendering system polish[/h2]

After the initial release of the new text rendering system this Sunday, it has received a lot of extra polish to fix-up various issues and improve the quality of the rendering. For example glyphs randomly becoming blank were fixed, as well as rounding errors in calculations causing some of the symbols to become deformed or smaller than they should be, especially characters like dashes, hyphens, commas, periods, semicolons and so on.

Unicode coverage has been improved as well, the system now handles characters outside the Basic Multilingual Plane as well, assuming they don’t require complex shaping (e.g. Arabic) and extra fallback fonts (Noto Symbols 2 and Noto Math) were added for various symbols, like arrows, shapes and other small graphics.

[previewyoutube][/previewyoutube]

Parsing of some missing tags and parameters was also fixed up as well as some line-wrapping layout issues, so most of the existing text should work the same way it did before with no manual fixes necessary. If you still encounter any issues, let us know in the Discord!

The system should be pretty reliable now, so the work has already started on designing a new UI framework around it to replace the current system and allow us to start on the UI/UX overhaul.

[h2]Community Highlights[/h2]

Secret Santa and Christmas Megajam!



The Secret Santa gift giving ceremony has gone underway and all of the participants have opened their presents, with many others joining in as well!

Frooxius got a bowl of delicious fruit from ProbablePrime, which he promptly scarfed down!



So many more gifts were given this night, and you can see all of the presents being unwrapped here in the Christmas Megajam video!



[previewyoutube][/previewyoutube]

Kurotori’s Japanese Keyboard

We’ve had a user making great use of our new Text Rendering engine, utilizing LogiX and built-in buttons to make a keyboard for typing in Japanese. Awesome job, Kurotori!



[h2]What’s next?[/h2]
The work has already begun on engineering a brand new UI framework for Neos, that will replace the current one built around the Unity UI, which is currently one of the oldest and ugliest parts of the codebase.

Similarly to the text rendering, this will be a fully custom system, designed to be asynchronous, multi-threaded and generally highly performant.

It will also give us a full level of control over the UI and ability to quickly and easily build new UI elements and interactions. Using this we’ll be able to overhaul the actual look and feel of Neos’ UI and ensure long term growth of the platform.

We thank you all for being with us throughout this year! The Neos team is hoping you’ve had a happy holiday season and wish you to have a very happy New Year. We’re all very excited to see what the next year and even the next decade brings, and we’re so happy to have you along with us!