Sampling Static Texture pixels in LogiX, Subtitle Import, Security Report Policy
Hello everyone and welcome to our weekly update!
This week we have a number of smaller, but hopefully very useful additions. Notably, you can now sample colors from static 2D and 3D textures from LogiX! This opens up new options for content creation, by letting you encode various data into texture assets and then read them at runtime. Similarly you can now also sample Animation tracks as well.
Neos also has native subtitle support, letting you import srt, sub, ssa, ttml and vtt directly into Neos (or dynamically from a URL) as a native Neos animation. Those can not only be synced with video and audio files (with new playback sync components to help), but sampled with LogiX as well, providing another building block for your experiences.
We added a number of other features as well, like Parent Data Link mechanism, Tube Wire procedural mesh, components for easily playing sound effects on button presses or parented objects and more.
Along with a number of security improvements based on community reports, we also published a Security Policy document on our GitHub, which details how to best make reports and to clear up some circulating misconceptions, notably that reporting security issues will lead to a ban - we want to encourage responsible security reporting, so we now offering CDFT bounties! To learn more, please give the document a read: https://github.com/Neos-Metaverse/NeosPublic/blob/master/.github/SECURITY.md
We have also been devoting some time to an educational project over past week, implementing things like a CIrcuit Simulator and part of The Universe experience. Those are mostly side projects for educational purposes right now though, but we'll be bringing portions of them to the broader community in the future. If you're curious to see some progress on them regardless, check out the videos below!
[h2]August Recap Livestream - MMC, Neos Festa, New Team Member, BEPUv2, Medra Interview[/h2]
On our last Friday livestream we did our regular beginning of month recap of the previous month's events, as well as any upcoming happenings. If you missed it, you can watch the archival footage below.
We covered the MMC and Neos Festa events and had a special guest - Medra joined us to talk about the currently ongoing Metaverse Maker Competition that's returning for its second year.
We covered the BEPUv2 upgrade and what it brought us as well what's coming in the future, as well as other events, like bringing Dante into the Neos Team to help bridge communication between moderation and the team and help with a number of other things.
[previewyoutube][/previewyoutube]
[h2]Sampling static textures (2D and 3D) in LogiX[/h2]
You can now read texture data from LogiX! With a new set of nodes for working with asset files it is now possible to get a reference to a texture and read both its metadata (format and size) and individual pixels.
You can either perform bilinear sampling using a UV(W) coordinate, which will interpolate between the pixels and allow you to use normalized coordinates without considering the actual resolution or use another node to get a specific pixel.
[previewyoutube][/previewyoutube]
Both 2D and 3D static textures are supported. In order to read the texture data, it needs to be marked as “Readable” and not use block compression. This will keep the texture data in-memory in an accessible format, increasing the RAM usage, so it’s recommended to only use it for textures that need it.
This feature can open up new options for building maps and tools. For example you can encode any data into a texture and sample based on the user's position in the world to vary some parameters - like making a Geiger counter!
[h2]Sampling animations in LogiX[/h2]
Similarly to textures, it’s now possible to also read animation data directly in LogiX! By referencing an animation asset, you can read the values of its animation tracks at a given position in time.
We have added nodes to help you find the right animation track by its name as well. In order for this to work, you need to make sure that you’re sampling the track with correct data type - for example float3 for positions, floatQ for rotations and so on.
If you’re animating some parameters it might be better to use the traditional Animator for better performance, but this feature should open some new options for building content in Neos.
[h2]Native subtitle support[/h2]
Neos now has native subtitle support! You can import srt, sub, ssa, ttml and vtt files directly into Neos, to have them converted into a native Neos animation, that can be synced with a video or audio file or any other playback in Neos.
This allows you to easily bring external subtitles not only for videos, but also as a building block for your own experiences. They can be sampled with the new LogiX nodes for sampling animations as well if you need more fine-grained control over them.
There is also a special component DynamicSubtitleProvider, which will seamlessly import subtitles from a given URL and provide them as animation files, without having to manually import the file by hand first.
[h2]Parent Data Link mechanism[/h2]
Recent build has added a new mechanism for dynamically linking data based on hierarchy. It is now possible to dynamically drive values and references from its parent with ParentValue, ParentReference and ParentValueLink and ParentReferenceLink components.
[previewyoutube][/previewyoutube]
This system is akin to a very lightweight version of the dynamic variables. Instead of the entire hierarchy it links up data between the immediate parent and its children and it’s only one way - the child values are always driven from the parent (although they do support writeback).
This can be useful for a number of workflows, where a child object can get data based on where it’s parented from. For example snapping objects under a certain parent can link up a reference, allowing you to build things like virtual wires that interconnect elements.
[h2]Fixing time desync on headless sessions[/h2]
Recently we discovered a long standing problem with headless that was responsible for a number of strange behaviors on any headless session, that were mostly looked over. Most notable effect was video playback ending up significantly out of sync for multiple people.
We found that the root cause was due to the world time (a “master clock” to which everything else is synced) never being synced on headless sessions, resulting in different users having a chance of a random drift. This was due to the time corrections being done from stream packets, but headless never has any of its own!
This would not only affect videos, but any other kind of playback - audio files, animations, effects. Other behaviors would be affected too - any time based LogiX, stopwatch or time elapsed nodes and more.
We have released a fix that ensures that time synchronization packets are always sent, which fixes the whole range of issues and we have been tuning and improving the system to eliminate other resulting sources of random jumps and skips from the synchronization system based on reports from our community.
If you find any more issues with time synchronization, please let us know on our GitHub!
[h2]Other additions and quality of life improvements[/h2]
We’ve made a number of other smaller additions over the course of the week. There is now a new procedural mesh TubeWireMesh, which is ideal for making things like cables and there’s a mechanism for duplicating other objects when a particular object gets duplicated.
We added components for easily playing sounds on button presses or when children get parented and unparented under a Slot. PhysicalButtons also have a new “Hold” mode which makes them behave like the physical toggle switches - you press them once and they stay pressed in, you press them again and they pop out!
[previewyoutube][/previewyoutube]
Some other new components include VectorSwizzleDrivers (they let you rearrange components of any vector into another vector) and various bug fixes for remaining issues after the BEPUv2 upgrade.
If you’d like to see all the changes, check out the release notes on Steam or in our official Discord!
[h2]Security improvements[/h2]
We’ve made a number of security improvements based on reports by our community. The SimpleAvatarProtection and world delete confirmations have now been completely moved to userspace to prevent potential tampering and extra protections have been made for the laser offset from worldspace.
We also covered cases of missing validations when adding generic components with Nullable types or when trying to access extremely long components paths. Neos will now also send an email whenever you reset your password, in case someone makes an unauthorized change.
[h2]Official Security Policy[/h2]
To help better understand our security policy and how to report security issues and exploits, we have created a new document on our GitHub detailing the process and how to best report them. If you’d like to read more about it, please give it a read here.
While we have already been receiving a number of security reports through our Moderation Ticket system that helped improve Neos’ security, we have been noticing a number of misconceptions circulating in the community, for example that reporting security issues will result in a ban, which is untrue.
Hopefully the document will help clarify those misconceptions and offer some incentives in the form of CDFT as well. It also explains how to make a good quality report to ensure we have all the necessary information to implement a fix.
[h2]Work in progress on educational project - Circuit Simulator[/h2]
Neos has its origins as an educational project and is still actively used as such in a number of schools and universities. As part of an ongoing educational project, we have been recently required to focus on a few new educational features, notably for teaching electronics.
Over the past week, we have been integrating a functional circuit simulation with Neos, that will allow students to learn about electronics in a more intuitive manner, by being able to see the voltages and current flowing through a virtual model of educational circuit board.
We are looking into the possibility of releasing the simulation components used to build the circuit board for everyone, so anyone using Neos can toy around with those and build their own virtual circuits, since it can be quite fun in the multiplayer setting and with all the other tools and visualizations that Neos offers.
[previewyoutube][/previewyoutube]
However currently we are not certain whether it’s possible or not. In any case if you’re curious about what’s possible to do with Neos to help improve education, you can check out the following videos! They showcase a number of working circuits, notably flip flop circuits made out of NAND gates and NPN transistors.
https://www.youtube.com/watch?v=2iW4cLKuuCQ
https://www.youtube.com/watch?v=gvHqKhhhL7A
----------------------------------------
Anyway this is all for this week! And as usual, big thanks to everyone for supporting this project and helping it move forward every week, we couldn't do it without you!
This week we have a number of smaller, but hopefully very useful additions. Notably, you can now sample colors from static 2D and 3D textures from LogiX! This opens up new options for content creation, by letting you encode various data into texture assets and then read them at runtime. Similarly you can now also sample Animation tracks as well.
Neos also has native subtitle support, letting you import srt, sub, ssa, ttml and vtt directly into Neos (or dynamically from a URL) as a native Neos animation. Those can not only be synced with video and audio files (with new playback sync components to help), but sampled with LogiX as well, providing another building block for your experiences.
We added a number of other features as well, like Parent Data Link mechanism, Tube Wire procedural mesh, components for easily playing sound effects on button presses or parented objects and more.
Along with a number of security improvements based on community reports, we also published a Security Policy document on our GitHub, which details how to best make reports and to clear up some circulating misconceptions, notably that reporting security issues will lead to a ban - we want to encourage responsible security reporting, so we now offering CDFT bounties! To learn more, please give the document a read: https://github.com/Neos-Metaverse/NeosPublic/blob/master/.github/SECURITY.md
We have also been devoting some time to an educational project over past week, implementing things like a CIrcuit Simulator and part of The Universe experience. Those are mostly side projects for educational purposes right now though, but we'll be bringing portions of them to the broader community in the future. If you're curious to see some progress on them regardless, check out the videos below!
[h2]August Recap Livestream - MMC, Neos Festa, New Team Member, BEPUv2, Medra Interview[/h2]
On our last Friday livestream we did our regular beginning of month recap of the previous month's events, as well as any upcoming happenings. If you missed it, you can watch the archival footage below.
We covered the MMC and Neos Festa events and had a special guest - Medra joined us to talk about the currently ongoing Metaverse Maker Competition that's returning for its second year.
We covered the BEPUv2 upgrade and what it brought us as well what's coming in the future, as well as other events, like bringing Dante into the Neos Team to help bridge communication between moderation and the team and help with a number of other things.
[previewyoutube][/previewyoutube]
[h2]Sampling static textures (2D and 3D) in LogiX[/h2]
You can now read texture data from LogiX! With a new set of nodes for working with asset files it is now possible to get a reference to a texture and read both its metadata (format and size) and individual pixels.
You can either perform bilinear sampling using a UV(W) coordinate, which will interpolate between the pixels and allow you to use normalized coordinates without considering the actual resolution or use another node to get a specific pixel.
[previewyoutube][/previewyoutube]
Both 2D and 3D static textures are supported. In order to read the texture data, it needs to be marked as “Readable” and not use block compression. This will keep the texture data in-memory in an accessible format, increasing the RAM usage, so it’s recommended to only use it for textures that need it.
This feature can open up new options for building maps and tools. For example you can encode any data into a texture and sample based on the user's position in the world to vary some parameters - like making a Geiger counter!
[h2]Sampling animations in LogiX[/h2]
Similarly to textures, it’s now possible to also read animation data directly in LogiX! By referencing an animation asset, you can read the values of its animation tracks at a given position in time.
We have added nodes to help you find the right animation track by its name as well. In order for this to work, you need to make sure that you’re sampling the track with correct data type - for example float3 for positions, floatQ for rotations and so on.
If you’re animating some parameters it might be better to use the traditional Animator for better performance, but this feature should open some new options for building content in Neos.
[h2]Native subtitle support[/h2]
Neos now has native subtitle support! You can import srt, sub, ssa, ttml and vtt files directly into Neos, to have them converted into a native Neos animation, that can be synced with a video or audio file or any other playback in Neos.
This allows you to easily bring external subtitles not only for videos, but also as a building block for your own experiences. They can be sampled with the new LogiX nodes for sampling animations as well if you need more fine-grained control over them.
There is also a special component DynamicSubtitleProvider, which will seamlessly import subtitles from a given URL and provide them as animation files, without having to manually import the file by hand first.
[h2]Parent Data Link mechanism[/h2]
Recent build has added a new mechanism for dynamically linking data based on hierarchy. It is now possible to dynamically drive values and references from its parent with ParentValue, ParentReference and ParentValueLink and ParentReferenceLink components.
[previewyoutube][/previewyoutube]
This system is akin to a very lightweight version of the dynamic variables. Instead of the entire hierarchy it links up data between the immediate parent and its children and it’s only one way - the child values are always driven from the parent (although they do support writeback).
This can be useful for a number of workflows, where a child object can get data based on where it’s parented from. For example snapping objects under a certain parent can link up a reference, allowing you to build things like virtual wires that interconnect elements.
[h2]Fixing time desync on headless sessions[/h2]
Recently we discovered a long standing problem with headless that was responsible for a number of strange behaviors on any headless session, that were mostly looked over. Most notable effect was video playback ending up significantly out of sync for multiple people.
We found that the root cause was due to the world time (a “master clock” to which everything else is synced) never being synced on headless sessions, resulting in different users having a chance of a random drift. This was due to the time corrections being done from stream packets, but headless never has any of its own!
This would not only affect videos, but any other kind of playback - audio files, animations, effects. Other behaviors would be affected too - any time based LogiX, stopwatch or time elapsed nodes and more.
We have released a fix that ensures that time synchronization packets are always sent, which fixes the whole range of issues and we have been tuning and improving the system to eliminate other resulting sources of random jumps and skips from the synchronization system based on reports from our community.
If you find any more issues with time synchronization, please let us know on our GitHub!
[h2]Other additions and quality of life improvements[/h2]
We’ve made a number of other smaller additions over the course of the week. There is now a new procedural mesh TubeWireMesh, which is ideal for making things like cables and there’s a mechanism for duplicating other objects when a particular object gets duplicated.
We added components for easily playing sounds on button presses or when children get parented and unparented under a Slot. PhysicalButtons also have a new “Hold” mode which makes them behave like the physical toggle switches - you press them once and they stay pressed in, you press them again and they pop out!
[previewyoutube][/previewyoutube]
Some other new components include VectorSwizzleDrivers (they let you rearrange components of any vector into another vector) and various bug fixes for remaining issues after the BEPUv2 upgrade.
If you’d like to see all the changes, check out the release notes on Steam or in our official Discord!
[h2]Security improvements[/h2]
We’ve made a number of security improvements based on reports by our community. The SimpleAvatarProtection and world delete confirmations have now been completely moved to userspace to prevent potential tampering and extra protections have been made for the laser offset from worldspace.
We also covered cases of missing validations when adding generic components with Nullable types or when trying to access extremely long components paths. Neos will now also send an email whenever you reset your password, in case someone makes an unauthorized change.
[h2]Official Security Policy[/h2]
To help better understand our security policy and how to report security issues and exploits, we have created a new document on our GitHub detailing the process and how to best report them. If you’d like to read more about it, please give it a read here.
While we have already been receiving a number of security reports through our Moderation Ticket system that helped improve Neos’ security, we have been noticing a number of misconceptions circulating in the community, for example that reporting security issues will result in a ban, which is untrue.
Hopefully the document will help clarify those misconceptions and offer some incentives in the form of CDFT as well. It also explains how to make a good quality report to ensure we have all the necessary information to implement a fix.
[h2]Work in progress on educational project - Circuit Simulator[/h2]
Neos has its origins as an educational project and is still actively used as such in a number of schools and universities. As part of an ongoing educational project, we have been recently required to focus on a few new educational features, notably for teaching electronics.
Over the past week, we have been integrating a functional circuit simulation with Neos, that will allow students to learn about electronics in a more intuitive manner, by being able to see the voltages and current flowing through a virtual model of educational circuit board.
We are looking into the possibility of releasing the simulation components used to build the circuit board for everyone, so anyone using Neos can toy around with those and build their own virtual circuits, since it can be quite fun in the multiplayer setting and with all the other tools and visualizations that Neos offers.
[previewyoutube][/previewyoutube]
However currently we are not certain whether it’s possible or not. In any case if you’re curious about what’s possible to do with Neos to help improve education, you can check out the following videos! They showcase a number of working circuits, notably flip flop circuits made out of NAND gates and NPN transistors.
https://www.youtube.com/watch?v=2iW4cLKuuCQ
https://www.youtube.com/watch?v=gvHqKhhhL7A
----------------------------------------
Anyway this is all for this week! And as usual, big thanks to everyone for supporting this project and helping it move forward every week, we couldn't do it without you!