Paradiddle Update: Mixed Reality Streaming/Recording Support
I'm excited to announce that Paradiddle finally has mixed reality support! To use this feature, you'll have to be on the beta branch of the app. You can do this by right clicking Paradiddle in Steam, selecting Properties, and selecting "beta" under the Betas tab.
For any VR streamers/YouTubers out there, or anyone who'd like to better showcase their VR jamming skills, I'm hoping that this'll be a huge help. You can see an example of what this looks like in this video (note that this was a relatively quick test, and with a good enough lighting and green screen set up, you can spend some more time tweaking the parameters to get rid of any weird green screen artifacts you can see in the video): https://www.youtube.com/watch?v=hVUgO1jznpU MR support was added thanks to LIV, who are making it super easy for developers to add this functionality in their apps. They also have the LIV Client on Steam, which will let you easily composite the video and get the result in the YouTube video I posted above. You can find the instructions of this setup here: https://www.youtube.com/watch?v=Uj2bbgmVPgs&feature=youtu.be I'd also highly recommend joining their Discord for additional support and guidance, the developers are super friendly, and they're on there to answer people's questions pretty much all the time. I'm also setting up a channel in the Paradiddle Discord to support anyone who's trying to get this running.
With all of that being said, the reason this is in beta is that the LIV SDK for Unreal Engine currently has a few limitations, and I've had to hack my way around them a bit to get around them until these get fixed. Here's what to remember and the hacks you need to do to get this properly working:
The steps involved with getting the Oculus mixed reality set up working are a little weird for now. This is mostly because the LIV MR integration is only supposed to work with SteamVR, but for the time being I was able to find a workaround with the Oculus SDK. In upcoming updates, I'll introduce actual SteamVR support for Oculus and make sure the steps are the same as Oculus as they are for Vive users.
That's everything to keep in mind! Feel free to reach out to me on Discord or the Steam Forums if you're having trouble setting this up, or you can also talk to any of the LIV devs if you're having trouble getting this running using their software. As these limitations and issues are resolved, I'll make sure to update this post with the latest instructions on how to get everything running. Also, just a reminder that the 1st official Paradiddle VR Drum Cover Contest is accepting submissions until May 27th! If you'd like to share your VR drumming skills while winning some cool prizes, you should definitely consider sending in a drum cover.
Thanks!
Emre
[email protected]
Official Paradiddle website: http://paradiddleapp.com
Paradiddle drum covers Youtube playlist (your cover could be on here too!)
FAQ: http://paradiddleapp.com/faq
Join us on Discord!: https://discordapp.com/invite/wjHnTVr
Reddit: http://reddit.com/r/paradiddle
For any VR streamers/YouTubers out there, or anyone who'd like to better showcase their VR jamming skills, I'm hoping that this'll be a huge help. You can see an example of what this looks like in this video (note that this was a relatively quick test, and with a good enough lighting and green screen set up, you can spend some more time tweaking the parameters to get rid of any weird green screen artifacts you can see in the video): https://www.youtube.com/watch?v=hVUgO1jznpU MR support was added thanks to LIV, who are making it super easy for developers to add this functionality in their apps. They also have the LIV Client on Steam, which will let you easily composite the video and get the result in the YouTube video I posted above. You can find the instructions of this setup here: https://www.youtube.com/watch?v=Uj2bbgmVPgs&feature=youtu.be I'd also highly recommend joining their Discord for additional support and guidance, the developers are super friendly, and they're on there to answer people's questions pretty much all the time. I'm also setting up a channel in the Paradiddle Discord to support anyone who's trying to get this running.
Limitations/Hacks
With all of that being said, the reason this is in beta is that the LIV SDK for Unreal Engine currently has a few limitations, and I've had to hack my way around them a bit to get around them until these get fixed. Here's what to remember and the hacks you need to do to get this properly working:
Vive
- Currently only the 1st Vive Tracker in your system can be used as the external camera. If you don't have a Vive Tracker, don't worry, because LIV provides a virtual tracker for you that you can use as a static camera.
- This point specifically is for people looking to use the LIV Virtual Tracker as their external camera. Before you calibrate your camera, launch the LIV Client, open the Compositor, and make sure your LIV Virtual Tracker is enabled. You can check this by going to the Camera Tab: Check Enabled under Virtual Camera, select Device: Static(Virtual), and Class: Tracker. The LIV Tracker icon in your SteamVR Dashboard should now light up green. Now close the compositor window but keep the LIV Client window open.
- When you do your device calibration in LIV: vivr, make sure the tracker you select under "Camera Tracker" shows up as Connected and Tracking. So if you were using the LIV Virtual Tracker, you'd select the one that says "Generic Tracker - liv virtual camera". This is why we made sure that the virtual tracker was lighting up green in the SteamVR window in the previous step.
- After you finish your calibration through LIV: vivr, the resulting externalcamera.cfg file you get will be set up for Unity. Here's how you get the one you need for Paradiddle: Open up the LIV Client, and under Camera, press Edit on the camera profile you had created. In the Calibration section on the Device tab that should have opened up, click Import -> From File and select the externalcamera.cfg you exported from LIV: vivr. Now click Export -> To File -> For Unreal and save this to C:\Program Files (x86)\Steam\steamapps\common\Paradiddle\Paradiddle\Binaries\Win64 (or the equivalent directory under wherever your Steam installation of Paradiddle sits). Now the app should run with the 4 quadrant view that you need to get the mixed reality output through LIV.
Oculus
The steps involved with getting the Oculus mixed reality set up working are a little weird for now. This is mostly because the LIV MR integration is only supposed to work with SteamVR, but for the time being I was able to find a workaround with the Oculus SDK. In upcoming updates, I'll introduce actual SteamVR support for Oculus and make sure the steps are the same as Oculus as they are for Vive users.
- The Oculus MR integration currently only supports the LIV virtual controller/tracker as an external camera. Before you calibrate your camera, launch the LIV Client, open the Compositor, and make sure your LIV Virtual Tracker is enabled. You can check this by going to the Camera Tab: Check Enabled under Virtual Camera, select Device: Static(Virtual), and Class: Tracker. The LIV Tracker icon in your SteamVR Dashboard should now light up green. Now close the compositor window but keep the LIV Client window open.
- When you do your device calibration in LIV: vivr, make sure the tracker you select under "Camera Tracker" shows up as Connected and Tracking. So if you were using the LIV Virtual Tracker, you'd select the one that says "Generic Tracker - liv virtual camera". This is why we made sure that the virtual tracker was lighting up green in the SteamVR window in the previous step.
- After you finish your calibration through LIV: vivr, the resulting externalcamera.cfg file you get will be set up for Unity. Here's how you get the one you need for Paradiddle: Open up the LIV Client, and under Camera, press Edit on the camera profile you had created. In the Calibration section on the Device tab that should have opened up, click Import -> From File and select the externalcamera.cfg you exported from LIV: vivr. Now click Export -> To File -> For Unreal and save this to C:\Program Files (x86)\Steam\steamapps\common\Paradiddle\Paradiddle\Binaries\Win64 (or the equivalent directory under wherever your Steam installation of Paradiddle sits). Don't launch the app yet, finish the rest of the steps or your camera will show up at the wrong spot.
- IMPORTANT: Go through the previous calibration steps, without the virtual tracker/controller enabled this time. Export this file out of vivr, give it a different name (like externalcamera_heightonly.cfg), export it as an Unreal calibration file just like the previous step outlined (you don't have to save this one in your Steam Paradiddle directory). Now open this Unreal version of the file, take the value next to z= , and replace the z= value in the Unreal version of the externalcamera.cfg file you got in your first calibration, when the virtual tracker/controller was enabled.
- For instance, when I calibrated my camera with my virtual controller enabled and exported that as an Unreal file, the z= value I got was around -0.5. When I did the calibration and exported without my virtual camera enabled, the value I got was around 1.68. So I open up the initial externalcamera.cfg, and replace the -0.5 with 1.68. That's the externalcamera.cfg file that should be placed next to your Paradiddle executable. Now you can launch Paradiddle, and the app should run with the 4 quadrant view that you need to get the mixed reality output through LIV. Currently for this hacky Oculus workaround it won't matter whether your virtual controller/tracker is enabled or not as you run the app, but in future updates, you'll want to keep that enabled.
That's everything to keep in mind! Feel free to reach out to me on Discord or the Steam Forums if you're having trouble setting this up, or you can also talk to any of the LIV devs if you're having trouble getting this running using their software. As these limitations and issues are resolved, I'll make sure to update this post with the latest instructions on how to get everything running. Also, just a reminder that the 1st official Paradiddle VR Drum Cover Contest is accepting submissions until May 27th! If you'd like to share your VR drumming skills while winning some cool prizes, you should definitely consider sending in a drum cover.
Thanks!
Emre
[email protected]
Official Paradiddle website: http://paradiddleapp.com
Paradiddle drum covers Youtube playlist (your cover could be on here too!)
FAQ: http://paradiddleapp.com/faq
Join us on Discord!: https://discordapp.com/invite/wjHnTVr
Reddit: http://reddit.com/r/paradiddle