Tutorial: Turn Oculus Quest VR into AR/XR!


4 min read
22 Jul
22Jul

Many people asked me how I did it in my YouTube videos without a single line of code and any software / hardware hacks, so here’s the tutorial!As you may know, Oculus (Facebook) has blocked video recording on AR-Passtrhough mode (starting from the latest V30 update). So I found a creative solution for real-time AR/XR Pas-through mode, and it even enables you to record your environment in FULL COLORS + XR Holograms on top of it! 



Now you can create XR prototypes in Tvori or Unity, XR apps, and new use-cases without buying an expensive MR/XR glasses such as Varjo, or a real MR/AR glasses such as the Microsoft HoloLens and the Magic Leap one. How cool is that?!


I use my old Galaxy S8 with a broken screen, as an External Camera for my Quest 2


What you’ll need?

  1. A Smartphone or an IP Camera – To attach to your Oculus Quest 2 (I use my old Galaxy S8 with a broken screen).
  2. OBS Studio – To capture both of the streams from the Quest 2 and the Phone / IP Camera simultaneously.
  3. Install scrcpy – To wirelessly stream your phone to your PC/Mac/Linux. I’ll demonstrate how to do it on Mac & Linux via Terminal, Windows is a bit different, check SCRCPY’s GitHub page for more details about Windows.


On your phone (Similar on Mac & Linux via Terminal):

  1. Connect your phone to your PC via USB cable to run ADB (if ADB is not installed, follow SCRCPY Github page).
  2. Connect your phone to the same Wi-Fi as your computer.
  3. Open a Terminal on your computer and type: “adb tcpip 5555”. hit ENTER! And unplug your phone.
  4. Then type: adb connect X.X.X.X:5555 (replace XXXX with your phone’s IP).
  5. Type “scrcpy -b2M -m800 –max-fps 15” in your terminal and hit ENTER.
  6. You should see your phone on your computer screen and even control it with a mouse and keyboard.
  7. Download OpenCamera from Google play store on your phone and open the app on your phone (on Landscape).
  8. Find the best & secure way yo attach your phone to your Quest 2 without blocking Quest’s B&W tracking cameras!


On your Oculus Quest & Quest 2:

  1. Go your Settings > Experimental Features > Display Theme, and set it to “Light” (the device will reboot).
  2. Go to quick settings and enable Passthrough Home.
  3. Go to the Sharing Menu, click “Cast” and then select “Computer” & click “next”.
  4. Open up Google Chrome on your computer and go to: oculus.com/casting
  5. Stick a tape on your Quest’s Proximity Sensor (to keep it awake).

On your computer:You should see both streams from your Quest and Your phone on you computer screen. Now, Open up OBS Studio and configure it exactly like this:

  1. Click the “+” icon under “Sources” and select “Window Capture”, call it “Phone” and click OK.
    Now, next to “Window” within the drop-down select your phone by the device name / model and click OK.
  2. Click the “+” icon under “Sources” and select “Window Capture”, call it “Quest2” and click OK. Now, next to “Window” within the drop-down select “Oculus – Google Chrome” and click OK.
  3. Right Click on the “Quest2” source you just made, within the modal click on the “+” icon and select “Chroma-Key”. Set it to “Custom” and fill these parameters:
Combine both Live-Streams and apply Chroma-Key in OBS Studio


Key Color: #00007F | Similarity: 1 | Smoothness: 270 | Key Color Spill: 1 | Opacity: 1.0000 | Contract: 0.02

  • Click “Close” to apply! Now you can click “Start Recording” on OBS Studio and experience XR Passthrough Mode 
  • If you don’t have 2 monitors on your PC, go back (Alt+Tab) to Oculus stream on Chrome and click “Full screen”.
  • To make full screen 16:9 accept ratio, click on the Cog icon and change the toggle to “Widescreen (crop).



One more thing…

  • Black / Dark background in VR apps will be “removed” on OBS studio. It will be treated as a Transparent AR app 
  • You’ll still see the world in Black & White on Quest’s AR-Pass-through mode, But the recording will be in full colors!
  • OBS will save both profiles / sources for future use, so you won’t have to configure it again every time you use it.

For Non-techie creators:If you are not a technical person, you can skip all of the steps above, and simply attach your Phone to your VR headset. Record a video feed on both of them separately, and later on copy it to your computer in order sync and combine them with your favorite Editing tool (Premiere, Final Cut, KdeNlive, etc.), but make sure to set Chroma-Key, just like I did!


Personally, I find the OBS method more effective because it’s a Live Stream & recording of everything I see and experience in real time as a User. For example – If I’m testing a new app, Unity 3D project, or Tvori Prototype, I need to be able to test it in real time as I’m wearing the headset, and also live-stream it on the internet or to clients / colleagues in order to get feedback.

But, eventually, it’s up to you as a creator. You can choose the best method for you. I tried the Video Editing method and it looks like this: 



I hope you’ll find it useful, and I really hope that Facebook (Oculus) will take a different & Open approach and let us to do whatever we want to do with our VR headsets, while they also do their best in terms of “Privacy”. Recording AR Passtr-hough mode with SCRCPY was a decent solution for Pro-Users / Creators and without any privacy issues.Alternative – You can take a look at ZED Mini Mixed Reality cameras. But, I haven’t tried it myself…Cheers. 


You should also check out the following articles:

  1. Apple’s first headset will focus on “high-quality” games, reporter claims
  2. Pets, people, and Android Phone Notifications are now visible in Quest 2.
  3. 30 The top virtual reality games in 2021
  4. November 26, 2021 is Black Friday. Deals on Video Games and Virtual Reality in 2021
  5. Facebook wants to build a metaverse. Microsoft is creating something even more ambitious.
  6. Metaverse’s Cryptocurrency Leaps in Price After Facebook Rebrands as Meta
  7. Best VR Gifts for Christmas in 2022
  8. What is Augmented reality (AR) ?
  9. How to succeed in the virtual reality world of tomorrow?
  10. Best VR Headset cyber Monday 2021
  11. Books you must read about virtual reality
  12. Best New Augmented Reality Books To Read In 2021
  13. virtual reality Life 2029


Subscribe now to our YouTube channel


Subscribe now to our Facebook Page


Subscribe now to our twitter page


Subscribe now to our Instagram


Subscribe To my personal page on linkedin


Subscribe To my personal page on tiktok page for those who love to dance :)


Don't forget to be my friend. Sign up for my friend's letter. So I can tell you ALL about the news from the world of VR&AR, plus as my new friends I will send you my new article on how to make money from VR&AR for free.

Comments
* The email will not be published on the website.