Turn Oculus Quest VR into AR/XR with this tutorial!

Many people asked me how I did it in my YouTube videos without a single line of code and any software / hardware hacks, so here’s the tutorial!As you may know, Oculus (Facebook) has blocked video recording on AR-Passtrhough mode (starting from the latest V30 update). So I found a creative solution for real-time AR/XR Pas-through mode, and it even enables you to record your environment in FULL COLORS + XR Holograms on top of it! 

Now you can create XR prototypes in Tvori or Unity, XR apps, and new use-cases without buying an expensive MR/XR glasses such as Varjo, or a real MR/AR glasses such as the Microsoft HoloLens and the Magic Leap one. How cool is that?!

What you’ll need?

1. A Smartphone or an IP Camera–To attach to your Oculus Quest 2 (I use my old Galaxy S8 with a broken screen).
2. OBS Studio – To capture both of the streams from the Quest 2 and the Phone / IP Camera simultaneously.
3. Install scrcpy–To wirelessly stream your phone to your PC/Mac/Linux. I’ll show how to do it on Mac & Linux via Terminal, Windows is different, check SCRCPY’s GitHub page for more details about Windows.
4. 
   

On your phone (Similar on Mac & Linux via Terminal):

1. Connect your phone to your PC via USB cable to run ADB (if ADB is not installed, follow SCRCPY GitHub page).
2. Connect your phone to the same Wi-Fi as your computer.
3. Open a Terminal on your computer and type: “adb tcpip 5555”. hit ENTER! And unplug your phone.
4. Then type: adb connect X.X.X.X:5555 (replace XXXX with your phone’s IP).
5. Type “scrcpy -b2M -m800–max-fps 15” in your terminal and hit ENTER.
6. You should see your phone on your computer screen and even control it with a mouse and keyboard.
7. Download OpenCamera from Google play store on your phone and open the app on your phone (on Landscape).
8. Find the best & secure way yo attach your phone to your Quest 2 without blocking Quest’s B&W tracking cameras!

On your Oculus Quest & Quest 2:

1. Go to your Settings > Experimental Features > Display Theme, and set it to “Light” (the device will reboot).
2. Go to quick settings and enable Passthrough Home.
3. Go to the Sharing Menu, click “Cast” and then select “Computer” & click “next”.
4. Open up Google Chrome on your computer and go to oculus.com/casting
5. Stick a tape on your Quest’s Proximity Sensor (to keep it awake).

On your computer: You should see both streams from your quest and your phone on your computer screen. Now, Open up OBS Studio and configure it exactly like this:

1. Click the “+” icon under “Sources” and select “Window Capture”, call it “Phone” and click OK.
   Now, next to “Window” within the drop-down, select your phone by the device name / model and click OK.
2. Click the “+” icon under “Sources” and select “Window Capture”, call it “Quest2” and click OK. Now, next to “Window” within the drop-down, select “Oculus–Google Chrome” and click OK.
3. Right Click on the “Quest2” source you just made. Within the modal, click on the “+” icon and select “Chroma-Key”. Set it to “Custom” and fill these parameters:

Key Color: #00007F | Similarity: 1 | Smoothness: 270 | Key Color Spill: 1 | Opacity: 1.0000 | Contract: 0.02

• Click “Close” to apply! Now you can click “Start Recording” on OBS Studio and experience XR Passthrough Mode 
• If you don’t have 2 monitors on your PC, go back (Alt+Tab) to Oculus stream on Chrome and click “Full screen”.
• To make full screen 16:9 accept ratio, click on the Cog icon and change the toggle to “Widescreen (crop).

One more thing…

• Black / Dark background in VR apps will be “removed” on OBS studio. It will be treated as a Transparent AR app 
• You’ll still see the world in black & White on Quest’s AR-Pass-through mode, but the recording will be in full colors!
• OBS will save both profiles / sources for future use, so you won’t have to configure it again every time you use it.

For Non-techie creators:If you are not a technical person, you can skip all the steps above, and simply attach your phone to your VR headset. Record a video feed on both of them separately, and later on copy it to your computer in order sync and combine them with your favorite Editing tool (Premiere, Final Cut, KdeNlive, etc.), but make sure to set Chroma-Key, just like I did!

Personally, I find the OBS method more effective because it’s a Live Stream & recording of everything I see and experience in real time as a User. For example – If I’m testing a new app, Unity 3D project, or Tvori Prototype, I need to be able to test it in real time as I’m wearing the headset, and also live-stream it on the internet or to clients / colleagues in order to get feedback.

But, eventually, it’s up to you as a creator. You can choose the best method for you. I tried the Video Editing method and it looks like this: 

I hope it is useful, and I really hope that Facebook (Oculus) will take a different & Open approach and let us to do whatever we want to do with our VR headsets, while they also do their best in terms of “Privacy”. Recording AR Passtr-hough mode with SCRCPY was a decent solution for Pro-Users / Creators and with no privacy issues.Alternative–You can look at ZED Mini Mixed Reality cameras. But, I haven’t tried it myself… Cheers. 

Do you like our news? Give us a 5-star rating and review on Google to show your support for the website.

article by Amit Caesar:

You should also check out the following articles:

1. VR BUYING GUIDE AND THE BEST VR HEADSET FOR 2022
   
2. A meta market opportunity: The metaverse could soon be worth $1 trillion
   
3. Facebook wants to build a metaverse. Microsoft is creating something even more ambitious.
   
4. How to succeed in the virtual reality world of tomorrow?
5. Books you must read about virtual reality
6. Best New Augmented Reality Books To Read In 2021
7. US$ 4.7 Billion- The global augmented reality gaming market
8. The smart glasses revolution is about to get real
9. Consumer Brands Reinventing Marketing in the Metaverse
   
10. Imagine Making Money in Rec Room
11. The biggest AR and VR predictions of 2022
12. Apple hired Meta's AR communications lead ahead of the 2022 launch of the headset.  
13. Who Will Be in Charge of the Metaverse?
14. The Kingdom of Abraham: The first Jewish metaverse  
15. Gen Z are planning to spend thousands on cryptocurrency, NFTs and metaverse
16. According to Goldman Sachs, the metaverse must run on blockchain  
17. What You Can Create With a Small Piece of Land in the Metaverse
18. How To Buy Land In The Metaverse?