5 min read
what is mixed reality ?

Mixed reality is the next wave in computing followed by mainframes, PCs, and smartphones. Mixed reality is going mainstream for consumers and businesses. It liberates us from screen-bound experiences, by offering instinctual interactions with data in our living spaces, among our things, and with our friends. Online explorers, in hundreds of millions around the world, have experienced mixed reality through their handheld devices. Mobile AR offers the most mainstream mixed reality solutions today on social media. People may not even realize that the AR filters they use on Instagram are Mixed Reality experiences. Microsoft Mixed Reality is to take all these user experiences to the next level with a combination of truly stunning holographic representations of people, high fidelity holographic 3D models and the real world around them. 

Mixed Reality is a blend of physical and digital worlds, unlocking natural and intuitive 3D human, computer, and environment interactions. This new reality is based on advancements in computer vision, graphical processing, display technologies, input systems, and cloud computing. The term Mixed Reality was introduced in a 1994 paper by Paul Milgram and Fumio Kishino, "A Taxonomy of Mixed Reality Visual Displays." Their paper explored the concept of a virtuality continuum and the taxonomy of visual displays. Since then, the application of Mixed Reality has gone beyond displays to include:

  • Environmental understanding: spatial mapping and anchors.
  • Human understanding: hand-tracking, eye-tracking, and speech input.
  • Spatial sound.
  • Locations and positioning in both physical and virtual spaces.
  • Collaboration on 3D assets in mixed reality spaces.

Environmental input and perception

In the past decades, the relationship between humans and computers has continued to evolve by the means of input methods. A new discipline emerged, which is known as human-computer interaction or HCI. Human input can now include keyboards, mice, touch, ink, voice, and Kinect skeletal tracking.

Advancements in sensors and processing power are creating new computer perceptions of environments based on advanced input methods. This is why API names in Windows that reveal environmental information are called the perception APIs. Environmental inputs can capture:

  • a person's body position in the physical world (head tracking)
  • objects, surfaces, and boundaries (spatial mapping and scene understanding)
  • ambient lighting and sound
  • object recognition
  • physical locations

Image: The interactions between computers, humans, and environments.

A combination of the three essential elements sets the stage for creating true Mixed Reality experiences:

  • Computer processing powered by the cloud
  • Advanced input methods
  • Environmental perceptions

As we move through the physical world, our movements are mapped in a digital reality. Physical boundaries influence mixed reality experiences in the digital world, such as games or task-based guidance in a manufacturing facility. With environmental input and perceptions, experiences start to blend between physical and digital realities.

The Mixed Reality spectrum

Mixed Reality blends both physical and digital worlds. These two realities mark the polar ends of a spectrum known as the virtuality continuum. We refer to this spectrum of realities as the Mixed Reality spectrum. On the end of the spectrum, we have the physical reality that we as humans exist. On the other end of the spectrum, we have the corresponding digital reality.

Augmented vs. virtual reality

Most mobile phones on the market today have little to no environmental perception capabilities. The experiences these phones offer can't mix physical and digital realities.

The experiences that overlay graphics, video streams, or holograms in the physical world are called augmented reality. The experiences that occlude your view to present a fully immersive digital experience are virtual reality. The experiences that can transition between augmented and virtual realities form Mixed Reality, where you can:

  • Place a digital object, such as a hologram, in the physical world, as if it were physically present.
  • Be personally and digitally present in the physical world, in the form of an avatar, to asynchronously collaborate with others at different points in time.
  • In virtual reality, physical boundaries like walls and furniture appear digitally within the experience for users to avoid running into physical obstacles.

Most augmented reality and virtual reality experiences available today represent a small subsets of the larger Mixed Reality spectrum. Windows 10 is built with the entire spectrum in mind, and allows blending digital representations of people, places, and things with the real world.

Devices and experiences

There are two main types of devices that deliver Windows Mixed Reality experiences:

  1. Holographic devices are characterized by the device's ability to place digital content in the real world as if it were there.
  2. Immersive VR devices are characterized by the device's ability to create a sense of presence by blocking out the physical world and replacing it with a fully immersive digital experience.

CharacteristicHolographic devicesImmersive devices
Example deviceMicrosoft HoloLens

Microsoft HoloLens 2 image
Samsung HMD Odyssey+

Samsung HMD Odyssey+ image
DisplaySee-through display. Allows the user to see the physical environment while wearing the headset.Opaque display. Blocks out the physical environment while wearing the headset.
MovementFull six-degrees-of-freedom movement, both rotation and translation.Full six-degrees-of-freedom movement, both rotation and translation.

Mixed Reality experiences are the result of technological advancements. There aren't any devices today that can run experiences across the entire spectrum. Windows 10 provides a common Mixed Reality platform for both device manufacturers and developers. Any given device today can support a specific range within the Mixed Reality spectrum. In the future, new devices with more expansive range are expected: holographic devices will be more immersive, and immersive devices will be more holographic. 

As an application or game developer, what type of experiences do you want to create? The experiences will typically target a specific point or part on the spectrum. You will need to consider the capabilities of devices to target. Experiences that rely on the physical world will run best on HoloLens.

  • Towards the left (near physical reality). Users remain present in their physical reality, and aren't made to believe they have left that reality.
  • In the middle (fully Mixed Reality). These experiences blend the real world and the digital world. For example, in the movie Jumanji, the physical structure of the house where the story took place was blended with a jungle environment.
  • Towards the right (near digital reality). Users experience a digital reality and are unaware of the physical reality around them.

You should also check out the following articles:

  1. Apple’s first headset will focus on “high-quality” games, reporter claims
  2. Pets, people, and Android Phone Notifications are now visible in Quest 2.
  3. 30 The top virtual reality games in 2021
  4. November 26, 2021 is Black Friday. Deals on Video Games and Virtual Reality in 2021
  5. Facebook wants to build a metaverse. Microsoft is creating something even more ambitious.
  6. Metaverse’s Cryptocurrency Leaps in Price After Facebook Rebrands as Meta
  7. Best VR Gifts for Christmas in 2022
  8. What is Augmented reality (AR) ?
  9. How to succeed in the virtual reality world of tomorrow?
  10. Best VR Headset cyber Monday 2021
  11. Books you must read about virtual reality
  12. Best New Augmented Reality Books To Read In 2021
  13. virtual reality Life 2029
  14. Imagine Making Money in Rec Room
  15. The smart glasses revolution is about to get real
  16. Squid Games are now playing in virtual reality

Subscribe now to our YouTube channel

Subscribe now to our Facebook Page

Subscribe now to our twitter page

Subscribe now to our Instagram

Subscribe To my personal page on linkedin

Subscribe To my personal page on tiktok page for those who love to dance :)

Don't forget to be my friend. Sign up for my friend's letter. So I can tell you ALL about the news from the world of VR&AR, plus as my new friends I will send you my new article on how to make money from VR&AR for free.