No conversation about Mixed Reality HMDs is ever complete until someone brings up Apple. Yes, the company has no available or announced devices in this space — but there isn’t a soul following technology trends that doesn’t believe the Cupertino giant isn’t a) working on some kind of AR glasses, b) acquiring companies that make lenses and materials that are perfect for some kind of AR glasses, and c) talking about AR as a transformational technology every time a microphone appears. The rumor mill says Apple’s glasses could appear in 2019, but we have our money on a 2020 launch, alongside a newly redesigned and reimagined iPhone that moves away from the “iPhone X design” for the first time since it launched. Stay tuned.
With a multiplicity of emerging hardware and software options, the future of wearables is unfolding but yet unknown. Concepts such as the HTC Vive Pro Eye, Oculus Quest and Playstation VR are leading the way, but there are also players like Google, Apple, Samsung, Lenovo and others who may surprise the industry with new levels of immersion and usability. Whomever comes out ahead, the simplicity of buying a helmet-sized device that can work in a living-room, office, or factory floor has made HMDs center stage when it comes to Virtual Reality technologies.
Special input devices are required for interaction with the virtual world. These include the 3D mouse, the wired glove, motion controllers, and optical tracking sensors. Controllers typically use optical tracking systems (primarily infrared cameras) for location and navigation, so that the user can move freely without wiring. Some input devices provide the user with force feedback to the hands or other parts of the body, so that the human being can orientate himself in the three-dimensional world through haptics and sensor technology as a further sensory sensation and carry out realistic simulations. Additional haptic feedback can be obtained from omnidirectional treadmills (with which walking in virtual space is controlled by real walking movements) and vibration gloves and suits.
In 1999, entrepreneur Philip Rosedale formed Linden Lab with an initial focus on the development of VR hardware. In its earliest form, the company struggled to produce a commercial version of "The Rig", which was realized in prototype form as a clunky steel contraption with several computer monitors that users could wear on their shoulders. The concept was later adapted into the personal computer-based, 3D virtual world program Second Life. https://www.aliexpress.com/item/New-Style-Camera-360-4k-360-degree-Panoramic-VR-Camera-Build-in-WiFi-Mini-Sports-Action/32702459981.html
This is more difficult than it sounds, since our senses and brains are evolved to provide us with a finely synchronised and mediated experience. If anything is even a little off we can usually tell. This is where you’ll hear terms such as immersiveness and realism enter the conversation. These issues that divide convincing or enjoyable virtual reality experiences from jarring or unpleasant ones are partly technical and partly conceptual. Virtual reality technology needs to take our physiology into account. For example, the human visual field does not look like a video frame. We have (more or less) 180 degrees of vision and although you are not always consciously aware of your peripheral vision, if it were gone you’d notice. Similarly when what your eyes and the vestibular system in your ears tell you are in conflict it can cause motion sickness. Which is what happens to some people on boats or when they read while in a car. https://www.youtube.com/watch?v=YYy1cCpjOW0&t=1s
There's no wires here at all and no sensors to unlock the room scale tracking. Instead, you can draw out your VR space using the bundled Touch controllers, which are the same ones included with the Rift. The ability to draw multiple Guardians (as Oculus calls it) means you can freely move from different spaces and the headset will remember your safe VR territory.
The Oculus Rift was the first big name in the current wave of VR, and Oculus is still a major player. The Rift S has a higher resolution than the Vive (but not as high as the Vive Pro or, strangely, the Oculus Quest) and newer and lighter Oculus Touch motion controllers, and doesn't need external sensors to work. It does, however, need DisplayPort; if your PC only has an HDMI output, you might want to hunt for the previous Rift and deal with the extra cables. https://www.aliexpress.com/item/Viper-Panoramic-Lens-360-Degrees-Capturing-Camera-AR-VR-Wide-Angle-Fisheye-Lens-Panoclip-for-iPhone/32891616697.html
Basically, these AR headsets have transparent lenses that let you look at your surroundings, instead of completely replacing your vision with a computer-generated image. They can still project images over whatever you're looking at, but those images are designed to complement and interact with the area around you. You can make a web browser pop up in the middle of a room, for instance, or watch animals run around your coffee table. It's fascinating technology that could hint at the future of computing.
Alright, I’m going to walk you through my specific process, although some people may do things in a slightly different order. Let’s jump back to the import. The camera I used records to 2 separate Micro SD cards, so I have to import both. In the “Ingested Footage” folder for that day’s shoot, I put a folder for one card, “Front” and one card, “Back”. Then I use the stitching software and export final stitches for every clip to a third folder I call “Stitched”. Then I dropped each of those stitched files into Adobe After Effects and mask myself, other, gear, and tripod out of the shot. This way I have clean video with only the model in it. I export these as individual files as well. https://www.youtube.com/watch?v=YFrYq9IMzEY&t=1s
Modern VR headsets now fit under one of three categories: Mobile, tethered, or standalone. Mobile headsets are shells with lenses into which you place your smartphone. The lenses separate the screen into two images for your eyes, turning your smartphone into a VR device. Mobile headsets like the Samsung Gear VR and the Google Daydream View are relatively inexpensive at around $100, and because all of the processing is done on your phone, you don't need to connect any wires to the headset. https://www.youtube.com/watch?v=iHCEGDeQjsQ&t=1s
HTC's Vive is a comprehensive package that includes a headset, two motion controllers, and two base stations for defining a "whole-room" VR area. It's technically impressive, and can track your movements in a 10-foot cube instead of just from your seat. It also includes a set of motion controllers more advanced than the PlayStation Move. PC-tethered VR systems like the Vive need plenty of power, with HTC recommending at least an Intel Core i5-4590 CPU and a GeForce GTX 970 GPU. https://www.youtube.com/watch?v=A6aRkhlqWuE&t=1s
After doing this over and over, revision after revision, eventually I have a final draft to export. In the export window, it is important that I ensure the “VR” is still enabled. What this does is put that specification in the metadata file so that sites like YouTube and Vimeo know to play it as 360° VR footage. Then I can upload it and ensure the playback is the way I want it. https://www.youtube.com/watch?v=ZVUfnJipFh0&t=1s
In 1992, Nicole Stenger created Angels, the first real-time interactive immersive movie where the interaction was facilitated with a dataglove and high-resolution goggles. That same year, Louis Rosenberg created the virtual fixtures system at the U.S. Air Force's Armstrong Labs using a full upper-body exoskeleton, enabling a physically realistic mixed reality in 3D. The system enabled the overlay of physically real 3D virtual objects registered with a user's direct view of the real world, producing the first true augmented reality experience enabling sight, sound, and touch.
The Oculus Rift was the first big name in the current wave of VR, and Oculus is still a major player. The Rift S has a higher resolution than the Vive (but not as high as the Vive Pro or, strangely, the Oculus Quest) and newer and lighter Oculus Touch motion controllers, and doesn't need external sensors to work. It does, however, need DisplayPort; if your PC only has an HDMI output, you might want to hunt for the previous Rift and deal with the extra cables.
Vive's Tracker might not look like much, but stick it on something and that "thing" becomes a controller. Yes, HTC is redefining how we think of mixed reality with this accessory, and now anyone can buy one. Developers have been busily finding exciting and unique ways to use the Tracker in games, which will require mounts for certain props, like guns and baseball bats.
One place Microsoft has succeeded in how easy it is to set up its WMR headsets. It uses inside-out tracking, so you won't need to go plug in a bunch of sensors to make sure your headset is being seen. These headsets have built-in cameras and sensors that handle all of that. They'll even track the Windows Mixed Reality controllers, which suffer a bit because they rely on the headset being able to see them clearly to properly track. https://www.aliexpress.com/item/New-14-Joint-360-Degree-Rotation-Resin-Nude-Naked-Female-Body-For-Bar-bie-Doll/32884974971.html
According to CEO Stanislas Chesnais, the 3DRudder was designed for "existing games where two hands aren't enough [for] VR, where you don't actually have any solutions today to move in a nice way while sitting." A circular device that you control with both feet by tilting in the direction you want to move, the 3DRudder's latency-free design means that response to your movements is instant and requires little thought. "It's like in real life," says Chesnais. "When you do something [like] set the table… you don't think about your feet moving around. Your hands are what matter." https://www.youtube.com/watch?v=LD4XfM2TZ2k&t=1s
The big upside to Oculus is that its ecosystem is pretty robust. There are over 1,000 apps and games available to you for the Go. Now you're not going to get some of the more high-end experiences, like Robo Recall, but you're not going to be hurting for content. Oculus' home screen is also probably the best in VR, even on the standalone Go. Finally, you're going to get about 1.5 hours of battery life if you're gaming and 2.5 hours if you're watching videos. The basic version comes with 32GB of storage, you camp bump up to 64GB of storage for a cool £50 more.
Microsoft HoloLens is shaping up to be another formidable competitor in the HMD market. Unlike VR tech, Microsoft based their display on holographic technology. The original Hololens was more proof of concept than consumer device, and the recently announced Hololens 2 is again eschewing the public in favor of enterprise/government uses. The Hololens 2 will feature upgraded visuals, including a much-expanded field of view — music to fans of the original device, who settled on the limited FOV as the main drawback of the device. https://www.youtube.com/watch?v=FNq-8bWfAwQ&t=1s
To a degree, I’ve made it sound harder than it actually is since software has compensated for much of this, but these were some of the questions I was asking when I was first learning. I then started learning that some standard editing procedures could still be done. For example, I can still create image masks in 360°. So in truth, I still stood behind the camera the whole time along with others and gear and then masked myself out with a separate cut of footage showing the empty room. And for the motion graphics, (I use Adobe software) the software has built-in accommodations for editing and viewing in the 3D space so as to create accurate graphics. So in truth, once I learned, it wasn’t that hard, but boy is it time-consuming. https://www.youtube.com/watch?v=TNbxXSOsmKs&t=1s
So far, Apple has been very cool on VR, but that's slowly starting to change, at least from a software development side. OS X High Sierra enables VR development on three major VR software platforms: Steam, Unity, and Unreal. It also uses Apple's Metal 2 framework, which the company says provides the performance necessary for VR. No plans for any Apple-branded VR headset have been announced—we'll much more likely see Rift or Vive compatibility added to Macs.
In aviation, medicine, and the military, Virtual Reality training is an attractive alternative to live training with expensive equipment, dangerous situations, or sensitive technology. Commercial pilots can use realistic cockpits with VR technology in holistic training programs that incorporate virtual flight and live instruction. Surgeons can train with virtual tools and patients, and transfer their virtual skills into the operating room, and studies have already begun to show that such training leads to faster doctors who make fewer mistakes. Police and soldiers are able to conduct virtual raids that avoid putting lives at risk.
Virtual reality (VR) is an experience taking place within simulated and immersive environments that can be similar to or completely different from the real world. Applications of virtual reality can include entertainment (i.e. gaming) and educational purposes (i.e. medical or military training). Other, distinct types of VR style technology include augmented reality and mixed reality.
Google and Samsung still offer phone-based VR headsets in the form of the Daydream View and the Gear VR, and even Nintendo has gotten into the game with its Labo VR Kit for the Nintendo Switch. However, these shell-like headsets, which require a phone or some other device physically inserted into them, feel like novelties next to more powerful headsets that can provide more immersive experiences. https://www.youtube.com/watch?v=gFiZ7cZcuQo&t=1s
Although much more time and data intensive, I do foresee 360° and VR as being a growing trend of the future. I think the tools and software will continue to improve and the real world applications will start to become more obvious (i.e. behind-the-scenes where you could actually see everything, car cams so that every angle was captured, real estate videos, etc). I’m excited to continue learning about it and continue shooting on my 360° camera.
The Vive Pro sports a 2,880 x 1,600 resolution on a dual-OLED display, up from the Vive's 2,160 x 1,200 resolution. There's also dual cameras on the front that'll help track your hands and a 90Hz refresh rate with 70 sensors, giving you 360-degree head-tracking. It's not a leap to say that the Vive Pro is a monster. This is easily the most advanced VR headset out there, but it's also incredibly expensive, sitting at £799. If you don't need that added clarity, the Vive has all the same tricks. https://www.aliexpress.com/item/Insta360-Nano-360-Degree-Dual-3K-lens-VR-Video-Camera-Real-Time-Seamless-Stitching-for-iPhone/32819111754.html
Answering “what is virtual reality” in technical terms is straight-forward. Virtual reality is the term used to describe a three-dimensional, computer generated environment which can be explored and interacted with by a person. That person becomes part of this virtual world or is immersed within this environment and whilst there, is able to manipulate objects or perform a series of actions. https://www.youtube.com/channel/UC48zebw9GZ8WHGprugU73lg
In Virtual Reality, the computer uses similar sensors and math. However, rather than locating a real camera within a physical environment, the position of the user’s eyes are located within the simulated environment. If the user’s head turns, the graphics react accordingly. Rather than compositing virtual objects and a real scene, VR technology creates a convincing, interactive world for the user. https://www.youtube.com/watch?v=hTFe95oFnDM&t=1s
The PlayStation VR is compelling thanks to Sony backing development for it and the affordability and availability of the PlayStation 4 compared with gaming PCs. All you need is the headset, a PlayStation 4, and a PlayStation Camera (now included with most PlayStation VR bundles). There are some excellent games on PS VR like Moss, Rez Infinite, and Until Dawn: Rush of Blood, with Sony and other developers working on several more (including Five Nights at Freddy's VR).
Expand your media adventures and social gaming communities with accessories and peripherals created for virtual reality. For example, a VR controller offers seamless integration with virtual reality games to give you more control over your virtual environment. Beyond PC gaming, virtual reality experiences also encompass other forms of video. Virtual reality-compatible cameras capable of filming in 360 degrees offer you the chance to view and edit your footage, along with other VR-ready components. https://www.youtube.com/watch?v=S3vO8E2e6G0&t=1s
Visuals are delivered via an OLED display that offers 1,440 x 1,660 resolution per eye. There's also a built-in speaker with the option to plug in headphones for a more isolated experience. While the VR experience might be on par with those PC-powered headsets, it's still very good and a big step up from the Oculus Go. Games run nice and smooth, though the smartphone-like processing power does mean some titles can take slightly long to load up. https://www.aliexpress.com/item/Wall-Mount-Tablet-Holder-Aluminum-Alloy-Wall-Hang-Bracket-Screen-360-Degree-Rotation-Tablets-Stand-for/32891486912.html
Fast Five and Star Trek Beyond director Justin Lin put together this short film for Google's Spotlight Story program in 2016, and it's still one of the more ambitious live action VR shorts out there. Lin opts to use 360 cameras to put you in what feels like a big budget blockbuster, with an alien hellbent on chasing you through the subway. It's absolutely thrilling.
It's also got a bunch of doodads to help you customise your mobile experience. Adjust the lenses all you want, click two buttons to interact with your content; pop out the front window in case you ever need to use it for mixed/augmented reality purposes. Speaking of which, Merge now also sells the AR Merge cube, which lets you put mixed reality experiences in the palm of your hand. https://www.youtube.com/watch?v=L0bzBxxDQzs&t=1s