Sniper Elite VR: Winter Warrior brings Rebellion's shooter back to virtual reality, arriving next month on Quest.
A sequel to 2021's Sniper Elite VR, Rebellion partnering with Just Add Water once more. Playing as a former Italian resistance soldier called 'the Partisan,' Winter Warrior sees you embark on a story-driven campaign to find and destroy Nazi 'wonder-weapons.' Promising a "wide range of difficulty, comfort and control options" alongside bHaptics and ProTube VR support, here's the reveal trailer:
0:00
/0:50
Beyond the main campaign, Sniper Elite VR: Winter Warrior includes two new game modes. 'Sniper Hunt' is described as "an intense game of cat and mouse" that pits you against expert Nazi snipers, taking down enemy patrols to lure out the snipers before relocating. 'Last Stand' sounds more like a survival mode, placing you against increasing enemy waves as you gather ammunition and weapons while setting traps.
We enjoyed Rebellion's previous entry in our 2021 Sniper Elite VR review. We believed that "some of the series’ staple elements are really enhanced by the platform," praising the stealth segments and highlighting how aiming down the sniper rifle scope is a "calculated and convincing process with perfectly streamlined authenticity."
However, don't expect to see Winter Warrior on PC VR or PSVR 2. When asked about a potential multiplatform release like the original Sniper Elite VR had, Rebellion provided UploadVR with the following statement:
We were approached by our partners at Meta to create more Sniper Elite VR content, exclusively for Quest. Those discussions progressed to become Winter Warrior.
Sniper Elite VR: Winter Warrior arrives on November 30 on the Meta Quest platform for $15.
The Asus ROG Zephyrus G14 with the Nvidia RTX 4050 graphics card is down to $1,100 from Best Buy, for a chance to get one of the best gaming laptops for cheap.
A Sony exec reaffirmed that PSVR 2 is "important to us," but explained it's not the 'core proposition' this season.
In a recent interview with Barron's (paywall), Eric Lempel, Senior Vice President, Head of Global Marketing, Sales & Business Operations at PlayStation, was questioned about the sales outlook for PS5 and PSVR 2 this holiday season. Calling the PS5 outlook "really strong," he calls VR a "category that can help with innovation" before calling PS5 the core focus.
Here's the full quote:
The VR category is important to us. It’s a category that can help us with innovation. It’s never going to be the only way people play games, but I’m happy that we’re in it. There are great experiences to be had and consumers really like it. But it’s a nascent business for us. It’s something that we want to be a part of but it’s not the core proposition we have this season. PlayStation 5 is the core.
Considering VR still fights for mainstream adoption, Lempel's comments aren't particularly surprising. In May, Sony confirmed PSVR 2 sold 600k units from its late February launch up until April, while PS5 surpassed 40 million units in July after launching in November 2020. The two aren't directly comparable despite PSVR 2 requiring PS5, but it's not hard to see why Sony is prioritizing the latter.
Despite a strong launch line-up with Horizon, Gran Turismo 7, Resident Evil Village and more, there's been vocal criticism about PSVR 2's library ever since. Firewall Ultra received mixed reviews in August, there are currently no revealed upcoming 1st party games and recent last-minute indefinite delays for Vertigo 2 and Phasmophobia will only deepen frustrations.
With the recent Quest 3 launch putting these issues into greater perspective, Lempel's comments will likely do little to encourage existing PSVR 2 owners.
The Alienware Aurora R15 gaming desktop with the Nvidia GeForce RTX 4090 graphics card and 32GB of RAM is currently available from Dell with an $500 discount.
In this week's episode of the Between Realities VR Podcast, Alex and Skeeva host Will Cloxton of Vankrupt Games, developers of Pavlov and Pavlov Shack.
Will gives the insider's perspective on Pavlov Shack's upcoming official Quest store release. Topics include reasons behind Pavlov's game design choices and the importance of mods and community created content.
Apple could reportedly use Vision Pro to detect health issues.
The Information's Wayne Ma cites "people with direct knowledge of the matter" as saying that Apple is discussing using the face and eye tracking sensors on Vision Pro to "measure a person’s facial expressions, using them to detect depression, anxiety, stress or post-traumatic stress disorder".
Ma's reporting suggests Apple is hiring health experts to research whether the company could connect face and eye tracking data to health-related features of the Vision Pro. To treat mental health, Vision Pro could "display images and sounds that might improve the wearer’s emotions", Ma writes. When announcing Vision Pro, Apple showed a "full space" mindfulness app that darkened most of the room to "create a moment of calm".
0:00
/0:19
Apple's Mindfulness visionOS app.
Another possible feature reportedly being considered is using the eye tracking cameras to detect blood vessel swelling, which can be an early symptom of heart failure, Ma writes.
Apple has put a strong focus on fitness and health in Apple Watch. It can detect unusual heart rate, irregular rhythm, low cardio fitness, blood oxygen level, falls, and car crashes. Newer models can even capture an electrocardiogram (ECG).
In 2019, Tim Cook told CNBC that Apple's "greatest contribution to mankind" would be to health.
Apple reportedly cut full body tracking from Vision Pro "years ago".
Vision Pro has downwards-facing cameras, announced as being for hand tracking. Last year The Information's Wayne Ma reported that the headset - still only a rumor at the time - would also use these cameras to track your body, including your legs, but Apple didn't announce this. The same report described the headset's OpticID iris authentication system, which did turn out to be true.
0:00
/0:23
Apple Vision Pro's sensor array includes downwards cameras.
In a new report this week, Ma claims Apple actually "cut full body tracking from the Vision Pro several years ago because engineers couldn’t make it reliable enough".
Apple was planning to use body tracking for health features and fitness, Ma writes. Movement data could have been analyzed over time to detect Parkinson’s disease, for example. Despite rumors of an Apple Fitness+ service for Vision Pro, Apple didn't discuss fitness at all during the product's lengthy announcement in June.
Meanwhile, Meta plans to add inside-out upper body tracking to Quest 3 in a software update in December, which will utilize Quest 3's downwards-facing side cameras. This feature won't track your legs, though, just everything above. Meta plans to release "Generative Legs" in the same update to estimate your legs with AI.
0:00
/1:11
Quest 3's upcoming inside-out upper body tracking feature.
Meta is considered a leader in computer vision and machine learning, which may have given it an advantage over Apple when it comes to shipping features like this. Alternatively, Apple executives may have a higher quality bar, one which they felt current technology doesn't surpass yet.
Ma's report doesn't say whether Apple is still working on body tracking or has abandoned work on it until future hardware, however Apple has a long history of adding significant new software features to its products over time.
A YouTuber tried BlockVerse's mixed reality mode at "field scale" outdoors, describing it as "Minecraft in real life".
BlockVerse is a Minecraft clone for Meta Quest headsets with a twist: it supports mixed reality, not just virtual reality. The default mixed reality view presents a chunk of the world as a miniature diorama floating in front of you, but it also supports a full-scale mixed reality mode.
To use this mode practically, you'll need a warehouse or flat outdoor space. You'll probably also want to enable developer mode to disable the headset's safety boundary, since it only supports 15x15 meter spaces at most.
YouTuber Brandon Ballinger (AwakenToast) tried out exactly this with his Quest 3, and the result is a glimpse of the enormous barely-tapped potential of outdoor mixed reality gaming:
Ballinger starts by digging a virtual hole in the physical ground, expressing amazement at the "sense of depth" looking down into it. Throughout the video you see him carefully avoid falling into virtual holes he digs, stepping around them despite the fact that, of course, they're not really there.
He goes on to build a virtual house, noting that it seems to stay in place even as he roams around the large grassy area, suggesting Quest 3's positional tracking system has little-to-no drift even at this scale.
When he adds a ceiling to the house it becomes clear just how blurred the line between virtual and mixed reality can be. Seeing his only view of the real world through the empty doorway highlights how mixed reality can be a spectrum of merged spaces rather than a binary on/off button.
Quest 3's lack of dynamic occlusion (it's currently experimental and can't be shipped in apps yet) is also clearly on show here, however. At several points in the video you can see Ballinger walk in front of trees, yet his virtual creations are still rendered in front of it. What's even more jarring is that when he's joined by children at the end of the video, they should appear as if in his virtual house, but instead are only visible through the doorway as if half in the ground.
The jarring results of the lack of dynamic occlusion.
On a more positive note, what strikes me about this video was just how compelling games like this could be with local colocated multiplayer. Yes, the now-defunct Minecraft Earth tried this on mobile. But seeing virtual objects in the real world in a headset is a fundamentally different and better experience to just seeing them on your phone camera preview screen.
Developers and users are exploring mixed reality at a pace we've never seen before, and we'll be following developments closely. If you see something interesting being done with mixed reality please let us know about it with an email to tips@uploadvr.com.
XREAL opened preorders for the Air 2 series of media viewing smart glasses.
XREAL, formerly Nreal, is a China-based startup that launched the first consumer AR glasses in the US, Nreal Light, in late 2021. Light was capable of positioning virtual objects in your real environment, but was $600 and only compatible with a few specific flagship Android phones.
XREAL told UploadVR it "probably won't" release another Light, and is now focused on its Air series of non-AR smart glasses with the release of the second generation.
Air glasses lack tracking cameras. Instead, they show a virtual TV view of any device with USB-C display-out capability, such as Android phones and iPads. The 39° horizontal field of view is equivalent to a 65-inch TV 2 meters in front of you.
0:00
/0:06
With just the glasses, content is 0DoF head-locked.
With the glasses alone the virtual TV will be fully locked to your head, as if part of a HUD. With the optional $120 XREAL Beam dongle you can anchor it in front of your body, so it will stay in place when you rotate your head.
The Beam dongle also has ports to enable the use of lightning iPhones and HDMI-out devices, such as the Nintendo Switch.
0:00
/0:08
With the Beam dongle, content stays in place rotationally.
The original Air glasses launched in 2022 at $380. The new Air 2 is priced at $400 and offers 20% higher brightness, improved comfort, enhanced color accuracy, and higher quality audio.
But the more interesting new feature arrives in Air 2 Pro. It features configurable dimming. By tapping the dimming adjust button on the side you can toggle between 0% dimming to see the real world clearly, 35% dimming to focus on the content while still being aware of your surroundings, or 99.9% dimming to be highly immersed. Air 2 Pro is priced at $450.
Air 2 and Air 2 Pro preorders are available now, and both will ship in November in the US and UK then in December in Germany, France, and Italy.
There are many wondrous things to discover in Super Mario Wonder, including our old friend Captain Toad. Here are all the places you can find the explorer.
Quest 3's passthrough will improve over time with software updates, according to Meta's CTO.
When asked about potential passthrough improvements in an Instagram AMA (ask me anything) session yesterday, Andrew Bosworth replied:
"Yes, it will continue to improve as we continue to get real world lighting conditions and information from the headsets that have been picked up, we start to tune the algorithms that drive it more effectively.
And so I do think it will continue to improve - modestly - from here for a little while as we do a better job depth estimating with where your hands are and working with the distortion around that, and things like that.
So yeah, we're gonna continue to work on it as we have with the Quest Pro."
While Quest 3 is arguably the first consumer headset with passthrough you'd want to spend more than a few minutes in, in our review we pointed out the noticeable flaws that make it feel very far from a transparent optic.
While the passthrough is depth and scale correct for objects and scenery further away than arm's length, and has impressively low latency and stability, at close range it exhibits geometric warping and double-imaging on moving objects. This can be clearly seen on your hands and arms, and means that while the camera resolution is just about good enough to read text, you won't actually want to use your phone for long because it appears significantly distorted in a way that's different for each eye.
0:00
/0:25
The warping and ghosting looks even worse in the headset than in a recording.
This warping is partially a result of the very low resolution of the depth estimation field the system generates to reproject the color camera views. It works by comparing the views from the two lower latency greyscale cameras. Quest Pro's depth estimation outputs 10,000 points per frame, and Meta hasn't yet given a figure for Quest 3.
Bosworth's comments suggest that Quest 3's depth estimation accuracy or resolution may be improved in upcoming software updates. If that happens, it would also improve the quality of the currently experimental dynamic occlusion feature.
Still, Bosworth tempered expectations with the word “modestly” in his reply. While software can improve, Meta's passthrough stack will still be limited by the cameras on the Quest 3 headset and the Snapdragon XR2 Gen 2 chipset. My colleague Ian reports that distortion wasn't visible in his brief time with Vision Pro, but Apple is using a dedicated secondary chipset to process passthrough and more (likely higher quality) sensors. Of course, this is also part of why it will start at seven times the price.
Don't miss this chance to get the Nintendo Switch OLED - The Legend of Zelda: Tears of the Kingdom Edition from Walmart, where it's available for only $319.
If you want a powerful gaming PC that won't have issues running today's video games, go for the Alienware Aurora R15, which is on sale from Dell for $1,600.
Samba De Amigo: Virtual Party marks SEGA’s first major entry into VR, bringing the rhythm series to Quest. Here is our full review.
In swapping to a first-person perspective, Samba De Amigo: Virtual Party feels like a natural fit for VR. We're hardly lacking for choice with VR rhythm games but even beyond its returning characters, Amigo's latest outing feels refreshing. High production values and a vivid presentation help Virtual Party sing, boosted by a straightforward control scheme that's easy to pick up and play.
Samba De Amigo: Virtual Party – The Facts
What is it?: A rhythm sequel to 1999’s Samba De Amigo on Dreamcast. Platforms:Quest 2, Quest Pro, Quest 3 (Review conducted on Quest 3) Release Date: Out Now Developer: SEGA Price: $29.99
Similar to the Dreamcast and Wii entries, Virtual Party features six rings in sets of two that are placed high, middle and low. All that's required is shaking your controllers in position with the rings as rhythm balls approach. It's a simple approach that perfectly suits motion controls, and pulling off long combos on harder difficulties always feels satisfying.
Most levels also require matching an on-screen pose or copying a certain movement, with more precise motions earning a better score. Several sections involve tracing colorful arrows that appear and combined, these moments keep gameplay varied. For most songs, this aligns with the rhythm well and I found myself working up a sweat on harder difficulties.
With over 40 base game songs, Virtual Party packs a varied selection of licensed artists ranging from Lady Gaga, Rina Sawayama, Ricky Martin and Bon Jovi. There's something for everyone, though I wish SEGA stuck more closely to the original game's Hispanic focus. Despite mostly being locked behind DLC, it also benefits from SEGA's back catalog with songs from Sonic the Hedgehog, Like a Dragon, Space Channel 5 and even Rhythm Thief.
Virtual Party features five gameplay modes and ‘Rhythm Game’ is your standard quick-play option. 'StreamiGo!' adds campaign-style missions as you try to become a famous online influencer – I suddenly understood why Samba De Amigo is inspired by TikTok. I wasn't too interested in building up a following, though the mission goals, like pulling off 10 consecutive perfects, add a nice challenge.
Like many Quest 3 launch window games, Virtual Party also supports mixed reality, with two modes to choose from. 'MR Party' lets you pick individual songs like Rhythm Game, starting in passthrough mode and gradually building towards a fully immersive VR environment. An interesting but gimmicky approach that wears out pretty quickly.
'MR Multiverse' is the better choice here, placing you through three consecutive stages tied together by a brief story. The narrative isn't much to write home about, though MR Multiverse makes better use of passthrough. Seeing an active volcano beneath you while you dance away makes for some entertaining environmental effects.
It’s worth comparing Virtual Party here with its Nintendo Switch counterpart, Party Central. Both games share the soundtrack but there are major differences. Local multiplayer is understandably gone, though the online 'World Party' mode remains intact with a three-stage format. Waiting 2-3 minutes each time I jumped into these 20-person lobbies isn't great, though the knockout format offers an enjoyable competitive touch that gradually eliminates low-scoring competitors.
However, that's where multiplayer support ends and I'm highly disappointed that Virtual Party doesn't include the four-player online option with friends. World Party lacks an invite feature, so it's entirely reliant on random players. Online leaderboards for both friends and everyone with each song are appreciated but that isn't a replacement.
Samba De Amigo: Virtual Party - Comfort
Samba De Amigo doesn’t include comfort settings, though it doesn't really need them. Every song takes place in one stationary position, negating the need for movement settings or vignettes. The only notable options let you adjust the ring height and closeness to you, and it's a comfortable experience for newcomers.
Still, Virtual Party gains one major advantage through its controls. Having played the Switch version during this review, I found the Quest 3 Touch Plus controllers provide noticeably improved feedback over detached Switch Joy-Cons. Touch Plus didn't suffer the same motion tracking issues either and the larger size makes them considerably nicer to hold.
Samba De Amigo: Virtual Party Review - Final Verdict
Samba De Amigo: Virtual Party benefits greatly from an impressive VR transition. The vibrant presentation, varied soundtrack and enjoyable rhythm gameplay kept me invested, though the comparative lack of multiplayer options is disappointing compared to Party Central. More responsive controls make the VR edition preferable and Amigo's latest outing is well-suited for casual sessions. For anyone seeking a solo rhythm game, Virtual Party comes recommended.
UploadVR focuses on a label system for reviews, rather than a numeric score. Our reviews fall into one of four categories: Essential, Recommended, Avoid and reviews that we leave unlabeled. You can read more about our review guidelines here.
VRChat is now listed as "coming soon" on the app store on Pico 4.
VRChat remains the leading social VR platform, with millions of users. On Meta Quest headsets it's almost always one of the top 10 most popular apps, and on Steam (where it also supports non-VR) it typically has tens of thousands of concurrent players.
Most PC worlds and avatars are too detailed to run performantly on mobile chipsets though, so VRChat takes a fragmented approach. All worlds and avatars must have a mobile-optimized version if the creator wants Quest & phone players to be able to join or use them. Creators can upload two versions of the same content, one for PC and another for mobile.
Given it uses the same Snapdragon XR2 Gen 1 mobile chipset as Quest 2, Pico 4 will almost certainly support these mobile versions too.
Pico 4's specifications and price put it squarely between Quest 2 and Quest 3, offering pancake lenses and 2K panels while using the original XR2 Gen 1 chip.
We've reached out to VRChat to confirm that the Pico Store listing is accurate and to ask for more details. We'll update this article if we get a response.
The co-founders of Upstream Arcade talk to us about Hellboy: Web of Wyrd's comic book visuals, Hellboy's fighting style, and working with Lance Reddick.
A new Squingle mixed reality update adds trippy fractals across your home, available next week on Quest.
Offering a psychedelic puzzle experience with over 100 levels, Squingle previously introduced mixed reality support last year with its passthrough update. Free for all existing owners, the MR update builds upon that by introducing "trippy fractals that emanate across users’ walls and furniture."
The mixed reality update also introduces Quest 3 enhancements. Speaking to UploadVR, creator Ben Outram explained that he "darkened the passthrough layer to bring out the action of the game" on Quest 2 but confirms this no longer made sense with Quest 3's improved passthrough "as it’s fun to be more connected to your real environment and lighting."
Squingle is out now on the Meta Quest platform via App Lab. It's also available on Pico 4, Vive XR Elite and SteamVR, though non-Quest platforms won't receive Squingle's new MR features. A multiplayer update is also planned for release next year.
Prowler Stashes are packed with valuable resources in Spider-Man 2. These are well hidden all across the map, so we will help you find them all with no trouble.
Professional sports is one potent sector of the entertainment industry that virtual and augmented reality has yet to truly crack.
Much of the focus has been on watching existing sports by placing you next to the action, like sitting court-side at a basketball game, with varying degrees of success.
Last weekend, Psychic VR debuted a new project called Air Race X, representing a bold vision for what sports might look like in immersive mixed reality formats.
The project isn't centered around viewing an existing sport in VR, nor creating an entirely new sport for VR/AR mediums either. Instead, Air Race X attempts to take an existing professional sport and partially digitize it, merging real world data and athletic performance into a mixed reality immersive presentation.
While a fascinating concept, the results were ultimately more mixed in execution than reality. Here's how it went down.
Investors, media, local government officials and other guests gather to watch Air Race X in Shibuya, Tokyo on Sunday October 15, 2023.
Getting Psychic
Air Race X was just one facet of a dazzling display of technology, concepts and in-development projects presented to me by Psychic VR Lab during a three-day media tour last week – our second visit to the company.
In 2019 we noted that Psychic was a incredibly bullish company when it came to the future of VR and AR. Four years later, that remains entirely true.
During that previous visit, Psychic demonstrated the early capabilities of Styly, the company's all-in-one creation and viewing platform for immersive content.
Conceptually akin to other creation tools like Unity – though infinitely less advanced in terms of features, infrastructure and scale – Styly is a platform where users can create scenes and experiences that can then be published to the platform's public library and viewed by others on AR/VR devices. Creators can share their experiences via QR codes, which can be displayed at physical events like Air Race X for easy access on compatible devices.
Four years on, Psychic VR remains committed to developing and growing the Styly platform. It forms the basis for almost everything the company has in the works, aiming to support a wealth of different experiences at a variety of scales.
Standard VR/AR scenes created in Styly can be viewed anywhere, but the app also supports location-specific content that uses virtual positioning systems to anchor an experience around a physical location, such as Tokyo's iconic Shibuya Crossing.
This was the plan for Air Race X, Psychic's first iteration of a mixed reality air race event built using the Styly platform.
Bringing In Yoshi
World Championship-winning pilot Yoshi Muroya at Fukushima Air Park.
Yoshi Muroya is a world championship-winning air pilot, previously positioned as high as 6th place in global rankings. On the second day of our media tour, Psychic takes us to Fukushima Sky Park to meet Yoshi and watch him complete an air race course, the data of which would be used the following day for the Air Race X event.
Traditional air racing competitions see pilots set times one-by-one across progressive runs of the same course, competing for the best time with as few penalties as possible. Traditionally, the pilots will all get together in one place to compete in global events such as the Red Bull World Championship.
World Championship-winning pilot Yoshi Muroya at Fukushima Air Park.
However, Red Bull pulled out of the sport in 2019, discontinuing its World Championship series and stating that it "did not attract the level of outside interest as many other Red Bull events across the world."
Four years on, Air Race X is Psychic's bid to revive competitive air racing with a new format that also happens to reduce some of the logistical and economic commitments involved. Instead of gathering pilots in one place for an air race event, the pilots will be able to fly the proposed course – in this case mapped to fit over Shibuya – before the event, whenever and wherever they want. The data from their performances will be compiled and sent over to Psychic for use in Air Race X, which will see their flight paths digitally recreated and visible in mixed reality over Tokyo.
0:00
/0:14
Yoshi Muroya takes to the skies at Fukushima Air Park. The data from the flight is used the next day in Air Race X, where his flight path is recreated and displayed over Tokyo's Shibuya district.
We watched Yoshi complete an impressive display of twists, turns and various other maneuvers in the sky above Fukushima Sky Park. He was just one of several competitive pilots who flew the same course remotely from various cities around the globe. Their times and flight path data (copied to a USB plugged into the plane's black box) are then converted for use in Air Race X – no international travel required.
On Your Marks... Get Set... Air Race?
After watching Yoshi complete his course at Fukushima on Saturday, the stage was set for Air Race X to take place in Tokyo the following day.
The competition was presented as a live event, available to watch online or across three physical locations in the Shibuya district. The race course itself was centered around the famous Shibuya Crossing, with the respective runs of four pilots recreated and displayed in mixed reality simultaneously across VR and AR devices. The digital nature of the planes also allows for previously impossible scenarios, such as two planes running the course at the same time without the risk of crashing into each other.
We observed the race from an in-person event held for media, investors, local government officials and other guests on the fifteenth floor of a building overlooking Shibuya Crossing. Yoshi Muroya was in attendance and spoke on stage before and after the event, alongside the three other competing international pilots who joined via a Zoom call.
0:00
/0:25
One of the viewing options was a traditional sports feed simulcast to YouTube, which used a mixture of environmental cameras, digital overlays of the planes, and cockpit footage captured from pilots' runs.
As the race began, prospective viewers had a few options. The physical screens around the room displayed a live feed (simulcast publicly on YouTube) overlaying the digital planes onto camera views positioned around the city at key points on the virtual course. For those seeking something more immersive, Pico 4E headsets were available for viewing the race in mixed reality while gazing out of the room's windows via the headset's passthrough camera.
Pico 4 headsets were available for users to observe the race overlaid onto Shibuya using the headset's passthrough mixed reality capabilities.
A similar presentation was available in augmented reality through phones and tablets in the Styly app, allowing users to move their device around to track the digital planes and watch them fly in mobile AR.
Digital content displayed over the city included the planes themselves, course markers, plane path trails, digital billboards and celebratory fireworks. There were speakers that projected audio across the room as planes flew by and commentators provided analysis alongside the action, just like a live sport event.
Feeling Occluded
It was an impressive array of content, but there was one major problem evident from the minute the race started: the experience had absolutely no occlusion on any platform.
0:00
/0:57
Screen recording capture of Air Race X running in augmented reality on an iPhone 12, showing planes and the digital course overlaid on Shibuya without occlusion.
This meant that walls, window frames and other buildings in the Shibuya surroundings were all displayed underneath the AR content. The planes, race course and all virtual objects appeared overlaid on the camera or passthrough view.
While Pico 4E headsets were available at the event for some guests, I wasn't able to view the main race in VR (only AR). However, earlier in the day I tried out a demo of the race on Quest 3 in mixed reality at one of the other event locations.
Admittedly, Quest 3's improved and balanced passthrough image provided for the most striking demonstration of the concept. However it nonetheless ran into the same problems with occlusion. Observing in mixed reality from an open rooftop, there were brief moments – when looking directly at the race area with nothing in the way – where the real promise of Air Race X came to life. But the illusion didn't last long – the planes twisted around a building and should have disappeared behind it. Instead, they appeared overlaid in front of it.
A separate location offered mixed reality demos of the race running on Quest 3. While impressive when looking at the sky and open areas, the experience still suffered from a lack of occlusion with buildings and other parts of the environment.
I'm told that the Pico 4 version of the experience would have been equivalent in terms of the lack of occlusion.
While a single problem amid an admittedly impressive technical and logistical feat, the lack of any occlusion in Air Race X was ultimately difficult to overcome. It's hard to feel excited about a virtual air race when the planes aren't actually weaving around buildings. It simply doesn't feel like the race is happening in physical space, which is the entire point. Without occlusion, the entire concept fails to come to fruition.
0:00
/0:10
Air Race X displayed in AR on an iPad.
Part of the problem stems from hardware and software limitations. Most devices used for the experience, like standard model iPhones, iPads and Pico headsets, don't include depth sensors, which can help devices and developers occlude nearby objects in mixed reality experiences. That said, experiences would likely still rely on system-level APIs to achieve this. As an example of how far out this technology is, Meta only just shipped an experimental depth API for mixed reality occlusion on Quest 3, a headset marketed entirely around mixed reality.
Even then, the level of occlusion and world understanding needed to pull off a concept like Air Race X goes far beyond just nearby objects – it would also need to occlude far away buildings and architecture well outside the range of a depth sensor.
Psychic is using geometric data of the cityscape, made available for use by the Japanese government, to correctly plan and position the content among the streets of Shibuya, but it's unclear whether that same data could be used to help solve the large-scale occlusion problem.
There's a strong argument to be made that, on a conceptual level, an experience like Air Race X shouldn't even be enticed at the moment, given the lack of essential features needed to properly bring it to life. Instead, we were left with a fantastic concept executed on technology that is not nearly ready for something of this scale and complexity.
Screenshots showing the lack of occlusion for digital objects when viewing Air Race X on a mobile device.
I reached out to Psychic after the event to inquire more about the lack of occlusion. Though occlusion is supported on the Styly platform, it was intentionally disabled for Air Race X, even on devices equipped with depth/LiDAR sensors (like the Pro series iPhones and iPads. Psychic found it caused wider misalignment issues with the overall experience when enabled for objects like window frames.
Psychic also stated that implementing such occlusion might restrict audience visibility and "create 'hard-to-see areas' and 'non-visible areas' for the airplane," which I'd argue is the whole point.
Other Reservations
Psychic hopes this is the first of many Air Race X events, with dreams of the Shibuya Crossing course in Tokyo becoming a staple of the sport akin to iconic Formula 1 locations around the world. However, occlusion issues aside, there remain some other reservations about the experience as a whole.
This first Air Race X event was made possible through a combination of large investment, sponsorships and other forms of funding. However, future iterations will need to find an audience – and ways to monetize that audience – in order to remain viable.
The obvious path will be paid access, advertisements and sponsorships. The foundation for those is already in place – during the finals event, sponsors like Lexus had their logos displayed on the virtual billboards across the immersive course, for example.
However, it remains to be seen whether there's genuine public interest in such an immersive experience, especially since a lack of interest is what Red Bull cited when it pulled support for the real thing four years ago.
Even so, the best part of any competitive sport is that it takes places live, in real time. Anything can happen in live sport, which fuels tension, excitement and authenticity. While the audience doesn't know the result of an Air Race X event ahead of time, it is still by nature a pre-recorded event.
A betting system was in place for last weekend's finals race, though there was no real money involved. Most forms of gambling are illegal in Japan, so users were just betting with free tokens and competing for prizes instead of cash. However, actual gambling could be a potential revenue stream for Air Race X events in other countries, even if it raises ethical questions when the competition is ultimately pre-recorded.
Getting Over The Line
At the end of the day, Yoshi Muroya took away first place in the Air Race X finals, awarded the trophy by the major of Shibuya. The result wasn't a huge surprise, given I had heard raucous cheers from Yoshi's team downstairs at Fukushima Air Park the day before, about 20 minutes after he completed his course for us.
Air Race X was just one of many in-development projects shown to me by Psychic VR, all of which seemingly funded by large swathes of continuing investment from local corporations and media companies.
While a interesting concept, it's hard to see the bet paying off while the race isn't properly displayed in the environment with full occlusion – and it might be a long time until that's actually feasible.
However, Air Race X is one side of a coin – the other is CityXR. Another of Psychic's ongoing projects, CityXR aims for Styly to transform city space into virtual immersive playgrounds for developers and users alike. While another bold vision from an unusual company, the project comes with its own set of difficulties that run parallel to those of Air Race X.
We'll discuss CityXR more next week in a second piece covering our weekend in Tokyo exploring the future of augmented and virtual reality with Psychic VR Lab.
Disclosure: Flights, accommodation and food were paid by Psychic VR for the three-day media tour.
Rare Tech Parts aren't named so because they're harder to find than anything else in Spider-Man 2, but are in short supply. Here is how to get them efficiently.
Jamie Feltham
