I have an iOS app that uses RealityView to display some models and interact with them, and the app uses regular iOS app navigations, then a challenge I'm facing is how to maintain multiple RealityView on multiplescreens. For example Screen A has a RealityView, and then I navigate to Screen B (also has a RealityView) using stack based navigation, when I do so I got a crash -[MTLDebugRenderCommandEncoder validateCommonDrawErrors:]:5970: failed assertion `Draw Errors Validation Fragment Function(fsRealityPbr): argument envProbeTable[0] from Buffer(7) with offset(0) and length(16) has space for 16 bytes, but argument has a length(864). Fragment Function(fsRealityPbr): incorrect type of texture (MTLTextureType2D) bound at Texture binding at index 20 (expect MTLTextureTypeCubeArray) for envProbeDiffuseArray[0]. Interestingly this crash only happens when debugging with Xcode, not happens when the app runs on its own. I'm not sure what I'm doing is anti-pattern or it's some Xcode debugging limitation.

Hi, I would like clarification on whether the new hover effects feature introduced in vision os 26 supported pinch gestures through the psvr 2 controllers. In your sample application, I found that this was not working. Pulling the trigger on the controller whilst looking at the 3d object did not activate the hover effect spatial event in the sample application. (The object is showing the highlight though), only pinch clicking with my fingers seem to be registering/triggering the spatial event. I am using Vision OS 26.3 This is inconsistent with how the psvr2 controller behaves on swift ui views and ui view elements, where the trigger press does count as a button click. The sample I used was this one: https://developer.apple.com/documentation/compositorservices/rendering_hover_effects_in_metal_immersive_apps

Hi, Is there a resource or sample code about how to draw an outline around a mesh in RealityKit? Typically, this is useful for visualizing a selection, like in Reality Composer Pro. How to achieve such effect? A shader material? A post-process effect in ARView or RealityRenderer? Methods such as duplicating the entity mesh, scaling it, and using material.faceCulling = .front did not look good in my experiments. Thank you.

Hi, I’m wondering whether RealityKit has its own scene management system, since it uses ARView (backed by ARKit) to present AR content. Does RealityKit manage scenes independently, or does it rely entirely on ARKit’s scene handling? Thank you.

Hi, we have been working on an application that attaches a hand model to the users hands. Apple provides an animating hand models in visionOS project that is a useful starting point. https://developer.apple.com/documentation/visionOS/animating-hand-models-in-visionOS We have been trying to create our own hand model to attach but have had some issues with how it is attaching to the hand. For our hand model we want to include the forearm all the way up to the users elbow. I have attached a sample project of what our code currently looks like so you can run it. Just select show immersive space to attach the models. The left hand model is the space glove that we were trying to mirror. The right hand model is our model that we have been using. I have mapped each of the joints to the pertaining joint name on our model. The first issue we are having seems to be based around the placement of the forearm. It attaches itself at the wrist. The second issue seems to be around rotation. Our team is looking for some guidance on what needs to change in order to map this model correctly. Thanks in advance!

I use ARKit for motion tracking. I get the skeleton joint coordinates and use them for animation. I didn't make any changes to the code, but I updated the iOS version from 18 to 26, and modelTransform now always returns nil. https://developer.apple.com/documentation/arkit/arskeleton3d/modeltransform(for:) For example bodyAnchor.skeleton.modelTransform(for: .init(rawValue: "head_joint")) bodyAnchor is ARBodyAnchor. I see the default skeleton on the screen, but now I can't get the coordinates out of it. I'm using an example from Apple's WWDC presentation. https://developer.apple.com/documentation/arkit/capturing-body-motion-in-3d Are there any changes in the API? Or just bug?

When rendering an equirectangular video on a sphere using VideoMaterial and MeshResource.generateSphere(), there is a visible black seam line running vertically on the sphere. This appears to be at the UV seam where the texture coordinates wrap from 1.0 back to 0.0. The same video file plays without any visible seam in other 360° video players on Vision Pro, so the issue is not with the video content itself. Here is the relevant code: private func createVideoSphere(content: RealityViewContent, player: AVPlayer) { let sphere = MeshResource.generateSphere(radius: 1000) let material = VideoMaterial(avPlayer: player) let entity = ModelEntity(mesh: sphere, materials: [material]) entity.scale *= .init(x: -1, y: 1, z: 1) // Flip to render on inside content.add(entity) player.play() } The setup is straightforward: MeshResource.generateSphere(radius: 1000) generates the sphere mesh VideoMaterial(avPlayer:) provides the video texture X scale is flipped to -1 so the texture renders on the inside of the sphere The video is a standard equirectangular 360° MP4 file What I've tried: I attempted to create a custom sphere mesh using MeshDescriptor with duplicate vertices at the UV seam (longitude 0°/360°) to ensure proper UV continuity. However, VideoMaterial did not render any video on the custom mesh (only audio played), and the app eventually crashed. It seems VideoMaterial may have specific mesh requirements. Questions: Is the black seam a known limitation of MeshResource.generateSphere() when used with VideoMaterial for 360° video? Is there a recommended way to eliminate this UV seam — for example, a texture addressing mode or a specific mesh configuration that works with VideoMaterial? Is there an official sample project or code example for playing 360° equirectangular video in a fully immersive space on visionOS? That would be extremely helpful as a reference. Any guidance would be greatly appreciated. Thank you!

My visionOS 26.3 app displays a diorama-like scene in a RealityView in a mixed immersive space, about 1 meter square, with view attachments floating above the scene. Each view attachment fades out after user interaction, by animating the view's opacity. What I'm observing is that depending on the position of a view attachment relative to the scene and the camera, an unwanted cutout effect is observed (presumably because of draw order issues), as shown in the right column in the screenshots below. YouTube video link of these sequences: https://youtu.be/oTuo0okKCkc (19 seconds) My question: How does visionOS determine the view attachment draw order relative to the RealityView scene? If I better understood how the draw order is determined, I could modify my scene to ensure that the view attachments were always drawn after the scene, fixing the unwanted cutout effect. I've successfully used ModelSortGroupComponent to control the draw order of entities within the RealityView scene, but my understanding is that this approach cannot be used with view attachments. I've submitted FB22014370 about this issue. Thank you.

I have a simple visionOS app that creates an Entity, writes it to the device, and then attempts to load it. However, when the entity file get overwritten, it affects the ability for the app to load it correctly. Here is my code for saving the entity. import SwiftUI import RealityKit import UniformTypeIdentifiers struct ContentView: View { var body: some View { VStack { ToggleImmersiveSpaceButton() Button("Save Entity") { Task { // if let entity = await buildEntityHierarchy(from: urdfPath) { let type = UTType.realityFile let filename = "testing.\(type.preferredFilenameExtension ?? "bin")" let documentsURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0] let fileURL = documentsURL.appendingPathComponent(filename) do { let mesh = MeshResource.generateBox(size: 1, cornerRadius: 0.05) let material = SimpleMaterial(color: .blue, isMetallic: true) let modelComponent = ModelComponent(mesh: mesh, materials: [material]) let entity = Entity() entity.components.set(modelComponent) print("Writing \(fileURL)") try await entity.write(to: fileURL) } catch { print("Failed writing") } } } } .padding() } } Every time I press "Save Entity", I see a warning similar to: Writing file:///var/mobile/Containers/Data/Application/1140E7D6-D365-48A4-8BED-17BEA34E3F1E/Documents/testing.reality Failed to set dependencies on asset 1941054755064863441 because NetworkAssetManager does not have an asset entity for that id. When I open the immersive space, I attempt to load the same file: import SwiftUI import RealityKit import UniformTypeIdentifiers struct ImmersiveView: View { @Environment(AppModel.self) private var appModel var body: some View { RealityView { content in guard let type = UTType.realityFile.preferredFilenameExtension else { return } let documentsURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0] let fileURL = documentsURL.appendingPathComponent("testing.\(type)") guard FileManager.default.fileExists(atPath: fileURL.path) else { print("❌ File does not exist at path: \(fileURL.path)") return } if let entity = try? await Entity(contentsOf: fileURL) { content.add(entity) } } } } I also get errors after I overwrite the entity (by pressing "Save Entity" after I have successfully loaded it once). The warnings that appear when the Immersive space attempts to load the new entity are: Asset 13277375032756336327 Mesh (RealityFileAsset)URL/file:///var/mobile/Containers/Data/Application/1140E7D6-D365-48A4-8BED-17BEA34E3F1E/Documents/testing.reality/Mesh_0.compiledmesh failure: Asset provider load failed: type 'RealityFileAsset' -- RERealityArchive: Failed to open load stream for entry 'assets/Mesh_0.compiledmesh'. Asset 8308977590385781534 Scene (RealityFileAsset)URL/file:///var/mobile/Containers/Data/Application/1140E7D6-D365-48A4-8BED-17BEA34E3F1E/Documents/testing.reality/Scene_0.compiledscene failure: Asset provider load failed: type 'RealityFileAsset' -- RERealityArchive: Failed to read archive entry. AssetLoadRequest failed because asset failed to load '13277375032756336327 Mesh (RealityFileAsset)URL/file:///var/mobile/Containers/Data/Application/1140E7D6-D365-48A4-8BED-17BEA34E3F1E/Documents/testing.reality/Mesh_0.compiledmesh' (Asset provider load failed: type 'RealityFileAsset' -- RERealityArchive: Failed to open load stream for entry 'assets/Mesh_0.compiledmesh'.) The order of operations to make this happen: Launch app Press "Save Entity" to save the entity "Open Immersive Space" to view entity Press "Save Entity" to overwrite the entity "Open Immersive Space" to view entity, failed asset load request Also Launch app, the entity should still be save from last time the app ran "Open Immersive Space" to view entity Press "Save Entity" to overwrite the entity "Open Immersive Space" to view entity, failed asset load request NOTE: It appears I can get it to work slightly better by pressing the "Save Entity" button twice before attempting to view it again in the immersive space.

I want to: Run ARKit on the main rear camera, and while it's running shoot high resolution pictures on the wide camera, without disturbing the AR tracking. Is this possible?

This is a very exciting feature in 26.4 beta. But from the document, it seems can only integrate with NVIDIA CloudXR™ SDK. I'm wondering if it's possible to use this tool to stream immersive video from Mac to Vision Pro?

When using the new RealityKit Manipulation Component on Entities, indirect input will never translate the entity - no matter what settings are applied. Direct manipulation works as expected for both translation and rotation. Is this intended behaviour? This is different from how indirect manipulation works on Model3D. How else can we get translation from this component? visionOS 26 Beta 2 Build from macOS 26 Beta 2 and Xcode 26 Beta 2 Attached is replicable sample code, I have tried this in other projects with the same results. var body: some View { RealityView { content in // Add the initial RealityKit content if let immersiveContentEntity = try? await Entity(named: "MovieFilmReel", in: reelRCPBundle) { ManipulationComponent.configureEntity(immersiveContentEntity, allowedInputTypes: .all, collisionShapes: [ShapeResource.generateBox(width: 0.2, height: 0.2, depth: 0.2)]) immersiveContentEntity.position.y = 1 immersiveContentEntity.position.z = -0.5 var mc = ManipulationComponent() mc.releaseBehavior = .stay immersiveContentEntity.components.set(mc) content.add(immersiveContentEntity) } } }

My application calculates three distinct Meesus Double [x, y, z] Radian values to light a sphere in RealityKit with DirectionalLight. It is my understanding that I must use (simd_quatf) for each radian value to properly light the sphere in the view. The code correctly [orientates] the sphere with the combined (simd_quatf) DirectionalLight in the view, but the illumination (Z-axis) fails to properly illuminate the sphere with the expected result, compared to associated Meesus web page images. For the moment, I do not know how to correct the (Z-axis). Curious for a suggestion ... :] // Location values. let theLatitude: Double = 51.13107260 let theLongitude: Double = -114.01127910 let currentDate: Date = Date() struct TheCalculatedMoonPhaseTest_ContentView: View { var body: some View { VStack { if #available(macOS 15.0, *) { RealityView { content in let moonSphere_Entity = Entity.createSphere(radius: 0.90, color: .black) moonSphere.Entity.name = "MoonSphere" moonSphere.Entity.position = SIMD3<Float>(x: 0, y: 0, z: 0) content.add(moonSphere.Entity) let sunLight_Entity = createDirectionalLight(latitude: theLatitude, longitude: theLongitude, date: currentDate) content.add(sunLight_Entity) } // End of [RealityView] } else { // Earlier version required. } // End of [if #available(macOS 15.0, *)] } // End of [VStack] .background(Color.black) } // End of [var body: some View] // MARK: - 🟠🟠🟠🟠 [SET THE BACKGROUND COLOUR] 🟠🟠🟠🟠 var backgroundColor: Color = Color.init(.black) // MARK: - 🟠🟠🟠🟠 [CREATE THE DIRECTIONAL LIGHT FOR THE SPHERE] 🟠🟠🟠🟠 func createDirectionalLight(latitude: Double, longitude: Double, date: Date) -> Entity { let directionalLight = DirectionalLight() directionalLight.light.color = .white directionalLight.light.intensity = 1000000 directionalLight.shadow = DirectionalLightComponent.Shadow() directionalLight.shadow?.maximumDistance = 5 directionalLight.shadow?.depthBias = 1 // MARK: 🟠🟠🟠🟠 Retrieve the [MEESUS MOON AGE VALUES] from the [CONSTANT FOLDER] 🟠🟠🟠🟠 let theMeesusMoonAge_LunarAgeDaysValue = 25.90567592898601 if theMeesusMoonAge_LunarAgeDaysValue >= 23.10 && theMeesusMoonAge_LunarAgeDaysValue < (29.530588853 - 1.00) { let someCalculatedX_WestEastRadian: Float = Float(1.00) // Identify the sphere’s DirectionalLight Tilt Angle (Y) radian value :: // Note :: The following Tilt Angle is corrected to [Zenith] with the [MeesusCalculatedTilt_Angle] minus the [MeesusCalculatedPar_Angle]. let someCalculatedY_TiltAngleRadian: Float = Float(1.3396086) // Identify the sphere’s DirectionalLight Illumination (Z) radian Value :: // Note :: The Meesus calculated illumination fraction is converted to degrees, then converted to a radian value. let someCalculatedZ_IlluminationAngleRadian: Float = Float(0.45176168630244457) // <=== 14.3800% Illumination. // Define rotation angles in radians for X, Y, and Z axes. let x_Radians = someCalculatedX_WestEastRadian let y_Radians = someCalculatedY_TiltAngleRadian let z_Radians = someCalculatedZ_IlluminationAngleRadian // Identify and separate the quaternion [simd_quatf] for each Radian. let q_X = simd_quatf(angle: x_Radians, axis: SIMD3<Float>(1, 0, 0)) let q_Y = simd_quatf(angle: y_Radians, axis: SIMD3<Float>(0, 1, 0)) let q_Z = simd_quatf(angle: z_Radians, axis: SIMD3<Float>(0, 0, 1)) // Apply and combine the rotations, where order matters. let combinedRotation = q_Z * q_Y * q_X // Identify the [Combined Rotation]. // The [MyMoonMeesus] :: [WANING CRESCENT] calculated [combinedRotation] :: simd_quatf(real: 0.73715997, imag: SIMD3<Float>(0.24427173, 0.61516714, -0.13599981)) ° Radians // Normalize the [combinedRotation]. let theNormalizesRotation = simd_normalize(combinedRotation) // Identify the [Normalized Combined Rotation]. // The [MyMoonMeesus] :: [WANING CRESCENT] calculated [normalizedRotation] :: simd_quatf(real: 0.73715997, imag: SIMD3<Float>(0.24427173, 0.61516714, -0.13599981)) ° Radians // Assume the [theNormalizesRotation] appears reversed. let theCorrectedRotation = theNormalizesRotation.inverse // Identify the [Reversed Combined Rotation]. // The [MyMoonMeesus] :: [WANING CRESCENT] calculated [correctedRotation] :: simd_quatf(real: 0.73715997, imag: SIMD3<Float>(-0.24427173, -0.61516714, 0.13599981)) ° Radians // Apply the [Corrected Rotation] to the entity. directionalLight.transform.rotation *= theCorrectedRotation // Add the [directionalLight] to the scene :: let anchor = AnchorEntity() anchor.addChild(directionalLight) } // End of [if theMeesusMoonAge_LunarAgeDaysValue >= 23.10 && theMeesusMoonAge_LunarAgeDaysValue < (29.530588853 - 1.00)] return directionalLight } // End of [func createDirectionalLight(latitude: Double, longitude: Double, date: Date) -> Entity] } // End of [struct TheCalculatedMoonPhaseTest_ContentView: View] // MARK: 🟠🟠🟠🟠 [ENTITY HELPER EXTENSION] 🟠🟠🟠🟠 extension Entity { static func createSphere(radius: Float, color: NSColor) -> Entity { let mesh = MeshResource.generateSphere(radius: radius) var material = PhysicallyBasedMaterial() material.baseColor = .init(tint: color) let modelComponent = ModelComponent(mesh: mesh, materials: [material]) let entity = Entity() entity.components.set(modelComponent) entity.components.set(Transform()) return entity } // End of [static func createSphere(radius: Float, color: NSColor) -> Entity] } // End of [extension Entity] // Application Image :: Calgary // Website Image :: timeanddate // mooncalc.org

I am a developer working on developing a space journal application. During the development process, I encountered several crucial strategic and technical decisions, and I would like to hear the experiences of those who have gone through similar situations. Here are the simplified versions of several questions I have. Resource allocation: Which problem should I address first? Design direction: In terms of interaction and UI design, how should I balance "immersion" and "usability"? Market selection: Was it easier for a business to survive in the early stages as a B2B or B2C entity? Cost estimation: How can I reasonably present to my investors the development costs of this project? In order to avoid relying solely on intuition in my decisions, I created a short questionnaire, hoping to gather more structured opinions from my colleagues. If you are also exploring VisionOS, I sincerely hope you can take a few minutes to fill it out. The results are extremely important to me, and I would be more than happy to share the final summary findings with you.

Hi, I'm trying to use AnchorEntity for horizontal surfaces. It works when the entity is being created, but I'm looking for a way to snap this entity to the nearest surface, after translating it, for example with a DragGesture. What would be the best way to achieve this? Using raycast, creating a new anchor, trackingMode to continuous etc. Do I need to use ARKitSession as I want continuous tracking?

The following sample code project does not seem to work as expected: https://developer.apple.com/documentation/visionos/implementing-shareplay-for-immersive-spaces-in-visionos Have tried to get this project working with a client, but while we were able to see nearby users and make facetime calls, the color changing cube experience always remained a single color. Are there step-by-step instructions that Apple has used to verify this sample code so I can try to recreate this sample code's expected behavior for both nearby participants and those in a Facetime call?

Hi, is there a way to track feet in a visionOS app in an immersive space? I want the whole body to be visible in VR, and I want to know if the user touches an object with their foot.

I’ve seen, mainly in discussions with AIs, that ARFaceTrackingConfiguration uses the same technology as Face ID and therefore should work in complete darkness. However, I haven’t been able to achieve this. Does anyone know if this is actually true? I'm using an iPhone 16 to test, and the Face ID works well in darkness.

I have already add liscene and entitlement for Passthrough in screen capture, but still get black background instaed of real world. Do you have any idea ? I've been stuck with this question for a long time 😢

Hello, I'm struggling trying to interact with a RealityKit entity nested (or at least visually nested) in a second one. I think I've the same issue as mentioned on this StackOverflow post, but I can't manage to reproduce the solution. https://stackoverflow.com/questions/79244424/how-to-prioritize-a-specific-entity-when-collision-boxes-overlap-in-realitykit What I'd like to achieve is to translate the red box using a DragGesture, while still be able to interact using a TapGesture on the sphere. Currently, I can only do one at a time. Does anyone know the solution? extension CollisionGroup { static let parent: CollisionGroup = CollisionGroup(rawValue: 1 << 0) static let child: CollisionGroup = CollisionGroup(rawValue: 1 << 1) } struct ImmersiveView: View { var body: some View { RealityView { content in let boxMesh = MeshResource.generateBox(size: 0.35) let boxMaterial = SimpleMaterial(color: .red.withAlphaComponent(0.25), isMetallic: false) let boxEntity = ModelEntity(mesh: boxMesh, materials: [boxMaterial]) let sphereMesh = MeshResource.generateSphere(radius: 0.05) let sphereMaterial = SimpleMaterial(color: .blue, isMetallic: false) let sphereEntity = ModelEntity(mesh: sphereMesh, materials: [sphereMaterial]) content.add(sphereEntity) content.add(boxEntity) boxEntity.components.set(InputTargetComponent()) boxEntity.components.set( CollisionComponent( shapes: [ShapeResource.generateBox(size: SIMD3<Float>(repeating: 0.35))], isStatic: true, filter: CollisionFilter( group: .parent, mask: .parent.subtracting(.child) ) ) ) sphereEntity.components.set(InputTargetComponent()) sphereEntity.components.set(HoverEffectComponent()) sphereEntity.components.set( CollisionComponent( shapes: [ShapeResource.generateSphere(radius: 0.05)], isStatic: true, filter: CollisionFilter( group: .child, mask: .child.subtracting(.parent) ) ) ) } } }