How to play 360 video on the iOS device

Question

Looking through different web sites and analyzing different resources I found out that for playing 360 videos on the iPhone you should use 3-d party lib (Panorama). But I'm really interested in how it is possible to do it by your own. Because standard iOS elements does not support such functionality.

Please give some advices about approaches that should i use to create own player for 360 videos.

Integrate

I just ask direction where should i investigate – Integrate

Disregard my first comment. I misread your post. You want to do it yourself. – Granddaughter

Hi Oleg, Do you find a good solution? Any free sdk? Or did yo do it yourself (hard work)? – Spurling

Answer 1

You can do it using scenekit, without any external libraries or dependencies.

Create a SCNScene and map your camera to the device movements. The cameras are a bit offsetted as so to map one per eye and create a 3D stereoscopic effect.

override func viewDidLoad() {
    super.viewDidLoad()
    leftSceneView?.backgroundColor = UIColor.blackColor()
    rightSceneView?.backgroundColor = UIColor.whiteColor()

    // Create Scene
    scene = SCNScene()
    leftSceneView?.scene = scene
    rightSceneView?.scene = scene

    // Create cameras
    let camX = 0.0 as Float
    let camY = 0.0 as Float
    let camZ = 0.0 as Float
    let zFar = 50.0

    let leftCamera = SCNCamera()
    let rightCamera = SCNCamera()

    leftCamera.zFar = zFar
    rightCamera.zFar = zFar

    let leftCameraNode = SCNNode()
    leftCameraNode.camera = leftCamera
    leftCameraNode.position = SCNVector3(x: camX - 0.5, y: camY, z: camZ)

    let rightCameraNode = SCNNode()
    rightCameraNode.camera = rightCamera
    rightCameraNode.position = SCNVector3(x: camX + 0.5, y: camY, z: camZ)

    camerasNode = SCNNode()
    camerasNode!.position = SCNVector3(x: camX, y:camY, z:camZ)
    camerasNode!.addChildNode(leftCameraNode)
    camerasNode!.addChildNode(rightCameraNode)

    camerasNode!.eulerAngles = SCNVector3Make(degreesToRadians(-90.0), 0, 0)

    cameraRollNode = SCNNode()
    cameraRollNode!.addChildNode(camerasNode!)

    cameraPitchNode = SCNNode()
    cameraPitchNode!.addChildNode(cameraRollNode!)

    cameraYawNode = SCNNode()
    cameraYawNode!.addChildNode(cameraPitchNode!)

    scene!.rootNode.addChildNode(cameraYawNode!)

    leftSceneView?.pointOfView = leftCameraNode
    rightSceneView?.pointOfView = rightCameraNode

    // Respond to user head movement. Refreshes the position of the camera 60 times per second.
    motionManager = CMMotionManager()
    motionManager?.deviceMotionUpdateInterval = 1.0 / 60.0
    motionManager?.startDeviceMotionUpdatesUsingReferenceFrame(CMAttitudeReferenceFrame.XArbitraryZVertical)

    leftSceneView?.delegate = self

    leftSceneView?.playing = true
    rightSceneView?.playing = true        
}

Update the camera position in the sceneRenderer:

func renderer(aRenderer: SCNSceneRenderer, updateAtTime time: NSTimeInterval){

    // Render the scene
    dispatch_async(dispatch_get_main_queue()) { () -> Void in
        if let mm = self.motionManager, let motion = mm.deviceMotion {
            let currentAttitude = motion.attitude

            var orientationMultiplier = 1.0
            if(UIApplication.sharedApplication().statusBarOrientation == UIInterfaceOrientation.LandscapeRight){ orientationMultiplier = -1.0}

            self.cameraRollNode!.eulerAngles.x = Float(currentAttitude.roll * orientationMultiplier)
            self.cameraPitchNode!.eulerAngles.z = Float(currentAttitude.pitch)
            self.cameraYawNode!.eulerAngles.y = Float(currentAttitude.yaw)

        }
    }
}

Here is some code to add a SCNSphere displaying an AVPlayer.

func play(){

    //let fileURL: NSURL? = NSURL(string: "http://www.kolor.com/360-videos-files/noa-neal-graffiti-360-music-video-full-hd.mp4")
    let fileURL: NSURL? = NSURL.fileURLWithPath(NSBundle.mainBundle().pathForResource("vr", ofType: "mp4")!)

    if (fileURL != nil){
        videoSpriteKitNode =  SKVideoNode(AVPlayer: AVPlayer(URL: fileURL!))
        videoNode = SCNNode()
        videoNode!.geometry = SCNSphere(radius: 30)

        let spriteKitScene = SKScene(size: CGSize(width: 2500, height: 2500))
        spriteKitScene.scaleMode = .AspectFit

        videoSpriteKitNode!.position = CGPoint(x: spriteKitScene.size.width / 2.0, y: spriteKitScene.size.height / 2.0)
        videoSpriteKitNode!.size = spriteKitScene.size

        spriteKitScene.addChild(videoSpriteKitNode!)

        videoNode!.geometry?.firstMaterial?.diffuse.contents = spriteKitScene
        videoNode!.geometry?.firstMaterial?.doubleSided = true

        // Flip video upside down, so that it's shown in the right position
        var transform = SCNMatrix4MakeRotation(Float(M_PI), 0.0, 0.0, 1.0)
        transform = SCNMatrix4Translate(transform, 1.0, 1.0, 0.0)

        videoNode!.pivot = SCNMatrix4MakeRotation(Float(M_PI_2), 0.0, -1.0, 0.0)
        videoNode!.geometry?.firstMaterial?.diffuse.contentsTransform = transform
        videoNode!.position = SCNVector3(x: 0, y: 0, z: 0)

        scene!.rootNode.addChildNode(videoNode!)
        videoSpriteKitNode!.play()

        playingVideo = true   
    }
}

I've put together a project on github to show how, with instructions that should be clear !

Works in VR too with a google cardboard.

https://github.com/Aralekk/simple360player_iOS

Rabi

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review – Offing

Done ! Learning Stackoverflow still – Rabi

Answer 2

Apart from Aralekk's repo, I've found hanton's repo useful when creating my own video 360° player. Here is the link to my repo and here is related blogpost.

Enhanced

Answer 3

You can use this fame work for 360 degree player pod 'NYT360Video' in Objective C and Swift their is a For Objective default Example is provided For Swift use the same Frame work

/// For 360 degree view

 // just import the frame work 
 import NYT360Video    

 // declare the nyt360VC global fro the view controller 
 var nyt360VC: NYT360ViewController!

/// Player implementation
let videoURL = Bundle.main.url(forResource: "360Video", withExtension: "mp4")!
let player = AVPlayer(url: videoURL)
let playerLayer = AVPlayerLayer(player: player)
playerLayer.frame = self.view.bounds
self.view.layer.addSublayer(playerLayer)
player.play()

/// 360 Degree VC implementation along with manager 
let manager: NYT360MotionManagement = NYT360MotionManager.shared()
self.nyt360VC = NYT360ViewController.init(avPlayer: player2,
                                          motionManager: manager)

// Adding the 360 degree view controller to view
self.addChildViewController(nyt360VC)
self.view.addSubview(self.nyt360VC.view)
self.nyt360VC.didMove(toParentViewController: self)

Nautch

Answer 4

I implemented a similar solution using SceneKit for iOS here: ThreeSixtyPlayer.

In my opinion this code is a bit simpler and more scalable than other solutions out there. It is however just the basics (no stereoscopic playback, only supports sphere geometry, doesn't yet support cardboard, etc.).

Lorrettalorri