Swift 2 : AVAssetReader and NSInputStream Audio Graph
Asked Answered
J

1

7

I'm trying to convert an example from Bob McCune's Learning AVFoundation book and having some issues using AVAssetReader and NSInputStream. The graph should be a pure sine wave but the values seem reflected on the X-axis somehow.

I've tried every iteration of byte swapping I could think of and that didn't work.

Playground posted to github here: https://github.com/justinlevi/AVAssetReader

//: Playground - noun: a place where people can play

import UIKit
import AVFoundation
import XCPlayground

func plotArrayInPlayground<T>(arrayToPlot:Array<T>, title:String) {
  for currentValue in arrayToPlot {
    XCPCaptureValue(title, value: currentValue)
  }
}

class SSSampleDataFilter {
  var sampleData:NSData?

  init(data:NSData) {
    sampleData = data
  }

  func filteredSamplesForSize(size:CGSize) -> [Int]{
    var filterSamples = [UInt16]()

    if let sampleData = sampleData {
      let sampleCount = sampleData.length
      let binSize = CGFloat(sampleCount) / size.width

      let stream = NSInputStream(data: sampleData)
      stream.open()

      var readBuffer = Array<UInt8>(count: 16 * 1024, repeatedValue: 0)
      var totalBytesRead = 0

      let size = sizeof(UInt16)
      while (totalBytesRead < sampleData.length) {
        let numberOfBytesRead = stream.read(&readBuffer, maxLength: size)
        let u16: UInt16 = UnsafePointer<UInt16>(readBuffer).memory

        var sampleBin = [UInt16]()
        for _ in 0..<Int(binSize) {
          sampleBin.append(u16)
        }

        filterSamples.append(sampleBin.maxElement()!)
        totalBytesRead += numberOfBytesRead
      }

      //plotArrayInPlayground(filterSamples, title: "Samples")
    }

    return [0]

  }
}

let sineURL = NSBundle.mainBundle().URLForResource("440.0-sine", withExtension: "aif")!
let asset = AVAsset(URL: sineURL)
var assetReader:AVAssetReader

do{
  assetReader = try AVAssetReader(asset: asset)
}catch{
  fatalError("Unable to read Asset: \(error) : \(__FUNCTION__).")
}

let track = asset.tracksWithMediaType(AVMediaTypeAudio).first
let outputSettings: [String:Int] =
  [ AVFormatIDKey: Int(kAudioFormatLinearPCM),
    AVLinearPCMIsBigEndianKey: 0,
    AVLinearPCMIsFloatKey: 0,
    AVLinearPCMBitDepthKey: 16,
    AVLinearPCMIsNonInterleaved: 0]

let trackOutput = AVAssetReaderTrackOutput(track: track!, outputSettings: outputSettings)

assetReader.addOutput(trackOutput)
assetReader.startReading()

var sampleData = NSMutableData()

while assetReader.status == AVAssetReaderStatus.Reading {
  if let sampleBufferRef = trackOutput.copyNextSampleBuffer() {
    if let blockBufferRef = CMSampleBufferGetDataBuffer(sampleBufferRef) {
      let bufferLength = CMBlockBufferGetDataLength(blockBufferRef)
      var data = NSMutableData(length: bufferLength)
      CMBlockBufferCopyDataBytes(blockBufferRef, 0, bufferLength, data!.mutableBytes)
      var samples = UnsafeMutablePointer<Int16>(data!.mutableBytes)
      sampleData.appendBytes(samples, length: bufferLength)
      CMSampleBufferInvalidate(sampleBufferRef)
    }
  }
}

let view = UIView(frame: CGRectMake(0, 0, 375.0, 667.0))
//view.backgroundColor = UIColor.lightGrayColor()

if assetReader.status == AVAssetReaderStatus.Completed {
  print("complete")

  let filter = SSSampleDataFilter(data: sampleData)
  let filteredSamples = filter.filteredSamplesForSize(view.bounds.size)
}

//XCPShowView("Bezier Path", view: view)
XCPSetExecutionShouldContinueIndefinitely(true)

Here's what the graph should look like (taken from Audacity) Sine wave graph of audio file

Here's what the graph looks like in the playground

enter image description here

Jordaens answered 21/8, 2015 at 20:43 Comment(0)
B
2

Unfortunately your playground doesn't render anything for me in Xcode7b5, however you're asking the AVAssetReaderTrackOutput to give you signed 16bit ints, yet your code treats them as unsigned UInt16s (and your Audacity file uses floats).

Changing all instances of UInt16 to Int16 in your playground seems to print sensible looking sinusoidal data.

Burnaby answered 22/8, 2015 at 0:39 Comment(5)
Odd that it didn't render in Xcode7b5 for you. Did you grab the version from github that included the AIFF audio file? Sure enough, as soon as I changed all instances of UInt16 to 'Int16` everything seems to be working.Jordaens
Stared at that for hours yesterday. Thank youJordaens
Follow up question though - Any idea why aif, caf, m4a all work fine but an mp3 version of the same audio does not? I need to go back to the docs and see if I can understand why but I though the AVassetReader took care of the mp3 compression as wellJordaens
What happens with mp3?Burnaby
Hard to say in the playground but the error is "Execution was interrupted, reason: EXC_BAD_INSTRUCTION (code=EXC_i386_INVOP, subcode=0x0). Seems to be working fine in an iOS project that I'll post to my github account though so seems like it's not worth figuring out :) Thanks again.Jordaens

© 2022 - 2024 — McMap. All rights reserved.