How to use AVAssetReader and AVAssetWriter for multiple tracks (audio and video) simultaneously?
Asked Answered
S

3

11

I know how to use AVAssetReader and AVAssetWriter, and have successfully used them to grab a video track from one movie and transcode it into another. However, I'd like to do this with audio as well. Do I have to create and AVAssetExportSession after I've done with the initial transcode, or is there some way to switch between tracks while in the midst of a writing session? I'd hate to have to deal with the overhead of an AVAssetExportSession.

I ask because, using the pull style method - while ([assetWriterInput isReadyForMoreMediaData]) {...} - assumes one track only. How could it be used for more than one track, i.e. both an audio and a video track?

Spat answered 9/3, 2011 at 2:10 Comment(1)
Would you be able to point me in the right direction to learn how to transcode using AVAssetWriter? Just looking to transcode to lower bitrate and resolution?Isle
L
8

AVAssetWriter will automatically interleave requests on its associated AVAssetWriterInputs in order to integrate different tracks into the output file. Just add an AVAssetWriterInput for each of the tracks that you have, and then call requestMediaDataWhenReadyOnQueue:usingBlock: on each of your AVAssetWriterInputs.

Here's a method I have that calls requestMediaDataWhenReadyOnQueue:usingBlock:. I call this method from a loop over the number of output/input pairs I have. (A separate method is good both for code readability and also because, unlike a loop, each call sets up a separate stack frame for the block.)

You only need one dispatch_queue_t and can reuse it for all of the tracks. Note that you definitely should not call dispatch_async from your block, because requestMediaDataWhenReadyOnQueue:usingBlock: expects the block to, well, block until it has filled in as much data as the AVAssetWriterInput will take. You don't want to return before then.

- (void)requestMediaDataForTrack:(int)i {
  AVAssetReaderOutput *output = [[_reader outputs] objectAtIndex:i];
  AVAssetWriterInput *input = [[_writer inputs] objectAtIndex:i];

  [input requestMediaDataWhenReadyOnQueue:_processingQueue usingBlock:
    ^{
      [self retain];
      while ([input isReadyForMoreMediaData]) {
        CMSampleBufferRef sampleBuffer;
        if ([_reader status] == AVAssetReaderStatusReading &&
            (sampleBuffer = [output copyNextSampleBuffer])) {

          BOOL result = [input appendSampleBuffer:sampleBuffer];
          CFRelease(sampleBuffer);

          if (!result) {
            [_reader cancelReading];
            break;
          }
        } else {
          [input markAsFinished];

          switch ([_reader status]) {
            case AVAssetReaderStatusReading:
              // the reader has more for other tracks, even if this one is done
              break;

            case AVAssetReaderStatusCompleted:
              // your method for when the conversion is done
              // should call finishWriting on the writer
              [self readingCompleted];
              break;

            case AVAssetReaderStatusCancelled:
              [_writer cancelWriting];
              [_delegate converterDidCancel:self];
              break;

            case AVAssetReaderStatusFailed:
              [_writer cancelWriting];
              break;
          }

          break;
        }
      }
    }
  ];
}
Lueck answered 9/4, 2011 at 1:20 Comment(2)
Whats with the [self retain]? Won't that just keep incrementing the retain count on the object an arbitrary amount? Who is releasing this object?Wraith
The intention may have been for readingCompleted to release the object. That being said, it may be unnecessary as the presence of the [self readingCompleted] call should cause the runtime to retain self automatically.Lueck
S
1

Have you tried using two AVAssetWriterInputs and pushing the samples through a worker queue? Here is a rough sketch.

processing_queue = dispatch_queue_create("com.mydomain.gcdqueue.mediaprocessor", NULL);

[videoAVAssetWriterInput requestMediaDataWhenReadyOnQueue:myInputSerialQueue usingBlock:^{
    dispatch_asyc(processing_queue, ^{process video});
}];

[audioAVAssetWriterInput requestMediaDataWhenReadyOnQueue:myInputSerialQueue usingBlock:^{
    dispatch_asyc(processing_queue, ^{process audio});
}];
Struble answered 14/3, 2011 at 6:36 Comment(5)
Thanks for the suggestion. I've tried it, but couldn't figure out how to properly end the writing session. With one track, I just see if copyNextSampleBuffer returns empty, then end the session. With multiple tracks, I tried creating a flag for when the audio or video was done, and if both were complete I would end the session. However, even though I did a check for the asset reader status being AVAssetReaderStatusReading, I kept getting the error: [AVAssetReaderTrackOutput copyNextSampleBuffer] cannot copy next sample buffer unless the asset reader is in the 'reading' state'Spat
I will bet testing a strategy much like this in the near future. If I find a solution I will post back.Struble
Are you calling [AVAssetWriterInput markAsFinished]; in your requestMediaDataWhenReadyOnQueue method? I would not call that method at all, but rather call finishWriting on the AVAssetWriter when everything is complete.Struble
sorry for the late reply, I wasn't notified of the comment. I only call finishWriting, just as you're doing.Spat
By the way, your "'reading' state" error may actually be a codec issue. If the inputs and outputs are not compatible formats that's the way it will sometimes surface.Lueck
D
0

You can use dispatch groups!

Check out the AVReaderWriter example for MacOSX...

I am quoting directly from the sample RWDocument.m:

- (BOOL)startReadingAndWritingReturningError:(NSError **)outError
{
    BOOL success = YES;
    NSError *localError = nil;

    // Instruct the asset reader and asset writer to get ready to do work
    success = [assetReader startReading];
    if (!success)
        localError = [assetReader error];
    if (success)
    {
        success = [assetWriter startWriting];
        if (!success)
            localError = [assetWriter error];
    }

    if (success)
    {
        dispatch_group_t dispatchGroup = dispatch_group_create();

        // Start a sample-writing session
        [assetWriter startSessionAtSourceTime:[self timeRange].start];

        // Start reading and writing samples
        if (audioSampleBufferChannel)
        {
            // Only set audio delegate for audio-only assets, else let the video channel drive progress
            id <RWSampleBufferChannelDelegate> delegate = nil;
            if (!videoSampleBufferChannel)
                delegate = self;

            dispatch_group_enter(dispatchGroup);
            [audioSampleBufferChannel startWithDelegate:delegate completionHandler:^{
                dispatch_group_leave(dispatchGroup);
            }];
        }
        if (videoSampleBufferChannel)
        {
            dispatch_group_enter(dispatchGroup);
            [videoSampleBufferChannel startWithDelegate:self completionHandler:^{
                dispatch_group_leave(dispatchGroup);
            }];
        }

        // Set up a callback for when the sample writing is finished
        dispatch_group_notify(dispatchGroup, serializationQueue, ^{
            BOOL finalSuccess = YES;
            NSError *finalError = nil;

            if (cancelled)
            {
                [assetReader cancelReading];
                [assetWriter cancelWriting];
            }
            else
            {
                if ([assetReader status] == AVAssetReaderStatusFailed)
                {
                    finalSuccess = NO;
                    finalError = [assetReader error];
                }

                if (finalSuccess)
                {
                    finalSuccess = [assetWriter finishWriting];
                    if (!finalSuccess)
                        finalError = [assetWriter error];
                }
            }

            [self readingAndWritingDidFinishSuccessfully:finalSuccess withError:finalError];
        });

        dispatch_release(dispatchGroup);
    }

    if (outError)
        *outError = localError;

    return success;
}
Dispersion answered 16/9, 2012 at 15:54 Comment(2)
Notice the dispatch group notify command at the end to know when all audio/video is finished... this is offscreen rendering... not realtimeDispersion
"Offscreen" and "real-time" are not opposites; "rendered" and "off-screen" are (and do). "Real-time" is subjective to an observer, and describes the display rate of video frames as they are acquired from their source. It has no opposite, not even "non-real-time." You wouldn't want a transcoder to work at a real-time pace. Transcoding should happen as quickly as possible; there are dedicated devices for this in the broadcast industry. Nothing goes slower than normal real-time, anyway. Even playback software has to pace video to keep it from displaying faster than it can be perceived.Creativity

© 2022 - 2024 — McMap. All rights reserved.