How to disable libav autorotate display matrix

Question

I have a video taken from my mobile in portrait mode. Here is the dumped info about the video:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video.MOV':
Metadata:
major_brand     : qt  
minor_version   : 0
compatible_brands: qt  
creation_time   : 2017-05-04 02:21:37
Duration: 00:00:06.91, start: 0.000023, bitrate: 4700 kb/s
Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 90 kb/s (default)
Metadata:
  creation_time   : 2017-05-04 02:21:37
  handler_name    : Core Media Data Handler
Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720, 4602 kb/s, 29.98 fps, 29.97 tbr, 600 tbn, 1200 tbc (default)
Metadata:
  rotate          : 90
  creation_time   : 2017-05-04 02:21:37
  handler_name    : Core Media Data Handler
  encoder         : H.264
Side data:
  displaymatrix: rotation of -90.00 degrees
  5.78 A-V: -0.028 fd=   1 aq=   14KB vq=  351KB sq=    0B f=0/0   

I am using libav api to decode/encode my video. After encoding, I get the rotated version of the video by -90 degrees.

How can I stop decoder to prevent auto-rotation?

Auklet

See the source code related to the -noautorotate option for the cli tool. – Protochordate

@LordNeckbeard, I did so, but I still have a problem with disabling autorotae in libav api. Can you provide details about it? – Auklet

Answer 1

After a long time dealing with problems caused by display matrix side data, I came across the following solutions:

Copy the metadata and side data along with video stream. This code does the trick

AVStream *in_stream; //input stream
AVStream *out_stream; //output stream
....        
if(in_stream->side_data!=NULL){

  av_log(NULL, AV_LOG_ERROR, "side data size: %d , size: %d\n", in_stream->side_data->size, sizeof(uint8_t*));

        if(av_stream_get_side_data(in_stream , AV_PKT_DATA_DISPLAYMATRIX, sd_size)!=NULL){

            uint8_t* resp=(uint8_t*) av_mallocz(in_stream->side_data->size+
                                                AV_INPUT_BUFFER_MIN_SIZE);


            resp  = av_stream_get_side_data(in_stream , AV_PKT_DATA_DISPLAYMATRIX, sd_size);

              av_log(NULL, AV_LOG_DEBUG,"side data detected, size :%d vs %d, "
                                        "nb_side_data %d, sizeof data %d,"
                                        " sizeof resp %d\n",
                     *sd_size, in_stream->side_data->size ,in_stream->nb_side_data,
                     in_stream->side_data->size*sizeof(uint8_t*),sizeof(resp));
              in_stream->side_data->data = (uint8_t*) av_mallocz(
                          in_stream->side_data->size*sizeof(uint8_t*)
                          );
              hasRotation = true; /* it will be used for 
              something like ffmpeg does with noautorotate flag */
              av_stream_add_side_data(out_stream, AV_PKT_DATA_DISPLAYMATRIX, resp, *sd_size);

        }
    }

The resulting video will have a display matrix, and some players will show them right.

Using simple filters As demonstrated above, one can detect the video is rotated. In my case the rotation is -90, so I use transpose=clock filter in initiating my filtergraph. Consider transoding example in ffmpeg 3.4.1. I change the code as follow to disable autorotate by -90 degrees.

static int init_filters(void)
{
    const char *filter_spec;
    unsigned int i;
    int ret;
    filter_ctx = (FilteringContext*) av_malloc_array(ifmt_ctx->nb_streams, sizeof(*filter_ctx));
if (!filter_ctx)
    return AVERROR(ENOMEM);
     for (i = 0; i < ifmt_ctx->nb_streams; i++) {
       filter_ctx[i].buffersrc_ctx  = NULL;
       filter_ctx[i].buffersink_ctx = NULL;
       filter_ctx[i].filter_graph   = NULL;
       if (!(ifmt_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO
            || ifmt_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO))
        continue;
        if (ifmt_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO)
        filter_spec = isRotated ?"transpose=clock":"null"; /* passthrough (dummy) filter for video */
    else
        filter_spec = "anull"; /* passthrough (dummy) filter for audio */
    ret = init_filter(&filter_ctx[i], stream_ctx[i].dec_ctx,
            stream_ctx[i].enc_ctx, filter_spec);
    if (ret)
        return ret;
}
return 0;
}

Note that the encoder dimensions should also be swapped. It means:

enc_ctx->width = dec_ctx->height; 
enc_ctx->height = enc_ctx->width;

I hope it would help others.

Auklet