Retrieve album art using FFmpeg

Question

I'm developing an Android application that relies on FFmpeg to retrieve audio metadata. I know it's possible to retrieve album art programmatically using FFMpeg. However, once you have decoded the art (a video frame within an MP3) how do generate an image file (a PNG) for use within an application? I've search all over but can't seem to find a working example.

Edit, here is the solution:

#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>

void retrieve_album_art(const char *path, const char *album_art_file) {
    int i, ret = 0;

    if (!path) {
        printf("Path is NULL\n");
        return;
    }

    AVFormatContext *pFormatCtx = avformat_alloc_context();

    printf("Opening %s\n", path);

    // open the specified path
    if (avformat_open_input(&pFormatCtx, path, NULL, NULL) != 0) {
        printf("avformat_open_input() failed");
        goto fail;
    }

    // read the format headers
    if (pFormatCtx->iformat->read_header(pFormatCtx) < 0) {
        printf("could not read the format header\n");
        goto fail;
    }

    // find the first attached picture, if available
    for (i = 0; i < pFormatCtx->nb_streams; i++)
        if (pFormatCtx->streams[i]->disposition & AV_DISPOSITION_ATTACHED_PIC) {
            AVPacket pkt = pFormatCtx->streams[i]->attached_pic;
            FILE* album_art = fopen(album_art_file, "wb");
            ret = fwrite(pkt.data, pkt.size, 1, album_art);
            fclose(album_art);
            av_free_packet(&pkt);
            break;
        }

    if (ret) {
        printf("Wrote album art to %s\n", album_art_file);
    }

    fail:
        av_free(pFormatCtx);
        // this line crashes for some reason...
        //avformat_free_context(pFormatCtx);
}

int main() {
    avformat_network_init();
    av_register_all();

    const char *path = "some url";
    const char *album_art_file = "some path";

    retrieve_album_art(path, album_art_file);

    return 0;
}

Ljoka

Answer 1

To use ffmpeg programmatically, I think you would have to call read_apic() in libavformat (which is part of ffmpeg).

From the commandline, you can apparently do this:

ffmpeg -i input.mp3 -an -vcodec copy cover.jpg

-an: disables audio
-vcodec codec: force video codec ('copy' to copy stream)

The commandline behaviour implies that the cover art image is seen as just another video stream (containing just one frame), so using libavformat in the usual way you would to demux the video part of a stream should produce that image.

Sample code for demuxing: ffmpeg/docs/examples/demuxing.c The first (and only) AVPacket that would be obtained from demuxing the video stream in an mp3 would contain the JPEG file (still encoded as JPEG, not decoded).

AVFormatContext* fmt_ctx;
// set up fmt_ctx to read first video stream
AVPacket pkt;
av_read_frame(fmt_ctx, &pkt);
FILE* image_file = fopen("image.jpg", "wb");
int result = fwrite(pkt.data, pkt.size, 1, image_file);
fclose(image_file);

If there are multiple images, I think they would be seen as separate video streams, rather than as separate packets in the same stream. The first stream would be the one with the largest resolution.

All this is probably implemented internally in terms of read_apic().

The ID3v2 spec allows for any image format, but recommends JPEG or PNG. In practice all images in ID3 are JPEG.

EDIT: Moved some of the less useful bits to postscript:

P.S. ffmpeg -i input.mp3 -f ffmetadata metadata.txt will produce an ini-like file containing the metadata, but the image is not even referred to in there, so that is not a useful approach.

P.S. There may be multiple images in an ID3v2 tag. You may have to handle the case when there is more than one image or more than one type of image present.

P.S. ffmpeg is probably not the best software for this. Use id3lib, TagLib, or one of the other implementations of ID3. These can be used either as libraries (callable from the language of your choice) or as commandline utilities. There is sample C++ code for TagLib here: How do I use TagLib to read/write coverart in different audio formats? and for id3lib here: How to get album art from audio files using id3lib.

Dejesus

@WilliamSeemann: I don't understand why this doesn't meet your needs. I did describe two different ways to get the images from an mp3 file using ffmpeg programmatically: One by calling read_apic(), and the other by calling av_read_frame() as shown in the demuxing.c example. – Dejesus

"I need them in file rather than rendered directly from a buffer." Just to clarify, the AVPacket you would get from av_read_frame() contains literally just one JPEG file (still compressed), not the decoded image or anything like that. You can straightforwardly save that to a file if needed, I'm writing some sample code for you right now. – Dejesus

@WilliamSeemann: "packet that is written to a file without being decoded is pretty useless" - why? It is actually a JPEG file, not a "packet". You did say "I don't really care what format the images are, but I need them in file" and this way you get the image in a file in JPEG format. – Dejesus

It wasn't immediately apparent to me that I could just simply write the packet directly to a file. Thanks for the help. – Ljoka

Thanks It Helped Me Too. – Motivate

Answer 2

As an addition the answer above, I needed a way to resize the output image as well so I found the below command while experimenting with the command in the current answer:

ffmpeg -i input.mp3 -filter:v scale=-2:250 -an output.jpeg

So this basically scales the output image to whatever ratio or value you want.

Interlunation

Answer 3

I see it's solved, but if that can help, I think your crash there:

// this line crashes for some reason...
//avformat_free_context(pFormatCtx);

Is because you free the AVPacket earlier:

AVPacket pkt = pFormatCtx->streams[i]->attached_pic;
....
av_free_packet(&pkt);

But you don't reference it, copy it, or whatever, so when you try to avformat_free_context() it iterates over the streams and their parts, and tries to double-free the ->attached_pic.

Otherwise, thanks for the example code, it helped me!

Camelot