## An ffmpeg and SDL TutorialPage 1 2 3 4 5 6 7 8 End Prev Home Next  Printable version Text version## Tutorial 02: Outputting to the ScreenCode: tutorial02.c### SDL and VideoTo draw to the screen, we're going to use SDL. SDL stands for Simple DirectLayer, and is an excellent library for multimedia, is cross-platform, and isused in several projects. You can get the library at the official website oryou can download the development package for your operating system if there isone. You'll need the libraries to compile the code for this tutorial (and forthe rest of them, too). SDL has many methods for drawing images to the screen, and it has one inparticular that is meant for displaying movies on the screen - what it calls aYUV overlay. YUV (technically not YUV but YCbCr) *** A note: **There is agreat deal of annoyance from some people at the convention of calling "YCbCr""YUV". Generally speaking, YUV is an analog format and YCbCr is a digitalformat. ffmpeg and SDL both refer to YCbCr as YUV in their code and macros. isa way of storing raw image data like RGB. Roughly speaking, Y is thebrightness (or "luma") component, and U and V are the color components. (It'smore complicated than RGB because some of the color information is discarded,and you might have only 1 U and V sample for every 2 Y samples.) SDL's YUVoverlay takes in a raw array of YUV data and displays it. It accepts 4different kinds of YUV formats, but YV12 is the fastest. There is another YUVformat called YUV420P that is the same as YV12, except the U and V arrays areswitched. The 420 means it is subsampled at a ratio of 4:2:0, basicallymeaning there is 1 color sample for every 4 luma samples, so the colorinformation is quartered. This is a good way of saving bandwidth, as the humaneye does not percieve this change. The "P" in the name means that the formatis "planar" -- simply meaning that the Y, U, and V components are in separatearrays. ffmpeg can convert images to YUV420P, with the added bonus that manyvideo streams are in that format already, or are easily converted to thatformat. So our current plan is to replace the SaveFrame() function from Tutorial 1,and instead output our frame to the screen. But first we have to start byseeing how to use the SDL Library. First we have to include the libraries andinitalize SDL:             #include <SDL.h>    #include <SDL_thread.h>        if(SDL_Init(SDL_INIT_VIDEO | SDL_INIT_AUDIO | SDL_INIT_TIMER)) {      fprintf(stderr, "Could not initialize SDL - %s\n", SDL_GetError());      exit(1);    }    SDL_Init() essentially tells the library what features we're going to use.SDL_GetError(), of course, is a handy debugging function. ### Creating a DisplayNow we need a place on the screen to put stuff. The basic area for displayingimages with SDL is called a **surface**:             SDL_Surface *screen;        screen = SDL_SetVideoMode(pCodecCtx->width, pCodecCtx->height, 0, 0);    if(!screen) {      fprintf(stderr, "SDL: could not set video mode - exiting\n");      exit(1);    }    This sets up a screen with the given width and height. The next option is thebit depth of the screen - 0 is a special value that means "same as the currentdisplay". (This does not work on OS X; see source.) Now we create a YUV overlay on that screen so we can input video to it:             SDL_Overlay     *bmp;        bmp = SDL_CreateYUVOverlay(pCodecCtx->width, pCodecCtx->height,                               SDL_YV12_OVERLAY, screen);    As we said before, we are using YV12 to display the image. ### Displaying the ImageWell that was simple enough! Now we just need to display the image. Let's goall the way down to where we had our finished frame. We can get rid of allthat stuff we had for the RGB frame, and we're going to replace theSaveFrame() with our display code. To display the image, we're going to makean AVPicture struct and set its data pointers and linesize to our YUV overlay:              if(frameFinished) {        SDL_LockYUVOverlay(bmp);            AVPicture pict;        pict.data[0] = bmp->pixels[0];        pict.data[1] = bmp->pixels[2];        pict.data[2] = bmp->pixels[1];            pict.linesize[0] = bmp->pitches[0];        pict.linesize[1] = bmp->pitches[2];        pict.linesize[2] = bmp->pitches[1];            // Convert the image into YUV format that SDL uses        img_convert(&pict, PIX_FMT_YUV420P,                        (AVPicture *)pFrame, pCodecCtx->pix_fmt,                 pCodecCtx->width, pCodecCtx->height);                SDL_UnlockYUVOverlay(bmp);      }        First, we lock the overlay because we are going to be writing to it. This is agood habit to get into so you don't have problems later. The AVPicture struct,as shown before, has a data pointer that is an array of 4 pointers. Since weare dealing with YUV420P here, we only have 3 channels, and therefore only 3sets of data. Other formats might have a fourth pointer for an alpha channelor something. linesize is what it sounds like. The analogous structures in ourYUV overlay are the pixels and pitches variables. ("pitches" is the term SDLuses to refer to the width of a given line of data.) So what we do is pointthe three arrays of pict.data at our overlay, so when we write to pict, we'reactually writing into our overlay, which of course already has the necessaryspace allocated. Similarly, we get the linesize information directly from ouroverlay. We change the conversion format to PIX_FMT_YUV420P, and we useimg_convert just like before. ### Drawing the ImageBut we still need to tell SDL to actually show the data we've given it. Wealso pass this function a rectangle that says where the movie should go andwhat width and height it should be scaled to. This way, SDL does the scalingfor us, and it can be assisted by your graphics processor for faster scaling:            SDL_Rect rect;          if(frameFinished) {        /* ... code ... */        // Convert the image into YUV format that SDL uses        img_convert(&pict, PIX_FMT_YUV420P,                        (AVPicture *)pFrame, pCodecCtx->pix_fmt,                 pCodecCtx->width, pCodecCtx->height);                SDL_UnlockYUVOverlay(bmp);    	rect.x = 0;    	rect.y = 0;    	rect.w = pCodecCtx->width;    	rect.h = pCodecCtx->height;    	SDL_DisplayYUVOverlay(bmp, &rect);      }    Now our video is displayed! Let's take this time to show you another feature of SDL: its event system. SDLis set up so that when you type, or move the mouse in the SDL application, orsend it a signal, it generates an **event**. Your program then checks forthese events if it wants to handle user input. Your program can also make upevents to send the SDL event system. This is especially useful whenmultithread programming with SDL, which we'll see in Tutorial 4. In ourprogram, we're going to poll for events right after we finish processing apacket. For now, we're just going to handle the SDL_QUIT event so we can exit:            SDL_Event       event;            av_free_packet(&packet);        SDL_PollEvent(&event);        switch(event.type) {        case SDL_QUIT:          SDL_Quit();          exit(0);          break;        default:          break;        }    And there we go! Get rid of all the old cruft, and you're ready to compile. Ifyou are using Linux or a variant, the best way to compile using the SDL libsis this:             gcc -o tutorial02 tutorial02.c -lavutil -lavformat -lavcodec -lz -lm \    `sdl-config --cflags --libs`    sdl-config just prints out the proper flags for gcc to include the SDLlibraries properly. You may need to do something different to get it tocompile on your system; please check the SDL documentation for your system.Once it compiles, go ahead and run it. What happens when you run this program? The video is going crazy! In fact,we're just displaying all the video frames as fast as we can extract them fromthe movie file. We don't have any code right now for figuring out _when_ weneed to display video. Eventually (in Tutorial 5), we'll get around to syncingthe video. But first we're missing something even more important: sound! _**>>** Playing Sound_* * *Function Reference  Data Referenceemail:dranger at gmail dot comBack to dranger.comThis work is licensed under the Creative Commons Attribution-Share Alike 2.5License. To view a copy of this license, visithttp://creativecommons.org/licenses/by-sa/2.5/ or send a letter to CreativeCommons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA.    Code examples are based off of FFplay, Copyright (c) 2003 Fabrice Bellard, anda tutorial by Martin Bohme.