Welcome!

Welcome to the official BlackBerry Support Community Forums.

This is your resource to discuss support topics with your peers, and learn from each other.

inside custom component

Native Development

Reply
Developer
Posts: 1,041
Registered: ‎07-16-2008
My Device: ಠ_ಠ

Re: Options for displaying video

[ Edited ]

No idea what sdl is.

 

You have to draw the frames manually. Look in ffbbdec.cpp at the bottom:

https://github.com/hardisonbrewing/libffbb/blob/master/src/ffbbdec.cpp

 

I'm using mpeg2 with yuv frames. I don't know what you'll need, but its probably similar enough.

BlackBerry Development Advisor
Posts: 683
Registered: ‎11-29-2011
My Device: PRIV
My Carrier: Rogers

Re: Options for displaying video

will mm-renderer not play the media files you want to play?

Developer
Posts: 1,041
Registered: ‎07-16-2008
My Device: ಠ_ಠ

Re: Options for displaying video

[ Edited ]

AFAIK he's still doing a live stream over sockets. mmrenderererer doesn't take in a raw data.

 

But maybe I missed something in the convo.

 

@surashid mentioned something about a loopback url... no idea what that is though:

https://twitter.com/surashid/status/243694482787291138

BlackBerry Development Advisor
Posts: 683
Registered: ‎11-29-2011
My Device: PRIV
My Carrier: Rogers

Re: Options for displaying video

ah ok.. missed that it was the same guy Smiley Happy  thought this was someone new jumping on an old thread.

Trusted Contributor
Posts: 160
Registered: ‎09-13-2012
My Device: 9900
My Carrier: vodafone

Re: Options for displaying video

Thanks guys for your answers, right now as mentioned i would just like to see a video being played using ffmpeg/ndk as that would be an acheivement after the struggles of the last week. Then I will look more into feeding bytes - fifo - other options to enhance performance. I would have a bit more understanding after this exercise also.

 

I have read the ffbbdec.cpp code mreed, I am being thrown a little with references to ffd_context as it seems like a class/object in itself but can't seem to locate it in src.

 

My lack of understanding right now can be boiled down to 2 questions - what are the main steps when using ffmpeg once it has the video in memory - I see references to avframes and avpacket but don't quite fuly understand what the process is.

 

Also once we have the data - if i am using screen how is the data put from say avpacket to the screen - what is the key function?

 

I'm sorry if these are simple questions, im still learning basics of c whilst taking on advanced challenges.

Trusted Contributor
Posts: 160
Registered: ‎09-13-2012
My Device: 9900
My Carrier: vodafone

Re: Options for displaying video

Also, im thought i would be ok with the following to learn ffmpeg with c using http://dranger.com/ffmpeg/ffmpegtutorial_all.html but the av_open_input_file and some other functions are not available in this version of c/ffmpeg. Is there a better guide that anyones knows of ?

BlackBerry Development Advisor
Posts: 683
Registered: ‎11-29-2011
My Device: PRIV
My Carrier: Rogers

Re: Options for displaying video

Does ffmpeg allows you to define your own buffer allocator function.  If so, you can create a screen window with n buffers (say 2 or 3) and then get the pointers to those buffers, the width, height and stride, and fill in an appropriate AVFrame descriptor for ffmpeg to use as output.

When ffmpeg spits out one of those frames, you then just call screen_post_window() to make that buffer visible.

 

I suggest taking a browse through the NDK-Samples repository... specifically this one seems to have most of the set-up you would need:

https://github.com/blackberry/NDK-Samples/tree/master/ScreenTemplate

 

The important part for performance will be to make sure that your screen window is created with the same colour-space that ffmpeg will be outputting (eg. ARGB8888, NV12, etc.)

 

Cheers,

Sean

Trusted Contributor
Posts: 160
Registered: ‎09-13-2012
My Device: 9900
My Carrier: vodafone

Re: Options for displaying video

I have been trying to implement what has been suggested since last night, I have overcome many errors such as removing audio from video stream (for now) to remove bad headers error, av_find_stream_info(&pFormatCtx) never returned so i have set codec manually, it is now reading frames continously from stream but i can't see them! 

 

I would really appreciate someone looking at my code (only for testing currently) to see if im missing something obvious in this setup.

 

Thanks for anything you can suggest.

 

int main(int argc, char *argv[]) {
    int rc;
    int exit_application = 0;

  
    screen_context_t screen_cxt;
        screen_window_t screen_window;
        screen_buffer_t screen_buffer[1];
        screen_buffer_t screen_pixel_buffer;
    av_register_all();
     // Couldn't find stream information
    //Create a screen context that will be used to create an EGL surface to to receive libscreen events
    screen_create_context(&screen_cxt, 0);
    printf("after av registerall\n");
    //Initialize BPS library
    bps_initialize();

    //Use utility code to initialize EGL for rendering with GL ES 1.1
    if (EXIT_SUCCESS != bbutil_init_egl(screen_cxt)) {
        fprintf(stderr, "bbutil_init_egl failed\n");
        bbutil_terminate();
        screen_destroy_context(screen_cxt);
        return 0;
    }
  if (EXIT_SUCCESS != initialize()) {
        fprintf(stderr, "initialize failed\n");
        bbutil_terminate();
        screen_destroy_context(screen_cxt);
        return 0;
    }

    //Signal BPS library that navigator and screen events will be requested
    if (BPS_SUCCESS != screen_request_events(screen_cxt)) {
        fprintf(stderr, "screen_request_events failed\n");
        bbutil_terminate();
        screen_destroy_context(screen_cxt);
        return 0;
    }

    if (BPS_SUCCESS != navigator_request_events(0)) {
        fprintf(stderr, "navigator_request_events failed\n");
        bbutil_terminate();
        screen_destroy_context(screen_cxt);
        return 0;
    }

    //Signal BPS library that navigator orientation is not to be locked
    if (BPS_SUCCESS != navigator_rotation_lock(false)) {
        fprintf(stderr, "navigator_rotation_lock failed\n");
        bbutil_terminate();
        screen_destroy_context(screen_cxt);
        return 0;
    }

    while (!exit_application) {
        //Request and process all available BPS events
        bps_event_t *event = NULL;

        for(;;) {
            rc = bps_get_event(&event, 0);
            assert(rc == BPS_SUCCESS);

            if (event) {
                int domain = bps_event_get_domain(event);

                if (domain == screen_get_domain()) {
                    handleScreenEvent(event);
                } else if ((domain == navigator_get_domain())
                        && (NAVIGATOR_EXIT == bps_event_get_code(event))) {
                    exit_application = 1;
                }
            } else {
                break;
            }
        }
        render();
        avformat_network_init();
          AVFormatContext* pFormatCtx = avformat_alloc_context();
          avformat_open_input(&pFormatCtx, filename, NULL, NULL);
         // int64_t duration = pFormatCtx->duration;
          // etc

         // screen_window_t screen_window;
             screen_create_window_type(&screen_window, screen_cxt, SCREEN_CHILD_WINDOW);
            // screen_join_window_group(screen_window, groupArr.constData());
             //screen_set_window_property_cv(screen_window, SCREEN_PROPERTY_ID_STRING, idArr.length(), idArr.constData());

             int usage = SCREEN_USAGE_NATIVE;
             screen_set_window_property_iv(screen_window, SCREEN_PROPERTY_USAGE, &usage);

             int video_size[] = { 900,500 };
             screen_set_window_property_iv(screen_window, SCREEN_PROPERTY_BUFFER_SIZE, video_size);
             screen_set_window_property_iv(screen_window, SCREEN_PROPERTY_SOURCE_SIZE, video_size);

             int z = -1;
             screen_set_window_property_iv(screen_window, SCREEN_PROPERTY_ZORDER, &z);

             int pos[] = { 0, 0 };
             screen_set_window_property_iv(screen_window, SCREEN_PROPERTY_POSITION, pos);

             screen_create_window_buffers(screen_window, 1);

             screen_pixmap_t screen_pix;
             screen_create_pixmap(&screen_pix, screen_cxt);

             usage = SCREEN_USAGE_WRITE | SCREEN_USAGE_NATIVE;
             screen_set_pixmap_property_iv(screen_pix, SCREEN_PROPERTY_USAGE, &usage);

             int format = SCREEN_FORMAT_YUV420;
             screen_set_pixmap_property_iv(screen_pix, SCREEN_PROPERTY_FORMAT, &format);

             screen_set_pixmap_property_iv(screen_pix, SCREEN_PROPERTY_BUFFER_SIZE, video_size);

             screen_create_pixmap_buffer(screen_pix);

             screen_buffer_t screen_pixel_buffer;
             screen_get_pixmap_property_pv(screen_pix, SCREEN_PROPERTY_RENDER_BUFFERS, (void**) &screen_pixel_buffer);

             int stride;
             screen_get_buffer_property_iv(screen_pixel_buffer, SCREEN_PROPERTY_STRIDE, &stride);
             AVFrame *frame = avcodec_alloc_frame();
             AVFrame *pFrameRGB = avcodec_alloc_frame();
             AVPacket packet;
             AVCodec *codec;
             uint8_t *buffer;
               int numBytes = avpicture_get_size(PIX_FMT_YUV420P, 900, 500);
               buffer = (uint8_t *) av_malloc(numBytes*sizeof(uint8_t));


               //assign appropriate parts of buffer to image panes in pFrameRGB
             avpicture_fill((AVPicture*) pFrameRGB, buffer, PIX_FMT_YUV420P, 900,500);

             av_init_packet(&packet);
             int got_frame = 0;

             codec = avcodec_find_decoder(CODEC_ID_FLV1);
             if (!codec) {
                 fprintf(stderr, "codec not found\n");
                 exit(1);
                }
             AVCodecContext *c = avcodec_alloc_context3(codec);


             if(codec->capabilities&CODEC_CAP_TRUNCATED)
                 c->flags|= CODEC_FLAG_TRUNCATED; /* we do not send complete frames */


             if (avcodec_open(c, codec) < 0) {
               fprintf(stderr, "could not open codec\n");
                exit(1);
                }

             while(av_read_frame(pFormatCtx, &packet) >= 0)
               {
            	 printf("before decode \n");
            	 fflush( stdout );
            	 int got_frame = 0;
             int decode_result = avcodec_decode_video2(c, frame, &got_frame, &packet);
             printf("after decode \n");
                         fflush( stdout );
                        if (decode_result < 0)
                        {
                            fprintf(stderr, "Error while decoding video\n");


                        }

                        if(got_frame)
                        {
                        	printf("got frame \n");
                        	fflush( stdout );
                        uint8_t *srcy = frame->data[0];
                            uint8_t *srcu = frame->data[1];
                            uint8_t *srcv = frame->data[2];
                            unsigned char *ptr = NULL;
                               screen_get_buffer_property_pv(screen_pixel_buffer, SCREEN_PROPERTY_POINTER, (void**) &ptr);
                            unsigned char *y = ptr;
                            unsigned char *u = y + (500 * stride);
                            unsigned char *v = u + (500 * stride) / 4;
                            int i = 0;
                            for ( i = 0; i < 500; i++)
                            {
                                int doff = i * stride;
                                int soff = i * frame->linesize[0];
                                memcpy(&y[doff], &srcy[soff], 900);
                            }

                            for ( i = 0; i < 500 / 2; i++)
                            {
                                int doff = i * stride / 2;
                                int soff = i * frame->linesize[1];
                                memcpy(&u[doff], &srcu[soff], 900 / 2);
                            }

                            for ( i = 0; i < 500 / 2; i++)
                            {
                                int doff = i * stride / 2;
                                int soff = i * frame->linesize[2];
                                memcpy(&v[doff], &srcv[soff], 900 / 2);
                            }
                            printf("before posttoscreen \n");
                            fflush( stdout );
                            screen_buffer_t screen_buffer;
                            screen_get_window_property_pv(screen_window, SCREEN_PROPERTY_RENDER_BUFFERS, (void**) &screen_buffer);

                            int attribs[] = { SCREEN_BLIT_SOURCE_WIDTH, 900, SCREEN_BLIT_SOURCE_HEIGHT, 500, SCREEN_BLIT_END };
                            screen_blit(screen_cxt, screen_buffer, screen_pixel_buffer, attribs);

                            int dirty_rects[] = { 0, 0, 900, 500 };
                            screen_post_window(screen_window, screen_buffer, 1, dirty_rects, 0);
                            printf("after posttoscreen  \n");
                            fflush( stdout );
                            av_free_packet(&packet);
                           // av_init_packet(&packet);
                            }

                           printf("after freepacket  \n");
                           fflush( stdout );
                           //av_init_packet(&packet);
               }

             //avformat_free_context(pFormatCtx);//Initialize application logic

    }

    //Stop requesting events from libscreen
    screen_stop_events(screen_cxt);

    //Shut down BPS library for this process
    bps_shutdown();

    //Use utility code to terminate EGL setup
    bbutil_terminate();

    //Destroy libscreen context
    screen_destroy_context(screen_cxt);
    return 0;
}

 

 

 

Developer
Posts: 1,041
Registered: ‎07-16-2008
My Device: ಠ_ಠ

Re: Options for displaying video

[ Edited ]

You're mixing a whole bunch of code into one method... and a) thats confusing, b) are you sure you're not blocking the ui thread, or that there even is a ui thread? You have all the bps/render stuff further up in the method, but thats not going to get hit once you start looping on the frames. I haven't seen this type of ndk sample, which one did it come from?

Trusted Contributor
Posts: 160
Registered: ‎09-13-2012
My Device: 9900
My Carrier: vodafone

Re: Options for displaying video

Thanks mreed, im sorry about the confusing code, i will start a new project as soon as i understand what im doing, I was worried if i started editing the code before posting it I wouldnt show the problem properly.

 

The application was first created using egl sample i think (a few days ago now!), it just creates a screen with a rotating square. I try to add the video frames on top for now. Just grasping concepts still.

 

I could be blocking ui, not really researched that yet in c, i did see reference to pthread in camera sample for decode function - is that what i should be doing to show the frames - spawning a new thread?

 

I understand about the bps stuff at the top - i was thinking i could even just remove it and it would'nt make a difference until i start wanting to receive screen events.