Native Development

[cjonesy]

thanks kboone, you were spot on. I moved the thread creation a couple of lines earlier and my network code and screen push both happen! 

Now to see the best way to put the video frames from ffmpeg on the qt screen!

[cjonesy]

I am now facing exactly what kboone mentioned - ui thread and other threads.

I have my code working perfectly using pthreads from earlier c project and I'm desperate to keep it that way.

I'm now seeing QThreads and slots and signals are maybe going to be the only way to get the class to update the ui which

main.cpp creates

and extends QMainWindow

and contains the thread to decode the ffmpeg frames (using pthread).

Am I wrong? Is there no equivalent to java where we grab the ui thread by another method within my mywindow.cpp? I ask this as I have just spent hours trying to get some code going to put the video frames on the qt display which is mywindow.show(). I'll leave some code in case anyway can help me. Still trying to grasp qt.

if(got_frame)
        {
        	QByteArray bytes;
        	int imgWidth = 1024;
        	int imgHeight = 600;

        	bytes.resize(1024 * 600 * 4);
        	yuv420_rgb(decodedFrame, bytes.data(), QRect(0, 0, 1024, 600));
        	QImage image = QImage(reinterpret_cast<const uchar*>(bytes.constData()), 1024, 600, QImage::Format_RGB32);

        	    paintScreen(image);
        	    //painter.drawImage(QRect(10, 37, width, height), image);
        }

//////////

void paintScreen(QImage img)
{
	QLabel myLabel;
	myLabel.setText("test");//this doesn't show anything
	    //myLabel.setPixmap(QPixmap::fromImage(img));//neither does this but no errors
	    myLabel.setGeometry(0,0,1024,600);

}

[kboone]

My gut feeling -- and that's all it is -- is that you'd have more success letting your ffmpeg decoder draw directly on the native screen, over the top of whatever interface Qt provides. Qt would just leave a space (maybe the whole screen) for the native code to write to. You could do that in a separate thread, and Qt wouldn't care about it -- except to the extent that you'd need to control the decoder thread to some extent. You'd also need Qt to tell the native thread how big a screen area it has to paint in, and where it is.

The other approach, which I think you're trying, is to have ffmpeg decode into a memory block, and somehow signal the UI thread that there is data to display. You could do this by using the slot-signal approach, which should be thread-safe, so long as all you're passing across the slot is primitive data types (or just nothing at all). My gut feeling is that passing a buffer full of decoded image data across a slot is asking for trouble, but I don't know, because I've never tried.

My worry about this approach is that I'm not sure that Qt image-handling functions will be fast enough to keep pace with the decoder. They certainly weren't when I tried to do this on the Archos, although that was five years ago, and I guess hardware has moved on a bit since then.

On the Archos I had Qt draw its output into one screen layer, and the video decoder into a different one. The video hardware composited the layers into one display. As a result, the UI and the decoder were almost independent. From a comment that somebody posted I got the impression that the BB screen has this layering/compositing capacity, but I'm afraid I don't have the faintest idea how to use it :/

[mreed]

Why are you trying to make a new image for every frame? Does the foreign window not work for you on playbook?

[cjonesy]

mreed just to quickly reply, I could find no reference/sample other than the camera sample this morning when thinking how to do what i wanted - view the frames using the qt/screen.h window approach.

I therefore went with the way I could figure out myself without having to ask again. That was a result from searching but if I could figure out the other way then brilliant. I just couldnt find what was the relvent part of code in cam sample to link screen.h and QtMainWindow.

[cjonesy]

btw I still have all the screen code in the class ready to integrate with qt (always hoping this is the solution after being told no and then maybe yes).

How do I learn how to implement foreign window for myself?

p.s just rememebered that when looking through cam sample it was bb 10/cascades so that threw me a bit more.

p.p.s kboone, thanks for your reply. I will read over a few times, I would really like to implement the screen.h approach as that certainly is fast enough to keep up and I did think qt would be ideal for just buttons, textboxes, etc.

[mreed]

The libffbb is using screen.h (libscreen). It shouldn't be Cascades specific if Playbook has libscreen too.

screen_context_t screen_context;
screen_window_t screen_window;
screen_buffer_t screen_buffer[1];
screen_buffer_t screen_pixel_buffer;

In the cpp look at:

ffdec_error ffdec_context::create_view(QString group, QString id, screen_window_t *window);
void ffdec_context::display_frame(AVFrame *frame);

https://github.com/hardisonbrewing/libffbb/blob/master/public/ffbbdec.h

https://github.com/hardisonbrewing/libffbb/blob/master/src/ffbbdec.cpp

I adapted this from the HelloForeignWindow sample... which is under Cascades-Samples... but not all of it requires Cascades.

https://github.com/blackberry/Cascades-Samples/tree/master/helloforeignwindow

[cjonesy]

mreed thanks for your reply,  im sorry this is such a tough task to learn, when I actually created my code from libffbb and included the ffmpeg files, I rememeber I tried a few times to understand the create_view trail but I eventually just pasted the code, commented the part with group name out and the rest of the code was ok as it was. I was just happy to see the frames back then!

So trying to understand it again for a new reason (qt) with a bit more knowledge im still struggling, I need to understand the concept a little more and whats going on with the 2 screens, then ill prob figure out the code fully. That would be an acheivement.

I see ffcamerasampleapp calls create_view passing a fresh new screen_window_t which is then initialised as a child - i presume this is the foreign window. There is no initialising of a parent screen so im thinking is it grabbed via a slot?

maybe this is how i get my parent screen (and maybe even the group names io should bind to??)?..

QObject::connect(mViewfinderWindow, SIGNAL(windowAttached(unsigned long,
const QString &, const QString &)), this, SLOT(onWindowAttached(unsigned long,
const QString &,const QString &)));

Right now, i know my code is showing a red qt window and the ffmpeg thread is correctly decoding the video frames but i just cannot see the video. 

I initialise the screen for the video using the following..

void initConnectionScreen()
{
cout << "init connection screen";
cout.flush();
//screen_create_window(&connection_screen_window, screen_cxt);
screen_create_window_type(&connection_screen_window, screen_cxt, SCREEN_CHILD_WINDOW);

screen_join_window_group(connection_screen_window, window_group_name);
screen_set_window_property_cv(connection_screen_window, SCREEN_PROPERTY_ID_STRING, sizeof(window_group_name), window_group_name);
if (screen_create_window_group(connection_screen_window, window_group_name) != 0) {
cout << "windowgroup problem";
cout.flush();
}
int usage = SCREEN_USAGE_NATIVE;
screen_set_window_property_iv(connection_screen_window, SCREEN_PROPERTY_USAGE, &usage);

int video_size[] = { 1024,600 };
screen_set_window_property_iv(connection_screen_window, SCREEN_PROPERTY_BUFFER_SIZE, video_size);
screen_set_window_property_iv(connection_screen_window, SCREEN_PROPERTY_SOURCE_SIZE, video_size);
cout << "init connection screen";
cout.flush();
int z = 10;
screen_set_window_property_iv(connection_screen_window, SCREEN_PROPERTY_VISIBLE, &z);

int pos[] = { 0, 0 };
screen_set_window_property_iv(connection_screen_window, SCREEN_PROPERTY_POSITION, pos);

screen_create_window_buffers(connection_screen_window, 1);

screen_pixmap_t screen_pix;
screen_create_pixmap(&screen_pix, screen_cxt);

int usage2 = SCREEN_USAGE_WRITE | SCREEN_USAGE_NATIVE;
int format2 = SCREEN_FORMAT_YUV420;;
screen_set_pixmap_property_iv(screen_pix, SCREEN_PROPERTY_USAGE, &usage2);

//int format2 = SCREEN_FORMAT_YUV420;
screen_set_pixmap_property_iv(screen_pix, SCREEN_PROPERTY_FORMAT, &format2);

screen_set_pixmap_property_iv(screen_pix, SCREEN_PROPERTY_BUFFER_SIZE, video_size);

screen_create_pixmap_buffer(screen_pix);

screen_get_pixmap_property_pv(screen_pix, SCREEN_PROPERTY_RENDER_BUFFERS, (void**) &connection_screen_pixel_buffer);

screen_get_buffer_property_iv(connection_screen_pixel_buffer, SCREEN_PROPERTY_STRIDE, &stride);

}

[mreed]

Yeah, its not really a child in the usual sense. You can set the size and position through the window properties, and then it just draws it there.

Are you using YUV420 for your video frames, or are you still converting to RGB? Your window pixel format needs to match what ur passing in, if its not.

[cjonesy]

Yes, I am passing YUV420 to my encoder when creating the video.

Somehow I must have to make sure the window group name/id matches what the window group name is for the window created by qt - thats got to be their connection but I don't know how to set/get anything to do with these properties in qt windows. Thats what im not getting the connection.