Native Development

[sketch34]

I'm using bbndk-10.0.4-beta with a Dev Alpha device. I want to have a Cascades UI rendering over the top of an OpenGL ES 2.0 window. The Cascades samples web page indicates this is possible. 

To see if this would work, I've adapted the HelloForeignWindow sample so that the child window is bound to an OpenGL ES 2.0 context. Essentially I've merged the GLES20Template sample with the HelloForeignWindow sample, and adapted bbutil_init_egl() so that it calls:

screen_create_window_type(&screen_win, screen_ctx, SCREEN_CHILD_WINDOW)

Instead of screen_create_window().

All the GLES and screen_* initialisation code runs fine, and is called from createForeignWIndow() which is triggered by a UI event. Note that this means the code below will run in a different thread from the code in HelloForeignWindowApp::run() where I want to do my rendering:

bool HelloForeignWindowApp::createForeignWindow(const QString &group, const QString id, int x,
        int y, int width, int height)
{
 
    QByteArray groupArr = group.toAscii();
    QByteArray idArr = id.toAscii();
 
    // You must create a context before you create a window.
    if (screen_create_context(&mScreenCtx, SCREEN_APPLICATION_CONTEXT) != 0) {
        returnfalse;
    }
 
          // Adaptation of normal bbutil_init_egl()
    if(bbutil_init_egl_child(mScreenCtx, idArr.constData(), idArr.length(), groupArr.constData(), groupArr.length()) != 0)
    {
    return false;
    }
 
    if(initializegl() != 0)
    {
    return false;
    }
    return true;
}

The problem is when I call glClear() from inside HelloForeignWindowApp::run() I get a crash due to segmentation fault:

• Thread [10] (Suspended : Signal : SIGSEGV: Segmentation fault)
• glClear() at 0x78009ee4
• render() at helloforeignwindowapp.cpp:275 0x104110
• HelloForeignWindowApp::doNoise() at helloforeignwindowapp.cpp:306 0x1042fc
• HelloForeignWindowApp::run() at helloforeignwindowapp.cpp:88 0x103908

The segmentation fault made me suspect that OpenGL ES didn't like me initialising it in one thread and calling it in another thread. So I tried putting the initialisation and everything inside HelloForeignWindowApp::run(), which would now call HelloForeignWindowApp::initForeignWindow(). That resulted in this error:

• ApplicationPrivate::screenSupport: ERROR called from non-UI thread HelloForeignWindowApp(0xaf484)
• ApplicationPrivate::screenSupport: ERROR called from non-UI thread

from inside initForeignWindow() during the call to createForeignWindow(), although createForeignWindow() didn't actually get called before the error was raised.

If you're still with me at this point, thanks! My questions are:

• What's the proper way to integrate an OpenGL window into the HelloForeignWindow sample?
• Why does glClear() cause a segmentation fault in my first attempt?
• In my second attempt why does the "ERROR called from non-UI thread" get raised? 
• Where is the documentation I can study to understand the Cascades threading model in more detail? (e.g. how Application::exec() works exactly, how the ::run() call is threaded etc.)

Thanks a lot guys.

ps I tried to attach a zip of my project but I can't find the option. 

[smcveigh]

Regarding your second approach:

initForeignWindow() in the sample does some screen stuff AND some Cascades stuff.

If you are getting complaints about not running in a UI thread, then I recommend separating the cascades code from the screen code thread-wise.

I previously hacked up one of our internal OpenGL samples (gles1-vsync) to run in a foreign window as an example for someone else.  I never ran into this sort of problem.  The original code was a standalone opengl example.. all I did was rename it's main() to gl_main() and added some extra args and let it run in it's own thread when you hit the "start" button.

The cascades wrapper code looked something like the following.  Maybe it will help you out.

Cheers,

Sean

App::App()
{
    qDebug() << "Hello OpenGL!";

    // create our foreign window
    // Using .id() in the builder is equivalent to mViewfinderWindow->setWindowId()
    mGlWindow = ForeignWindow::create()
        .id(QString("glWindow"));
    mButton = Button::create("start");
    QObject::connect(mButton,
                     SIGNAL(clicked()), this, SLOT(onButtonClicked()));

    QObject::connect(mGlWindow,
                     SIGNAL(windowAttached(unsigned long,
                                           const QString &, const QString &)),
                     this,
                     SLOT(onWindowAttached(unsigned long,
                          const QString &,const QString &)));

    Container* container = Container::create()
        .layout(AbsoluteLayout::create())
        .add(mGlWindow)
        .add(mButton);

    Application::setScene(Page::create().content(container));
}


void App::onWindowAttached(unsigned long handle,
                           const QString &group,
                           const QString &id)
{
    qDebug() << "onWindowAttached: " << handle << ", " << group << ", " << id;
    screen_window_t window = (screen_window_t)handle;
    // make window visible and position it behind cascades
    int i = 1;
    screen_set_window_property_iv(window, SCREEN_PROPERTY_VISIBLE, &i);
    i = -1;
    screen_set_window_property_iv(window, SCREEN_PROPERTY_ZORDER, &i);
    // all good, right?
}

void App::onButtonClicked()
{
    qDebug() << "onButtonClicked";
    // only let the button be clicked once
    mButton->setEnabled(false);
    // spawn a thread to do the opengl stuff
    pthread_create(&mTid, NULL, &glThread, (void*)this);
}


void* App::glThread(void* arg)
{
    App* inst = (App*)arg;
    qDebug() << "starting main";
    int err = gl_main(0, NULL, inst->mGlWindow->windowId().toStdString().c_str(), inst->mGlWindow->windowGroup().toStdString().c_str());
    qDebug() << "gl_main() error %d " << err;
    return NULL;
}

[sketch34]

Thanks I'll try your suggestion.

Just one question, where did you put the OpenGL initialisation code like 

eglInitialize(), eglCreateWindowSurface() etc?

[smcveigh]

All of the opengl stuff in it's entirety was in gl_main() which was a standalone sample, so everything ran in that new thread except the cascades stuff.

I will see if I can get the OK to post it.. I can't imagine there would be a problem given that it has been a published sample in the QNX SDP previously -- it's just been updated to work in a libscreen window.

[sketch34]

That would be awesome thanks.

[sketch34 · Accepted solution]

Solved.

After looking at your example, I modified my second attempt and got it working. The cause of the "ERROR called from non-UI thread" was the call to ForeignWindow::mainWindowGroupId() from within the HelloForeignWindowApp::run() thread. I put that call into the contructor, stored the resulting QString as a member and then did all the OpenGL and screen initialisation in HelloForeignWindowApp::run() thread as before.

I think this is equivalent to the example your provided in terms of thread use.

For people wanting a code sample of this, see the Cascades Community Samples on the developer blog. The Macadamian community sample mixes Cascades and OpenGL rendering. 

So it seems that the constructor is executed on the "UI thread" in this instance, which is a little confusing to me since it is called from main(), and I thought a feature of Cascades was that the UI was rendered off the default main thread.

    // We complete the transaction started in the app constructor and start the client event loop here...
    return Application::exec();
} 

Can you explain this comment some more or point me to a document that goes into the threading setup for Cascades and how it decides what constitutes a "UI thread"?

[sketch34]

I've encountered another issue: Cascades seems to cause poor performance.

The OpenGL window animates smoothly until I touch the screen. But if I touch anywhere on the screen and drag my finger around, the OpenGL animation slows to a crawl (probably 5x slower). As soon as I stop dragging my finger and lift it off the screen, performance shoots back up to normal.

I tried debug and release builds, and disconnecting the USB but the slowdown occurs in all cases.

[smcveigh]

Regarding your threading question.. you might try asking in the Cascades forum if someone can explain the threading model.  Unless you spawn new threads yourself, your code should all be running in the main thread.  This includes your constructors, main(), etc.  I believe the Application::exec() call gets the Cascades rendering thread started up and then parks the main thread in its event loop.

Regarding performance during eventing, I think you should make sure that your opengl rendering is being done on its own thread, and not the main thread.  Out of curiosity -- what drives the tick on that thread?

If your sample is simple enough, you could consider emailing it to me and I could have someone take a look.

Cheers,

Sean

[sketch34]

The OpenGL rendering is all being done inside HelloForeignWindowApp::run() which is its own QThread.

It seems like the events might be being fired on the same thread, but then why would I get the "ERROR called from non-UI thread" then? That error implies that run() is *not* a UI thread, which I would presume means it shouldn't receive touch UI events that I haven't registered to listen for.

Update:

I just tried changing the z-order of the OpenGL window to put it in front of the Cascades UI. It renders much faster now than when it is behind Cascades either with or without touching the screen.

I need the OpenGL window to be high performance so it's looking like layering Cascades over the top isn't a good idea, unless there is something fundametally wrong with my setup. I've sent you a dropbox link to my code. Thanks for the help, appreciated.

Update 2:

I just stumbled upon the Cascades Community Samples hidden away on the developer blog. The Macadamian community sample mixes Cascades and OpenGL rendering, similar to what I want to do. I just downloaded and ran it on the Dev Alpha device and it suffers the exact same performance issue I describe above when touching the screen.

[smcveigh]

Could just be beta performance issues then.

Please start up a new discussion thread on the cascades forum to discuss your OpenGL + Cascades performance issue.

Cheers,

Sean