Welcome!

Welcome to the official BlackBerry Support Community Forums.

This is your resource to discuss support topics with your peers, and learn from each other.

inside custom component

Native Development

Reply
Developer
Posts: 1,635
Registered: ‎07-14-2008
My Device: Z10
My Carrier: Fido

Re: Display Capture API


JohnWS wrote:

Thanks Sean, I really appreciate your response.

 

I have been investigating further, I have been able to accomplish what the screen recording app does, it uses the camera api to record a video passing CAMERA_UNIT_DESKTOP to camera_open and then  just records a video as normal. Never thought to check the camera docs once 10.2 was released!

 


 


JohnWS:  is there a special permission needed to use CAMERA_UNIT_DESKTOP? I tried to open the camera with camera_open() and getting error.  thanks

Highlighted
Developer
Posts: 121
Registered: ‎02-08-2013
My Device: 9900
My Carrier: verizon

Re: Display Capture API

The permission needed is use_camera_desktop.

 

I think you have to use video for this, im using camera_start_video_viewfinder after camera_open and converting the nv12 frames to QImages in my callback.

 

Hope that helps.

BlackBerry Development Advisor
Posts: 683
Registered: ‎11-29-2011
My Device: PRIV
My Carrier: Rogers

Re: Display Capture API

FYI, I think you can get RGB out of this as well.  If you query camera_get_videovf_frame_types() it will probably tell you as much.

I believe the colour channels are in reverse order though (gpu-native), so they will come through as BGR, which will still need some twiddling in your QImage code.

Developer
Posts: 121
Registered: ‎02-08-2013
My Device: 9900
My Carrier: verizon

Re: Display Capture API

Thanks Sean,

 

I am working close to my limits here and the code I was using to convert nv12 frame to QImage wasn't my own, I've searched and can find no equivalent code to convert rgb8888. I don't actually need a QImage - I just want to send the bytes to the client socket but I was just converting to QImage for simplicity.

 

Do you know of any equivalent code to get a byte array for rgb8888?

 

Here's the code I am using but now crashes since changing to rgb8888.

 

 

 

static QImage convertFrameToImage(camera_buffer_t *buffer, BluetoothWorker* ob)
{
    if (buffer->frametype != CAMERA_FRAMETYPE_RGB8888)
    {
    	qDebug() << "convert frame image not rgb8888";
        return QImage();

    }
    if (buffer->framemetasize == 65536) return QImage();
	qDebug() << "convert frame is rgb8888";
   const unsigned int width = buffer->framedesc.rgb8888.width;
    const unsigned int height = buffer->framedesc.rgb8888.height;

  
    const unsigned long long size = width*height + width*height/2;
    unsigned char *data = new unsigned char[size];

    unsigned char *source = buffer->framebuf;
    unsigned char *dest = data;
    for (uint row = 0; row < height; ++row) {
        memcpy(dest, source, width);
        source += buffer->framedesc.rgb8888.stride;
        dest += width;
    }

    source = buffer->framebuf;// + buffer->framedesc.rgb8888.offset;
   for (uint row = 0; row < height/2; ++row) {
        memcpy(dest, source, width);
        source += buffer->framedesc.rgb8888.stride;
        dest += width;
    }

    QImage image(width, height, QImage::Format_ARGB32);

    unsigned char *dataPtr = data;
    unsigned char *uvDataPtr = 0;
    unsigned int uv_base_offset = width*height;
    int yValue = 0;
    int uValue = 0;
    int vValue = 0;
    int bValue = 0;
    int gValue = 0;
    int rValue = 0;
    unsigned char *rowStart = 0;

    unsigned char *imageDest = 0;
    for (unsigned int y = 0; y < height; ++y) {
        imageDest = const_cast<unsigned char*>(image.constScanLine(y));
        rowStart = data + (uv_base_offset + (width*qFloor(y/2.0)));
        for (unsigned int x = 0; x < width; ++x) {
          uvDataPtr = rowStart + (qFloor(x/2)*2);

            yValue = ((*dataPtr++) - 16) * 1.164;
            uValue = ((*uvDataPtr++) - 128);
            vValue = ((*uvDataPtr) - 128);

            bValue = yValue + 2.018 * uValue;
            gValue = yValue - 0.813 * vValue - 0.391 * uValue;
            rValue = yValue + 1.596 * vValue;

            *imageDest = qBound(0, bValue, 255);
           imageDest++;
           *imageDest = qBound(0, gValue, 255);
            imageDest++;
            *imageDest = qBound(0, rValue, 255);
            imageDest++;
            *imageDest = 255;
            imageDest++;
        }
    }

    delete [] data;
    camera_stop_video_viewfinder(ob->mCameraHandle);
    return image;
}

 

BlackBerry Development Advisor
Posts: 683
Registered: ‎11-29-2011
My Device: PRIV
My Carrier: Rogers

Re: Display Capture API

hmm, let's see.

the rgb8888 image format is just width * height * 4 bytes of picture data, with stride bytes per line.

You still seem to have a lot of weird nv-12 and yuv code in there.

 

walking the pixels is more or less:

for (int y=0; y<height; y++) {
    uint8_t* src=&buf->framebuf[y * stride];
    for (int x=0; x<width; x++) {
         uint8_t b = src[0];
         uint8_t g = src[1];
         uint8_t r = src[2];
         src+=4;
    }
}

I *might* have B and R transposed, but I'm pretty sure that this is the ordering.

 

Also, I don't think there is any reason for you to have to do a memcpy() to duplicate the buffer before processing since it doesn't seem like you have any need to de-stride the data.

 

You can probably speed things up a bit my working 32 or 64 bits at a time and doing some bit-twiddling to do the channel transpose...

 

eg.

uint64_t bgr64 = *(uint64_t*)src;
uint64_t rgb64 = bgr64 & 0x00ff000000ff0000;
rgb64 |= (bgr64 & 0xff000000ff000000) >> 16;
rgb64 |= (bgr64 & 0x0000ff000000ff00) << 16;

This would process 2 pixels at a time and swap from BGRA to RGBA.  I leave the details of shoving that into a QImage up to you Smiley Happy

 

Cheers,

Sean

 

Developer
Posts: 121
Registered: ‎02-08-2013
My Device: 9900
My Carrier: verizon

Re: Display Capture API

Thanks so much.

 

Developer
Posts: 71
Registered: ‎02-06-2013
My Device: 9900
My Carrier: Rogers

Re: Display Capture API

Would I be right in assuming that this can't be accomplished in pure QML?

 

'qmlCameraObj.open(CameraUnit.Desktop)' doesn't work.

The only valid members for CameraUnit in QML are Front, Rear and none.

BlackBerry Development Advisor
Posts: 683
Registered: ‎11-29-2011
My Device: PRIV
My Carrier: Rogers

Re: Display Capture API

That's correct.

There is no QML interface for this presently.