01-19-2013 01:42 AM
I'm trying to use libimg to load and display an image. It's 3/4 working. Specifically, the image is in RGB888 format, and the Cascades ImageData object needs RGBX, so I get a cool squished and somewhere in between mono and colour effect.
I'm trying to use img_convert_data() to convert the data, but it doesn't seem to do anything. The destination buffer I give it comes out unchanged. If I memset the buffer to 0xD0 before calling img_convert_data(), then the image is displayed as a solid light grey. img_convert_data() returns IMG_ERR_OK, so I don't even have some error to work with...
An abbreviated version of what I've got is:
img_t img;
img_format_t new_format = IMG_FMT_RGBA8888;
uint8_t *new_data;
size_t new_bpl;
int rc;
rc = img_load_file(ilib, filename, NULL, &img);
new_bpl = IMG_FMT_BPL(new_format, img.w);
new_data = (uint8_t *) malloc(new_bpl * img.h);
rc = img_convert_data(img.format, img.access.direct.data, new_format, new_data, N);
I've tried a few numbers for N ("the number of samples to convert"), since I'm not sure what's required. 1 doesn't work, img.h doesn't work, new_bpl * img.h segfaults...
What am I missing?
Solved! Go to Solution.
01-20-2013 04:31 AM
A much simpler solution -- specify the format in the img_t structure passed to img_load_file(), and if you plan on using img_rotate_ortho(), be sure the destination img_t's format is also set.
Alternately, if you plan on resizing, img_resize_fs() also takes care of the conversion well.
One oddity -- when I called img_resize_fs() and then img_rotate_ortho(), the destination img_t given to img_rotate_ortho() didn't need to have the format specified. However, if I did not resize the image, I did need to specify the destination img_t's format before giving it to img_rotate_ortho().
(This is with BB10 NDK 10.0.9.2318.)
01-21-2013 10:25 AM
Hey, I work on this image library at QNX. The issue you are having is you are passing in the number of bytes to img_convert_data() instead of the number of "samples" i.e., pixels. For example, IMG_FMT_RGBA8888 has four bytes per pixel, so you are asking it to convert four times more data than there is present.
I tried my best to reproduce your example with the following program:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
#include <img/img.h>
int main(int argc, char ** argv)
{
if (argc != 2) {
printf("Usage: %s [filename]\n", argv[0]);
printf(" e.g. %s /path/to/some/photo.jpg\n", argv[0]);
return 1;
}
// attach image library (loads codecs)
img_lib_t ilib;
int rc = img_lib_attach(&ilib);
if (rc != IMG_ERR_OK) {
printf("img_lib_attach() failed: %d\n", rc);
return 1;
}
// load image
img_t img;
memset(&img, 0, sizeof(img_t));
// I'm putting this here to make sure I already have an RGB image to convert up to RGBA later
img.format = IMG_FMT_RGB888;
img.flags |= IMG_FORMAT;
rc = img_load_file(ilib, argv[1], NULL, &img);
if (rc != IMG_ERR_OK) {
printf("img_load_file() failed: %d\n", rc);
return 1;
}
printf("img_load_file() succeeded\n");
// convert image
img_format_t new_format = IMG_FMT_RGBA8888;
size_t new_bpl = IMG_FMT_BPL(new_format, img.w);
uint8_t * new_data = malloc(new_bpl * img.h);
rc = img_convert_data(img.format, img.access.direct.data, new_format, new_data, img.h * img.w);
if (rc != IMG_ERR_OK) {
printf("img_convert_data() failed: %d\n", rc);
return 1;
}
printf("img_convert_data() succeeded\n");
return 0;
}I tested this with a PNG I have and the conversion worked successfully.
In general though, it's probably more idiomatic to specify the destination format prior to loading the image to let lib img do the conversion for you. The benefit is that the img_t will be updated with the data, as well as the correct stride, format flags, etc
Chris
01-21-2013 10:29 AM - edited 01-21-2013 10:30 AM
torpesco wrote:One oddity -- when I called img_resize_fs() and then img_rotate_ortho(), the destination img_t given to img_rotate_ortho() didn't need to have the format specified. However, if I did not resize the image, I did need to specify the destination img_t's format before giving it to img_rotate_ortho().
Can you share a short example of how this fails? img_rotate_orth() should only get mad if you have not specified the correct fields in the src image, not the dst image. I'll take a look just to make sure though, in case the doc is incorrect.
01-21-2013 10:34 AM
01-22-2013 03:50 AM - edited 01-22-2013 03:54 AM
Thanks for the clarification on the 'n' parameter. Yeah, once I figured out that the img parameter for opening a file was an in/out one, it clearly made more sense to specify the desired format there.
As for rotation, I threw some more debug into my program. Possibly a problem with IMG_FMT_PKLE_XBGR8888?
Since Cascades wants either PixelFormat::RGBX or PixelFormat::RGBA_Premultiplied, I request IMG_FMT_RGBA8888 from img_load_file() and tell Cascades it's PixelFormat::RGBX.
I see that IMG_FMT_RGBA8888 is an alias for IMG_FMT_PKLE_ABGR8888 (at least on the simulator). After loading the image (a JPEG), img.format has been set to IMG_FMT_PKLE_XBGR8888 instead.
So in the sample above, request IMG_FMT_RGBA8888. Then instead of the conversion, do a rotate:
/* sorry for any errors -- adapted from my rotation wrapper function */
img_fixed_t angle = get_image_orientation(argv[1]); /* via EXIF orientation tag */
img_t di;
int rc;
if (!angle)
goto no_rotation; /* actually a return in my function */
memset(&di, 0, sizeof di);
di.format = IMG_FMT_RGBA8888;
di.flags |= IMG_FORMAT;
printf("req f %s\n", imgfmt(di.format));
rc = img_rotate_ortho(&img, &di, angle);
if (rc != IMG_ERR_OK) {
printf("img_rotate_ortho error %d\n", rc);
return 1;
}
free(img.access.direct.data);
printf("old f %s\n", imgfmt(img.format));
img = di;
printf("new f %s\n", imgfmt(img.format));
no_rotation:
...
What I see from the debug is:
req f IMG_FMT_PKLE_ABGR8888
old f IMG_FMT_PKLE_XBGR8888
new f IMG_FMT_PKLE_ABGR8888
If I comment out the di.format and di.flags assignment lines, then img_rotate_ortho() returns 5, or IMG_ERR_NOSUPPORT.
Likely when I saw it work after a resize call, I had probably specified the desired format in the destination image, along with the dimension to scale for, and thus img.format had already been modified to IMG_FMT_PKLE_ABGR8888. (When I add debug in my resize wrapper, I see similar output to the above, and then the call to rotate shows req/old/new formats all being the same.)
So... no support for rotating IMG_FMT_PKLE_XBGR8888, and requesting IMG_FMT_RGBA8888 results in a conversion happening prior to rotation?
If that shouldn't happen in theory, maybe a version clash? I've been building using 2318 and running on simulator 1673. For whatever reason, blackberry-deploy (and thus Momentics) can't load apps onto the 2318 simulator for me.
01-22-2013 11:12 AM
torpesco wrote:I see that IMG_FMT_RGBA8888 is an alias for IMG_FMT_PKLE_ABGR8888 (at least on the simulator). After loading the image (a JPEG), img.format has been set to IMG_FMT_PKLE_XBGR8888 instead.
You are correct about the alias, this is in img.h:
#define IMG_FMT_BGRA8888 IMG_FMT_PKLE_ARGB8888 #define IMG_FMT_RGBA8888 IMG_FMT_PKLE_ABGR8888
However, we don't have any codec that outputs IMG_FMT_PKLE_XBGR8888. In fact, the jpeg decoder only decodes to IMG_FMT_PKLE_ARGB8888. To get XBGR (or any other format), you have to request that either by providing an img_decode_choose_format_f() callback or setting the format/flag right before img_load_file(). When you do that, libimg automatically detects that the decoder can't output the requested format, and performs an auto conversion for you.
Is it possible the imgfmt() function you've created is printing the wrong enum name?
In any case, I investigated whether or not there was a problem going from IMG_FMT_PKLE_XBGR8888 to
IMG_FMT_RGBA8888 and tested the four rotations on a JPEG I had like so:
int main(int argc, char ** argv) {
if (argc != 2) {
printf("Usage: %s [filename]\n", argv[0]);
printf(" e.g. %s /path/to/some/photo.jpg\n", argv[0]);
return 1;
}
// attach image library (loads codecs)
img_lib_t ilib;
int rc = img_lib_attach(&ilib);
if (rc != IMG_ERR_OK) {
printf("img_lib_attach() failed: %d\n", rc);
return 1;
}
// load image
img_t img;
memset(&img, 0, sizeof(img_t));
img.format = IMG_FMT_PKLE_XBGR8888;
img.flags |= IMG_FORMAT;
rc = img_load_file(ilib, argv[1], NULL, &img);
if (rc != IMG_ERR_OK) {
printf("img_load_file() failed: %d\n", rc);
return 1;
}
int i;
img_t di;
img_fixed_t angle[4] = {0, IMG_ANGLE_90CW, IMG_ANGLE_90CCW, IMG_ANGLE_180};
for (i = 0; i < 4; i++) {
printf("=== Test %d - angle = 0x%X\n", i, angle[i]);
printf("source image format=0x%X\n", img.format);
memset(&di, 0, sizeof di);
di.format = IMG_FMT_RGBA8888; // == IMG_FMT_PKLE_ABGR8888 on little-endian systems
di.flags |= IMG_FORMAT;
printf("destination image requested format=0x%X\n", di.format);
rc = img_rotate_ortho(&img, &di, angle[i]);
if (rc != IMG_ERR_OK) {
printf("img_rotate_ortho() failed: %d\n", rc);
return 1;
}
printf("destination image actual format=0x%X\n", di.format);
}
if (img.flags & IMG_DIRECT) {
free(di.access.direct.data);
}
if (img.flags & IMG_DIRECT) {
free(img.access.direct.data);
}
return 0;
}
The results were:
=== Test 0 - angle = 0x0 source image format=0x1001120 destination image requested format=0x1001520 destination image actual format=0x1001520 === Test 1 - angle = 0x1921F source image format=0x1001120 destination image requested format=0x1001520 destination image actual format=0x1001520 === Test 2 - angle = 0x4B65F source image format=0x1001120 destination image requested format=0x1001520 destination image actual format=0x1001520 === Test 3 - angle = 0x3243F source image format=0x1001120 destination image requested format=0x1001520 destination image actual format=0x1001520
The source format checks out as IMG_FMT_PKLE_XBGR8888, a combination of 0x20 (32 bits) | IMG_FMT_PKLE | IMG_FMT_RGB | IMG_FMT_RGB_ORDER. The destination format checks out as IMG_FMT_RGBA8888 (i.e., IMG_FMT_PKLE_ABGR8888) which is basically just (IMG_FMT_PKLE_XBGR8888 | IMG_FMT_ALPHA).
Most likely there is something else going wrong. Can you expand your example to include details of how img is created?
These are the possible reasons for getting IMG_ERR_NOSUPPORT from img_rotate_ortho():
1. angle is not one of 0, IMG_ANGLE_90CW, IMG_ANGLE_90CCW, IMG_ANGLE_180
2. You have specified a destination format different than the source format, and there is no available conversion function (common errors: the img_t has the format flag set but the format is invalid, or there is uninitialized junk in the img_t)
3. Same as 2., but there is no available copy function (this is highly unlikely unless you are using low bits-per-pixel formats)
4. Same as 2., but there is no available expansion function (applies only to palette images)
5. Trying to convert from non-palette to palette
#3,4,5 are unlikely in your case if you are working with jpeg.
PS - in the case of rotation angle = 0, it's still worth calling img_rotate_ortho() if you are going to set a destination format, because you will get the conversion for free.
01-23-2013 02:00 AM
No codec that outputs IMG_FMT_PKLE_XBGR8888? That makes the results I get very strange. The imgfmt() function was correct, but I switched back to integer display (rather than wasting space pasting that whole enum->const char * switch statement
).
My load function is:
bool App::load_photo(const QString &name, img_t &img)
{
int rc;
TRACE();
memset(&img, 0, sizeof img);
set_img_format(img);
printf("request format 0x%08x\n", img.format);
rc = img_load_file(_ilib, name.toUtf8().data(), NULL, &img);
printf("loaded format 0x%08x\n", img.format);
if (rc != IMG_ERR_OK) {
qDebug() << __FUNCTION__ << rc << name;
return false;
}
return true;
}
set_img_format() is:
static void set_img_format(img_t &img)
{
img.format = IMG_FMT_RGBA8888;
img.flags |= IMG_FORMAT;
}
The output is:
load_photo request format 0x01001520 loaded format 0x01001120
0x400 is the alpha flag, of course, and it looks like it's being dropped. It looks like this is the cause of the issues with img_resize_fs and img_rotate_ortho.
Noting that img.format is now 0x01001120: For the rotations, when I request IMG_FMT_RGBA8888 as the destination format, it works just fine. If I just memset the structure and do not request a specific format, that's when I get IMG_ERR_NOSUPPORT:
void App::rotate_photo(const QString &name, img_t &img)
{
img_fixed_t angle = get_image_orientation(name.toUtf8().data());
img_t di;
int rc;
if (!angle)
return;
TRACE();
angle = IMG_ANGLE_90CW; /* override for sake of clarity in example */
memset(&di, 0, sizeof di);
printf("image format 0x%08x\n", img.format);
/* set_img_format(di); */
printf("request format 0x%08x\n", di.format);
rc = img_rotate_ortho(&img, &di, angle);
printf("rotated format 0x%08x\n", di.format);
if (rc != IMG_ERR_OK) {
qDebug() << __FUNCTION__ << rc;
return;
}
free(img.access.direct.data);
img = di;
}
(get_image_orientation() does only return one of the pre-defined angles, but since the image I'm rotating just needs 90CW, I explicitly set that for the sake of this test.)
output:
load_photo request format 0x01001520 loaded format 0x01001120 rotate_photo image format 0x01001120 request format 0x00000000 rotated format 0x01001120 rotate_photo 5
If I uncomment the set_img_format() line, the output is:
load_photo request format 0x01001520 loaded format 0x01001120 rotate_photo image format 0x01001120 request format 0x01001520 rotated format 0x01001520
Similarly, if I add img_resize_fs() back into the mix (similar structure to the rotation) and don't give it a destination image format, I get the following:
load_photo request format 0x01001520 loaded format 0x01001120 resize_photo image format 0x01001120 request format 0x00000000 resized format 0x01001120 resize_photo 5
So it looks like these functions don't like IMG_FMT_PKLE_XBGR8888, but if a different destination format is requested, the conversion will be done first and then rotating or resizing works.
The chunk of code calling these basically looks like:
...
if (!load_photo(name, img)) {
/* handle the error */
goto could_not_load_photo;
}
// resize_photo(img);
rotate_photo(name, img);
...
01-25-2013 03:54 PM
Working offline, we've determined that there are two issues:
1. The jpeg decoder in the NDK is different than the one shipping on devices. The NDK decoder outputs RGB888 instead of an ARGB format, which caused the confusion in this discussion. We'll be fixing this soon.
2. libimg does not have full support for XBGR formats, resulting in some function calls returning IMG_ERROR_NOSUPPORT when they should not. We'll also be fixing this soon.
Workarounds for the time being:
1. If you are specifically asking for an XBGR format, request ABGR instead.
2. If you have requested ABGR and got XBGR returned, you can simply |= the IMG_FMT_ALPHA flag back into the format. The two formats are essentially identical except X means padding and A means alpha data. They both take the same amount of space.
Again, we'll have this fixed as soon as we can, so for most people this workaround will not be required.
Thanks,
Chris