Welcome!

Welcome to the official BlackBerry Support Community Forums.

This is your resource to discuss support topics with your peers, and learn from each other.

inside custom component

Native Development

Reply
Developer
peter9477
Posts: 6,447
Registered: ‎12-08-2010
My Carrier: none

Re: Display Capture API

[ Edited ]

Apparently this is now possible. I thought we'd been told we (lowly third-party devs) would not be given access, but it would appear that is not true.

If this app actually works (and reviews and other people claim it does) then it's now possible, at least under 10.2: https://appworld.blackberry.com/webstore/content/39679887  Edit: I've now seen the results of it working myself, so there's no question about that.

I'm not sure if this is done just with the screen_read_display() that Sean mentioned above, looping to encode each frame using the low-level media library. Or is there an entirely separate API?


Peter Hansen -- (BB10 and dev-related blog posts at http://peterhansen.ca.)
Author of White Noise and Battery Guru for BB10 and for PlayBook | Get more from your battery!
Please use plain text.
Trusted Contributor
JohnWS
Posts: 115
Registered: ‎02-08-2013
My Carrier: verizon

Re: Display Capture API

I contacted the developer of this screen recording app and apparently he knows how to capture the display because he knows somebody at BlackBerry and therefore has preferential treatment.

 

We, mere developers of BlackBerry, who don't have close contacts in BB due to spending most of our time developing apps for BlackBerry, have no way of knowing about how to do this.

 

Seems very unjust to me.

Please use plain text.
Developer
peter9477
Posts: 6,447
Registered: ‎12-08-2010
My Carrier: none

Re: Display Capture API

He doesn't just know someone there, he works there, as far as I can tell. I'm still working to discover whether it really is the case that the permissions available are not available to the rest of us.

It's fine if his internal knowledge is what led him to be able to do this. It's not so fine if any of us who may figure out the same thing would have the same app rejected.

This is an extremely useful feature for automated testing, which I've been pestering them to make available for two years... would be a shame if a single "third"-party app is the only way we can do this.

Peter Hansen -- (BB10 and dev-related blog posts at http://peterhansen.ca.)
Author of White Noise and Battery Guru for BB10 and for PlayBook | Get more from your battery!
Please use plain text.
Trusted Contributor
JohnWS
Posts: 115
Registered: ‎02-08-2013
My Carrier: verizon

Re: Display Capture API

I have tried everything I can think of to capture the device display without having a popup asking to allow or deny a screenshot request.

 

It doesn't seem possible unless you have root access.

 

I attempted a way this morning from qnx documentation using screen_read_display(), creating a screen_context with SCREEN_DISPLAY_MANAGER_CONTEXT as a parameter, but this returns -1. That's with the use_camera_desktop permission accepted on the device. It seems root access is required to create a context in this way.

 

I just expect a level playing field and after seeing the app record the device display, it's obvious we don't have a level playing field.

 

I wouldn't mind as much but I had an app which was number 1 for months on Playbook and to port this app to BB 10 I need to know how to capture the display. It's BB customers who benefit if we are allowed a level playing field, and BB customers who lose out if we aren't - it's as simple as that.

Please use plain text.
Developer
peter9477
Posts: 6,447
Registered: ‎12-08-2010
My Carrier: none

Re: Display Capture API

I've been exploring use_camera_desktop as well, but I've been informed that this is a restricted restricted permission, and apps submitted with it would be rejected anyway.

I haven't seen anything official confirming that yet. I'm working towards getting that, or a rebuttal of the claim it would be rejected.

Peter Hansen -- (BB10 and dev-related blog posts at http://peterhansen.ca.)
Author of White Noise and Battery Guru for BB10 and for PlayBook | Get more from your battery!
Please use plain text.
Trusted Contributor
JohnWS
Posts: 115
Registered: ‎02-08-2013
My Carrier: verizon

Re: Display Capture API

Well ScreenRecorder def uses the capture screen permission which I believe is use_camera_desktop and it also uses the camera permission - I wonder why the camera permission is there - a hint maybe?

 

I also noticed that the video ScreenRecorder has recorded is in line with the video names in the videos application, i.e vid_000009.mp4 is the name of file the app creates when my last video is vid_000008.mp4 so I also think that is a hint.

 

I have just done a little more investigation..

 

The app won't capture screen if you deny the permission to capture screenshot and when i do deny the permission within the app  I see an error displayed - cannot open camera error code 13.

 

Denying the camera permisison doesn't affect how the app works at all it seems.

 

I think a hack using the camera is how this is done.

Please use plain text.
BlackBerry Development Advisor
smcveigh
Posts: 651
Registered: ‎11-29-2011
My Carrier: other

Re: Display Capture API

Hi,

The API is available publicly (I designed it), however due to security concerns, it was restricted at the last minute.  We certainly should be allowing legitimate screen-capture apps to use this, but you would have to contact BlackBerry to get permission.  Talk to Dev Rel most likely to sort this out.  With new headless apps and such, we just felt that the risk of malware was high enough to take a case-by-case approach to this one.

 

If the developer in question actually works at BlackBerry, then you should also know that they are under a restrictive IP agreement to even participate in app world, so this would very likely be considered a "2nd party" or "close partner" sort of arrangement.  By all means, please point out the app to me and I can provide a bit more insight as to what process they followed to get approval, so that you can try the same route. 

 

Cheers,

Sean

Please use plain text.
BlackBerry Development Advisor
smcveigh
Posts: 651
Registered: ‎11-29-2011
My Carrier: other

Re: Display Capture API

JohnWS, what is it your are trying to accomplish in your app?

 

SCREEN_DISPLAY_MANAGER_CONTEXT is indeed root-restricted, as using it would allow you to bypass any perimeter and privacy settings, which is not good.  The non-root proxy into this context is via the desktop camera device for video recording (which imposes a bunch of privacy stuff on top), and also the prompted screenshot api, however as mentioned, security deemed that we should only dish that out on a case-by-case basis.  I am not aware of what the procedure is for requesting the capability for your app in app world, and this was raised as a concern previously.  I will have to follow-up with the appropriate people to see if they have a process in place yet.

 

FYI, the screen-shot popup for non-system apps is meant to let the user decide whether they are okay with sharing what is on the screen with the named app. Without this, any app that was granted the capability (and let's be honest -- sometime people just click through the permissions screen without even reading it or scrolling it, etc.) could capture the details of your banking, personal emails, etc.  Possibly even your password if they are sneaky enough.

One thing I would like to mention, in case it wasn't apparent, is that the app still has complete control over the timing of that screen-shot.  The image is captured before the pop-up is presented, so you won't miss whatever is on-screen when you make the function call.

 

What app did you have on PlayBook which was capable of capturing the entire desktop?  If all you need to do is capture your app's window, that has always been possible.

 

Cheers,

Sean

 

 

 

Please use plain text.
Trusted Contributor
JohnWS
Posts: 115
Registered: ‎02-08-2013
My Carrier: verizon

Re: Display Capture API

Thanks Sean, I really appreciate your response.

 

I have been investigating further, I have been able to accomplish what the screen recording app does, it uses the camera api to record a video passing CAMERA_UNIT_DESKTOP to camera_open and then  just records a video as normal. Never thought to check the camera docs once 10.2 was released!

 

I am now working on getting a jpg from a video recorded for 1 sec just to get me on the way to streaming images to another device. Not ideal but still researching this part of cascades.

 

I suspect that the security vulnerability you mention still exists using this way because the user doesn't know that the app is recording the screen as there is no popup making the user aware. Would that mean an app using this method would be denied in app world?

 

Now, regarding my app which was very popular for Playbook - it is called Phone Remote and allowed users to control their Blackberry os 5, 6 & 7 devices using their playbook or even their desktop. Even for me this is a great feature to have/use.

 

I would really like to bring this application to BlackBerry 10 and thought 10.2 would allow me to do that but then I found that nowhere is inject_events discussed since 10.2 release and until today I had no way to access the display data other than using screenshot api (which the user wouldn't be able to allow a screenshot as they are controlling the device remotely). 

 

Will I ever be able to inject events into the phone like I did with java sdk and will I ever be able to get an image of the screen without having to record a video each time? I will do anything to get this app released.

 

Sorry btw for assuming a private api was being used - it wasn't - I was wrong.

 

 

 

 

 

 

Please use plain text.
BlackBerry Development Advisor
smcveigh
Posts: 651
Registered: ‎11-29-2011
My Carrier: other

Re: Display Capture API


JohnWS wrote:

I suspect that the security vulnerability you mention still exists using this way because the user doesn't know that the app is recording the screen as there is no popup making the user aware. Would that mean an app using this method would be denied in app world?


Security in video streaming is handled differently.  There is a "sensitive information" object in the system which knows when something like a password dialog is on screen and will not permit video streaming while that condition persists.  Same for perimeter mismatches, etc.  You should be able to see this in action if you go into BBM Desktop Sharing, connect to someone and then do something that causes a password prompt to appear (eg. go into settings and try to change your password).

 

Since the screencapture service was not hooked up the same way (it is out of the loop regarding the privacy stuff), I had to resort to a user confirmation prompt.

 

 

Regarding injecting events -- I have talked with the input/graphics folks about this before, and I believe there was interest, but not sure where it currently stands as a priority.

 

Cheers,

Sean

 

 

Please use plain text.