Welcome!

Welcome to the official BlackBerry Support Community Forums.

This is your resource to discuss support topics with your peers, and learn from each other.

inside custom component

Native Development

Reply
Developer
Posts: 145
Registered: ‎05-31-2013
My Device: BlackBerry Z10 Limited Edition

Re: Merging multiple MP4 H264 encoded files with camera_api and Qt

Forgot to say that discrepancy of duration value probabyl comes out of STTS. I calculate the duration by multiplying and summing up all samples in STTS.

 

A sum of sample_count(i) * sample_delta(i) where i goes from 1 to entry_count.

BlackBerry Development Advisor
Posts: 668
Registered: ‎11-29-2011
My Device: developer

Re: Merging multiple MP4 H264 encoded files with camera_api and Qt

[ Edited ]

ah, still broken on Z10 is interesting.  didn't try that.  let us know when you have a file with STSC fixed..

there could possibly be other things wrong :smileyhappy:  my colleague just hex edited the file by hand.

Developer
Posts: 145
Registered: ‎05-31-2013
My Device: BlackBerry Z10 Limited Edition

Re: Merging multiple MP4 H264 encoded files with camera_api and Qt

Fixed STSC samples are uploaded to same Dropbox folder. :smileyhappy:

Both audio AAC and audio NONE samples and concat results.

Dropbox folder with fixed STSC samples

 

Now that I have been taught how to offset them with count of STCO chunks, the math fix in code was easy.

For a begginer like me in terms of MP4, details like that # of chunks in STCO are very valuable, they are never ever documented anywhere that I have seen.

 

Other small things I noticed:
- stts entry for 1st frame after a new segment – you may want to insert some time here to space out properly the last frame of the previous segment from the first frame of the new segment, as simply copying the value ends up with a very small delta between those 2 frames that does not match the frame rate.  You would need to add the same amount of time to audio and video and to the durations.  Likely not noticeable by any users, but would be nice to fix.

Is there a way I can calculate the exact value of that delta and where exactly (which atom) should I add it?

Adding it both to video and audio duration is not a problem. :smileyhappy:

BlackBerry Development Advisor
Posts: 668
Registered: ‎11-29-2011
My Device: developer

Re: Merging multiple MP4 H264 encoded files with camera_api and Qt

Hey.. we found the other problem with your first file.  response from my colleague is...

 

I debugged it quickly on the device and quickly found from the slog2 of mm-renderer that the issue was now with 'stss' being incorrectly merged.

Segment 1: 133 frames (stsz.sampleCount), stss of 1, 31, 61, 91, 121
Segment 2: 46 frames, stss of 1, 31

Combined 179 frames, stss of 1, 31, 61, 91, 121, 151, 181

This is obviously wrong, as the last entry in stss of 181 is bigger then # of frames in the file.  This is why our device refuses to play the file.

Fixed combined stss of 1, 31, 61, 91, 121, 134, 164

You cannot blindly add 30 here in between each, but you need to read the data of the file and merged it properly.  At merge point, take 1 corresponds to 134th frame from the combined file.

Attached the fixed file with both fixes and now plays well on device and in QuickTime.

 

Let me know if you need further help.  Also, thanks Christian! :smileyhappy:

 

Cheers,

Sean

Developer
Posts: 145
Registered: ‎05-31-2013
My Device: BlackBerry Z10 Limited Edition

Re: Merging multiple MP4 H264 encoded files with camera_api and Qt

I understand the corelation of STSS and STSZ now.

Out of 10 tests, 9 are a success and video is merged and plays on device.

 

1 of 10 fails and that is only when I go nuts with small short clips recorded (which users might to - in berserk mood).

 

I uploaded that 1/10 case to Dropbox, if it isn't a trouble, please look at it. I'd like to make this as perfect as it gets. :smileyhappy:

 

Also what about that delta, can you elaborate please? :smileyhappy:

Developer
Posts: 145
Registered: ‎05-31-2013
My Device: BlackBerry Z10 Limited Edition

Re: Merging multiple MP4 H264 encoded files with camera_api and Qt

Sean,

I noticed some truly big differences between recording and rendering between STL100-1 and STL100-2.

STL100-1 is faster and more responsive. They are both on 10.2 OS.

Is there anything else I should know about modules and APIs, or should I just make the code even more lean and mean? :smileyhappy:

BlackBerry Development Advisor
Posts: 668
Registered: ‎11-29-2011
My Device: developer

Re: Merging multiple MP4 H264 encoded files with camera_api and Qt

I will pass along the latest video.

 

Yes, STL100-1 and STL100-2/3/4 (and Q10, Q5, Z30) are different hardware variants.  There is a buffer-premapping requirement when starting a video recording on the latter models, which will eat up a bit of time.  Though I would have expected this to only happen on the first recording, so there may be something else at play -- we can look into that.  Needless to say, it's not likely something you can fix on your end without a major change in your approach, so we would have to look at optimizing in the next release.

By the way, consider using camera_init_video_encoder() after you've got the viewfinder started -- it will speed up the 1st recording you do by pre-loading some libraries.

 

Some future design tips:

 

The best way to do this would ultimately be to have access to the encoder offline, so that you could cache your 10-seconds worth of uncompressed video and then concatenate it all and pass it through an encoder at the end.  This would eat up a couple seconds after you're finished, but would give zero latency in terms of starting & stopping clips.   Unfortunately, we don't have access to the encoder available as a general purpose API at this time.

 

The next best approach (arguably better in some ways) would be to use camera_start_encode() which is similar to camera_start_video(), but doesn't write to a file.  It just hands you all the MDAT frames for both audio & video.  You could then cache the frames you want to keep, and discard the rest (again, zero-latency start & stop).  You can also force the injection of a keyframe at any time using camera_force_video_keyframe(), so that you can safely start a new subclip at that frame.  The downside here is that you would need to have some code to generate the MP4 file structure, which it seems like you are 90% of the way to understanding already :smileyhappy:  Consider this for an optimized 2.0 release perhaps!

 

Cheers,

Sean

 

 

 

BlackBerry Development Advisor
Posts: 668
Registered: ‎11-29-2011
My Device: developer

Re: Merging multiple MP4 H264 encoded files with camera_api and Qt

Regarding your "delta" question:

smcveigh wrote:

Hey.. we found the other problem with your first file.  response from my colleague is...

 

Segment 1: 133 frames (stsz.sampleCount), stss of 1, 31, 61, 91, 121
Segment 2: 46 frames, stss of 1, 31

Combined 179 frames, stss of 1, 31, 61, 91, 121, 151, 181

This is obviously wrong, as the last entry in stss of 181 is bigger then # of frames in the file.  This is why our device refuses to play the file.

Fixed combined stss of 1, 31, 61, 91, 121, 134, 164

You cannot blindly add 30 here in between each, but you need to read the data of the file and merged it properly.  At merge point, take 1 corresponds to 134th frame from the combined file.


What he's saying is that it looks like the stss set you were building was somehow assuming that there were always 30 frames between entries, which is not the case unless all of your clips are multiples of 30 frames long.
The correct set should be 1,31,61,91,121,134,164 (note that between 121 & 134 there are not 30 frames).  This is because your first clip is only 133 frames long.

 

BlackBerry Development Advisor
Posts: 668
Registered: ‎11-29-2011
My Device: developer

Re: Merging multiple MP4 H264 encoded files with camera_api and Qt

or maybe I misunderstood your "delta" question.. can you please rephrase as a complete question?  :smileyhappy:  are you referring to this stts recommendation?

- stts entry for 1st frame after a new segment – you may want to insert some time here to space out properly the last frame of the previous segment from the first frame of the new segment, as simply copying the value ends up with a very small delta between those 2 frames that does not match the frame rate.  You would need to add the same amount of time to audio and video and to the durations.  Likely not noticeable by any users, but would be nice to fix.

 

 

Cheers,

Sean

Developer
Posts: 145
Registered: ‎05-31-2013
My Device: BlackBerry Z10 Limited Edition

Re: Merging multiple MP4 H264 encoded files with camera_api and Qt

[ Edited ]

Thank you for your clarification on STL differences. Yes, I already took your advice before and preload libs with _init on app start. It did make latency a lot smaller. 

Yes, I was referring to that last delta post ^. 

 

I'm gonna look into working with mdats only, I already wrote the code that writes MP4 files. I just need to see if it's viable for this release or an update. Users are already impatient. :smileyhappy: