Welcome!

Welcome to the official BlackBerry Support Community Forums.

This is your resource to discuss support topics with your peers, and learn from each other.

inside custom component

Java Development

Reply
Developer
dx22
Posts: 402
Registered: ‎11-26-2010
My Device: Torch

Re: RIMM streaming file format

@floydian it seems that we are on the same page with you :smileyhappy:

I suppose you are using custom implementation of Datasource with some sort of buffer where you store the incoming data right? Can you explain what error exactly are you getting ?
I did a voip app for audio calls only and in my experience when the format of the data is not what the player expects you end up with MediaException with message similar to 'media unloaded while initializing'. Is this what you are getting?
----------------------------------------------------------------------
Press the button to give kudos if I helped you :smileyhappy:
Please use plain text.
Contributor
floydian
Posts: 16
Registered: ‎10-12-2009
My Device: Not Specified

Re: RIMM streaming file format

[ Edited ]

Yep, I am talking about this error... :-)

 

@dx22, look at the first post from this discussion. "aloka" proposed one interesting workaround in the second point (creating small files from RTP packets). In the past, I tried a similar  solution with audio stream and that glitch, "aloka" writes about, was really occurring . I imagine, this problem might be really disturbing during video playback...

 

Maybe, it's going to be a satisfactory solution for your needs though... good luck :smileyhappy: 

Please use plain text.
Developer
dx22
Posts: 402
Registered: ‎11-26-2010
My Device: Torch

Re: RIMM streaming file format

Hi again,

 

I found this code here really useful:

 

http://docs.blackberry.com/en/developers/deliverables/11942/CS_Parsing_a_RIM_proprietary_video_file_...

 

Using it I managed to get a better look on the rimm container and I have the following questions:

 

1) I see there are key and config frames. How important are they? I mean if I am going to transfer them through UDP some of them might be lost and I am wondering how this might affect the playback? Will it continue smoothly for the next frames which will follow?

 

2) The same goes for the video /audio frames - if some of them are lost during network transport will this cause the player to crash? I also noticed that the size of different video frames is not the same but I guess this has more to do with the h.264 codec than the container.

----------------------------------------------------------------------
Press the button to give kudos if I helped you :smileyhappy:
Please use plain text.
BlackBerry Development Advisor (Retired)
BlackBerry Development Advisor (Retired)
BVP
Posts: 150
Registered: ‎10-19-2010
My Device: Not Specified
My Carrier: Rogers

Re: RIMM streaming file format

I am not sure how familair you are with MPEG-encoded video, but a short explanation answers most of your questions:

 

Keyframes indicate that the video frame is an I-Frame.  As an over-simplified explanation of what an I-Frame is, it is basically a wholly-defined frame of video.  Every frame of video that is not an I-Frame (key frame) is either a P-Frame or a B-frame, which in simple terms means it represents a part of a video frame.  To achieve high compression, not all video frames are stored as whole frames but instead as ‘the differences since the last or next whole video frame’.  These differences are the B- and P-frames I mentioned. 

 

I-Frames are usually inserted periodically as reference points, and to help with seeking.  For example, without them, when seeking to say frame X, that frame of video would need to be built from the last known I-Frame with all B- or P-Frames applied to it.  Adding more iFrames shortens this process (at the cost of size) by needing fewer B/P-Frame transformations to rebuild a future frame of video. 

 

This is also why the frame sizes you see are different - storing the ‘changes’ from the previous or next I-Frame requires much less space than storing an entire frame.  Things like lots of movement in the video or sudden changes in scenery increase the size of B- and P-Frames since the change from frame to frame is large requireing more data to store it, whereas little movement or change means each frame is similar to the next and requires less room to store.  Here is a good overview of MPEG video that talks more about this stuff if you're interested.

 

Finally, how much dropping frames will impact the resulting video will likely depend on how fault-tolerant the decoder is, among other things.  If you've ever watched an online video and noticed a time where the image looks a bit distorted (especially on a seek) and blurry then suddenly jumps to a clear image, this is likely similar to the effect you'd see - frames of video being built by applying P-Frames to the wrong I-Frame at the start and later when an I-Frame appears the video corrects itself suddenly as there is a new valid reference point from which you can reliably build video frames from.

 

P.S. Sorry if that was much longer and more technical than you were looking for!

Please use plain text.
Developer
dx22
Posts: 402
Registered: ‎11-26-2010
My Device: Torch

Re: RIMM streaming file format

@BVP:
This last post was really useful - thanks again. I will read the overview of the mpeg-4. For now I decided to stick with the rimm container. But I got stuck with this descriptor like @floydian. What I finally did was to create a sender which packs the rimm frames and sends them through udp, and on the other side the other device just reads them.

 

I took the descriptor from another video clip and it seems that there is a problem and this descriptor does not work well because I am getting "net.rim.device.internal.media.RimMediaException MSG:Media unloaded while initializing".

The sender sends video in 176x144 px, h.264, amr and the descriptor fits that part but the rest of the descriptor properties seem incorrect. Do you know how should these be configured?

 

I did as you said - the 8th byte is 0 and after that is the descriptor(75 bytes).

 

Here is just the part of the code where the header is passed(I am following the description here http://docs.blackberry.com/en/developers/deliverables/11942/RIM_proprietary_video_format_1001586_11....):

 

byte[] header = "RIMM0000".getBytes();
byte[] descr = new byte[]{
		//audio frames
		0, 0, 0, 79,
		//video frames
		0, 0, 0, 78,
		//audio key frames
		0, 0, 0, 5,
		//video key frames
		0, 0, 0, 1,
		//audio frame rates
		0, 0, 0, 31,
		//video frame rates
		(byte)128, 0, 0, 0,
		//audio size
		32, 0, 0, 24,
		//video size
		11, 4, 0, 0,
		//video frame rate
		66, 0, 0, 0,
		//video max frame size
		28, 32, 0, 0,
		//audio duration millis
		80, 20, 0, 0,
		//video duration millis
		91, 20, 0, 0,
		//reserved, i have no idea what values should be placed here
		0x50, 0x14, 0x00, 0x00, 0x5B, 0x14, 
		0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 
		0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
		//video width
		(byte)176,0,
		//video height in px
		(byte) 144, 0,
		//video codec 6=h.264
		6,0,
		//audio codec 7=amr
		7
		};
		buff.getOutputStream().write(header);
		buff.getOutputStream().write(descr);
		UDPDatagramConnection conn = (UDPDatagramConnection) Connector.open("udp://:1234;interface=wifi;deviceside=false");
		Datagram dgram = conn.newDatagram(2048);
		while(running){
			conn.receive(dgram);
			buff.getOutputStream().write(dgram.getData());
		}

 Also the content type of the SourceStream is set to video/3gpp.

 

 

----------------------------------------------------------------------
Press the button to give kudos if I helped you :smileyhappy:
Please use plain text.
Developer
dx22
Posts: 402
Registered: ‎11-26-2010
My Device: Torch

Re: RIMM streaming file format

Did anyone have any success creating the descriptors here?
----------------------------------------------------------------------
Press the button to give kudos if I helped you :smileyhappy:
Please use plain text.
BlackBerry Development Advisor (Retired)
BlackBerry Development Advisor (Retired)
BVP
Posts: 150
Registered: ‎10-19-2010
My Device: Not Specified
My Carrier: Rogers

Re: RIMM streaming file format

Hi dx22!  Sorry for the late reply, it's only a week since I told you I would get back to you... :Whistling:

 

I wouldn't expect taking the header from one file woould work for another.  I don't work on the RIMM parser specifically, so I don't know why it needs many of these things.

 

Let's look at the descriptor:

 

The # of duaio and video frames will likely be different for your file, to get this value correct you need to know how many frames of each are in your video.  If you are not sure, you can try setting them to relatively high values but it's not something I've tried - I'm not sure what would happen at the end of the stream.

 

The number of keyframes will be a subset of the total # of frames I just mentioned.  To get this, if you don't know it then you'll probably have to parse the H.264 data and either use the value it specifies (if it does) or count them.

 

As for the A/V frame rates, these are the # of audio/video frame rate changes + 1.  This I believe is moslty needed for audio/video synchronization.  Same with initial video frame rate.

 

Audio/video size is the total size in bytes of all audio/video frames, respectively. The 'video max frame size' is what you'd expect, the size in bytes of the largest video frame (likely a keyframe) and the durations for audio and video streams are in milliseconds.

 

I spoke to some developers who do work on this, and I have found out that you don't need to worry about the 20 bytes of reserved  space.  They get populated when the RIMM stream is recorded but they are not used in either playback or recording at all.  If you are curious, the values you see represent 'audio seek duration' (2 bytes), 'video seek duration' (2 bytes), 'seekOffset' (2 bytes), 'esdsData' (1 byte), and 'esdsLength' (2 bytes) in that order.  I don't know what they mean, but I've been told by those that know that they are never used - so zeroes should be fine.

 

To see what's wrong specifically, I'd need some code along with the video that you are trying...

 

Please use plain text.
Developer
dx22
Posts: 402
Registered: ‎11-26-2010
My Device: Torch

Re: RIMM streaming file format

Thanks @BVP I will do some tests taking into account the info from your last post and I will package the whole project and upload it here and of course I will try to make it as readable as possible in order no to waste your time.
Basically what I do is have 2 devices in the same wifi network and one starts the recorder and while it is recording I parse the video stream wrap each frame into datagram and send it directly to the other device where I have started a video player for playback with custom DataSource implementation.
So this is a live active process and I don't know any of the numbers like total video/audio frames, number of key frames etc.

I will try to upload here the project within the next few days.
----------------------------------------------------------------------
Press the button to give kudos if I helped you :smileyhappy:
Please use plain text.
Developer
dx22
Posts: 402
Registered: ‎11-26-2010
My Device: Torch

Re: RIMM streaming file format

Hi,

 

Sorry for the huge delay but I've been assigned to another project for some time. But I am now continueing with this.

Now the next step I will take is to parse the H.264 data which is actually the "data" field in the frame(http://docs.blackberry.com/en/developers/deliverables/11942/RIM_proprietary_video_format_1001586_11....)

 

@BVP can you confirm that the structure of the H.264 data is as described in the standard( http://www.itu.int/rec/T-REC-H.264-201003-S/en) I mean do you know if in RIM's implementation is based on that? 

Basically no matter whether I choose to go with RIMM or 3GP as a container I still need to be able to parse the H.264 data.

----------------------------------------------------------------------
Press the button to give kudos if I helped you :smileyhappy:
Please use plain text.
BlackBerry Development Advisor (Retired)
BlackBerry Development Advisor (Retired)
BVP
Posts: 150
Registered: ‎10-19-2010
My Device: Not Specified
My Carrier: Rogers

Re: RIMM streaming file format

H.264 and MPEG4 AVC (where AVC = Advanced Video Coding which is what you linked to) are technically identical.  As long as it is MPEG 4 simple profile (and most models now also support advanced profile as well) it should be playable from either RIMM or 3GP containers.

Please use plain text.