MPEG PES private-stream-1 UDP Packet Decoding

3 replies [Last post]
pixelrender
Offline
Joined: 08/31/2010

First off I'm sorry If this is a question that has been asked a million times but I cant for the life of me find a search option here. what gives?

I am trying to read the KLV encoded metadata sent from a Unmanned Air Vehicle over a UDP data stream. And I desperately need some help.

Also, sorry if my wording seems off, I have jumped head first into this problem with no prior experience.

The video feed is sent over PID:0x80 and the metadata is sent over PID:0x90.
How should I go about reading only 0x90 packets?

To get the packet payload I need to getByteArray right? But that needs and index, does that mean the header index? this confuses me.

I think I can use KLV.java to decode the payload once I get it but I need to know the KLV encoding KeyLength, do you think there is a way to figure that out in the packet details?

Again, Sorry for the randomness, but any help at all would be greatly appreciated. My mind is all fuzzy at the moment and I just need some new leads.

I will try and clarify my points in the morning.

Thank you!!
Matt.

Mark Bednarczyk
Mark Bednarczyk's picture
Offline
Joined: 03/22/2008
Added search button to toolbar

pixelrender wrote:
First off I'm sorry If this is a question that has been asked a million times but I cant for the life of me find a search option here. what gives?

Sorry about that. You had to be logged to see the search button. I added it to the main tool bar, which should make it easier to find.

pixelrender wrote:
I am trying to read the KLV encoded metadata sent from a Unmanned Air Vehicle over a UDP data stream. And I desperately need some help.

Also, sorry if my wording seems off, I have jumped head first into this problem with no prior experience.

The video feed is sent over PID:0x80 and the metadata is sent over PID:0x90.
How should I go about reading only 0x90 packets?

We don't have header definitions for MPEG (ISO/IEC 13818-1) or PES. This is something that would have to be written.

pixelrender wrote:
To get the packet payload I need to getByteArray right? But that needs and index, does that mean the header index? this confuses me.

There are a number of ways to retrieve payload from a packet. From jNetPcap's point of view, every type of header is usually followed by payload. So payload for ethernet is everything behind the Ethernet header in a packet, which will typically include Ip4 header, udp, etc..

You can retrieve a header for a udp packet, which in your case would contain the MPEG2/PES data as byte array:

Udp udp = new Udp();
if (packet.hasHeader(udp)) {
  byte[] udpPayload = udp.getPayload();
}

Javadoc: JHeader.getPayload()

Another way to get the last part of the packet that is undecoded, which would be the same result as above because jNetPcap won't be able to interpret anything behind the udp header for MPEG2/PES traffic, using a special builtin header type called Payload:

Payload payload = new Payload();
if (packet.hasHeader(payload)) {
  JBuffer noCopyPayload = (JBuffer)payload;
  byte[] copiedToByteArrayPayload = payload.getByteArray(0, payload.size());
}

There are several additional ways of getting the payload beyond these 2 methods above.

pixelrender wrote:
I think I can use KLV.java to decode the payload once I get it but I need to know the KLV encoding KeyLength, do you think there is a way to figure that out in the packet details?

It would be a little more involved then just that. You would also need to skip over the appropriate amount of data immediately following the UDP header because all of it would end up as payload.

pixelrender wrote:
Again, Sorry for the randomness, but any help at all would be greatly appreciated. My mind is all fuzzy at the moment and I just need some new leads.

I'm not sure how much help I have been here, but I hope that answers your direct questions.

I would love to work with this type of traffic if you would be willing to contribute some capture files of the target traffic. If you look at the shear length of ISO standards 138181-X this is a significant effort to properly add as a supported protocol to jNetPcap. Also, to properly support this type of traffic, you would also need to implement the Ip4-reassembler for fragmented Ip4 packets. TS traffic type is contained to 188 bytes, but PS and PES are not and therefore would need reassembly services which are coming a little later (http://jnetpcap.com/node/599). Another words, this isn't something that can easily or quickly be added to jNetPcap. I have plans for it, but it will take lots of time.

What people typically do with missing header definitions in jNetPcap that they need, is to write skeleton code just to get through specifically the type of traffic they have to deal with. This isn't of course production code, but its pretty easy to write and add new protocol definitions to jnetpcap, as private header definitions. Then when the full production quality definition is added, replace the private skeleton definition with the production one.

What I mean by skeleton, is the very simplest definition that just figures out the length of the header so that it can be accounted for and linked with other headers.

Sly Technologies, Inc.
R&D

pixelrender
Offline
Joined: 08/31/2010
Thank you very much. I will

Thank you very much. I will attempt to peruse this further today. I will let you know when I run into another brick wall.

The skeleton code idea is a bit confusing but I'll do a quick search and see if I can make one to fit my needs.

I tried to send a PM but It said I was "unable to email" you. if you give me an address I can send you a some of the captured packets.

Thank You!

Mark Bednarczyk
Mark Bednarczyk's picture
Offline
Joined: 03/22/2008
Just so everyone has access

Just so everyone has access to the latest so far of the MPEG2 header. This isn't complete definition but it does provide most of the functionality and access to MPEG headers:

/**
 * Copyright (C) 2010 Sly Technologies, Inc. This library is free software; you
 * can redistribute it and/or modify it under the terms of the GNU Lesser
 * General Public License as published by the Free Software Foundation; either
 * version 2.1 of the License, or (at your option) any later version. This
 * library is distributed in the hope that it will be useful, but WITHOUT ANY
 * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
 * A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more
 * details. You should have received a copy of the GNU Lesser General Public
 * License along with this library; if not, write to the Free Software
 * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
 */
package org.jnetpcap.protocol.iso;

import org.jnetpcap.packet.JHeader;
import org.jnetpcap.packet.annotate.Dynamic;
import org.jnetpcap.packet.annotate.Field;
import org.jnetpcap.packet.annotate.Header;
import org.jnetpcap.packet.annotate.ProtocolSuite;

/**
 * @author Mark Bednarczyk
 * @author Sly Technologies, Inc.
 */
@Header(length = 188, suite = ProtocolSuite.ISO, description = "ISO/IEC 13818-1")
public class MPEG2
    extends
    JHeader {

	private int length = 4;
	private int offset = 4;

	@Field(offset = 0, length = 4 * BYTE, format = "%x")
	public long header() {
		return getUInt(0);
	}
	
	@Dynamic(Field.Property.DESCRIPTION)
	public String header_syncByteDescription() {
		if (header_syncByte() == 0x47) {
			return String.format("correct (0x%X)", header_syncByte());
		} else {
			return "invalid";
		}
	}

	@Field(parent = "header", offset = 24, length = 8, display = "Sync Byte")
	public long header_syncByte() {
		return (header() >> 24) & 0xFF;
	}

	@Field(parent = "header", offset = 23, length = 1, display = "Transport Error Indicator")
	public long header_TransportErrorIndicator() {
		return (header() >> 23) & 0x1;
	}

	@Field(parent = "header", offset = 22, length = 1, display = "Payload Unit Start Indicator")
	public long header_PayloadUnitStart() {
		return (header() >> 22) & 0x1;
	}

	@Field(parent = "header", offset = 21, length = 1, display = "Transport Priority")
	public long header_TransportPriority() {
		return (header() >> 21) & 0x1;
	}
	
	@Dynamic(Field.Property.DESCRIPTION)
	public String header_PIDDescription() {
			return String.format("(0x%X)", header_PID());
	}


	@Field(parent = "header", offset = 8, length = 13, display = "PID")
	public long header_PID() {
		return (header() >> 8) & 0x12FF;
	}

	@Dynamic(Field.Property.DESCRIPTION)
	public String header_TransportScramblingControlDescription() {
		if (header_TransportScramblingControl() == 0) {
			return String.format("not scramboled (0x%X)", header_TransportScramblingControl());
		} else {
			return String.format("scramboled (0x%X)", header_TransportScramblingControl());
		}
	}
	
	@Field(parent = "header", offset = 6, length = 2, display = "Transport Scrambling Control")
	public long header_TransportScramblingControl() {
		return (header() >> 6) & 0x03;
	}
	
	@Dynamic(Field.Property.DESCRIPTION)
	public String header_AdaptionFieldControlDescription() {
		if (header_AdaptionFieldControl() == 0x1) {
			return String.format("payload only (0x%X)", header_AdaptionFieldControl());
		} else {
			return String.format("adaptation and payload (0x%X)", header_AdaptionFieldControl());
		}
	}
	

	@Field(parent = "header", offset = 4, length = 2, display = "Adaption Field Control")
	public long header_AdaptionFieldControl() {
		return (header() >> 4) & 0x03;
	}
	
	@Dynamic(Field.Property.DESCRIPTION)
	public String header_ContinuityCounterDescription() {
			return String.format("%d", header_ContinuityCounter());
	}


	@Field(parent = "header", offset = 0, length = 4, display = "Continuity Counter")
	public long header_ContinuityCounter() {
		return (header() >> 0) & 0x0F;
	}

	@Dynamic(Field.Property.LENGTH)
	public int payloadLength() {
		return this.length * BYTE;
	}
	
	@Dynamic(Field.Property.OFFSET)
	public int payloadOffset() {
		return this.offset * BYTE;
	}

	@Field(format = "#hexdump#")
	public byte[] payload() {
		return getByteArray(offset, length);
	}
	
	@Dynamic(field = "adaptationFieldLength", value = Field.Property.CHECK)
	public boolean adaptationFieldLengthCheck() {
		return adaptationFieldCheck();
	}
		
	@Field(offset = 4 * BYTE, length = 1 * BYTE, display = "field length")
	public int adaptationFieldLength() {
		return getUByte(4);
	}


	@Dynamic(field = "adaptationField", value = Field.Property.CHECK)
	public boolean adaptationFieldCheck() {
		return (header_AdaptionFieldControl() & 0x2) == 0x2;
	}

	@Field(offset = 5 * BYTE, length = 1 * BYTE)
	public int adaptationField() {
		return getUByte(5);
	}

	@Override
	protected void decodeHeader() {
		final int remaining = getPacket().size() - getOffset();
		length = (remaining >= 188) ? 188 - 4 : remaining;
		
		if (adaptationFieldCheck()) {
			length -= (adaptationFieldLength() + 1);
			offset += (adaptationFieldLength() + 1);
		}
	}

}

Sly Technologies, Inc.
R&D

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.