January 2009

I have the TCP analyzer working. It tracks seq/ack and a bunch of other stuff. It works currently well with the happy case, where everything goes smoothly between client and server.

Now I'm working on making it handle all the un-happy cases with duplicate ACKs, out of order segments, timedout segments, etc, etc..

Its a very complex protocol and from analyzer's perspective its even more complex. An analyzer sits between client and server, sender and receiver. It sees things that sender sends out but potentially receiver never receives. It has to keep virtual state for both client and sender and visa versa in opposite direction.

None the less its starting to look really well. Keeping track of seq and acks and receive windows. There is a ton of events and analysis it can attach. For now, every TCP packet gets a TcpDuplexStream analysis tag. This analysis object contains 2 TcpStream objects, one for each direction of the duplex stream. Each TcpStream keeps track of various tcp parameters and progress as tcp stream time ticks on (i.e. packets are received.) Errors detected will also be attached, as well as (if user requests it) a snapshot of the TCP stream progress for every packet. This would allow one to graph any of the TCP parameters on a stream by stream basis. For example you could graph how much unacknowledged data remains in the sender send buffer and its relation ship to receivers receive window. Time based analysis and graphing since you now know exactly how long it took to acknowledge send segments, retransmissions and how they occured. All on a per packet that is on a per stream basis.

I have Ip4 analyzer and reassemblers working. These are 2 separate classes. Ip4FragmentationAnalyzer tags ip packets with FragmentSequence analysis objects, while Ip4Reassembler takes those fragments, reassembles them in a new buffer and creates a new Ip PDU packet. Here is a code from jUnit testcase:

public void testIp4Reasemble() throws IOException {

	ip4Defrag = new Ip4Reassembler(ip4Analyzer);

	controller.add(new JPacketHandler

and output from the last reassembled packet:

Frame #243
Ip:  ******* Ip4 - "ip version 4" - offset=0 (0x0) length=20
Ip: 
Ip:          version = 4
Ip:             hlen = 5 [5 * 4 = 20 bytes, No Ip Options]
Ip:         diffserv = 0x0 (0)
Ip:                    0000 00.. = [0] code point: not set
Ip:                    .... ..0. = [0] ECN bit: not set
Ip:                    .... ...0 = [0] ECE bit: not set
Ip:           length = 5720
Ip:               id = 0x252 (594)
Ip:            flags = 0x2 (2)
Ip:                    0.. = [0] reserved
Ip:                    .1. = [1] DF: do not fragment: set
Ip:                    ..0 = [0] MF: more fragments: not set
Ip:           offset = 0
Ip:              ttl = 254 [time to live]
Ip:             type = 17 [next: udp]
Ip:         checksum = 0x0 (0)
Ip:           source = 131.151.1.146
Ip:      destination = 131.151.32.21
Ip: 

Have been working on analyzers and the new package org.jnetpcap.analysis.

Ip4FragmentAnalyzer is nearly complete. It keeps track of ip fragments and attaches FragmentSequence analysis information to each fragment. Tcp segments are going to be handled the same way, reusing the same analysis class. Ip4Reassembler is going to be very trivial, because all it does is monitor packets that have a complete FragmentSequence attached, when they do, the call FragmentSequence.getPacketSequence() returns a list of packets part of the complete fragment. All it has to do is copy them to a new buffer and scan as a new packet. All the analysis is already done by Ip4FragmentAnalyzer.

The hard part was actually how attach information to packets. Packets can be copied into a raw byte buffer or byte[] and copied out of them to create new packets that are fully scanned. I also like the "peer" API a lot with how the headers are bound today. JAnalysis objects are bound the same way now, even when they contain references to other java objects.

I will tidy things up tomorrow morning and check what I have in. Once Ip4 protocol is done with analysis and reassembly, I'll do TCP next.

The way you analyzers come into play is the user instantiates an AnalyzerController object, add whatever JAnalyzer based classes you want to it and then pass the controller over to Pcap.loop or dispatch. It implements the JPacketHandler interface. It also accepts a JHandlerHandler as a listener which is where the user gets to see the analyzed packets. It goes right in between the Pcap.loop and user's JPacketHandler. Analyzers can dispatch numerous different events based on what the analyzer is. That is another way to attach to the analyzers.

Added org.jnetpcap.nio.JReference class. This class is needed to track JNI global object references. For those that are familiar with how JNI works, once you allocate a JNI global reference, its upto your application to release it when its no longer needed.

JMemory class already keeps track of allocated memory very well, but it needed more to keep track of JNI global references. Especially with how packet_state_t and header_state_t structures are setup, you couldn't just call JNIEnv->NewGlobalRef and set the reference in packet or header state structures. Now JMemory works with JReference (allocating one if its needed) to keep track of JNI global references. During the peering process the memory owner's JReference object is passed to the peer. When all the objects that used those JNI references natively expire, JNI references are deleted by JReference destructor method.

Anyway, its a little complex, but the bottom line is that JNI global references to java objects can now be safely created and they will be released, just like native memory is released as well.

Also added JAnalysis jobject reference to packet_state_t and header_t structures. This allows hierarchal analysis objects to be attached to packets and headers.

Here is debug output from a test method that sets 2 JAnalysis objects using JPacket.State.setAnalysis() and JHeader.State.setAnalysis() methods:

JMemory: JMemory@af4baee: size=168 bytes
JMemory: owner=nio.JMemoryPool$Block.class(size=10240/offset=2590)
JMemory: references(capacity=3, @25c798, @256cf4, @0)
JPacket.State: sizeof(packet_state_t)=104
JPacket.State: sizeof(header_t)=16 and *4=64
JPacket.State: pkt_header_map=0x27
JPacket.State: pkt_header_count=4
JPacket.State: pkt_headers[0]=[hdr_id=1  ETHERNET       ,hdr_offset=0   ,hdr_length=14]
JPacket.State: pkt_headers[1]=[hdr_id=2  IP4            ,hdr_offset=14  ,hdr_length=20]