Captured Packet.Data.Length and udpPacket.PayloadData.Length threshold?

[charleslales]

Hi,

I'm currently facing an issue when capturing packet..
It does happen with a first app using
_sendSocket = new Socket(_ep.AddressFamily, SocketType.Dgram, ProtocolType.Udp);
_sendSocket.SendTo(message, msgLen, SocketFlags.None, ep);
but not with another test app using
_UdpClient = new UdpClient(port);
_UdpClient.Send(data, data.Length, ipe);

Capture code is samples one to get UDP info:
Device_OnPacketArrival(object o, CaptureEventArgs e)
var packet = PacketDotNet.Packet.ParsePacket(e.Packet.LinkLayerType, e.Packet.Data);
var udpPacket = packet.Extract<PacketDotNet.UdpPacket>();

first app:
114 obj -> 1472 bytes KO why???
113 obj -> 1472 bytes KO why???
112 obj -> 1463 bytes OK

test app:
114 obj -> 1489 bytes OK
113 obj -> 1476 bytes OK
112 obj -> 1463 bytes OK

Sorry if it is basic question..
Is there any other option / param I missed smwhere?

Kind regards,
Charles

[chmorgan]

What do you mean by KO vs OK?

…

On Fri, May 7, 2021 at 12:56 PM charleslales ***@***.***> wrote: Hi, I'm currently facing an issue when capturing packet.. It does happen with a first app using _sendSocket = new Socket(_ep.AddressFamily, SocketType.Dgram, ProtocolType.Udp); _sendSocket.SendTo(message, msgLen, SocketFlags.None, ep); but not with another test app using _UdpClient = new UdpClient(port); _UdpClient.Send(data, data.Length, ipe); Capture code is samples one to get UDP info: Device_OnPacketArrival(object o, CaptureEventArgs e) var packet = PacketDotNet.Packet.ParsePacket(e.Packet.LinkLayerType, e.Packet.Data); var udpPacket = packet.Extract<PacketDotNet.UdpPacket>(); first app: 114 obj -> 1472 bytes KO why??? 113 obj -> 1472 bytes KO why??? 112 obj -> 1463 bytes OK test app: 114 obj -> 1489 bytes OK 113 obj -> 1476 bytes OK 112 obj -> 1463 bytes OK Sorry if it is basic question.. Is there any other option / param I missed smwhere? Kind regards, Charles — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#280>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJH4ACZAXPXGA2WYDK5WXDTMQLVDANCNFSM44KSBL6Q> .

[charleslales]

Well, to parse properly n obj, I need to get n x 13 + 7 bytes from udpPacket.PayloadData.
The test app with _UdpClient.Send(data, data.Length, ipe); generates no issue. These are the OK. All bytes are Device_OnPacketArrival.
The first app, operational one in fact, using _sendSocket.SendTo(message, msgLen, SocketFlags.None, ep);, generates proper bytes array up to 112 obj, 1463 bytes, then is stuck to a threshold of 1472 bytes and parse fails :-o
It weird, or maybe a basic network udp concept I totally miss..

[kayoub5]

@charleslales You have an MTU of 1500, your network interface can't send an UDP payload larger than 1500 - 14 - 20 - 8 = 1458 where 14 = Ethernet header, 20 = IP header, 8 = UDP header, the numbers may vary a bit depending on the specifics of your setup.

[charleslales]

Thanks for hint, is it IP layer fragmentation of too large UPD packet?
(test app working fine is when selecting loopback device.. the other app is using an ethernet device..)
In this case, how can I proceed? Checking a flag to detect such fragmentation and reassemble payloads?

[kayoub5]

Loopback devices don't have an MTU limit so the UDP packet can get as big as needed.

Performing IP de-fragmentation is one way to solve the issue,
Do keep in mind that modern Ethernet devices perform automatic hardware accelerated fragmentation also called UDP fragmentation offload, that means that OS/Pcap library see sent UDP message as a whole, while in reality the hardware is fragmenting the UDP message before sending it.

[charleslales]

Ok, thanks for explanation.
Would you see a way to deal with that fragmentation with sharppcap?

[kayoub5]

Ok, thanks for explanation.
Would you see a way to deal with that fragmentation with sharppcap?

• Is it doable with SharpPcap? yes
• Is there something buitin in to do it? no
• Something similar previously done? https://github.com/chmorgan/packetnet-connections

[chmorgan]

Yeah an extension package or an addition to packetnet-connections (although in this case I guess its misnamed since udp is connectionless) would be pretty cool

[charleslales]

Done it... well, let's say implemented what I needed to grab all fragments and build back at the end the udp packets.
No extension methods with a nice Device_OnReassembledPacketArrival event... but some concurrent ConcurrentDictionaries to keep it a min thread safe and straight forward... Work great.
Can post it if you may think it is interesting.

[chmorgan]

I bet others would be interested. It could be a good addition to packdotnet-connections if you were interested.

…

On Fri, May 14, 2021 at 11:20 AM charleslales ***@***.***> wrote: Done it... well, let's say implemented what I needed to grab all fragments and build back at the end the udp packets. No extension methods with a nice Device_OnReassembledPacketArrival event... but some concurrent ConcurrentDictionaries to keep it a min thread safe and straight forward... Work great. Can post it if you may think it is interesting. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#280 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJH4ABEIXYR7U36PQLQTLLTNU5UPANCNFSM44KSBL6Q> .

[charleslales]

Hi,

Here it is, not fully functional but hopefully giving already elements to start with..
One more time, another Device_OnReassembledPacketArrival would be cleaner.

Let's consider these intermediate concurrent collections...

        private ConcurrentDictionary<int, ConcurrentDictionary<int, byte[]>> _fragmentedPackets = new ConcurrentDictionary<int, ConcurrentDictionary<int, byte[]>>();
        private ConcurrentDictionary<int, Tuple<ushort, ushort>> _ports = new ConcurrentDictionary<int, Tuple<ushort, ushort>>();
        private ConcurrentDictionary<int, int> _closure = new ConcurrentDictionary<int, int>();

... and see how to use them Device_OnPacketArrival

        private void Device_OnPacketArrival(object o, CaptureEventArgs e)
        {
            var time = e.Packet.Timeval.Date.ToLocalTime();
            var len = e.Packet.Data.Length;
            var packet = PacketDotNet.Packet.ParsePacket(e.Packet.LinkLayerType, e.Packet.Data);

            var ipV4Packet = packet.Extract<PacketDotNet.IPv4Packet>();
            if (ipV4Packet != null)
            {
                //udp standard packet
                if (ipV4Packet.FragmentFlags == 0 && ipV4Packet.FragmentOffset == 0)
                {
                    var udpPacket = packet.Extract<PacketDotNet.UdpPacket>();
                    if (udpPacket != null)
                    {
                        if (udpPacket.PayloadData != null && udpPacket.PayloadData.Length > 0)
                            DealWithPayloadData(time,
                                ipV4Packet.SourceAddress, udpPacket.SourcePort,
                                ipV4Packet.DestinationAddress, udpPacket.DestinationPort,
                                udpPacket.PayloadData);
                        else
                            Logger..
                    }
                    else
                        Logger..
                }
                else //udp fragmented packet.. need reassembly
                {
                    var udpPacket = ipV4Packet.Extract<PacketDotNet.UdpPacket>();
                    if (udpPacket != null)
                        _ports.TryAdd(ipV4Packet.Id, Tuple.Create(udpPacket.SourcePort, udpPacket.DestinationPort));

                    ConcurrentDictionary<int, byte[]> fragments;
                    if (!_fragmentedPackets.TryGetValue(ipV4Packet.Id, out fragments))
                    {
                        fragments = new ConcurrentDictionary<int, byte[]>();
                        _fragmentedPackets.TryAdd(ipV4Packet.Id, fragments);
                    }
                    fragments.TryAdd(ipV4Packet.FragmentOffset, udpPacket != null ? udpPacket.PayloadData : ipV4Packet.PayloadData);
                    if (ipV4Packet.FragmentFlags == 0)
                        _closure.TryAdd(ipV4Packet.Id, ipV4Packet.FragmentOffset);

                    //to deal with multithreading, each time have to check all
                    int lastFragmentOffset;
                    if (_closure.TryGetValue(ipV4Packet.Id, out lastFragmentOffset))
                    {
                        //test all there
                        var fragmentsArr = fragments.ToArray(); //copy safer
                        if (fragmentsArr.Length == (lastFragmentOffset / _fragmentOffset) + 1)
                        {
                            //reassemble
                            var allBytes = fragmentsArr
                                .OrderBy(kvp => kvp.Key)
                                .SelectMany(byteArr => byteArr.Value)
                                .ToArray();

                            Tuple<ushort, ushort> ports;
                            if (allBytes != null && allBytes.Length > 0)
                            {
                                if (_ports.TryGetValue(ipV4Packet.Id, out ports))
                                    DealWithPayloadData(time,
                                        ipV4Packet.SourceAddress, ports.Item1,
                                        ipV4Packet.DestinationAddress, ports.Item2,
                                        allBytes);
                            }
                            else
                                Logger..

                            //clear since single use
                            _fragmentedPackets.TryRemove(ipV4Packet.Id, out fragments);
                            _ports.TryRemove(ipV4Packet.Id, out ports);
                            _closure.TryRemove(ipV4Packet.Id, out lastFragmentOffset);
                        }
                    }
                }
            }
            else
                Logger..
        }

Feel free to comment and especialy suggest some fixes ;-)

Kind regards,
Charles