Tuesday, July 9, 2013

PCAP Link-Layer header type. (Follow-up on: A look at the pcap file format)

Yesterday, I was emailed by someone who read an article I have written last year about pcap file format, his email was something along the lines of:

> I'm writing to you because I've read your introduction to PCAP file format
> (http://www.kroosec.com/2012/10/a-look-at-pcap-file-format.html) but I still
> have some "advanced" doubts, mainly related to when I change the encapsulation
> type. Could you take some time to enlighten me further in this topic? I'd
> really appreciate it :) 

As my answer was longer than expected, and as Scott Hanselman advices, I am sharing my answer as a blog post, hopefully to be found useful by more people.

Hi X,

If I understood correctly, you have an issue with the meaning of the last 4 bytes in the global header, right ?

That value is not part of the "data" that is captured on the wire (ie. it is not sent or received), but is saved by libpcap / your capturing tool when you save the file. That value is used by tools such as Wireshark to decide on the type of the Link-Layer (Ethernet, Wlan...) header.

You can think of it like other "type" fields in various networking protocols. For instance, the IPv4 header has a "Protocol" field on 1 byte: 0x06 for TCP and 0x11 for UDP (See RFC 790 for all the values.)

Now, imagine you are writing a tool parsing one packet (sent over Ethernet/IPv4/UDP) pcap file. Obviously, you will start from the lower-layers and go up. Here are the question you will be asking each time:

Question: What is the Link-Layer type ?
Answer: Check the pcap global header's link-layer header type: Equals 0x10000000 for Ethernet.

Question: What is the Internet-Layer type ?
Answer: Check Ethernet header's EtherType field: Equals 0x8000 for IP.

Question: What is the transport-Layer type ?
Answer: Check IPv4 header's protocol field: Equals 0x11 for UDP.

At this point, you would have understood that the Link-Layer header type is stored in the pcap file's global header because, well, there is no "lower" layer to store it in.

Throwing a bit of "hands-on" may give you an "aha moment".
* Download the .cap file I have referenced in the article.
* Make two new copies.
* Use your favorite hex editor to edit the original value (0x01000000 == Ethernet) in the two copies to 0x030000000 and 0x69000000. From the referenced link in the article [1], these values are for AX.25 and 802.11 respectively.
* Open both files with Wireshark. It says the first (AX.25) is "not supported while the second is parsed as a 802.11 (Wireless LAN) (with erroneous result as you may expect) instead of 802.3 (Ethernet.)

You may also be wondering about support for Wireshark and other capturing tools or other subjects related to your capture adapter. For this, you may find various discussions like [2] on the subject to be of a help.

Hopefully, this will shed some light on your issue.

[1] http://www.tcpdump.org/linktypes.html
[2] http://www.wireshark.org/lists/wireshark-dev/200810/msg00270.html

Cheers,

Hani.

Monday, July 1, 2013

Building a Linux Firewall as a kernel module

 Recently, I finished reading Linux Kernel Development, 3rd (Robert Love) and started developing a kernel module to get some more hands-on experience with the kernel code and interfaces before delving further into the rabbit hole. A couple of days ago, I have pushed a first version of a lightweight Firewall on Github that I named Merwall.

 This blog post will summarize the work done and will include some notes that may be of use for anyone who may tackle a similar project.
  • Components
 The project has two main parts:
+ A kernel Module whose main task is network traffic filtering based on user-set rules as well as creating and managing the sysfs class/file (more on that later.)
+ The userspace administration tool (à la iptables command-line tool) that is used for various configuration tasks (rules setting, listing, deleting etc,.)
  • The Kernel module
 A nice thing about developing kernel code as a module is the low time needed for the build/test cycle. After modifying code, a fast "make" (With a simple Makefile like this) and an "insmod our_module.ko" is all it takes to test the code. It goes without saying that you should use Qemu, VirtualBox, KVM (or something similar) for code testing... unless you don't mind rebooting after each kernel panic. :)

 A kernel module should define two main functions that are called once, each: The initialization function, which will be called when the module is first inserted and the exit function which will be called when the module is removed. These could be specified with macros from <linux/init.h> header.
module_init(merwall_init);
module_exit(merwall_exit);
 In our case, the merwall_init function's role is to register the netfilter hooks, create the sysfs class and file and initialize any other global variables. The merwall_exit function on the other hand unregisters the netfilter hooks, destroy the sysfs class and frees allocated memory (used by the network filtering rules, among others things.) 
  • Rules list data structure
 The network filtering rules structures are kept in an ordered (by rule index) linked-list. The O(n) worst-case time complexity for inserting/deleting rules isn't much of an issue in this as the main use of the structure is matching every captured network packet with all the rules in the rules list. I was very positively surprised by the great Linked-lists API that the Linux kernel provides. Check this rather good explanation by KernelNewbies.org, or alternatively your favorite search engine.
  • Netfilter
 The Netfilter API allows a kernel module to manipulate incoming/outgoing network traffic. This could be released by using the nf_register_hook function call which takes as a sole argument a nf_hook_ops struct. Among information we add to the structure is the hook point (values NF_INET_PRE_ROUTING/NF_INET_POST_ROUTING are of interest for manipulating incoming/outgoing traffic respectively.)
 
 We also add a callback function (aka. hook) that will be called for every packet passing by the associated hook point in the structure. The passed sk_buff structure contains the packet data that the hook could parse, and match with the user-set filtering rules. The return value of the hook reflects the decision to be taken. Common actions in firewall rules have could be implemented with the following return values:
NF_ACCEPT  => Ignore/Log packet.
NF_DROP      => Block/Drop packet.
NF_STOLEN  => Unblock/Pass packet.
 In Merwall, two (statically declared) nf_hook_ops structures were used that differ in the hook point and the hook function (being a wrapper around packet handling function with a different "direction" values.) Initializing the values is simple as this:
/* Incoming packets */
nfhi.hook = merwall_hook_in;
nfhi.hooknum = NF_INET_PRE_ROUTING;
nfhi.pf = PF_INET;
nfhi.priority = NF_IP_PRI_FIRST;
nf_register_hook(&nfhi);

/* Outgoing packets */
nfho.hook = merwall_hook_out;
nfho.hooknum = NF_INET_POST_ROUTING;
nfho.pf = PF_INET;
nfho.priority = NF_IP_PRI_FIRST;
nf_register_hook(&nfho);
  • Sysfs virtual filesystem
 Sysfs is a virtual (in-memory) filesystem that allows exporting kernel code information to user-space as files and directories under the /sys directory. The hierarchy is better organized compared /proc. In Merwall, I used the sysfs class wrapper function class_create() to create what will be seen as directory in user space (/sys/class/merwall) and class_create_file() to add a class attribute (aka. file found under that directory.)
 
 class_create_file() takes a class_attribute structure as a 2nd argument which contains the file name, permissions and two functions to be called when the fileis read/written to.
  • Userspace
The big bulk of the userspace admin tool's job is to parse command-line arguments (using getopt), check the validity of the provided values and send/receive adequate information from the kernel module through the sysfs file.
For instance:
./merwall_admin --list
Outputs the content of /sys/class/merwall/merwall_file
./merwall_admin --delete 35
will write "1 35"
./merwall_admin --index 44 --direction OUT --proto UDP --action PASS --srcport 5555
--dstip 192.168.1.1
will write "0 44 1 2 0 2 0.0.0.0 192.168.1.1 5555 0"
 One nice benefit of using a file based solution was that I could get something that I could test efficiently early-on using simple tools like cat and echo/printf.

 That was pretty much all that came to my mind while writing this blog post, the code is available on github. Feel free to contact me with any critics, remarks or questions.