Saturday, November 26, 2011

Software Freedom Day Algeria 2011

Yesterday, November 26th, I attended the Software Freedom Day Algeria 2011 in Medea and I still don't know what I liked the most. We arrived around 9:30AM and were amazed by the beauty of the town and how large and architecturally gorgeous the university is.

First talk was by Taha Zerrouki on Open Source software in the service of the Arabic language. I didn't know that there were so much efforts and projects focusing on arabic vocalization ("chakl"). You could check Taha's work on his blog.
As the Debian Project talk was cancelled, it was time for Groupe Fedora Algérie presentation by Lamine Belbachir who was replacing Djelloul Bouida. I've met the guys at a GNU/Linux install party I was involved into last July and I can witness how great and helpful they're. Keep up the good work!

After that Our own Halim presented the Algiers Google Technology Group, I already know this presentation as it was done last month when the group was first launched. Following him was Madjda presenting Mozilla Algeria talking about the values of Mozilla, how it isn't just about a web browser and the role of local communities. As expected in the Q&A part, people were whining about how slow is Firefox compared to Chrome. :D
Next was Bilel Sam from the AAUJ (Algerian Association of Joomla Users) talking about... Joomla. Some interesting things he talked about were the work on localisation of Joomla to Algerian dialect. Though some of the audience didn't really like it and there was even one person who said that it's "ignorance promotion"... I couldn't disagree more with him.If you're ashamed of your culture and heritage, please spare us of your nonsense. Again, the expected question about Joomla's security was asked with the typical answer of doing your homework by updating as soon as possible and not installing random modules.


After lunch, it was time for conferences, workshops and booths visiting in parallel. I was supposed to be at the Fedora booth for the install party but as there wasn't much work to be done, I spent most time catching up with friends I've met at the event. I attended the talk about Information Technology Watch by Karim Kheliouati. Nice information about the use of social networks and different tools like Google Reader and Alerts. Fun talk, lots of laughs and great speaker. :D

It was a shame that I had to leave a bit early to not miss the last bus. There were many talks I was waiting for such as Ismail Chaib on OpenData, my friend Djamel from the recently created Algeria Java User Group on ThinkQuest and Djalal Harouni's workshop (who's very known for his work on Nmap) on Free Embedded Systems.

Overall, though some delay in the planning, it was a great event. Kudos to the organisers for the great work and being kind and helpful all day. Big kudos to Abdelhak Fareh, well done!

Tuesday, October 25, 2011

Book review: Penetration Tester's Open Source Toolkit, Third Edition

This book is a rather ok book for people new to the penetration testing and average for the more experienced ones. The chapters of the book follow the same methodology: Objectives, approach/technologies/tools, case study, hands-on challenge and close with a summary and an end note. This leads to some boring/useless repetition in some chapters.

Chapter 1 is a general introduction to the use of tools, but 29 pages is a bit too much given that it has a lot of filler text (why discuss LiveCD/LiveUSB creation and modification in details ?)

Chapter 2 Reconnaissance is probably the best written chapter of the book. It discusses the theory and the tools used to gather information in a solid way and covers many small and often forgotten details.

Chapter 3 Scanning and enumeration is average. that mostly looked like an average Nmap manual with listings on many pages and man pages copy pasting but still couldn't cover NSE in more than half a page. Oh, and the chapter (and whole book) has no reference to Scapy at all ? that's shameful ;)

Chapter 4 Client-side attacks and human weaknesses is an ok chapter. As expected it introduced to phishing attacks and how to make them more effective. The main discussed tool is with no surprise the Social-Engineering Toolkit. Nothing exceptional.

Chapter 5 Hacking databases services includes some good theory on MS SQL and Oracle RDBMS. Beside Nmap, the discussed tools are mainly Metasploit auxiliary modules.

Chapter 6 Web Server and Web Application testing. This is the worse chapter by a large margin. For such a large topic and a short number of pages (~40), it discusses stack and heap based buffer overflows... Not only this, but rather than a good introduction to HTTP and manual testing with intercepting proxies like ZAP/Webscarab/W3af, it briefly discusses the common web application vulnerabilities in a paragraph or two each and then goes on talking about tools: WAF detection with WAF00F (who's the audience again ?) and automatic scans with Nikto, Grendel and fimap, SQLiX etc... there are also some filler Nmap scans screenshots again.

Chapter 7 Network Devices has some rather ok theory on different networking protocols and use of different tools. Again, there's some Nmap filler...

Chapter 8 Enterprise application testing has good information on enterprise applications, the architecture and technologies used. Beside the Nmap filler that adds nearly nothing new, it discusses the use of tools like sapyto and soapUI.

Chapter 9 Wireless penetration testing covers well both theory and use of tools for wifi technologies. It also discusses briefly the bluethooth technology. Finally, a chapter with no Nmap filler.

Chapter 10 Building penetration test labs should have been moved to the beginning of the book or be a pointed to annexe. it discusses building home labs with virtualization tools, safety, reporting and penetration testing frameworks.

Rating: 3/5

Sunday, October 23, 2011

Library preloading for reverse engineering

  Many times when reversing a dynamically linked executable, we could benefit from knowing when calls to certain functions are made or/and alter its behavior in a faster and more elegant way without modifying the executable.
Library preloading is a great feature that allows us to inject functions from our own libraries in a program and override the duplicated functions.

  Let's take for example Wireshark. 

  ldd is a tool that outputs a program's dynamically linked dependencies, known as shared objects (.so) on *nix systems.
hani@JustD:~$ ldd /usr/bin/wireshark =>  (0x00007fff3f7ff000) => /usr/lib/wireshark/ (0x00007f4b095b5000) => /usr/lib/wireshark/ (0x00007f4b060d9000)
    ... => /lib/x86_64-linux-gnu/ (0x00007f4b03d57000)
Better yet than replacing a whole library, we could override only one or certain functions.
Let's replace getenv which is a function from stdlib.h that returns the value of an environment variable. We use the same original source code of getenv and only add a printf at the beginning to show us what variable is being looked for.
char *getenv(const char *name)
We then compile it and explicitly specify that we want it to be a shared object.
hani@JustD:~/space$ gcc -shared -fPIC mygetenv.c -o 
hani@JustD:~/space$ file
space/ ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
and finally we use the environnement variable LD_PRELOAD to preload our shared object
hani@JustD:~/space$ /usr/bin/wireshark
That's it! There are many variantes and uses to this technique that would could prove useful in many situations.

Thursday, August 18, 2011

Extracting TFTP data from pcap files with Scapy

  I've come across another challenge on This time it was about analyzing a pcap file. When we open the OperationNEPTUNE.pcap in Wireshark and going to Statistics->Conversations we see a couple of TFTP file transfers.

Usually we extract the data by following the TCP/UDP stream, specifying the direction on which the data is sent, "save as" then clean the added data with a hex editor (Okteta or MadEdit on Linux).

In this case however, every data packet payload sent begins with an TFTP opcode on 2 bytes (0x0003 == Data packet) followed by the block number on 2 bytes incremented each time. (0x0001, 0x0002, 0x0003 etc...)

This means that not only we have overhead data at the beginning of the stream but also in the middle.
00 03 00 01 data1 00 03 00 02 data2 00 03 00 03 data3 00 03 00 04 data etc...
Sometimes, 4 bytes for each 512 bytes of data is not a big deal like a .wav file in this pcap where It was readable and not really altered by removing 00 03 00 01 at the beginning only (It should be removed because each file begins with it's format's magic number which is the beginning of data1 in this case).
But in other cases, like a .7z file contained in the same pcap, the integrity of the data shouldn't be altered.

Obviously, it would be very non-productive to clean all the data stream manually with an editor.
In Wireshark we first filter out packets that don't interest us, applying this display filter:
tftp.opcode == 3 and tftp.destination_file == "IECache.7z"
Then we go to File->Save As..., select Displayed and choose a file name.

After that we fire Scapy and read the pcap file.
>>>packets = rdpcap("IECache.pcap")
<IECache.pcap: TCP:0 UDP:569 ICMP:0 Other:0>
Then we iterate over the packets list and write the data to the output file without the first 4 bytes.
>>> f = open("IECache.7z","w")
>>> for p in packets:
. . .        f.write(p.load[4:])
. . .
>>> f.close()
and that is it.
$ file IECache.7z
IECache.7z: 7-zip archive data, version 0.3
We could have used automated pcap extraction tools like NetworkMiner on Windows or Xplico on Linux (Both are Open Source and free), but nothing beats Scapy, the networking Swiss Army Knife. ;)

Friday, August 12, 2011

Convert hex dump into pcap

Recently, I've come across an old packet challenge on ismellpackets blog which consists of finding a secret in a 4 packets capture. The problem is that they were given as a hex dump of this form:
00 0c 29 4c 6d a6 00 0c 29 0e 66 bd 08 00 45 00
00 f6 04 38 40 00 80 06 b7 77 c0 a8 5e 80 c0 a8
5e 81 0d 0d 01 bd 42 d0 33 b5 d2 64 26 85 50 18
fa 97 0e ae 00 00 00 00 00 ca ff 53 4d 42 73 00
One possible way to convert it to pcap file format is to use text2pcap.
But we should first modify it into an appropriate format, adding the offset at the beginning of each line. (man text2pcap and man od for more info)
000000 00 0c 29 4c 6d a6 00 0c 29 0e 66 bd 08 00 45 00
000016 00 f6 04 38 40 00 80 06 b7 77 c0 a8 5e 80 c0 a8
000032 5e 81 0d 0d 01 bd 42 d0 33 b5 d2 64 26 85 50 18
000048 fa 97 0e ae 00 00 00 00 00 ca ff 53 4d 42 73 00
The offset could be in decimal, hexadecimal or octal.

Not a very hard task. I copied all the packets in a text file named input and made the changes with a quick Python script.
with open("input") as f:
    count = 0
    for line in f:
        if line=='\n':
            count = 0
            print "%06d" % count + " " + line
            count += len(line.split())
the if line=='\n' is used to reset the offset to 000000 as there are 4 packets separated with a blank line.
We can now pipe the result into text2pcap:
text2pcap <options> <infile> <outfile>
$python | text2pcap -o dec - output.pcap
The default offset format for text2pcap is hexadecimal. In our case, it would
have been 000000 000010 000020 000030 etc...
We use -o dec to precise that the offset is in decimal.
the - is to precise that the input is provided through the stdin.
output.pcap is the output file. We can open it in Wireshark.

we could have piped the output into tcpdump for instance to see:
$python | text2pcap -o dec - - | tcpdump -Xnnr -
That's it. There are many other ways and already existing scripts that could do the same job.

I came across another challenge in which the packet dump was given in this format:
4500 0527 0001 4000 4006 0000 c0a8 0102
c0a8 0101 2b67 0014 0000 006f 0000 006f
5018 0200 aa32 0000 ffd8 ffe0 0010 4a46
here's the modified version of the script which supports this format:
import sys
with open(sys.argv[1]) as f:
    count = 0
    for line in f:
        if line == '\n': pass
            string = [i[:2] for i in line.split()]
            for i in range(len(string)):
                string.insert(i*2+1, line.split()[i][2:])
            print "%06d" % count + " " + ' '.join(string)
            count += len(string)

Thursday, July 14, 2011

Not all proxies are the same

Proxy servers have many uses. One of them is to stay anonymous on the Internet.
Instead of communicating directly with a server, a client uses a proxy server as an intermediate.
Proxy servers are classified into different categories depending on how they modify the information they get from the client and send it to the server.

To demonstrate this, we could use a little PHP script to see what information a server gets when it receives an HTTP request from a client.
echo "*** IP: " . $_SERVER['REMOTE_ADDR'] ." ***<br/><br/>";

$headers = apache_request_headers();
echo $_SERVER['REQUEST_METHOD'] . " " .
$_SERVER['SERVER_PROTOCOL'] . "<br/><br/>";
foreach ($headers as $header => $value) {
echo "$header: $value <br/>\n";
if you get this error when requesting the script from your browser
Fatal error: Call to undefined function apache_request_headers() in ...
It means that (according to the PHP gurus out there) PHP is not installed as an Apache module.
you can solve this by adding this definition to the beginning of the script after the <?php.
if( !function_exists('apache_request_headers') ) {
function apache_request_headers() {
  $arh = array();
  $rx_http = '/\AHTTP_/';
  foreach($_SERVER as $key => $val) {
    if( preg_match($rx_http, $key) ) {
      $arh_key = preg_replace($rx_http, '', $key);
      $rx_matches = array();
      $rx_matches = explode('_', $arh_key);
      if( count($rx_matches) > 0 and strlen($arh_key) > 2 ) {
        foreach($rx_matches as $ak_key => $ak_val) $rx_matches[$ak_key] = ucfirst($ak_val);
        $arh_key = implode('-', $rx_matches);
      $arh[$arh_key] = $val;
  return( $arh );
When requesting the script with no proxy used we get an answer like this:
*** IP: 41.104.X.Y ***

GET /headers.php HTTP/1.1

USER-AGENT: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20100101 Firefox/5.0
HOST: ***
ACCEPT-CHARSET: ISO-8859-1,utf-8;q=0.7,*;q=0.7
CONNECTION: keep-alive
ACCEPT-LANGUAGE: en-us,en;q=0.5
DNT: 1
ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
CACHE-CONTROL: max-age=0
ACCEPT-ENCODING: gzip, deflate
(Host value is stripped)

Highly anonymous (aka. Elite) proxies:
These proxies are the best to stay anonymous on the internet.
The server sees the request as if it came from the proxy server as a client.

*** IP: ***

GET /headers.php HTTP/1.1

USER-AGENT: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20100101 Firefox/5.0
HOST: ***
ACCEPT-CHARSET: ISO-8859-1,utf-8;q=0.7,*;q=0.7
ACCEPT-LANGUAGE: en-us,en;q=0.5
DNT: 1
ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
ACCEPT-ENCODING: gzip, deflate

Using Hidemyass web-based proxy
*** IP: ***

GET /headers.php HTTP/1.1

USER-AGENT: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20100101 Firefox/5.0
HOST: ***
ACCEPT-CHARSET: ISO-8859-1,utf-8;q=0.7,*;q=0.7
ACCEPT-LANGUAGE: en-us,en;q=0.5
ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Anonymous proxies:
These reveal that they are proxy servers. Either by adding a recognizable user-agent or by using X-PROXY-ID and Via headers.
Like highly anonymous, they don't reveal the client's identity to the server.

Example: web based interface proxy.
*** IP: ***

GET /headers.php HTTP/1.0

HOST: ***
CONNECTION: keep-alive

Transparent proxies:
These are practicaly as good as nothing from a privacy point of view, as they not only reveal themselves to servers as being proxies but also reveal the client's identity by adding headers like X-FORWARDED-FOR with the client's IP as the value.
*** IP: ***

GET /headers.php HTTP/1.1

X-PROXY-ID: 566548825
USER-AGENT: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20100101 Firefox/5.0
HOST: ***
ACCEPT-CHARSET: ISO-8859-1,utf-8;q=0.7,*;q=0.7
VIA: 1.1 (Mikrotik HttpProxy)
ACCEPT-LANGUAGE: en-us,en;q=0.5
DNT: 1
ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
ACCEPT-ENCODING: gzip, deflate

Some proxies are called distorting and are considered anonymous as they add a random IP as value of the X-FORWARDED-FOR header.

For even more anonymity, you should remove the user-agent field, a good add-on for Firefox that helps you doing that is User Agent Switcher.
Next time, know which proxies to use, but always remember that they still know who you're. Even highly anonymous proxies aren't as much private as they seem.

Sunday, July 10, 2011

Introduction to Scapy

 Scapy is a powerful tool that can be used to manipulate and generate network traffic.
It can be either run interactively from the command line or used as a library in a Python script.

Scapy could be installed either through APT in Ubuntu:
$ sudo apt-get install python-scapy
Or the distutils way:
$ cd /tmp
$ wget
$ unzip
$ cd scapy-2.*
$ sudo python install
To launch Scapy:
$ scapy
There is pretty much everything that could be done with Python interpreter plus
the goodies brought by Scapy.
>>> new_packet = IP()
>>> new_packet
<IP  |>
The ls() command is used either without arguments to list supported protocols or
with a packet as an argument to list a protocol (or an instance of it) fields, types, actual values and default values.
>>> ls(new_packet)
version    : BitField             = 4               (4)
ihl        : BitField             = None            (None)
tos        : XByteField           = 0               (0)
len        : ShortField           = None            (None)
id         : ShortField           = 1               (1)
flags      : FlagsField           = 0               (0)
frag       : BitField             = 0               (0)
ttl        : ByteField            = 64              (64)
proto      : ByteEnumField        = 0               (0)
chksum     : XShortField          = None            (None)
src        : Emph                 = ''     (None)
dst        : Emph                 = ''     ('')
options    : PacketListField      = []              ([]) 
>>> new_packet.dst
>>> new_packet.dst = ''
>>> new_packet
<IP  dst= |>
del() is used to revert field values back to default.
>>> del(new_packet.dst)
>>> new_packet.dst
lsc() is used to list available commands.
to encapsulate protocols, the / operator is used.
>>> Ether()
<Ether  |>
>>> Ether()/IP()
<Ether  type=0x800 |<IP  |>>

>>> Ether()/IP()/TCP()
<Ether  type=0x800 |<IP  frag=0 proto=tcp |<TCP  |>>>
Notice that default fields values are changed accordingly.
a raw string of data can also be used as a layer.
>>> IP()/TCP()/"Payload !"
<IP  frag=0 proto=tcp |<TCP  |<Raw  load='Payload !' |>>>
str() function is used to get raw representation of a packet.
>>> a = IP()
>>> str(a)
>>> IP(str(a))
<IP  version=4L ihl=5L tos=0x0 len=20 id=1 flags= frag=0L ttl=64 proto=ip chksum=0x7ce7 src= dst= |>
the hide_defaults() method could be used to delete the default values.
>>> b = IP(str(a))
>>> b
<IP  version=4L ihl=5L tos=0x0 len=20 id=1 flags= frag=0L ttl=64 proto=ip chksum=0x7ce7 src= dst= |>
>>> b.hide_defaults()
>>> b
<IP  ihl=5L len=20 chksum=0x7ce7 src= |>
hexdump() is used to show hexadecimal representation of the packet, like in wireshark.

The real deal:
What are packets useful for if not sent and received ?
To be able to send traffic, Scapy should be run as root.
$sudo scapy
To send packets, there are two commands available: send() which sends packets at layer 3 and sendp() which sends packets at layer 2.
>>> send(IP(dst='')/ICMP())
Sent 1 packets.

>>> sendp(Ether()/IP(dst='')/ICMP(), iface="wlan0")
Sent 1 packets.
Multiple packets could be sent with varying fields values, for example:

>>> group =IP(dst='')/ICMP()
>>> group
<IP  frag=0 proto=icmp dst=Net('') |<ICMP  |>>
>>> [p for p in group]
[<IP  frag=0 proto=icmp dst= |<ICMP  |>>,
 <IP  frag=0 proto=icmp dst= |<ICMP  |>>,
 <IP  frag=0 proto=icmp dst= |<ICMP  |>>]

>>> send(IP(dst='')/ICMP())
Sent 3 packets.

>>> destinations = ['', '', '']

>>> send(IP(dst=destinations, ttl=(1,4))/ICMP())
Sent 12 packets.

To send and receive packets sr() and srp() are used (at layer 2 and 3).
sr1() and srp1() are variants that return only the first replied packet.

>>> response = sr1(IP(dst='')/ICMP())
Begin emission:
..Finished to send 1 packets.
Received 3 packets, got 1 answers, remaining 0 packets
>>> response
<IP  version=4L ihl=5L tos=0x0 len=28 id=31732 flags= frag=0L ttl=64 proto=icmp chksum=0x7b69 src= dst= options=[] |<ICMP  type=echo-reply code=0 chksum=0xffff id=0x0 seq=0x0 |>>
>>> response.summary()
'IP / ICMP > echo-reply 0'
###[ IP ]###
  version= 4L
  ihl= 5L
  tos= 0x0
  len= 28
  id= 31732
  frag= 0L
  ttl= 64
  proto= icmp
  chksum= 0x7b69
###[ ICMP ]###
     type= echo-reply
     code= 0
     chksum= 0xffff
     id= 0x0
     seq= 0x0

This is practically all that's needed to begin using Scapy. Coupled with TCP/IP knowledge, a lot of traditional tools and more could be replaced with Scapy. In future articles, we will try to make useful scripts with Scapy.
The project's website contains complete documentation, demos, mailing lists and all about Scapy.

Friday, January 7, 2011

Blog opening

Hello readers !
It's hard to talk about all the who, why and how concerning me and this blog. So, I will just say that am an infosec enthusiast, Pythonista and Open Source proponent. One day, when Mr. Free time will visit me, I will write more here.

Update: Mr. Free visited me after dinner and I updated the About page.