Thursday, July 14, 2011

Not all proxies are the same

Proxy servers have many uses. One of them is to stay anonymous on the Internet.
Instead of communicating directly with a server, a client uses a proxy server as an intermediate.
Proxy servers are classified into different categories depending on how they modify the information they get from the client and send it to the server.

To demonstrate this, we could use a little PHP script to see what information a server gets when it receives an HTTP request from a client.
<?php
echo "*** IP: " . $_SERVER['REMOTE_ADDR'] ." ***<br/><br/>";

$headers = apache_request_headers();
echo $_SERVER['REQUEST_METHOD'] . " " .
$_SERVER['REQUEST_URI'] . " " .
$_SERVER['SERVER_PROTOCOL'] . "<br/><br/>";
foreach ($headers as $header => $value) {
echo "$header: $value <br/>\n";
}
?>
if you get this error when requesting the script from your browser
Fatal error: Call to undefined function apache_request_headers() in ...
It means that (according to the PHP gurus out there) PHP is not installed as an Apache module.
you can solve this by adding this definition to the beginning of the script after the <?php.
if( !function_exists('apache_request_headers') ) {
function apache_request_headers() {
  $arh = array();
  $rx_http = '/\AHTTP_/';
  foreach($_SERVER as $key => $val) {
    if( preg_match($rx_http, $key) ) {
      $arh_key = preg_replace($rx_http, '', $key);
      $rx_matches = array();
      $rx_matches = explode('_', $arh_key);
      if( count($rx_matches) > 0 and strlen($arh_key) > 2 ) {
        foreach($rx_matches as $ak_key => $ak_val) $rx_matches[$ak_key] = ucfirst($ak_val);
        $arh_key = implode('-', $rx_matches);
      }
      $arh[$arh_key] = $val;
    }
  }
  return( $arh );
}
}
When requesting the script with no proxy used we get an answer like this:
*** IP: 41.104.X.Y ***

GET /headers.php HTTP/1.1

USER-AGENT: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20100101 Firefox/5.0
HOST: ***
ACCEPT-CHARSET: ISO-8859-1,utf-8;q=0.7,*;q=0.7
CONNECTION: keep-alive
ACCEPT-LANGUAGE: en-us,en;q=0.5
DNT: 1
ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
CACHE-CONTROL: max-age=0
ACCEPT-ENCODING: gzip, deflate
(Host value is stripped)

Highly anonymous (aka. Elite) proxies:
These proxies are the best to stay anonymous on the internet.
The server sees the request as if it came from the proxy server as a client.

Example:
*** IP: 123.30.183.119 ***

GET /headers.php HTTP/1.1

USER-AGENT: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20100101 Firefox/5.0
HOST: ***
ACCEPT-CHARSET: ISO-8859-1,utf-8;q=0.7,*;q=0.7
ACCEPT-LANGUAGE: en-us,en;q=0.5
DNT: 1
ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
ACCEPT-ENCODING: gzip, deflate

Using Hidemyass web-based proxy
*** IP: 67.159.5.242 ***

GET /headers.php HTTP/1.1

USER-AGENT: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20100101 Firefox/5.0
HOST: ***
ACCEPT-CHARSET: ISO-8859-1,utf-8;q=0.7,*;q=0.7
ACCEPT-LANGUAGE: en-us,en;q=0.5
ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Anonymous proxies:
These reveal that they are proxy servers. Either by adding a recognizable user-agent or by using X-PROXY-ID and Via headers.
Like highly anonymous, they don't reveal the client's identity to the server.

Example: Anonymouse.org web based interface proxy.
*** IP: 193.200.150.82 ***

GET /headers.php HTTP/1.0

USER-AGENT: http://Anonymouse.org/ (Unix)
HOST: ***
CONNECTION: keep-alive

Transparent proxies:
These are practicaly as good as nothing from a privacy point of view, as they not only reveal themselves to servers as being proxies but also reveal the client's identity by adding headers like X-FORWARDED-FOR with the client's IP as the value.
Example:
*** IP: 80.78.136.83 ***

GET /headers.php HTTP/1.1

X-PROXY-ID: 566548825
X-FORWARDED-FOR: 41.104.X.Y
USER-AGENT: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20100101 Firefox/5.0
HOST: ***
ACCEPT-CHARSET: ISO-8859-1,utf-8;q=0.7,*;q=0.7
VIA: 1.1 20.20.20.1 (Mikrotik HttpProxy)
ACCEPT-LANGUAGE: en-us,en;q=0.5
DNT: 1
ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
ACCEPT-ENCODING: gzip, deflate

Some proxies are called distorting and are considered anonymous as they add a random IP as value of the X-FORWARDED-FOR header.

For even more anonymity, you should remove the user-agent field, a good add-on for Firefox that helps you doing that is User Agent Switcher.
Next time, know which proxies to use, but always remember that they still know who you're. Even highly anonymous proxies aren't as much private as they seem.

Sunday, July 10, 2011

Introduction to Scapy

 Scapy is a powerful tool that can be used to manipulate and generate network traffic.
It can be either run interactively from the command line or used as a library in a Python script.

Scapy could be installed either through APT in Ubuntu:
$ sudo apt-get install python-scapy
Or the distutils way:
$ cd /tmp
$ wget http://www.secdev.org/projects/scapy/files/scapy-latest.zip
$ unzip scapy-latest.zip
$ cd scapy-2.*
$ sudo python setup.py install
To launch Scapy:
$ scapy
>>>
There is pretty much everything that could be done with Python interpreter plus
the goodies brought by Scapy.
>>> new_packet = IP()
>>> new_packet
<IP  |>
The ls() command is used either without arguments to list supported protocols or
with a packet as an argument to list a protocol (or an instance of it) fields, types, actual values and default values.
>>> ls(new_packet)
version    : BitField             = 4               (4)
ihl        : BitField             = None            (None)
tos        : XByteField           = 0               (0)
len        : ShortField           = None            (None)
id         : ShortField           = 1               (1)
flags      : FlagsField           = 0               (0)
frag       : BitField             = 0               (0)
ttl        : ByteField            = 64              (64)
proto      : ByteEnumField        = 0               (0)
chksum     : XShortField          = None            (None)
src        : Emph                 = '127.0.0.1'     (None)
dst        : Emph                 = '127.0.0.1'     ('127.0.0.1')
options    : PacketListField      = []              ([]) 
>>> new_packet.dst
'127.0.0.1'
>>> new_packet.dst = '192.168.1.1'
>>> new_packet
<IP  dst=192.168.1.1 |>
del() is used to revert field values back to default.
>>> del(new_packet.dst)
>>> new_packet.dst
'127.0.0.1'
lsc() is used to list available commands.
to encapsulate protocols, the / operator is used.
>>> Ether()
<Ether  |>
>>> Ether()/IP()
<Ether  type=0x800 |<IP  |>>

>>> Ether()/IP()/TCP()
<Ether  type=0x800 |<IP  frag=0 proto=tcp |<TCP  |>>>
Notice that default fields values are changed accordingly.
a raw string of data can also be used as a layer.
>>> IP()/TCP()/"Payload !"
<IP  frag=0 proto=tcp |<TCP  |<Raw  load='Payload !' |>>>
str() function is used to get raw representation of a packet.
>>> a = IP()
>>> str(a)
'E\x00\x00\x14\x00\x01\x00\x00@\x00|\xe7\x7f\x00\x00\x01\x7f\x00\x00\x01'
>>> IP(str(a))
<IP  version=4L ihl=5L tos=0x0 len=20 id=1 flags= frag=0L ttl=64 proto=ip chksum=0x7ce7 src=127.0.0.1 dst=127.0.0.1 |>
the hide_defaults() method could be used to delete the default values.
>>> b = IP(str(a))
>>> b
<IP  version=4L ihl=5L tos=0x0 len=20 id=1 flags= frag=0L ttl=64 proto=ip chksum=0x7ce7 src=127.0.0.1 dst=127.0.0.1 |>
>>> b.hide_defaults()
>>> b
<IP  ihl=5L len=20 chksum=0x7ce7 src=127.0.0.1 |>
hexdump() is used to show hexadecimal representation of the packet, like in wireshark.

The real deal:
What are packets useful for if not sent and received ?
To be able to send traffic, Scapy should be run as root.
$sudo scapy
>>>
To send packets, there are two commands available: send() which sends packets at layer 3 and sendp() which sends packets at layer 2.
>>> send(IP(dst='192.168.1.1')/ICMP())
.
Sent 1 packets.

>>> sendp(Ether()/IP(dst='192.168.1.1')/ICMP(), iface="wlan0")
.
Sent 1 packets.
Multiple packets could be sent with varying fields values, for example:

>>> group =IP(dst='192.168.1.1-3')/ICMP()
>>> group
<IP  frag=0 proto=icmp dst=Net('192.168.1.1-3') |<ICMP  |>>
>>> [p for p in group]
[<IP  frag=0 proto=icmp dst=192.168.1.1 |<ICMP  |>>,
 <IP  frag=0 proto=icmp dst=192.168.1.2 |<ICMP  |>>,
 <IP  frag=0 proto=icmp dst=192.168.1.3 |<ICMP  |>>]

>>> send(IP(dst='192.168.1.1-3')/ICMP())
.
Sent 3 packets.

>>> destinations = ['192.168.1.1', '192.168.1.5', '192.168.1.20']

>>> send(IP(dst=destinations, ttl=(1,4))/ICMP())
.
Sent 12 packets.

To send and receive packets sr() and srp() are used (at layer 2 and 3).
sr1() and srp1() are variants that return only the first replied packet.

>>> response = sr1(IP(dst='192.168.1.1')/ICMP())
Begin emission:
..Finished to send 1 packets.
*
Received 3 packets, got 1 answers, remaining 0 packets
>>> response
<IP  version=4L ihl=5L tos=0x0 len=28 id=31732 flags= frag=0L ttl=64 proto=icmp chksum=0x7b69 src=192.168.1.1 dst=192.168.1.50 options=[] |<ICMP  type=echo-reply code=0 chksum=0xffff id=0x0 seq=0x0 |>>
>>> response.summary()
'IP / ICMP 192.168.1.1 > 192.168.1.50 echo-reply 0'
>>> response.show()
###[ IP ]###
  version= 4L
  ihl= 5L
  tos= 0x0
  len= 28
  id= 31732
  flags=
  frag= 0L
  ttl= 64
  proto= icmp
  chksum= 0x7b69
  src= 192.168.1.1
  dst= 192.168.1.50
  \options\
###[ ICMP ]###
     type= echo-reply
     code= 0
     chksum= 0xffff
     id= 0x0
     seq= 0x0

This is practically all that's needed to begin using Scapy. Coupled with TCP/IP knowledge, a lot of traditional tools and more could be replaced with Scapy. In future articles, we will try to make useful scripts with Scapy.
The project's website http://www.secdev.org/projects/scapy/ contains complete documentation, demos, mailing lists and all about Scapy.