help

# MASSCAN: Mass IPv4 port scanner

This is a port scanner. It spews out packets at a high rate, then catches any

responses asynchronously. Because it's asynchronous, it's a lot faster than 

`nmap` -- and a lot less feature rich.

This is a port scanner. It spews out packets at a high rate, up to 10 million

packets-per-second, fast enough to scan the entire Internet for one port

in 6 minutes. I can do this because it's *asynchronous*: one thread transmits

packets, another thread receives them, without much communication between

the treads, or remembering *state* about each packet that was sent.

This program looks a lot like the most famous port scanner, `nmap`, but

because it's *asynchronous*, it's less feature rich (albeit 10,000 times

faster).

This is a 48-bit scanner: scanning all ports (16-bits) on all

IPv4 addresses (32-bits). It's also useful on smaller problems, such as the

10.x.x.x address space within a company.

It randomizes the IPv4+port combination, whereas `nmap only randomizes the

IPv4 address. This is so that we can send out 10-million packet per second

when scanning the entire Internet, but the owner of a Class C network will

only see 1 packet per second comming in.

This randomizes the IPv4+port combination, whereas `nmap` only randomizes the

IPv4 address. That means we don't produce complete output *per host*, but

output *per host+port* combination. By randomizing the port alongside the IP

address, target networks won't get overwhelmed.

# Building

	$ git clone https://github.com/robertdavidgraham/masscan

	$ cd masscan

	$ sudo apt-get install build-essential

	$ sudo apt-get install libpcap-dev

	$ make

	$ make regresss

This puts the program in the `masscan\bin` subdirectory.

* Windows: use the Visual Studio 2010 project in the `vs10` subdirectory

* Windows: MingGW should work

* Windows: cygwin shouldn't work

* Mac OS X: once you install the development tools, just `make`

* FreeBSD: doesn't work, probably, but I'm hoping to get around to it

This puts the program in the `masscan/bin` subdirectory. You'll have to

manually copy it to something like `/usr/local/bin` if you want to

install it elsewhere on the system.

While Linux is the primary target platform, the code runs well on many other

systems. Here's some additional info:

* Windows:Visual Studio: use the VS10 project in the `vs10` subdirectory

* Windows:MingGW: just type `make`

* Windows:cygwin: won't work, I hate cygwin

* Mac OS X: once you install the development tools, just type `make`

* FreeBSD: type `gmake`, probably will have some problems

* other: won't work, don't care

The code works with PF_RING. There are no special build instructions. After

(or before) building this project, follow the PF_RING directions to install.

Run Masscan with the `--pfring` option, and it will go try to use PF_RING 

instead of libpcap. If it can't find the driver (`pf_ring`) or the shared

library (`/usr/lib/libpfring.so`), it'll warn you. For me, even `make install`

didn't install things, so I had to manually install the kernel drivers and

shared library. With the PF_RING-customized driver `ixgbe` on an Intel 10gbps

network card, this program runs at 12-million packets/second.

Linux and Windows (both 64-bit) are what I use every day, so that's what's

likely to work best. If you are having some problem on another platform,

try going back a version or two.

## PF_RING

Because of Linux kernel overhead, the transmit rate is limited to about

2 million packets/second. To go faster, a "zero-overhead" driver is needed

that bypasses the Linux kernel. One such driver is known as PF_RING DNA. Using

this driver, the code can run at 10 million packets/second. And really, it's

only that slow because since I don't have an Internet connection that fast,

I haven't had a time to optimize it further. I'm pretty sure I coiuld get 

20 million packets/second with a minor amount of tuning.

The PF_RING drivers must be installed separately. You need to install the

`pf_ring.ko` driver, as well as replace the `ixgbe.ko` driver for Intel

cards. You need the shared library `/usr/lib/libpfring.so` in the

proper directory. You probably need to reconfigure things so that the 

drivers install automatically on bootup.

# Regression testing

As for `masscan`, no special build instructions are needed. Indeed, you

can use a binary built before the installation of any PF_RING files. The

program will automatically detect if PF_RING is available and use it. You

can force the issue with the `--pfring` command-line option, which will 

force `masscan` to fail if it can't use PF_RING, and print diagnostic 

information why.

## Regression testing

The project contains a built-in self-test:

an online test, where a second program listens on the network to verify

that what's transmitted is the same thing that was specified to be sent.

To test performance, run something like the following:

	$ bin/masscan 0.0.0.0/4 -p80 --rate 100000000 --router-mac 66-55-44-33-22-11

By setting a bogus MAC address for the local router, the packets won't

go anywhere. This will benchmark how fast the program will run on the 

local system, and will also stress test the local switch.

# Usage

dumps the current configuration and exits. This ouput can be used as input back

into the program:

	# masscan masscan -p80,8000-8100 10.0.0.0/8 --echo > xxx.conf

	# masscan -p80,8000-8100 10.0.0.0/8 --echo > xxx.conf

	# masscan -c xxx.conf --rate 1000

# Comparison with Nmap

## How to scan the entire Internet

The program is designed to scan everything. Therefore, you can do something

like the following:

	# masscan 0.0.0.0/0 -p0-65535

This actually won't work, warning you that you don't have any `--exclude`

ranges defined. That's because indiscriminate scanning of the entire Internet

quickly gets your IP address on ban lists, causing your IP address to get

filtered before you complete the scan. Thus, but excluding the ranges of

people who don't want to be scanned, you can avoid such bans. I hate it when

I do this accidentally, so I've put this warning mechanism in to prevent

accidental mistakes when scanning any range larger than a billion addresses.

Therefore, what your command will really look like is the following:

	# masscan 0.0.0.0/0 -p0-65535 --excludefile exclude.txt

But this just prints the results to the command-line. You probably want them

saved to a file instead. Therefore, you want something like:

	# masscan 0.0.0.0/0 -p0-65535 --excludefile exclude.txt -oX scan.xml

This saves the results in an XML file, allowing you to easily dump the

results in a database or something.

But, this only goes at the default rate of 100 packets/second, which will

take forever to scan the Internet. You need to speed it up as so:

	# masscan 0.0.0.0/0 -p0-65535 --excludefile exclude.txt -oX scan.xml --max-rate 100000

This increases the rate to 100,000 packets/second, which will scan the

entire Internet (minus excludes) in about 10 hours per port (or 655,360 hours

if scanning all ports).

The thing to notice about this command-line is that these are all `nmap`

compatible options. In addition, "invisible" options compatible with `nmap`

are also set for you: `-sS -Pn -n --randomize-hosts --send-eth`. Likewise,

the format of the XML file is inspired by `nmap`. There are, of course, a

lot of minor differences, because the *asynchronous* nature of the program

leads to a fundamentally different approach to the problem.

The above command-line is a bit cumbersome. Instead of putting everything

on the command-line, it can be stored in a file instead. The above settings

would look like this:

	# My Scan

	rate =  100000.00

	output-format = xml

	output-status = all

	output-filename = scan.xml

	ports = 0-65535

	range = 0.0.0.0-255.255.255.255

	excludefile = exclude.txt

To use this configuration file, use the `-c`:

	# masscan -c myscan.conf

This also makes things easier when you repeat a scan.

## Comparison with Nmap

Where reasonable, every effort has been taken to make the program familiar

to `nmap` users, even though it's fundamentally different. Two important

	# masscan --nmap

# Tips on reading the code

The file `main.c` contains the `main()` function, as you'd expect. Also,

this file contains the main scanning thread that spews packets, as well

as the catching thread that catches responses. This is the core functionality

of the program, everything else is secondary.

# Transmit rate (IMPORTANT!!)

## Transmit rate (IMPORTANT!!)

This program spews out packets very fast. On Windows, or from VMs,

it can do 300,000 packets/second. On a Linux (no virtualization) it'll

do 1.6 million packets-per-second. That's fast enough to melt most networks.

Note that it'll only melt your own network. It randomizes the target

IP addresses so that it shouldn't overwhelm any one network.

IP addresses so that it shouldn't overwhelm any distant network.

By default, the rate is set to 100 packets/second. To increase the rate to

a million use something like "--rate 1000000".

a million use something like `--rate 1000000`.

It's floating point. So if you want one packet ever 10 seconds, use the value

`--rate 0.1`.

# How it works

Here are some notes on the design.

# Design

## Spews out packets asynchronously

This is an *asynchronous* design. In other words, it is to `nmap` what

the `nginx` web-server is to `Apache`. It doesn't keep track of which

packets were sent. Instead, it puts a *syncookie* in the packets it

transmits, so that when it receives a response, it can figure out what

was originally transmitted. This allows the *transmit-thread* to

work completely independently from the *receive-thread*.

This is an **asynchronous** program. That means it has a single thread

that spews out packets indiscriminately without waiting for responses.

Another thread collects the responses.

All asynchronous port scanners share this basic design. Others that use

it are `scanrand`, `unicornscan`, and `ZMap`.

This has lots of subtle consequences. For example, you can't use this

program to scan the local subnet, because it can't ARP targets and 

wait for responses -- that's synchronous thinking.

The major benefit of `masscan` is speed. It can use a *zero-overhead* driver

in order to bypass the kernel. This allows it go at 10 million packets/second,

which is fast enough to scan the entire Internet for one port in about

six minutes. This assumes, of course, that you have an Internet connection

that supports such speeds. Actually, this limitation is purely arbitrary

because we are using only a single transmit-thread. We could create 

multiple transmit/recieve queues, and multiple threads, and run much

faster. This is a purely academic problem, since already the 10 million

rate is faster than networks support.

## Randomization

Also, the code is more portable. It runs on Windows and Macintosh as well

as Linux. This is mostly because Windows and Mac are friendlier development

environments to work from, they are both significantly slower than Linux

in terms of scan speed (though both can reach the 100,000 packets/second

speed, which you don't want to exceed if you want to avoid causing

problems in your network).

## Code Layout

The file `main.c` contains the `main()` function, as you'd expect. It also

contains the `transmit_thread()` and `receive_thread()` functions. These

functions are fairly large, trying to expose all the details you need to

worry about in order to see how the program works.

## Randomization (LCG)

Packets are sent in a random order, randomizing simultaneously the IPv4

address and the port.

16-bit port), but we accomplish this with only a few kilobytes of

memory.

# Authors

This tool created by Robert Graham:

void masscan_usage(void)

{

    printf("usage:\n");

    printf("masscan --echo\n");

    printf(" view default configuration\n");

    printf("masscan -p80,8000-8100 10.0.0.0/8 --rate=10000\n");

    printf(" scan some web ports on 10.x.x.x at 10kpps\n");

    printf("masscan --nmap\n");

    printf(" list all the options (nmap format)\n");

    printf("masscan --echo\n");

    printf(" list all options AND their current settings (non-nmap format)\n");

    printf(" list those options that are compatiable with nmap\n");

    exit(1);

}

            struct Range range;

            range = range_parse_ipv4(ranges, &offset, max_offset);

            if (range.begin == 0 && range.end == 0) {

                fprintf(stderr, "CONF: bad range spec: %s\n", ranges);

            if (range.end < range.begin) {

                fprintf(stderr, "ERROR: bad IP address/range: %s\n", ranges);

                break;

            }

    return 0;

}

void

masscan_help()

{

    printf(

"MASSCAN is a fast port scanner. The primary input parameters are the\n"

"IP addresses/ranges you want to scan, and the port numbers. An example\n"

"is the following, which scans the 10.x.x.x network for web servers:\n"

" masscan 10.0.0.0/8 -p80\n"

"The program auto-detects network interface/adapter settings. If this\n"

"fails, you'll have to set these manually. The following is an\n"

"example of all the parameters that are needed:\n"

" --adapter-ip 192.168.10.123\n"

" --adapter-mac 00-11-22-33-44-55\n"

" --router-mac 66-55-44-33-22-11\n"

"Parameters can be set either via the command-line or config-file. The\n"

"names are the same for both. Thus, the above adapter settings would\n"

"appear as follows in a configuration file:\n"

" adapter-ip = 192.168.10.123\n"

" adapter-mac = 00-11-22-33-44-55\n"

" router-mac = 66-55-44-33-22-11\n"

"All single-dash parameters have a spelled out double-dash equivelent,\n"

"so '-p80' is the same as '--ports 80' (or 'ports = 80' in config file).\n"

"To use the config file, type:\n"

" masscan -c <filename>\n"

"To generate a config-file from the current settings, use the --echo\n"

"option. This stops the program from actually running, and just echoes\n"

"the current configuration instead. This is a useful way to generate\n"

"your first config file, or see a list of parameters you didn't know\n"

"about. I suggest you try it now:\n"

" masscan -p1234 --echo\n");

    exit(1);

}

/***************************************************************************

 * Read the configuration from the command-line.

 * Called by 'main()' when starting up.

         */

        if (argv[i][0] == '-' && argv[i][1] == '-') {

            if (strcmp(argv[i], "--help") == 0)

                masscan_usage();

                masscan_help();

            else {

                char name2[64];

                char *name = argv[i] + 2;

                exit(1);

                return;

            default:

                fprintf(stderr, "unknown option: %s\n", argv[i]);

                LOG(0, "FAIL: unknown option: -%s\n", argv[i]);

                LOG(0, " [hint] try \"--help\"\n");

                LOG(0, " [hint] ...or, to list nmap-compatible options, try \"--nmap\"\n");

                exit(1);

            }

            continue;

        }

        if (!isdigit(argv[i][0])) {

            fprintf(stderr, "FAIL: unknown command-line parameter \"%s\"\n", argv[i]);

            fprintf(stderr, " [hint] did you want \"--%s\"?\n", argv[i]);

            exit(1);

        }

        /* If parameter doesn't start with '-', assume it's an

         * IPv4 range

         */

            );

    }

    if (*r_adapter_ip == 0) {

        fprintf(stderr, "FAIL: failed to detect IP of interface: \"%s\"\n", ifname);

        fprintf(stderr, "FAIL:... try something like \"--adapter-ip 192.168.100.5\"\n");

        fprintf(stderr, "FAIL: failed to detect IP of interface \"%s\"\n", ifname);

        fprintf(stderr, " [hint] did you spell the name correctly?\n");

        fprintf(stderr, " [hint] if it has no IP address, manually set with \"--adapter-ip 192.168.100.5\"\n");

        return -1;

    }

    }

    if (memcmp(adapter_mac, "\0\0\0\0\0\0", 6) == 0) {

        fprintf(stderr, "FAIL: failed to detect MAC address of interface: \"%s\"\n", ifname);

        fprintf(stderr, "FAIL:... try something like \"--adapter-mac 00-11-22-33-44\"\n");

        fprintf(stderr, " [hint] try something like \"--adapter-mac 00-11-22-33-44-55\"\n");

        return -1;

    }

    }

    if (memcmp(router_mac, "\0\0\0\0\0\0", 6) == 0) {

        fprintf(stderr, "FAIL: failed to detect router for interface: \"%s\"\n", ifname);

        fprintf(stderr, "FAIL:... try something like \"--router-mac 66-55-44-33-22-11\"\n");

        fprintf(stderr, " [hint] try something like \"--router-mac 66-55-44-33-22-11\"\n");

        return -1;

    }

    unsigned char router_mac[6];

    int err;

    /*

     * Initialize the task size

     */

    count_ips = rangelist_count(&masscan->targets);

    if (count_ips == 0) {

        LOG(0, "FAIL: target IP address list empty\n");

        LOG(0, " [hint] try something like \"--range 10.0.0.0/8\"\n");

        LOG(0, " [hint] try something like \"--range 192.168.0.100-192.168.0.200\"\n");

        return 1;

    }

    count_ports = rangelist_count(&masscan->ports);

    if (count_ports == 0) {

        LOG(0, "FAIL: no ports were specified\n");

        LOG(0, " [hint] try something like \"-p80,8000-9000\"\n");

        LOG(0, " [hint] try something like \"--ports 0-65535\"\n");

        return 1;

    }

    /* If the IP address range is very big, then require that that the 

     * user apply an exclude range */

    if (count_ips > 1000000000ULL && rangelist_count(&masscan->exclude_ip) == 0) {

        LOG(0, "FAIL: range too big, need confirmation\n");

        LOG(0, " [hint] to prevent acccidents, at least one --exclude must be specified\n");

        LOG(0, " [hint] use \"--exclude 255.255.255.255\" as a simple confirmation\n");

        exit(1);

    }

    /*

     * Turn the adapter on, and get the running configuration

    adapter_port = tcpkt_get_source_port(pkt);

    /*

     * Initialize the task size

     */

    count_ips = rangelist_count(&masscan->targets);

    if (count_ips == 0) {

        fprintf(stderr, "FAIL: no IPv4 ranges were specified\n");

        return 1;

    }

    count_ports = rangelist_count(&masscan->ports);

    if (count_ports == 0) {

        fprintf(stderr, "FAIL: no ports were specified, use \"-p<port>\"\n");

        return 1;

    }

    fprintf(stderr, "Scanning %u hosts [%u port%s/host]\n",

    LOG(0, "Scanning %u hosts [%u port%s/host]\n",

        (unsigned)count_ips, (unsigned)count_ports, (count_ports==1)?"":"s");

    /*

    if (masscan->resume.index && masscan->resume.seed && masscan->lcg.m

        && masscan->lcg.a && masscan->lcg.c) {

        if (masscan->lcg.m != count_ips * count_ports) {

            fprintf(stderr, "FAIL: corrupt resume data\n");

            LOG(0, "FAIL: corrupt resume data\n");

            exit(1);

        } else

            fprintf(stderr, "resuming scan...\n");

            LOG(0, "resuming scan...\n");

    } else {

        masscan->lcg.m = count_ips * count_ports;

        lcg_calculate_constants(

        gmtime_s(&x, &now);

        strftime(buffer, sizeof(buffer), "%Y-%m-%d %H:%M:%S GMT", &x);

        fprintf(stderr, "\nStarting masscan 1.0 (http://github.com/robertdavidgraham/masscan) at %s\n", buffer);

        LOG(0, "\nStarting masscan 1.0 (http://github.com/robertdavidgraham/masscan) at %s\n", buffer);

    }

    fprintf(stderr, " -- forced options: -sS -Pn -n --randomize-hosts -v --send-eth\n");

    fprintf(stderr, "Initiating SYN Stealth Scan\n");

    LOG(0, " -- forced options: -sS -Pn -n --randomize-hosts -v --send-eth\n");

    LOG(0, "Initiating SYN Stealth Scan\n");

    /*

     * Allocate packet buffers for sending

    syn_set_entropy(masscan->seed);

    /* If the IP address range is very big, then require that that the 

     * user apply an exclude range */

    if (rangelist_count(&masscan->targets) > 1000000000ULL

        && rangelist_count(&masscan->exclude_ip) == 0) {

            LOG(0, "FAIL: no --exclude specified\n");

            exit(1);

    }

    /*

     * Apply excludes. People ask us not to scan them, so we maintain a list

int

randlcg_selftest()

{

    int is_success;

    unsigned i;

    int is_success = 0;

    uint64_t m, a, c;

    m = 3015 * 3;

    for (i=0; i<5; i++) {

        a = 0;

        c = 0;

        m += 10 + i;

        lcg_calculate_constants(m, &a, &c, 0);

        is_success = lcg_verify(a, c, m, m);

        if (!is_success) {

            fprintf(stderr, "LCG: randomization failed\n");

            return 1; /*fail*/

    } else {

        return 0; /*success*/

        }

    }

    return 0; /*success*/

}