Merge branch 'master' of https://github.com/robertdavidgraham/masscan

## Getting output

The are two primary formats for output. The first is XML, which products

fairly large files, but is easy to import into anything. Just use the

parameter `-oX <filename>`. Or, use the parameters `--output-format xml` and

`--output-filename <filename>`.

The are five primary formats for output. 

1. xml: The default option also prodces fairly large files, but is easy 

	to import into anything. Just use the parameter `-oX <filename>`. 

	Or, use the parameters `--output-format xml` and `--output-filename <filename>`.

The second is the binary format. This produces much smaller files, so that

2. binary: This is the masscan builtin format. This produces much smaller files, so that

when I scan the Internet my disk doesn't fill up. They need to be parsed,

though. The command line option `--readscan` will read binary scan files.

Using `--readscan` with the `-oX` option will produce a XML version of the 

results file.

3. grepable: This is an implementation of the Nmap -oG

output and can be easily parsed by command-line tools. Just use the

parameter `-oG <filename>`. Or, use the parameters `--output-format grepable` and

`--output-filename <filename>`.

4. json: This saves the results in a json format. Just use the

parameter `-oJ <filename>`. Or, use the parameters `--output-format json` and

`--output-filename <filename>`.

5. list: This is a simple list with one host and port pair 

per line. Just use the parameter `-oL <filename>`. Or, use the parameters 

`--output-format list` and `--output-filename <filename>`. The format is:

	```

	<port state> <protocol> <port number> <IP address> <POSIX timestamp>  

	open tcp 80 XXX.XXX.XXX.XXX 1390380064

	```	

## Comparison with Nmap

.\" generated with Ronn/v0.7.3

.\" http://github.com/rtomayko/ronn/tree/0.7.3

.

.TH "MASSCAN" "8" "September 2013" "" ""

.TH "MASSCAN" "8" "January 2014" "" ""

.

.SH "NAME"

\fBmasscan\fR \- Fast scan of the Internet

\fB\-e <ifname>\fR, \fB\-\-adapter <ifname>\fR: use the named raw network interface, such as "eth0" or "dna1"\. If not specified, the first network interface found with a default gateway will be used\.

.

.IP "\(bu" 4

\fB\-\-adapter\-ip <ip\-address>\fR: send packets using this IP address\. If not specified, then the first IP address bound to the network interface will be used\.

\fB\-\-adapter\-ip <ip\-address>\fR: send packets using this IP address\. If not specified, then the first IP address bound to the network interface will be used\. Instead of a single IP address, a range may be specified\. NOTE: The size of the range must be an even power of 2, such as 1, 2, 4, 8, 16, 1024 etc\. addresses\.

.

.IP "\(bu" 4

\fB\-\-adapter\-port <ip\-address>\fR: send packets using this port number as the source\. If not specified, a random port will be chosen in the range 40000 through 60000\. This port should be filtered by the host firewall (like iptables) to prevent the host network stack from interfering with arriving packets\.

\fB\-\-adapter\-port <port>\fR: send packets using this port number as the source\. If not specified, a random port will be chosen in the range 40000 through 60000\. This port should be filtered by the host firewall (like iptables) to prevent the host network stack from interfering with arriving packets\. Instead of a single port, a range can be specified, like \fB40000\-40003\fR\. NOTE: The size of the range must be an even power of 2, such as the example above that has a total of 4 addresses\.

.

.IP "\(bu" 4

\fB\-\-adapter\-mac <mac\-address>\fR: send packets using this as the source MAC address\. If not specified, then the first MAC address bound to the network interface will be used\.

\fB\-\-nmap\-payloads <filename>\fR: read in a file in the same format as the nmap file \fBnmap\-payloads\fR\. This contains UDP payload, so that we can send useful UDP packets instead of empty ones\. Similar to \fB\-\-pcap\-payloads\fR\.

.

.IP "\(bu" 4

\fB\-\-open\-only\fR: report only open ports, not closed ports\.

.

.IP "\(bu" 4

\fB\-\-output\-format <fmt>\fR: indicates the format of the output file, which can be \fBxml\fR or \fBbinary\fR\. The option \fB\-\-output\-filename\fR must be specified\.

\fB\-\-http\-user\-agent <user\-agent>\fR: replaces the existing user\-agent field with the indicated value when doing HTTP requests\.

.

.IP "\(bu" 4

\fB\-\-output\-filename <filename>\fR: the file which to save results to\. If the parameter \fB\-\-output\-format\fR is not specified, then the default of \fBxml\fR will be used\.

\fB\-\-open\-only\fR: report only open ports, not closed ports\.

.

.IP "\(bu" 4

\fB\-\-pcap <filename>\fR: saves received packets (but not transmitted packets) to the libpcap\-format file\.

\fB\-\-rotate\-dir <directory>\fR: when rotating the file, this specifies which directory to move the file to\. A useful directory is \fB/var/log/masscan\fR\.

.

.IP "\(bu" 4

\fB\-\-seed <integer>\fR: an integer that seeds the random number generator\. Using a different seed will cause packets to be sent in a different random order\. Instead of an integer, the string \fBtime\fR can be specified, which seeds using the local timestamp, automatically generating a differnet random order of scans\.

\fB\-\-seed <integer>\fR: an integer that seeds the random number generator\. Using a different seed will cause packets to be sent in a different random order\. Instead of an integer, the string \fBtime\fR can be specified, which seeds using the local timestamp, automatically generating a differnet random order of scans\. If no seed specified, \fBtime\fR is the default\.

.

.IP "\(bu" 4

\fB\-\-regress\fR: run a regression test, returns \'0\' on success and \'1\' on failure\.

\fB\-sL\fR: this doesn\'t do a scan, but instead creates a list of random addresses\. This is useful for importing into other tools\. The options \fB\-\-shard\fR, \fB\-\-resume\-index\fR, and \fB\-\-resume\-count\fR can be useful with this feature\.

.

.IP "\(bu" 4

\fB\-oX <filename>\fR: sets the output format to XML and saves the output in the given filename\. This is equivelent to using the \fB\-\-output\-format\fR and \fB\-\-output\-filename\fR parameters\.

\fB\-\-interactive\fR: show the results in realtime on the console\. It has no effect if used with \-\-output\-format or \-\-output\-filename\.

.

.IP "\(bu" 4

\fB\-\-output\-format <fmt>\fR: indicates the format of the output file, which can be \fBxml\fR, \fBbinary\fR, \fBgrepable\fR, \fBlist\fR, or \fBJSON\fR\. The option \fB\-\-output\-filename\fR must be specified\.

.

.IP "\(bu" 4

\fB\-\-output\-filename <filename>\fR: the file which to save results to\. If the parameter \fB\-\-output\-format\fR is not specified, then the default of \fBxml\fR will be used\.

.

.IP "\(bu" 4

\fB\-oB <filename>\fR: sets the output format to binary and saves the output in the given filename\. This is equivelent to using the \fB\-\-output\-format\fR and \fB\-\-output\-filename\fR parameters\. The option \fB\-\-readscan\fR can then be used to read the binary file\. Binary files are mush smaller than their XML equivelents, but require a separate step to convert back into XML or another readable format\.

.

.IP "\(bu" 4

\fB\-\-readscan <filename>\fR: reads the binary scan results and displays to console\. If used with \fB\-oX a XML version of the binary file will be created\.

\fB\-oX <filename>\fR: sets the output format to XML and saves the output in the given filename\. This is equivelent to using the \fB\-\-output\-format xml\fR and \fB\-\-output\-filename\fR parameters\.

.

.IP "\(bu" 4

\fB\-oG <filename>\fR: sets the output format to grepable and saves the output in the given filename\. This is equivelent to using the \-\-output\-format grepable and \-\-output\-filename parameters\.

.

.IP "\(bu" 4

\fB\-oJ <filename>\fR: sets the output format to JSON and saves the output in the given filename\. This is equivelent to using the \-\-output\-format json and \-\-output\-filename parameters\.

.

.IP "\(bu" 4

\fB\-oL <filename>\fR: sets the output format to a simple list format and saves the output in the given filename\. This is equivelent to using the \-\-output\-format list and \-\-output\-filename parameters\.

.

.IP "\(bu" 4

\fB\-\-readscan <binary\-files>\fR: reads the files created by the \fB\-oB\fR option from a scan, then outputs them in one of the other formats, depending on command\-line parameters\. In other words, it can take the binary version of the output and convert it to an XML or JSON format\.

.

.IP "" 0

.

.P

You should be able to import the XML into databases and such\.

.

.P

The following example reads a binary scan results file called bin\-test\.scan and prints results to console\.

.

.IP "" 4

.

.nf

# masscan \-\-readscan bin\-test\.scan

.

.fi

.

.IP "" 0

.

.P

The following example reads a binary scan results file called bin\-test\.scan and creates an XML output file called bin\-test\.xml\.

.

.IP "" 4

.

.nf

# masscan \-\-readscan bin\-test\.scan \-oX bin\-test\.xml

.

.fi

.

.IP "" 0

.

.SH "ADVANCED EXAMPLES"

Let\'s say that you want to scan the entire Internet and spread the scan across three machines\. Masscan would be launched on all three machines using the following command\-lines:

.

  * `--open-only`: report only open ports, not closed ports.

  * `--output-format <fmt>`: indicates the format of the output file, which

    can be `xml` or `binary`. The option `--output-filename` must be

	specified.

  * `--output-filename <filename>`: the file which to save results to. If

    the parameter `--output-format` is not specified, then the default of

	`xml` will be used.

  * `--pcap <filename>`: saves received packets (but not transmitted

    packets) to the libpcap-format file.

	`--shard`, `--resume-index`, and `--resume-count` can be useful with

	this feature.

  * `-oX <filename>`: sets the output format to XML and saves the output in the

    given filename. This is equivelent to using the `--output-format` and

    `--output-filename` parameters.

  * `--interactive`: show the results in realtime on the console. It has 

    no effect if used with --output-format or --output-filename.		

  * `--output-format <fmt>`: indicates the format of the output file, which

    can be `xml`, `binary`, `grepable`, `list`, or `JSON`. The 

	option `--output-filename` must be specified.

  * `--output-filename <filename>`: the file which to save results to. If

    the parameter `--output-format` is not specified, then the default of 

	`xml` will be used.

  * `-oB <filename>`: sets the output format to binary and saves the output in

    the given filename. This is equivelent to using the `--output-format` and

    equivelents, but require a separate step to convert back into XML or

    another readable format.

  * `-oX <filename>`: sets the output format to XML and saves the output in the

    given filename. This is equivelent to using the `--output-format xml` and

    `--output-filename` parameters.

  * `-oG <filename>`: sets the output format to grepable and saves the output 

	  in the given filename. This is equivelent to using the --output-format grepable 

	  and --output-filename parameters.

  * `-oJ <filename>`: sets the output format to JSON and saves the output in 

	  the given filename. This is equivelent to using the --output-format json 

	  and --output-filename parameters.

  * `-oL <filename>`: sets the output format to a simple list format and saves 

	  the output in the given filename. This is equivelent to using 

	  the --output-format list and --output-filename parameters.

  *  `--readscan <binary-files>`: reads the files created by the `-oB` option

    from a scan, then outputs them in one of the other formats, depending

    on command-line parameters. In other words, it can take the binary

#include "templ-payloads.h"

#include "templ-port.h"

#include "crypto-base64.h"

#include "script.h"

#include <ctype.h>

#include <limits.h>

    printf(" scan some web ports on 10.x.x.x at 10kpps\n");

    printf("masscan --nmap\n");

    printf(" list those options that are compatiable with nmap\n");

    printf("masscan -p80 10.0.0.0/8 --banners -oB <filename>\n");

    printf(" save results of scan in binary format to <filename>\n");

    printf("masscan --open --banners --readscan <filename> -oX <savefile>\n");

    printf(" read binary scan results in <filename> and save them as xml in <savefile>\n");

    exit(1);

}

"  -n: Never do DNS resolution (default)\n"

"SCAN TECHNIQUES:\n"

"  -sS: TCP SYN (always on, default)\n"

"SERVICE/VERSION DETECTION:\n"

"  --banners: get the banners of the listening service if available. The\n"

"    default timeout for waiting to recieve data is 30 seconds.\n"

"PORT SPECIFICATION AND SCAN ORDER:\n"

"  -p <port ranges>: Only scan specified ports\n"

"    Ex: -p22; -p1-65535; -p 111,137,80,139,8080\n"

"TIMING AND PERFORMANCE:\n"

"  --max-rate <number>: Send packets no faster than <number> per second\n"

"  --connection-timeout <number>: time in seconds a TCP connection will\n"

"    timeout while waiting for banner data from a port.\n"

"FIREWALL/IDS EVASION AND SPOOFING:\n"

"  -S/--source-ip <IP_Address>: Spoof source address\n"

"  -e <iface>: Use specified interface\n"

"  --ttl <val>: Set IP time-to-live field\n"

"  --spoof-mac <mac address/prefix/vendor name>: Spoof your MAC address\n"

"OUTPUT:\n"

"  -oL/-oJ <file>: Output scan in List or JSON format, respectively,\n"

"     to the given filename.\n"

"  --output-format <format>: Sets output to binary/list/json/grepable/xml\n"

"  --output-file <file>: Write scan results to file. If --output-format is\n"

"     not given default is xml\n"

"  -oL/-oJ/-oG/-oB/-oX <file>: Output scan in List/JSON/Grepable/Binary/XML format,\n"

"     respectively, to the given filename. Shortcut for\n"

"     --output-format <format> --output-file <file>\n"

"  -v: Increase verbosity level (use -vv or more for greater effect)\n"

"  -d: Increase debugging level (use -dd or more for greater effect)\n"

"  --open: Only show open (or possibly open) ports\n"

"  -h: Print this help summary page.\n"

"EXAMPLES:\n"

"  masscan -v -sS 192.168.0.0/16 10.0.0.0/8 -p 80\n"

"  masscan 23.0.0.0/0 -p80 -output-format binary --output-filename internet.scan\n"

"  masscan 23.0.0.0/0 -p80 --banners -output-format binary --output-filename internet.scan\n"

"  masscan --open --banners --readscan internet.scan -oG internet_scan.grepable\n"

"SEE (https://github.com/robertdavidgraham/masscan) FOR MORE HELP\n"

"\n");

}

        while (*p && (p[strlen(p)-1] == '/' || p[strlen(p)-1] == '/'))

            p[strlen(p)-1] = '\0';

    } else if (EQUALS("script", name)) {

        fprintf(stderr, "nmap(%s): unsupported, it's too complex for this simple scanner\n", name);

        if (!script_lookup(value)) {

            fprintf(stderr, "FAIL: script '%s' does not exist\n", value);

            fprintf(stderr, "  hint: most nmap scripts aren't supported\n");

            fprintf(stderr, "  hint: use '--script list' to list available scripts\n");

            exit(1);

        }

        if (masscan->script.name != NULL) {

            if (strcmp(masscan->script.name, value) == 0)

                return; /* ok */

            else {

                fprintf(stderr, "FAIL: only one script supported at a time\n");

                fprintf(stderr, "  hint: '%s' is existing script, '%s' is new script\n",

                        masscan->script.name, value);

                exit(1);

            }

            masscan->script.name = script_lookup(value)->name;

        }

    } else if (EQUALS("scan-delay", name) || EQUALS("max-scan-delay", name)) {

        fprintf(stderr, "nmap(%s): unsupported: we do timing VASTLY differently!\n", name);

        exit(1);

#include "pixie-threads.h"      /* portable threads */

#include "templ-payloads.h"     /* UDP packet payloads */

#include "proto-snmp.h"         /* parse SNMP responses */

#include "proto-ntp.h"          /* parse NTP responses */

#include "templ-port.h"

#include "in-binary.h"          /* covert binary output to XML/JSON */

#include "main-globals.h"       /* all the global variables in the program */

#include "crypto-base64.h"      /* base64 encode/decode */

#include "pixie-backtrace.h"

#include "proto-sctp.h"

#include "script.h"

#include <assert.h>

#include <limits.h>

    time_t now = time(0);

    struct Status status;

    uint64_t min_index = UINT64_MAX;

    struct MassScript *script = NULL;

    memset(parms_array, 0, sizeof(parms_array));

    /*

     * Script initialization

     */

    if (masscan->script.name) {

        unsigned i;

        script = script_lookup(masscan->script.name);

        /* If no ports specified on command-line, grab default ports */

        if (rangelist_count(&masscan->ports) == 0)

            rangelist_parse_ports(&masscan->ports, script->ports, 0);

        /* Kludge: change normal port range to script range */

        for (i=0; i<masscan->ports.count; i++) {

            struct Range *r = &masscan->ports.list[i];

            r->begin = (r->begin&0xFFFF) | Templ_Script;

        }

    }

    /*

     * Initialize the task size

     */

         * scanning. Then, we adjust the template with additional features,

         * such as the IP address and so on.

         */

        parms->tmplset->script = script;

        template_packet_init(

                    parms->tmplset,

                    parms->adapter_mac,

            x += banner1_selftest();

            x += output_selftest();

            x += siphash24_selftest();

            x += ntp_selftest();

            x += snmp_selftest();

            x += payloads_selftest();

            x += blackrock_selftest();