html title banner (d7a9db41) · Commits · nimrod / masscan

doc/masscan.8.markdown

+78 −10

Original line number	Diff line number	Diff line
		@@ -7,7 +7,7 @@ masscan <ip addresses/ranges> -p <ports> <options>

		## DESCRIPTION

		masscan is an Internet-scale port scanner, useful for large scal surveys
		masscan is an Internet-scale port scanner, useful for large scale surveys
		of the Internet, or of internal networks. While the default transmit rate
		is only 100 packets/second, it can optional go as fast as 25 million
		packets/second, a rate sufficient to scan the Internet in 3 minutes for
		@@ -27,9 +27,9 @@ one port.
		* `--range <ip/range>`: the same as target range spec described above,
		except as a named parameter instead of an unnamed one.

		* `-p <ports`, `--ports <ports>`: specifies the port(s) to be scanned. A single
		port can be specified, like `-p80`. A range of ports can be specified,
		like `-p 20-25`. A list of ports/ranges can be specified, like
		* `-p <ports`, `--ports <ports>`: specifies the port(s) to be scanned. A
		single port can be specified, like `-p80`. A range of ports can be
		specified, like `-p 20-25`. A list of ports/ranges can be specified, like
		`-p80,20-25`. UDP ports can also be specified, like
		`--ports U:161,U:1024-1100`.

		@@ -38,12 +38,12 @@ one port.
		supported.

		* `--rate <packets-per-second>`: specifies the desired rate for transmitting
		packets. This can be very small numbers, like `0.1` for transmitting packets
		at rates of one every 10 seconds, for very large numbers like 10000000,
		which attempts to transmit at 10 million packets/second. In my experience,
		Windows and can do 250 thousand packets per second, and latest versions of
		Linux can do 2.5 million packets per second. The PF_RING driver is needed
		to get to 25 million packets/second.
		packets. This can be very small numbers, like `0.1` for transmitting
		packets at rates of one every 10 seconds, for very large numbers like
		10000000, which attempts to transmit at 10 million packets/second. In my
		experience, Windows and can do 250 thousand packets per second, and latest
		versions of Linux can do 2.5 million packets per second. The PF_RING driver
		is needed to get to 25 million packets/second.

		* `-c <filename>`, `--conf <filename>`: reads in a configuration file. The
		format of the configuration file is described below.
		@@ -186,6 +186,18 @@ one port.
		`--shard`, `--resume-index`, and `--resume-count` can be useful with
		this feature.

		* `-oX <filename>`: sets the output format to XML and saves the output in the
		given filename. This is equivelent to using the `--output-format` and
		`--output-filename` parameters.

		* `-oB <filename>`: sets the output format to binary and saves the output in
		the given filename. This is equivelent to using the `--output-format` and
		`--output-filename` parameters. The tool `scan2text` can then be used to
		read the binary file. Binary files are mush smaller than their XML
		equivelents, but require a separate step to convert back into XML or
		another readable format.


		## CONFIGURATION FILE FORMAT

		The configuration file uses the same parameter names as on the
		@@ -252,6 +264,62 @@ using the following command-lines:
		# masscan 0.0.0.0/0 -p0-65535 --shard 2/3
		# masscan 0.0.0.0/0 -p0-65535 --shard 3/3

		## SPURIOUS RESETS

		When scanning TCP using the default IP address of your adapter, the built-in
		stack will generate RST packets. This will prevent banner grabbing. There are
		are two ways to solve this. The first way is to create a firewall rule
		to block that port from being seen by the stack. How this works is dependent
		on the operating system, but on Linux this looks something like:

		# iptables -A INPUT -p tcp -i eth0 --dport 61234 -j DROP

		Then, when scanning, that same port must be used as the source:

		# masscan 10.0.0.0/8 -p80 --banners --adapter-port 61234

		An alternative is to "spoof" a different IP address. This IP address must be
		within the range of the local network, but must not otherwise be in use by
		either your own computer or another computer on the network. An example of this
		would look like:

		# masscan 10.0.0.0/8 -p80 --banners --adapter-ip 192.168.1.101

		Setting your source IP address this way is the preferred way of running this
		scanner.


		## ABUSE COMPLAINTS

		This scanner is designed for large-scale surveys, of either an organization,
		or of the Internet as a whole. This scanning will be noticed by those
		monitoring their logs, which will generate complaints.

		If you are scanning your own organization, this may lead to you being fired.
		Never scan outside your local subnet without getting permission from your boss,
		with a clear written declaration of why you are scanning.

		The same applies to scanning the Internet from your employer. This is another
		good way to get fired, as your IT department gets flooded with complaints as
		to why your organization is hacking them.

		When scanning on your own, such as your home Internet or ISP, this will likely
		cause them to cancel your account due to the abuse complaints.

		One solution is to work with your ISP, to be clear about precisely what we are
		doing, to prove to them that we are researching the Internet, not "hacking" it.
		We have our ISP send the abuse complaints directly to us. For anyone that asks,
		we add them to our "--excludefile", blacklisting them so that we won't scan
		them again. While interacting with such people, some instead add us to their
		whitelist, so that their firewalls won't log us anymore (they'll still block
		us, of course, they just won't log that fact to avoid filling up their logs
		with our scans).

		Ultimately, I don't know if it's possible to completely solve this problem.
		Despite the Internet being a public, end-to-end network, you are still
		"guilty until proven innocent" when you do a scan.


		## COMPATIBILITY

		While not listed in this document, a lot of parameters compatible with

src/main.c

+2 −0

Original line number	Diff line number	Diff line
		@@ -719,6 +719,8 @@ receive_thread(void *v)
		/*
		* cleanup
		*/
		if (tcpcon)
		tcpcon_destroy_table(tcpcon);
		dedup_destroy(dedup);
		output_destroy(out);
		if (pcapfile)

src/proto-banner1.c

+18 −12

Original line number	Diff line number	Diff line
		@@ -120,22 +120,28 @@ banner1_create(void)
		patterns[i].is_anchored);
		smack_compile(b->smack);

		/*
		* These match HTTP Header-Field: names
		*/
		b->http_fields = smack_create("http", SMACK_CASE_INSENSITIVE);
		for (i=0; http_fields[i].pattern; i++)
		smack_add_pattern(
		b->http_fields,
		http_fields[i].pattern,
		http_fields[i].pattern_length,
		http_fields[i].id,
		http_fields[i].is_anchored);
		smack_compile(b->http_fields);
		http_init(b);

		return b;
		}

		/***************************************************************************
		***************************************************************************/
		void
		banner_append(const void *vsrc, size_t src_len,
		void vbanner, unsigned banner_offset, size_t banner_max)
		{
		const unsigned char src = (const unsigned char )vsrc;
		unsigned char banner = (unsigned char )vbanner;
		size_t i;

		for (i=0; i<src_len; i++) {
		if (*banner_offset < banner_max)
		banner[(*banner_offset)++] = src[i];
		}
		}


		/***************************************************************************
		***************************************************************************/
		void

src/proto-banner1.h

+4 −0

Original line number	Diff line number	Diff line
		@@ -19,6 +19,7 @@ struct Banner1
		{
		struct SMACK *smack;
		struct SMACK *http_fields;
		struct SMACK *html_fields;
		};

		struct Patterns {
		@@ -41,6 +42,9 @@ banner1_parse(
		const unsigned char *px, size_t length,
		char banner, unsigned banner_offset, size_t banner_max);

		void
		banner_append(const void src, size_t src_len, void banner, unsigned *banner_offset, size_t banner_max);

		/**
		* Test the banner protocol-parsing system by reading
		* in a capture file

src/proto-http.c

+133 −10

Original line number	Diff line number	Diff line
		#include "proto-http.h"
		#include "proto-banner1.h"
		#include "smack.h"
		#include <ctype.h>
		@@ -6,18 +7,97 @@
		enum {
		HTTPFIELD_INCOMPLETE,
		HTTPFIELD_SERVER,
		HTTPFIELD_CONTENT_LENGTH,
		HTTPFIELD_CONTENT_TYPE,
		HTTPFIELD_VIA,
		HTTPFIELD_LOCATION,
		HTTPFIELD_UNKNOWN,
		HTTPFIELD_NEWLINE,
		};
		struct Patterns http_fields[] = {
		{"Server:", 7, HTTPFIELD_SERVER, SMACK_ANCHOR_BEGIN},
		//{"Content-Length:", 15, HTTPFIELD_CONTENT_LENGTH, SMACK_ANCHOR_BEGIN},
		//{"Content-Type:", 13, HTTPFIELD_CONTENT_TYPE, SMACK_ANCHOR_BEGIN},
		{"Via:", 4, HTTPFIELD_VIA, SMACK_ANCHOR_BEGIN},
		{"Location:", 9, HTTPFIELD_LOCATION, SMACK_ANCHOR_BEGIN},
		{":", 1, HTTPFIELD_UNKNOWN, 0},
		{"\n", 1, HTTPFIELD_NEWLINE, 0},
		{0,0,0,0}
		};
		enum {
		HTML_INCOMPLETE,
		HTML_TITLE,
		HTML_UNKNOWN,
		};
		struct Patterns html_fields[] = {
		{"<Title", 6, HTML_TITLE, 0},
		{0,0,0,0}
		};


		/*****************************************************************************
		*****************************************************************************/
		static void
		field_name(void banner, unsigned banner_offset, size_t banner_max, unsigned id, struct Patterns *http_fields)
		{
		unsigned i;
		if (id == HTTPFIELD_INCOMPLETE)
		return;
		if (id == HTTPFIELD_UNKNOWN)
		return;
		if (id == HTTPFIELD_NEWLINE)
		return;
		for (i=0; http_fields[i].pattern; i++) {
		if (http_fields[i].id == id) {
		if (*banner_offset != 0)
		banner_append("\n", 1, banner, banner_offset, banner_max);
		banner_append(http_fields[i].pattern
		+ ((http_fields[i].pattern[0]=='<')?1:0), /* bah. hack. ugly. */
		http_fields[i].pattern_length
		- ((http_fields[i].pattern[0]=='<')?1:0), /* bah. hack. ugly. */
		banner,
		banner_offset,
		banner_max);
		return;
		}
		}
		}

		/*****************************************************************************
		* Initialize some stuff that's part of the HTTP state-machine-parser.
		*****************************************************************************/
		void
		http_init(struct Banner1 *b)
		{
		unsigned i;

		/*
		* These match HTTP Header-Field: names
		*/
		b->http_fields = smack_create("http", SMACK_CASE_INSENSITIVE);
		for (i=0; http_fields[i].pattern; i++)
		smack_add_pattern(
		b->http_fields,
		http_fields[i].pattern,
		http_fields[i].pattern_length,
		http_fields[i].id,
		http_fields[i].is_anchored);
		smack_compile(b->http_fields);

		/*
		* These match HTML <tag names
		*/
		b->html_fields = smack_create("html", SMACK_CASE_INSENSITIVE);
		for (i=0; html_fields[i].pattern; i++)
		smack_add_pattern(
		b->html_fields,
		html_fields[i].pattern,
		html_fields[i].pattern_length,
		html_fields[i].id,
		html_fields[i].is_anchored);
		smack_compile(b->html_fields);

		}

		/***************************************************************************
		* BIZARRE CODE ALERT!
		@@ -51,7 +131,9 @@ banner_http( struct Banner1 *banner1,
		FIELD_NAME,
		FIELD_COLON,
		FIELD_VALUE,

		CONTENT,
		CONTENT_TAG,
		CONTENT_FIELD
		};

		state2 = (state>>16) & 0xFFFF;
		@@ -87,7 +169,8 @@ banner_http( struct Banner1 *banner1,
		if (px[i] == '\r')
		break;
		else if (px[i] == '\n') {
		state = STATE_DONE;
		state2 = 0;
		state = CONTENT;
		break;
		} else {
		state2 = 0;
		@@ -102,12 +185,16 @@ banner_http( struct Banner1 *banner1,
		banner1->http_fields,
		&state2,
		px, &i, (unsigned)length);
		i--;
		if (id == HTTPFIELD_NEWLINE) {
		state2 = 0;
		state = FIELD_START;
		} else if (id == SMACK_NOT_FOUND)
		; /* continue here */
		else if (id == HTTPFIELD_UNKNOWN) {
		/* Oops, at this point, both ":" and "Server:" will match.
		* Therefore, we need to make sure ":" was found, and not
		* a known field like "Server:" */
		size_t id2;

		id2 = smack_next_match(banner1->http_fields, &state2);
		@@ -116,7 +203,7 @@ banner_http( struct Banner1 *banner1,

		state = FIELD_COLON;
		} else
		state = STATE_DONE;
		state = FIELD_COLON;
		break;
		case FIELD_COLON:
		if (px[i] == '\n') {
		@@ -125,6 +212,7 @@ banner_http( struct Banner1 *banner1,
		} else if (isspace(px[i])) {
		break;
		} else {
		field_name(banner, banner_offset, banner_max, id, http_fields);
		state = FIELD_VALUE;
		/* drop down */
		}
		@@ -136,13 +224,48 @@ banner_http( struct Banner1 *banner1,
		state = FIELD_START;
		break;
		}
		if (id == HTTPFIELD_SERVER) {
		if (*banner_offset < banner_max) {
		banner[(*banner_offset)++] = px[i];
		switch (id) {
		case HTTPFIELD_SERVER:
		case HTTPFIELD_LOCATION:
		case HTTPFIELD_VIA:
		banner_append(&px[i], 1, banner, banner_offset, banner_max);
		break;
		case HTTPFIELD_CONTENT_LENGTH:
		if (isdigit(px[i]&0xFF)) {
		; /todo: add content length parsing /
		} else {
		id = 0;
		}
		break;
		}
		break;
		case CONTENT:
		id = smack_search_next(
		banner1->html_fields,
		&state2,
		px, &i, (unsigned)length);
		i--;
		if (id != SMACK_NOT_FOUND) {
		field_name(banner, banner_offset, banner_max, id, html_fields);
		banner_append(":", 1, banner, banner_offset, banner_max);
		state = CONTENT_TAG;
		}
		break;
		case CONTENT_TAG:
		for (; i<length; i++) {
		if (px[i] == '>') {
		state = CONTENT_FIELD;
		break;
		}
		}
		break;
		case CONTENT_FIELD:
		if (px[i] == '<')
		state = CONTENT;
		else {
		banner_append(&px[i], 1, banner, banner_offset, banner_max);
		}
		break;

		case STATE_DONE:
		default:
		i = (unsigned)length;