HP Tru64 UNIX and TruCluster Server Version 5.1.B-4 Patch Summary and Release Notes (13156)

3.2.2.3 Potential Security Vulnerability Identified
The industry standard TCP specification, RFC793, has a vulnerability in which an attacker can
reset established TCP connections using the TCP RST (Reset) or SYN (Synchronize) flags.
These packets need to have source and destination IP addresses that match the established
connection as well as the same source and destination TCP ports.
The fact that TCP sessions can be reset by sending suitable RST and SYN packets is a design
feature of TCP. According to RFC 793, an RST or SYN attack is only possible when the source IP
address and TCP port can be forged (also called spoofed). In that case TCP sessions, including
Telnet, SSH, SFTP and HTTP may be disconnected without warning. TCP sessions that have
been disconnected can be re-established.
Normally, a TCP SYN packet (request for a new connection) that arrives on a server using a
matching IP address, port number, and matching sequence number for an existing connection
causes a TCP RST packet to be returned to the client. An attacker can guess the proper sequence
number, along with the port and IP addresses, to cause an existing connection to be terminated
with a TCP RST.
When a client is rebooted without closing an old connection to the server, a subsequent attempt
to connect to the server that matches the old connection tuple and sequence number will require
a TCP RST in order to purge the old (stale) connection.
HP has addressed these potential vulnerabilities, called TCP RST attack and TCP SYN attack,
by providing two new kernel tunable variables, tcp_rst_win (TCP RST window) and
tcp_syn_win (TCP SYN window).
These variables mitigate the TCP reset attack by reducing the window sizes in which a TCP
RST/SYN packet will be accepted by the Tru64 UNIX system.
The attributes for these variables are described in a revised sys_attrs_inet(5) reference page
included in this kit.
After the patch kit is installed, you can adjust the variables using the sysconfig and
sysconfigdb commands, as described in the following sections.
3.2.2.3.1 Adjusting the tcp_rst_win Variable
You can adjust the TCP RST window variable, tcp_rst_win, as follows:
# sysconfig -q inet tcp_rst_win
inet:
tcp_rst_win = -1
# sysconfig -r inet tcp_rst_win=2048
tcp_rst_win: reconfigured
# sysconfig -q inet tcp_rst_win
inet:
tcp_rst_win = 2048
# sysconfig -q inet tcp_rst_win /tmp/tcp_rst_win_merge
# sysconfigdb -m -f /tmp/tcp_rst_win_merge inet
# sysconfigdb -l inet
inet:
tcp_rst_win = 2048
3.2.2.3.2 Adjusting the tcp_syn_win Variable
You can adjust the TCP SYN window variable, tcp_syn_win, as follows:
# sysconfig -q inet tcp_syn_win
inet:
3.2 Prior Release Notes 49