HP Tru64 UNIX and TruCluster Server Version 5.1B-5 Patch Summary and Release Notes (March 2009)

3.2.1.45 Protection Against Buffer Overflow Exploitation Added
This kit provides a security feature to prevent the execution of instructions that reside
in heap or other data areas of process memory. The result is additional protection
against buffer overflow exploits. This feature is similar in concept to Tru64 UNIX
executable stack protection.
This feature is implemented as a dynamic sysconfig tunable variable,
executable_data, in the proc subsystem. The supported settings allow system
administrators to cause requests from privileged processes for writable and executable
memory to fail, or to be treated as a request for writable memory, and to optionally
generate a message when such a request occurs.
In a buffer overflow exploitation, an attacker feeds a privileged program an
unexpectedly large volume of carefully constructed data through inputs such as
command line arguments and environment variables. If the program is not coded
defensively, the attacker can overwrite areas of memory adjacent to the buffer.
Depending upon the location of the buffer (stack, heap, data area), the attacker can
deceive these programs into executing malicious code that takes advantage of the
program's privileges or alter a security-sensitive program variable to redirect program
flow.
With some expertise, such an attack can be used to gain root access to the system.
Enabling the executable_data tunable changes a potential system compromise into,
at worst, a denial-of-service attack. A vulnerable program may still contain a buffer
overflow, but an exploit that writes an instruction stream into the buffer and attempts
to transfer control to those instructions will fail, because memory protection will prohibit
instruction execution from that area of memory.
Many applications never execute from the memory even though they unnecessarily
request write-execute memory directly or as a result of an underlying function acting
on their behalf. By substituting writable memory for the requested write-execute
memory, the executable_data tunable allows such applications to benefit from the
additional protection without requiring application modification. See sys_attrs_proc(5)
for more information.
Before enabling executable_data (changing it from the default value of 0), you must
run the /usr/sbin/javaexecutedata script. Otherwise, privileged Java™
applications will fail in unpredictable ways. See javaexecutedata(8) for more information.
60 Tru64 UNIX Patches