Computer security becomes esoteric when dealing with security vulnerabilities at a hardware level. Even the basic execution structure of computers has holes that hackers can exploit. Engineers must continually develop techniques to circumvent hacker attacks. To address memory manipulation exploits such as buffer overruns, engineers developed stack, or address space layout, randomization. This technique makes traditional and repeatable attacks less effective, and protects computers from basic security flaws.
The Call Stack
The generic data structure defined as a “stack” represents a basic list of items in which each added item appends to the end of the list, and each removed item comes from the end of the list in a last-in, first-out order. The call stack of a running program functions much in the same way. Instead of simple data items, however, the call stack organizes references in memory to executing functions in the program.
Buffer Overruns and Memory Vulnerability
One of the disadvantages of using the stack data structure to organize programs is related to how memory is organized. While each running function in a program sits in the call stack, they often sit side-by-side in physical memory. Accordingly, if an attacker overruns a variable buffer by providing too much data for a data structure to handle, for example, then the extra information can overwrite data in adjacent physical memory locations, affecting the flow of program execution.
To address security concerns, engineers developed a memory allocation technique known as stack randomization, or address space layout randomization. Typically, memory is allocated sequentially during program execution. When a program starts, it is allocated the next available memory space in memory. When using stack randomization, the computer determines a pseudo-random location based on a predetermined offset. The frame pointers on the stack are placed by this offset as well. The offset is determined at boot time by the operating system.
Stack Randomization and Security
Because of stack randomization, program data does not reside entirely consecutively in the system’s memory. This presents a problem for hackers attempting to use an attack such as a buffer overrun. While a buffer overrun can occur, the effects could not be predetermined. Before stack randomization, if a hacker knew how much data to use in a buffer overrun, he could use the same exploit on the same program, repeatedly. With randomization, no buffer attack is guaranteed success between each running instance of a program, even on the same computer.