Jon Bentley describes a case in which a thousand-line program spent 80 percent of its time in a five-line square-root routine. By tripling the speed of the square-root routine he doubled the speed of the program.
Bentley also reports the case of a team that discovered that half an operating system’s time was spent in a small loop. They rewrote the loop in microcode and made the loop 10 times faster, but it didn’t change the system’s performance — they had rewritten the system’s idle loop.

via Code Complete, First Edition.