It's possible to "confuse" the bootloader so that it never starts your sketch. In particular, if you send serial data to the board just after it resets (when the bootloader is running), it may think you're talking to it and never quit. In particular, the auto-reset feature on the Diecimila means that the board resets (and the bootloader starts) whenever you open a serial connection to it. To avoid this problem, you should wait for two seconds or so after opening the connection before sending any data. On the NG, the board doesn't reset when you open a serial connection to it, but when it does reset it takes longer - about 8-10 seconds - to timeout.
See the bootloader development page for information on burning a bootloader and other ways to configure a chip.
The bootloader is a small piece of software that we've burned onto the chips that come with your Arduino boards. It allows you to upload sketches to the board without external hardware.
When you reset the Arduino board, it runs the bootloader (if present). The bootloader pulses digital pin 13 (you can connect an LED to make sure that the bootloader is installed). The bootloader then listens for commands or data to arrive from the the computer. Usually, this is a sketch that the bootloader writes to the flash memory on the ATmega168 or ATmega8 chip. Then, the bootloader launches the newly-uploaded program. If, however, no data arrives from the computer, the bootloader launches whatever program was last uploaded onto the chip. If the chip is still "virgin" the bootloader is the only program in memory and will start itself again.
The use of a bootloader allows us to avoid the use of external hardware programmers. (Burning the bootloader onto the chip, however, requires one of these external programmers.)