I've installed DSM4.1 2668 on my AD2700-ITX without any problem. I use a 2G usb stick as boot media.
You may try the following steps:
1. restore to factory default.
2. set storage configuration to AHCI
3. Turn off all unnecessary items, such as parallel port, serial port etc.
I hope it helps. Do not use 4G or above as boot, I encountered some errors and I dont know why.