When you install ESXi to any device it will create a handful of small partitions for itself and (except on USB devices) another partition using the rest of the disk, formatted as a datastore.
If you are going to dedicate a HD for the datastore, the only practical benefit to install ESXi on a USB drive is that you will save the few GBs it needs for install and a slightly faster boot compared to a normal HD.
My advice in your case, is to install ESXi and its datastore in that dedicated HD, as HD are generally more reliable than USB drives.
As for the type of disk you need, SSD or not will not make a real difference for ESXi and XPEnoboot, as both are fully loaded in memory and small, so, performance wise, we are talking of an initial delay of a few seconds at startup. I made a test once and I THINK (was time ago) it was less than 2 seconds of difference.
However, the other virtual machines that you intend to start will greatly benefit from a SSD as their virtual disks will normally be allocated in the datastore.
Last, any reason for ESXi 5.5 when you have v6 freely available?