Speaker sensitivity and audio quality

Is there a reason all speakers don't have high efficiency? Does high efficiency speakers suffer any tradeoffs?