Headphone impedance is too low.
If you more or less exclusively use headphones, consider a dedicated headphone amp. Normal amps are designed to deliver their rated loads, not 1/1000 of it, and I doubt you're going to be betting the best sound quality this way.
Good headphone amps range from £20 - £200.
|