I think it's design. Wearing the sword kit without a support strap would look pretty silly because they'd just be floating, prettymuch. (You know, like Don's bo.

.) However, if he's not wearing his sword kit, there's no need to have the strap... maybe it's detachable. It might be difficult to remember when to draw it in if you're in a rush.