To be extra-awesome, I'd invert the logic here and peel_to_tag() only, and then get the object and peel that again, and only if it's not a tag, peel to a commit.
To streamline this, there is reference.peel_to_kind(gix::object::Kind::Tag), and if that fails, one can peel again to a Commit maybe. This method is mutating, so one won't lose progress - if it's not a tag it's a commit (usually), and it will do no work to realise that.

But if you want nice code, this is probably it, and I don't think this is a performance issue either.
Gosh, I shouldn't spend my time with premature performance optimisation!