Mindful About Annoyance

When you’re annoyed with someone is it because they’re actually being annoying, or is it because the voice in your mind is active? Is it because you’re frustrated about something unrelated to the person you’re annoyed with?

Is it true that they aren’t worth paying attention to in that moment? Would it be better for you to get away from them, or give them more attention?