I feel the same, it genuinely hurts when I think I let other down (despite they might not feel so).

Compassion is good, but we should probably reserve it for people who are close to us.

For those people who you probably will never meet a second time, close your heart to it.