Strong as the work family may be, HR and clients are particularly ruthless. To them, you're simply a number in the machine.
Late swing and graveyard workers get the shaft. All the fun things never happen if you're a barebones team that just holds the door open at night.
Additionally, if you're a bad agent - obviously, you'll be out of a job. If you're an exceptional agent - you won't move up very quickly. If you help pad their metrics that well, you will never advance.