[Note: Don't forget: I'll be on The Late Late Show with Craig Ferguson tonight!]
OK, so if you're still scratching your head over my earlier mathified Leap Day explanation, then here's a simpler one by Minute Physics that nonetheless hits all the high points:
Between my math and those animations, you've got it all now, right?
I sometimes wonder if, in the far future when we can terraform planets, we won't adjust every planet's day length to divide evenly into its year. That might be easier than adjusting the calendar!