I only take mine out when I'm offloading and haven't had any problems yet... who knows, maybe keeping it stored in there prevents dust and muck from getting into the connections?
Its ok to leave the p2 cards in. Actually, it will cut down on the wear and tear of putting in and taking out the cards. The only thing I would not leave attached to your hvx is the batteries. Take them off as soon as you are done using them.
Jester is so right there. I just leave mine in as I know where they are. Nothing worse than getting to a gig and not having ya cards. That's something you can leave to the guys shooting tape. YUCK!