Essentially, nothing. Both define areas and tell After Effects what to display on a layer, and to what amount.
I use masks primarily when building effects in program and rotoscoping. Mattes I use more in compositing and blending the multi-elements for plates and such.