Can I invite just my friends for a game?

Yes. But you need to tell ManageMyMatch who your friends are first. ManageMyMatch calls them "best buddies". Go to My buddies and you'll see a list of everyone you've played over the last year (or since you started). Click the yellow star button against each person to signify they are your best buddy.

You can also choose anyone from the membership list to be included as a best buddy. Conversely, there's a list below your buddy list that shows who has "best buddied" you.

Now, when you set up an invite, one of the options to choose who to invite for friendly matches is a best buddies button (this button has a 3-person icon).

Only people on your best buddies list will be able to take you up on your invite.