That's what happens for me too.
To be clear, when I start typing a name, the live Alfred results start coming in with "View Contact in Alfred" below each contact. But when I hit enter, or hit the command combination for the contacts further down the live list, it opens Apple's Contacts app to the last viewed contact in that app.
Alfred 5.0.3 [2087] on MacOS 11.7