Have you tried setting up Direct Connect between your Mac and your iPhone? You can do this by tapping the "Add" button in Alfred Remote, then choosing "Direct" in the top right of the connection screen.
When using Direct Connection, we recommend using a static IP address for a more predictable behaviour. You can also use your Mac's hostname (e.g. Veros-Mac.local).
Enter your IP address and the port number. Your IP address is set in OS X (usually automatically), while you can set the port in Alfred's server configuration.
Let me know if you need any help with this
Cheers,
Vero
