@jayhlee These are two different modes;
In Alfred's default results, Cmd + number actions the result (e.g. if it's an app, it'll launch the app, if it's a folder, it opens the folder, and if it's a custom search, it performs the search in your default browser)
In File Navigation mode (which is the mode you enter with the Previous Path hotkey), Enter navigates into the folder
As such, the preferences under File Search > Navigation only apply to the File Navigation mode.
If you want to navigate into a folder from Default Results, you can use Cmd + down to enter File Navigation mode and browse the contents of a folder.