FYI, I solved this by adding my entire home folder to the Search Scope. I navigated to /Users and then picked my home folder, and Alfred added it as "~"
With that, I didn't need to enter the ~/Library/CloudStorage folder to get results when I search for a file in the Dropbox folder.
I have "MacOS Applications folder and Folders in Home checked, though the later is redundant with ~ as an included path.
This seems to me like an Alfred bug, in that it's excluding everything from the ~/Library, even those things explicitly added, unless the whole home directory is present in the search scope.