to cache or not to cache?

[giovanni]

Hi all,

I am working on a workflow which 1) loads a large (43MB) JSON object and then 2) uses Alfred to filter results (with word matching). 

Alfred currently takes ~3 sec to load the object.

 

Here's my question: given that I am not fetching anything over the web or local network, would it still be beneficial (i.e. would make it faster) if I learn how to cache the JSON object? 

Or perhaps I should search via script and only serve the results to Alfred? 

 

Thanks!!

Giovanni

 

Edited November 3, 2020 by giovanni

[deanishe]

21 hours ago, giovanni said:

if I learn how to cache the JSON object?

 

If the JSON is already on disk, what are you planning to cache?

 

21 hours ago, giovanni said:

Or perhaps I should search via script and only serve the results to Alfred? 

 

That will make things much worse: Alfred can search a lot faster than a script.

 

One way or another, you're not going to get great performance with a 43MB JSON file. It's just too much data to load and search quickly.

 

You need to either reduce the amount of data or put it in a proper datastore that’s optimised for searching, like SQLite or MeiliSearch.

[giovanni]

Thanks! That's what I suspected but I wanted to check.

 

I also realized that one can start typing the search string in Alfred while the object is loading, so that also saves some time.

Once loaded, it is pretty fast for subsequent queries.  

 

Thanks for your great help as always!

Giovanni