Script filter - What to do when the JSON str is too long?

[Bernardo_V]

This is a workflow for filtering notes in Tinderbox. I have an applescript that will collect all the notes in the file with the URL and assemble it in a JSON format that Alfred can ingest. 

 

Example:

 

{ "items" : [{
					        "title": "Tinderbox-Acadêmico",
					        "subtitle": "",
					        "arg":  "tinderbox://Tinderbox-Acade%CC%82mico?view=outline+select=1618577108;",
					        },
{
					        "title": "TbxConfig",
					        "subtitle": "",
					        "arg":  "tinderbox://Tinderbox-Acade%CC%82mico?view=outline+select=1617846680;",
					        },
{
					        "title": "Hints",
					        "subtitle": "",
					        "arg":  "tinderbox://Tinderbox-Acade%CC%82mico?view=outline+select=1616867062;",
					        }
            ]}

 

I am using a script filter and the problem is that the whole string is 270.000 characters long, but Alfred truncates it at approx. 200.000 and never displays any result.

 

Is there any way to get this to work and still use Alfred to filter the results?

Edited April 16, 2021 by Bernardo_V

[deanishe]

I'd report it as a bug if I were you. There was a similar limit on STDERR (debugger messages) that was removed several years ago.

 

That said, it wouldn't surprise me if it's a deliberate limit for performance reasons: 200kB is a lot of items.

 

In terms of immediate solutions, you could do the filtering in your Script Filter instead of letting Alfred do it, so you aren't returning as many items to Alfred.

[Mr Pennyworth]

My emoji search workflow has a JSON file with following stats:

Filesize: 2.1 MB

Character Count: 1304946
Line Count: 1

 

Alfred loads it correctly without truncation.
Maybe it is worth investigating whether it is Alfred truncating your JSON file or something else causing the truncation...

[deanishe]

Might be an encoding issue, then? @Bernardo_V Have you tried saving the JSON as ASCII (i.e. escaping all non-ASCII characters).

 

In any case, I'm not sure we can say much more without the actual JSON file that's causing issues.

[Bernardo_V]

28 minutes ago, deanishe said:

Might be an encoding issue, then? @Bernardo_V Have you tried saving the JSON as ASCII (i.e. escaping all non-ASCII characters).

 

I did. The content doesn't seem to matter. Alfred truncates it at the precise same location every time (that is, after 200.094 characters).

 

6 hours ago, deanishe said:

In terms of immediate solutions, you could do the filtering in your Script Filter instead of letting Alfred do it, so you aren't returning as many items to Alfred.

Can you think of any workflow that I could look into to adapt for my situation? I was also thinking that it would be a good idea to build a cache so that the search is not performed every time.

 

3 hours ago, Mr Pennyworth said:

Alfred loads it correctly without truncation.

 Thanks, I will look into it to se if there is an applicable solution for me there.

[deanishe]

2 hours ago, Bernardo_V said:

The content doesn't seem to matter.

 

The fact that Alfred accepts Mr Pennyworth's 10x bigger file suggests that size isn't the issue. Especially as you say you're using AppleScript, which is notoriously poor at generating JSON.

 

Please upload a file that triggers the issue somewhere (Dropbox?) and post a link if you'd like us to help figure out what the issue is.

 

Without that, we're just guessing, which is a waste of everybody's time.

Edited April 17, 2021 by deanishe

[Bernardo_V]

On 4/17/2021 at 11:36 AM, deanishe said:

The fact that Alfred accepts Mr Pennyworth's 10x bigger file suggests that size isn't the issue. Especially as you say you're using AppleScript, which is notoriously poor at generating JSON.

Indeed and that proved to be the case.

 

For future reference, the JSON str will get truncated if it is too long, but this only happens in the debugger window. The problem was actually being caused by linefeeds present inside the some of the fields (despite having rules in place to fix this in my script).

 

Thank you @deanishe and @Mr Pennyworth for your willingness to help.

 

Edited April 18, 2021 by Bernardo_V