Search Apple/iCloud Notes

[sballin]

I've created a workflow to find and open Apple/iCloud notes in Notes.app. Get it on Packal and feel free to open issues/make pull requests on GitHub.

 

Edited December 29, 2019 by sballin

[shortbread]

Thanks for putting this together! I came across a bug in getting this to work, and in fixing it wound up making some modifications ...

 

Mainly I figured out how to extract the note contents from the sqlite db and add them to the JSON match field. The contents are gzipped in the db so I had to decompress and clean them up in order to use them for search matching ... I imagine performance could be a concern if you have a gajillion notes, but it runs very fast for me, pulling about 1000 notes down at a time.

 

Modified python file contents are below - I made some other changes that may or may not interest you, but I can do a PR if you like.

 

#!/usr/bin/python
import sqlite3
import json
import zlib
import re

# Open notes database
home = '/'.join(__file__.split('/')[:3])
conn = sqlite3.connect(
    home + '/Library/Group Containers/group.com.apple.notes/NoteStore.sqlite')
c = conn.cursor()

# Get uuid string required in full id
c.execute("SELECT z_uuid FROM z_metadata")
uuid = str(c.fetchone()[0])

# Get tuples of note title, folder code, snippet, modification date, & id#
# 432 is the zfolder id for 'Recently Deleted'
c.execute("""SELECT t1.ztitle1,t1.zfolder,t1.zsnippet,
    t1.zmodificationdate1,t1.z_pk,t1.znotedata,t2.zdata,t2.z_pk
FROM ziccloudsyncingobject AS t1
INNER JOIN zicnotedata AS t2
ON t1.znotedata = t2.z_pk
WHERE t1.ztitle1 IS NOT NULL AND t1.zfolder IS NOT 432 AND t1.zmarkedfordeletion IS NOT 1""")
matches = c.fetchall()
# Sort by title
matches = sorted(matches, key=lambda m: m[0], reverse=False)

# Get ordered lists of folder codes and folder names
c.execute("""SELECT z_pk,ztitle2
FROM ziccloudsyncingobject
WHERE ztitle2 IS NOT NULL AND zmarkedfordeletion IS NOT 1""")
folderCodes, folderNames = zip(*c.fetchall())

conn.close()

# Alfred results: title = note title, arg = id to pass on, subtitle = folder name, match = the note contents
items = [{"title": m[0],
          "arg": "x-coredata://" + uuid + "/ICNote/p" + str(m[4]),
          "subtitle": folderNames[folderCodes.index(m[1])],
          #  + ("  |  " + m[2] if type(m[2]) is unicode and len(m[2]) > 0 else ""),
          #
          #  decompress gzipped notes from the sqlite database, strip out gobbledygook footers.
          "match": zlib.decompress(m[6], 16+zlib.MAX_WBITS).split('\x1a\x10', 1)[0]}
         for m in matches]

# Do further clean up and additions to the match and subtitle fields.
for i, item in enumerate(items):
    # strip weird characters, title & weird header artifacts,
    # replace line breaks with spaces.
    txt = re.sub('^  ', '', re.sub('\n', ' ', re.sub('^.*\n', ' ', re.sub(r'[\x00-\x08\x0b\x0c\x0e-\x1f\x7f-\xff]', '', items[i]['match']))))
    items[i]['match'] = items[i]['title'] + " " + items[i]['subtitle'] + " " + txt
    items[i]['subtitle'] += "  |  " + txt[:100]

# Custom icons for folder names that start with corresponding emoji
icons = [u'\ud83d\udcd3', u'\ud83d\udcd5',
         u'\ud83d\udcd7', u'\ud83d\udcd8', u'\ud83d\udcd9']
for i in items:
    if any(x in i['subtitle'] for x in icons):
        subtitle = i['subtitle']
        icon = subtitle[:2]
        i['subtitle'] = subtitle[3:]
        i['icon'] = {'type': 'image', 'path': 'icons/' +
                     icon.encode('raw_unicode_escape') + '.png'}

output = {"items": items}
print json.dumps(output, sort_keys=True, indent=4, separators=(',', ': '))

 

Edited July 29, 2018 by shortbread

[sballin]

Thanks for your contribution, this is great! @Jasondm007 was requesting this feature. Seems like people are into the full text search so we could make that the default and "search titles only" as an option.

 

I'm on vacation for a week but will add this when I'm back. You can make a PR if you want and/or I'll make a list of contributors on the github page. 

[shortbread]

PR done ! cheers !

[MaxPetretta]

Any plans to add the option to create a new note with this workflow?  Maybe as a catchall option after any relevant results?  I might give it a go this weekend.

[wkoutre]

macOS 10.13.6

 

FYI, I'm getting:

[2018-08-09 13:55:48][input.scriptfilter] Queuing argument '(null)'

[2018-08-09 13:55:48][input.scriptfilter] Script with argument '(null)' finished

[2018-08-09 13:55:48][ERROR: input.scriptfilter] Code 1: Traceback (most recent call last):

  File "/Users/wkoutre/Library/Application Support/Alfred 3/Alfred.alfredpreferences/workflows/user.workflow.<key>/searchNotes.py", line 35, in <module>

    for m in matches]

TypeError: 'NoneType' object has no attribute '__getitem__'

 

Will look into it when I get a chance, just thought I'd share in the meantime.

Edited August 9, 2018 by wkoutre
Added pic and macOS version

[wkoutre]

In the meantime, I published a barebones JavaScript version: https://github.com/wkoutre/alfred-apple-notes-search/

[sballin]

On 8/2/2018 at 4:27 PM, MaxPetretta said:

Any plans to add the option to create a new note with this workflow?  Maybe as a catchall option after any relevant results?  I might give it a go this weekend.

 

That's a good idea, I'll put it on the to-do list!

 

@wkoutre and @40-02 this bug is now fixed by the PR from @shortbread  and an extra check I added. Packal and Github releases have been updated.

[sballin]

Updated the workflow with some better AppleScript methods, including one that searches note bodies. @deanishe the JSON is being output properly now using Foundation framework methods. Note bodies are obtained from AppleScript in HTML which I parse using regex (against all internet advice, but has been working so far—notes with HTML snippets are going to be less searchable). Let me know if there are better ways to grab the body text.

 

I tried using JXA but found it was slower for me than AppleScript where I'm careful to access large objects as references.

 

I looked into the new note fallback option but this seems difficult for script filters in Alfred, especially when Alfred handles narrowing the search results.

 

[deanishe]

On 8/19/2018 at 6:06 PM, sballin said:

which I parse using regex

 

If you're still using Python, HTMLParser can do a proper job of that.

[40-02]

On 8/10/2018 at 3:47 PM, sballin said:

 

That's a good idea, I'll put it on the to-do list!

 

@wkoutre and @40-02 this bug is now fixed by the PR from @shortbread  and an extra check I added. Packal and Github releases have been updated.

Unfortunately it doesn't work for me:

 

Starting debug for 'Search Notes.app'

[2018-08-23 00:32:04][ERROR: input.scriptfilter] Code 1: Traceback (most recent call last):
  File "/Users/uvaroff/Documents/! temp files archive/tempAlfred/Alfred.alfredpreferences/workflows/user.workflow.9E2B2CCF-C0BF-4FDA-9F5E-D44E798BA6C3/searchNotes.py", line 54, in <module>
    body = zlib.decompress(d[5], 16+zlib.MAX_WBITS).split('\x1a\x10', 1)[0]
zlib.error: Error -3 while decompressing data: incorrect header check
[2018-08-23 00:32:19][ERROR: input.scriptfilter] Code 1: Traceback (most recent call last):
  File "/Users/uvaroff/Documents/! temp files archive/tempAlfred/Alfred.alfredpreferences/workflows/user.workflow.9E2B2CCF-C0BF-4FDA-9F5E-D44E798BA6C3/searchNotes.py", line 54, in <module>
    body = zlib.decompress(d[5], 16+zlib.MAX_WBITS).split('\x1a\x10', 1)[0]
zlib.error: Error -3 while decompressing data: incorrect header check

 

[wkoutre]

@40-02 I made a version in JavaScript: https://github.com/wkoutre/alfred-apple-notes-search/tree/master if you want to try it out. Needs Node installed, other than that, should be good to go.

[sballin]

@40-02 have you tried the keywords a or b instead of n to search?

[40-02]

2 hours ago, sballin said:

@40-02 have you tried the keywords a or b instead of n to search?

Nope! Now it works!!! Thank you!!!

And sorry for absentmindedness

Edited August 23, 2018 by 40-02

[able]

Hell yeah, this is what I've been looking for forever. Thanks for making this @sballin!!

[dfay]

anyone try this on mojave beta?

[Diegobit]

It works for me ?

[Jasondm007]

@sballin Thanks for sharing this workflow. It's great, and has saved me a ton of time.

 

Out of curiosity, are there any small changes that could be made to your script filter so that it only searches for folders (i.e., as a new script filter based on the NS workflow)? Or, alternatively, is it possible to see folders in the results of the standard search (i.e., alongside/within the standard results for the individual notes)?

 

Thanks again!

[sballin]

@Jasondm007 Glad you're finding it useful! I chose to implement your first suggestion. In case you didn't know, the standard search already shows notes with matching folder names.

Edited November 18, 2018 by sballin

[Jasondm007]

Thanks @sballin!! I almost gave up on Notes before this workflow! This thing is a life saver.

 

Sorry if I wasn't clear in my previous post. I meant to ask whether the folders, themselves, could show up as individual results (i.e., (1) among the individual note results or (2) by themselves, if an argument was added to the main script filter)?

 

I ask because there are times when I want to open a folder without knowing which specific note that I might have placed the material in that I'm looking for (i.e., in cases where I don't have a good enough idea of what I'm even looking for). I'd also like to be able to use it as a fallback search for times when the usual workflow isn't returning the note that I am looking for, but I know the folder where it's located. In short, I'm just looking for an easy way to search for, and open, folders. 

 

Thanks again for sharing this workflow! It's great.

[sballin]

Gotcha. I'm hesitant to mix folders and notes in the same search results, which is why I implemented the folder-only search you suggested. You can download the latest version of the workflow to try it.

 

I'm also fairly sure the "fallback" behavior you're describing is currently implemented. If I have a folder named Aardvark containing notes Bob and Charlie, when I do the normal "n" search for Aardvark, the results will be Bob and Charlie.

[Jasondm007]

@sballin Thanks a ton! This feature is incredibly helpful.

 

If you live Stateside, I hope you and your family had a great Thanksgiving! All the best!

 

[Jasondm007]

@sballin I don't mean to hijack this thread, but I have a Notes-related question that I was hoping to bounce off you that is somewhat related to your workflow:

 

Namely, do you know how to create local file link for a specific note in the Notes app (i.e., one that could be inserted - like any other file link - into any text editor)?

 

Since your workflow searches for, and opens specific notes, I thought you might know the answer.

 

While I understand that I can generate a link for a note by using the Share option in the Notes app, I don't want to actually share the notes with other people. Instead, I'd just like to insert links in other notes and other local text editors that I could click to open within the Notes app (i.e., using the standard Add link... option). In short, I was hoping to create links that would operate like Finder (file:///path...) or Evernote's so-called "Classic Note Link" (evernote:///view/path...), but I have no idea how the Notes app stores its notes.

 

Ideally, I was hoping to create workflow that would copy a specific note's link/path to the clipboard based on either:

1. the note that is in the Notes app's frontmost window, OR
2. as an alternative action from your workflow's search results (e.g., using ⌥+↩︎ to copy, to the clipboard, the link of a note selected from your workflow's search results).

 

I suspect both options are incredibly difficult ... so please feel free to ignore this message. In any event, thanks for any help you can lend!!

Edited November 27, 2018 by Jasondm007
typo

[sballin]

@Jasondm007 Nice idea, I've added this feature in version 1.4. Let me know if it works for you. If it asks you to choose a program to open the notes:// links, you probably need to move [workflow directory]/Note Opener/Note Opener.app to /Applications.

[Jasondm007]

@sballin This is amazing!! I can't thank you enough!

 

11 hours ago, sballin said:

@Jasondm007 Nice idea, I've added this feature in version 1.4. Let me know if it works for you. If it asks you to choose a program to open the notes:// links, you probably need to move [workflow directory]/Note Opener/Note Opener.app to /Applications.

 

I was able to get it working just by opening the app in its current location inside of the workflow (and approving the permissions - re: unidentified developer).

 

It works perfectly!!