Alfred Markdown Notes

[Acidham]

Markdown Notes help to manage Markdown files in a directory with powerful full text search (supports & and |), tag search or search for todos ( - [ ] or * [ ]) . It also allows to quickly create new notes based on custom templates.

 

I started to write this to replace the ton of Note taking apps that I used a while ago. I spent a lot of time to get this to work and the setup is a bit complex but once you went thru the steps it can indeed replace your note taking app. 

 

The best MD Editor for this purpose is Typora but any other MD editor works as well. 

 

Download at Git as usual: https://github.com/Acidham/alfred-markdown-notes/releases/latest

Edited August 18, 2019 by Acidham

[Acidham]

v 0.9: Some code polish and added OneUpdater to update automatically.

[Acidham]

With v2.0 links to multiple markdown files (File Action) can be generated into a new markdown note e.g. to collect all invoices related to an insurance

 

Download at Git

[cands]

Thanks for the update! This is a quite powerful workflow, and may change my note taking habits. I have one issue, the workflow does not find notes with titles that use any of the Swedish characters å, ä and ö (if I search for a word that includes any of those characters, if I search for other words in the title the note will be found). I would guess other languages with special characters may also be affected. Could this be fixed? 

[Acidham]

53 minutes ago, cands said:

Thanks for the update! This is a quite powerful workflow, and may change my note taking habits. I have one issue, the workflow does not find notes with titles that use any of the Swedish characters å, ä and ö (if I search for a word that includes any of those characters, if I search for other words in the title the note will be found). I would guess other languages with special characters may also be affected. Could this be fixed? 

 

Thank you for reporting! I have the same issue I am German. Luckily I started to work on this today and so far I am able to get this fixed for for german umlauts but Swedish is missing. I need to work on it. 

The current solution is I am replacing e.g. ä with ae for search terms and within the notes content .. the search can then match. Is there a similar mapping in Swedish language?

 

Sorry but unicodes are pain in the ass with Python 2.x!

[deanishe]

7 hours ago, Acidham said:

The current solution is I am replacing e.g. ä with ae

 

It would make more sense to discard all the diacritics during search or make the code Unicode-aware.

 

Using German orthographic rules to ASCII-fy the text is questionable: A German-speaker is going to type "düsseldorf" as their query, not "duesseldorf", and a non-German is extremely unlikely to know the rules of German orthography. And that's assuming the text is German in the first place. Replacing "Motörhead" with "Motoerhead" is just wrong.

 

FWIW, I find you get the best results by stripping all diacritics if the query is ASCII-only, but preserving them if it isn't. And by normalising the Unicode to NFC, unless you're dealing with filepaths, in which case NFD is often better.

 

8 hours ago, Acidham said:

Sorry but unicodes are pain in the ass with Python 2.x!

 

Unicode is a massive pain in general (and Python 3 only takes care of the easy bit for you), but as someone who uses umlauts all the time, you really ought to know it

[Acidham]

@deanishe Agree, therefore I spent the whole Saturday to solve this issue. Over night I came to conclusion to unicodedata.normalize both content of the markdown and search terms and I did some tests so far. It seems to work because both encodings are normalised in the right way. 

 

Discard all was my first idea but I am not happy because it distorts the search and results.

 

I am running some further tests and will release hopefully today.

[Acidham]

Here we go v.2.2, let me know how it goes...

 

New Features

• Added ability to paste tag from tag search into frontmost app

• Added new flag to settings to change search behaviour to run exact match or match any string (exact_match)

Improvements

• Exchanged custom html to markdown converter to pandoc. The custom converter were not able to import a some webpages. Please ensure to install pandoc (the link can be found in mdhelp or README)

• rewrite of the search algorithm due to unicode issues

Bugfixes

• The Search is now unicode savy, means e.g. german umlauts will now be found in md notes

• Improved filename normalisation

Download via Git

Edited January 5, 2020 by Acidham

[cands]

Awesome, thank you very much for the quick and most valuable update! Now it works to search for words with Swedish characters, and it improves the usability a lot. 

[cands]

I have now been using the workflow quite much for a few weeks, it is awesome, thanks again! I have two feature requests I hope could be possible to implement, where the first is most important: 

1. Add a possibility to add multiple folders to the search scope. As far as I understand this is not possible now. For my use case I have several folders with markdown files, which is a structure I built over some time and rebuilding it would break a lot of stuff. It would help a lot if subfolders of the selected folder could be searched, although obviously it would be much more powerful to be able to provide a list of folders that is searched. 
2. Make it possible to search for tags everywhere in a file. For my use case I typically add #tags anywhere in the note, so using YAML front matter is not so convenient.

[deanishe]

On 1/5/2020 at 8:42 AM, Acidham said:

Discard all was my first idea but I am not happy because it distorts the search and results.

 

Fundamentally discarding diacritics isn't a good idea, but the method I suggested—ignore diacritics if the query doesn't contain any—tends to provide good results. (This is how Spotlight works.)

 

So in practice, someone who knows how to spell the word correctly (say, "résumé") isn't going to see any bogus results for "resume", but someone who has no idea how to type é on their keyboard can also find "résumé" by entering "resume".

 

This is especially important for typical (i.e. monolingual) English-speakers because diacritics are entirely optional in English.

[Acidham]

Quote

Add a possibility to add multiple folders to the search scope. As far as I understand this is not possible now. For my use case I have several folders with markdown files, which is a structure I built over some time and rebuilding it would break a lot of stuff. It would help a lot if subfolders of the selected folder could be searched, although obviously it would be much more powerful to be able to provide a list of folders that is searched. 

 

In my previous post I rejected to add subfolder scanning/searching but adding more than one folder would be a nice alternative. I will take a look into adding multiple folder in config which will be searched as well...

 

Quote

Make it possible to search for tags everywhere in a file. For my use case I typically add #tags anywhere in the note, so using YAML front matter is not so convenient.

 

I am using YAML front to ensure Tags: only in YAML Front will be searched for Tags. But I could add an option to allow yaml_tags_only=True. When set to False tags in the full content will be searched.

 

Alternatively you can search for #tagname in search. This will find tags in the text.

 

Thank you for your feedback, stay tuned ....

Edited January 28, 2020 by Acidham
added alternative for tagsearch

[Acidham]

@deanishe any idea how to exchange diacritics (e.g. é → e) in python?

[deanishe]

I think this is the best way (it should leave non-Latin characters alone):

import unicodedata
def strip_accents(s):
   return ''.join(c for c in unicodedata.normalize('NFD', s)
                  if unicodedata.category(c) != 'Mn')

Or if you want ASCII-only:

import unicodedata

def remove_accents(input_str):
    nfkd_form = unicodedata.normalize('NFKD', input_str)
    only_ascii = nfkd_form.encode('us-ascii', 'ignore')
    return only_ascii

Source: https://stackoverflow.com/questions/517923/what-is-the-best-way-to-remove-accents-in-a-python-unicode-string

[Acidham]

Thx @deanishe

 

@cands Done!

 

Quote

I am using YAML front to ensure Tags: only in YAML Front will be searched for Tags. But I could add an option to allow yaml_tags_only=True. When set to False tags in the full content will be searched.

 

[cands]

2 hours ago, Acidham said:

 

@cands Done!

 

 

 

Thank you very much! The tag search is very useful, especially that you also can choose to paste them to the frontmost app! 

[M0ll3art]

On 1/28/2020 at 7:41 AM, Acidham said:

 

In my previous post I rejected to add subfolder scanning/searching but adding more than one folder would be a nice alternative. I will take a look into adding multiple folder in config which will be searched as well...

 

 

really like your WF but I would really like to know if you made any progress on this one?  

[Acidham]

7 minutes ago, M0ll3art said:

 

really like your WF but I would really like to know if you made any progress on this one?  

 

I started to investigate what needs to be changed and the changes are require a lot of adoption. I am now thinking of just implementing read to additional folder but new notes will go to the default directory. With that I can leave current code-base as it is and just add a new search, aggregating additional directories.

thoughts?

[M0ll3art]

I think this is more than sufficient for the moment. 

 

If considering only to write to a default directory and read from many I could image it would be cool to be able to make customizable. I'm used to a kind of Triage directory and then I move it to the corresponding subfolders. 

 

Concerning the note creation,  I could think of using the YAML frontmatter to extend its keywords to allow for a special keyword to be set which could be used by the note creation script to move the the note to the subfolder. Of course this Keywords cannot be validated or whatever since it will identify a subfolder.. 

 

Hope this help .. and thanks for your time on this project. 

[Acidham]

Over the past days I ran some tests on implement  additional directories and sub-folders scanning but the performance loss is around 50 times slower. To get to same or equal performance experience I need to implement and index. For my personal requirements I don't need additional directories and therefore  I refrain from implementing additional directories and sub-folder scanning. Sorry!

[cands]

On 2/17/2020 at 7:46 PM, Acidham said:

additional directories and sub-folders scanning but the performance loss is around 50 times slower

 

Yes, that is a problem. Is it the sub-folders scanning that is taking time? I would be very happy if it was just possible to manually add some folders, like a static list of folders (e.g. defined in the path_to_notes variable separated by commas, or maybe picked up from a text file), so only these paths were searched but no subfolders. Maybe that would not affect performance noticeably? 

Edited February 26, 2020 by cands

[deanishe]

19 minutes ago, cands said:

Is it the sub-folders scanning that is taking time?

 

It's more likely the sheer number of files.

[Acidham]

1 hour ago, deanishe said:

 

It's more likely the sheer number of files.

I compared os.listdir (flat) vs os.walk (sub) and walk is 50 times slower. With listdir I tested with 100 k files and performance is good enough to serve md notes wf.

 

I am happy to accept PR with subfolder scanning (Or other solutions) with compareable performance 

[Acidham]

1 hour ago, cands said:

 

Yes, that is a problem. Is it the sub-folders scanning that is taking time? I would be very happy if it was just possible to manually add some folders, like a static list of folders (e.g. defined in the path_to_notes variable separated by commas, or maybe picked up from a text file), so only these paths were searched but no subfolders. Maybe that would not affect performance noticeably? 

Yep I will look into add additional folders readonly

[deanishe]

19 minutes ago, Acidham said:

I compared os.listdir (flat) vs os.walk (sub) and walk is 50 times slower.

 

Yeah, sorry. My stupid fault for guessing at performance without testing. os.walk is slow because it calls os.stat (repeatedly) on every file to see if it's a directory or not. Python 3 is 3x faster on my machine because it uses scandir under the hood.

 

25 minutes ago, Acidham said:

I am happy to accept PR with subfolder scanning (Or other solutions) with compareable performance

 

A propos of not very much, I've been using MeiliSearch in quite few workflows. It's great for cases like this because it's really smart, extremely fast, and very easy to use.