Pinboard Search (beta?)

[jdfwarrior]

There may be issues with this, I wanted to post it to let people test it. Special characters may still cause an issue.

 

http://d.pr/f/YzAy

 

To set it up, go log into your Pinboard account on pinboard.in. Then go to Settings, and Password. Toward the bottom they provide you with API key. Copy that value. In Alfred, use the pinboardsetup keyword to enter that key. So: pinboardsetup <key>

 

After that, just use: pb <query> 

 

It may take a second to sync your bookmarks to local. Subsequent calls should be fast. It will update the next time you search if the last time was more than 5 minutes ago. I still need to work on getting just the updated bookmarks and adding those in. I just wanted a few people to test this and play with it.

[Kennon]

It appears like it should accept tags as well as titles, descriptions, etc... Is that true? Currently tags don't seem to display any results.

[Jono]

At the moment it's not working for me. I entered my Pinboard API key which seemed fine, but nothing appears when I type a search.

 

I have over 4,000 bookmarks though, so not sure if that could choke it or it's just taking a long to to download/import the bookmarks.

 

[jdfwarrior]

Geez, this is getting ridiculous. Everything I make works fine on every machine I touch. Submit it for others and it goes haywire or doesn't work.

[dansherman]

Looks like the data synced ok (i.e. bookmarks.db has all my bookmarks), but the pb command comes up dry.

[dansherman]

If it helps, it looks like the script runs ok from the command line. It complains about UTF-8, but spits out valid XML. 

Might the XML error be the hold up?

 

> php -f search.php -- "alfred"

Warning: SimpleXMLElement::asXML(): string is not in UTF-8 in /Users/dansherman/Library/Application Support/Alfred 2/Alfred.alfredpreferences/workflows/user.workflow.313E934E-E195-49C0-A25F-C215C734171B/extension_utils.php on line 35
<?xml version="1.0"?>
<items><item uid="http://jdfwarrior.tumblr.com/post/40355737734/a-little-secret-in-alfred-2-youll-like" arg="http://jdfwarrior.tumblr.com/post/40355737734/a-little-secret-in-alfred-2-youll-like"><title>A Little Secret in Alfred 2 Youâ</title><subtitle></subtitle><icon>icon.png</icon></item><item uid="http://www.macstories.net/tutorials/screen-sharing-for-alfred-brings-vnc-to-your-fingertips/#more-28145" arg="http://www.macstories.net/tutorials/screen-sharing-for-alfred-brings-vnc-to-your-fingertips/#more-28145"><title>Screen Sharing For Alfred Brings VNC To Your Fingertips</title><subtitle></subtitle><icon>icon.png</icon></item></items>

[jdfwarrior]

If it helps, it looks like the script runs ok from the command line. It complains about UTF-8, but spits out valid XML. 

Might the XML error be the hold up?

 

> php -f search.php -- "alfred"

Warning: SimpleXMLElement::asXML(): string is not in UTF-8 in /Users/dansherman/Library/Application Support/Alfred 2/Alfred.alfredpreferences/workflows/user.workflow.313E934E-E195-49C0-A25F-C215C734171B/extension_utils.php on line 35
<?xml version="1.0"?>
<items><item uid="http://jdfwarrior.tumblr.com/post/40355737734/a-little-secret-in-alfred-2-youll-like" arg="http://jdfwarrior.tumblr.com/post/40355737734/a-little-secret-in-alfred-2-youll-like"><title>A Little Secret in Alfred 2 Youâ</title><subtitle></subtitle><icon>icon.png</icon></item><item uid="http://www.macstories.net/tutorials/screen-sharing-for-alfred-brings-vnc-to-your-fingertips/#more-28145" arg="http://www.macstories.net/tutorials/screen-sharing-for-alfred-brings-vnc-to-your-fingertips/#more-28145"><title>Screen Sharing For Alfred Brings VNC To Your Fingertips</title><subtitle></subtitle><icon>icon.png</icon></item></items>

 

Added the utf8 encoding and tweaked one or two other things.. try again?

[Kennon]

It looks like it's an encoding issue on my end. I have a few bookmarks with odd characters (imported by Safari from when I joined Pinboard), and it's choking on those (dashes and >> primarily). Once I got rid of the problematic characters everything started functioning fine.

 

EDIT: Did some investigating and it's the htmlentities() call that's the issue. It's causing certain characters to just go wrong. However, removing that causes the above error from "dansherman" to come back. I know this isn't the right way, but I added "error_reporting(0);" and removed the htmlentities() around title and description, and that seems to be ok. It has a few character issues, but I'm getting all my bookmarks returned.

[Jono]

Added the utf8 encoding and tweaked one or two other things.. try again?

Yes, that's definitely helped as I'm now seeing search results. But it's not returning all bookmarks if I do a search for a word I know is in the title, description or tags for a lot of my bookmarks. I see results for some of my tags, but not all of them though.

[chiggers]

David - you are the man. Thanks so much for this!

[epogue]

I found a possible fix that seems to be holding up to whatever weird entities I throw at it. I looked it up, and apparently SimpleXML will automatically do all the escaping it needs when using the assignment operator instead of addChild. Therefore, I tweaked the arrayToXml() function in extension_utils, replacing:

 

$c->addChild( $key, $b[$key] );

 

with:

 

$c->$key = $b[$key];

 

and deleted the wrapping utf8_encode(htmlentities()) calls in search.php.

 

Hope this works for you guys!

[Jono]

I found a possible fix that seems to be holding up to whatever weird entities I throw at it. I looked it up, and apparently SimpleXML will automatically do all the escaping it needs when using the assignment operator instead of addChild. Therefore, I tweaked the arrayToXml() function in extension_utils, replacing:

 

$c->$key = $b[$key];

 

and deleted the wrapping utf8_encode(htmlentities()) calls in search.php.

 

Hope this works for you guys!

 

Could you explain exactly what I should do for this second part? (please excuse my ignorance   )

 

[anas]

Could you explain exactly what I should do for this second part? (please excuse my ignorance   )

 

You need to edit the extension_utils.php in the workflow folder.

 

@epogue Thanks for the suggestion works for me now.

 

 

[Jono]

You need to edit the extension_utils.php in the workflow folder.

 

@epogue Thanks for the suggestion works for me now.

 

I realise that but wasn't sure exactly what to delete.

 

What line numbers should I delete, or if I shouldn't delete whole lines what exactly should I delete? (I'm not sure of the 'wrapping utf8_encode(htmlentities()) calls' part).

 

 

[epogue]

Could you explain exactly what I should do for this second part? (please excuse my ignorance   )

 

 

It's no problem. On line 30 in extension_utils.php, replace $c->addChild( $key, $b[$key] ); with

$c->$key = $b[$key];

 

 

Then, in search.php, on lines 38 - 41, remove the methods (utf8_encode and htmlentities) wrapping the attribute calls. For instance, instead of 'uid' => utf8_encode ( htmlentities ( $bookmark->url ) ), it should read:

'uid' => $bookmark->url,

[Jono]

It's no problem. On line 30 in extension_utils.php, replace $c->addChild( $key, $b[$key] ); with

$c->$key = $b[$key];

 

 

Then, in search.php, on lines 38 - 41, remove the methods (utf8_encode and htmlentities) wrapping the attribute calls. For instance, instead of 'uid' => utf8_encode ( htmlentities ( $bookmark->url ) ), it should read:

'uid' => $bookmark->url,

 

Thanks a lot! It now works for me 

 

I hope in the future the workflow is updated to allow mixed searches of (for example) a tag plus part of the title, a tag plus part of the description etc.

 

 

[jdfwarrior]

Yeah guys, I need to update several of mine. With the additions Andrew has been making and then me constantly changing my extensions_utils class to include more stuff and make it work better, I have lots to do.

 

Currently, the search does search the description, extended text, url, and tags. It performs a search looking for results where the title, description, extended, or tags match the search query. It sounds like what you're asking for though is for it to be more of a nested search? Kinda like, if you type one word it find matches, then if a second word is added, it performs a search on the currently returned items? Is that correct?

[Jono]

Yeah guys, I need to update several of mine. With the additions Andrew has been making and then me constantly changing my extensions_utils class to include more stuff and make it work better, I have lots to do.

 

Currently, the search does search the description, extended text, url, and tags. It performs a search looking for results where the title, description, extended, or tags match the search query. It sounds like what you're asking for though is for it to be more of a nested search? Kinda like, if you type one word it find matches, then if a second word is added, it performs a search on the currently returned items? Is that correct?

 

 

I'm not wanting nested searches. It just seems to not always find all bookmarks/tags when I type a search at the moment.

 

One thing I noticed when trying this out is if I type 2 non consecutive words from a bookmark title it won't show that bookmark (called fuzzy search?), but if I type the title 'properly' in order it finds the bookmarl.

 

For example if I have the bookmark titled 'Alfred App Community Forum' and type 'alfred forum' it won't find the bookmark, but if I type 'alfred app community' (or just ' alfred app') it will find the bookmark.

 

[jdfwarrior]

I'm not wanting nested searches. It just seems to not always find all bookmarks/tags when I type a search at the moment.

 

One thing I noticed when trying this out is if I type 2 non consecutive words from a bookmark title it won't show that bookmark (called fuzzy search?), but if I type the title 'properly' in order it finds the bookmarl.

 

For example if I have the bookmark titled 'Alfred App Community Forum' and type 'alfred forum' it won't find the bookmark, but if I type 'alfred app community' (or just ' alfred app') it will find the bookmark.

 

 

Ah yeah I see what you mean. What's doing is reading the entire input as one string and querying that so... say your searching for 'alfred app', it searches for that exactly and not alfred, and app individually. So what I need to do is go back and when I read the input, split it based on a space, and then alter the query to search for: name like "%arg1%" or name like "%arg2%" and so on... Thanks for pointing that out. I'll get it worked in. 

[Jono]

Ah yeah I see what you mean. What's doing is reading the entire input as one string and querying that so... say your searching for 'alfred app', it searches for that exactly and not alfred, and app individually. So what I need to do is go back and when I read the input, split it based on a space, and then alter the query to search for: name like "%arg1%" or name like "%arg2%" and so on... Thanks for pointing that out. I'll get it worked in. 

 

Yes, that's how I was trying to search for things. Thanks, I'll look forward to it 

 

[Luzzifus]

Hi,

 

I wrote a small modification for this awesome workflow (BIG thanks to David!  ) which allows to search for multiple words at once instead of treating all arguments as a single search string. So if you type "pb recipe apple", it returns all your bookmarks containing "recipe" and "apple" in any field. So "recipe" could be a tag and the description could contain "apple". That's basically what Jono was requesting, if I got that right.

 

So to achieve that behaviour, open "search.php" and find the line:

$matches = $db->get( 'bookmarks', 'url like "%'.$query.'%" or description like "%'.$query.'%" or extended like "%'.$query.'%" or tag like "%'.$query.'%"' );

 

Should be around line #35. Remove that line and replace it with:

$queries = explode(' ', $query);
$where_string = '';
foreach( $queries as $q ):
  $where_string .= (strlen($where_string) > 0) ? ' and ' : '';
  $where_string .= '(url like "%'.$q.'%" or description like "%'.$q.'%" or extended like "%'.$q.'%" or tag like "%'.$q.'%")';
endforeach;

$matches = $db->get( 'bookmarks', $where_string );

 

Save the file and that's it. Hope this helps! 

 

----------------------------------------

**edit:

 

Sometimes the subsequential syncs take some time, maybe I'd rather have the re-syncs manually triggered by its own command ("pbsync" or something like that).

Also it would be awesome to have an add-to-pinboard command.   

[drking]

this is awesome and works great!

[Jono]

Hi,

 

I wrote a small modification for this awesome workflow (BIG thanks to David!  ) which allows to search for multiple words at once instead of treating all arguments as a single search string. So if you type "pb recipe apple", it returns all your bookmarks containing "recipe" and "apple" in any field. So "recipe" could be a tag and the description could contain "apple". That's basically what Jono was requesting, if I got that right.

 

So to achieve that behaviour, open "search.php" and find the line:

$matches = $db->get( 'bookmarks', 'url like "%'.$query.'%" or description like "%'.$query.'%" or extended like "%'.$query.'%" or tag like "%'.$query.'%"' );

 

Should be around line #35. Remove that line and replace it with:

$queries = explode(' ', $query);
$where_string = '';
foreach( $queries as $q ):
  $where_string .= (strlen($where_string) > 0) ? ' and ' : '';
  $where_string .= '(url like "%'.$q.'%" or description like "%'.$q.'%" or extended like "%'.$q.'%" or tag like "%'.$q.'%")';
endforeach;

$matches = $db->get( 'bookmarks', $where_string );

 

Save the file and that's it. Hope this helps! 

 

----------------------------------------

**edit:

 

Sometimes the subsequential syncs take some time, maybe I'd rather have the re-syncs manually triggered by its own command ("pbsync" or something like that).

Also it would be awesome to have an add-to-pinboard command.   

 

Brilliant, thanks a lot. Work great 

[Jono]

…I keep forgetting to say. When I first use the workflow it takes about 20 seconds until it shows any results. If I do another search straight after the first one the results appear straight away. It feels as though if I haven't used the search for a while it has to load the database or something before it can show the search results.

 

Does anyone else have this problem?

[QuaCKeReD]

…I keep forgetting to say. When I first use the workflow it takes about 20 seconds until it shows any results. If I do another search straight after the first one the results appear straight away. It feels as though if I haven't used the search for a while it has to load the database or something before it can show the search results.

 

Does anyone else have this problem?

Mine took few seconds to return anything, yes.

 

Now though, subsequent searches show nothing, unless I delete last char from search value;

i.e.

pb test - nothing

pb tes - results