Jump to content

Pass search query through Python script


Recommended Posts

I want make a quick search for phpBB forums websites one of them is www.cracked.to when I search on it, it generate a new SID for each search result so the simple of putting {query} isnt the option, I was wondering if I can pass it through python:

 

I tried this on windows terminal, its working, but I am not sure pass how to pass it on web browser through Alfred. Here is the script: 

 

import requests
import sys

query = sys.argv[1]
payload = {'action': 'do_search', 'keywords' : query, 'postthread' :'2',  'matchusername' : '1', 'forums[]': 'all', 'findthreadst':'1', 'numreplies':'', 'postdate':'0', 'pddir':'1', 'threadprefix[]': 'any', 'sortby':'lastpost', 'sortordr':'desc', 'showresults':'threads', 'submit':'Search'}

if len(query) > 0:
    r = requests.post('https://cracked.to/search.php', data=payload)
    print(r.cookies)
    print(r.url)

Here is the workflow: https://d.pr/f/vEVw9K

 

Any ideas?

Edited by selfmade69
Link to post
2 hours ago, vitor said:

Get rid of the print(r.cookies) line. Then connect an Open URL Action to the end of your Run Script and you’re done.

 

 

 

OK, I got rid of the said line and connected the Open URL action but it doesn’t pass anything to the browser. Does it work for you? https://d.pr/f/UHIoDh

 

Here is what the debugger says: 

[2018-07-24 18:57:27][ERROR: action.script] File "/Users/UN/Library/Caches/com.runningwithcrayons.Alfred-3/Workflow Scripts/B562468F-248D-4FC2-B734-E940EB042E08", line 4
    import sys
    ^
SyntaxError: invalid syntax

 

 

2 hours ago, vitor said:

Keep in mind that if you want to package the Workflow for redistribution, you’ll have to bundle requests with it.

 

Please excuse my ignorance but how do I "bundle requests"?

Edited by selfmade69
Updated debugger detail
Link to post
45 minutes ago, selfmade69 said:

Non-ASCII character '\xef'

 

Search online for that error. It’s to do with Python 2’s unicode issues. The code did work for me, but I used Python 3 (which doesn’t come bundled with macOS and is thus not a great choice for the Workflow).

 

48 minutes ago, selfmade69 said:

Please excuse my ignorance but how do I "bundle requests"?

 

Ignorance that one wants to fix is nothing to apologise for!


Should be something like pip install --target "{{custom_dir}}" requests. Search for how to install Python eggs to custom locations. You’ll have to install it to the Workflows directory and then tell the script to use it.

Link to post
11 hours ago, selfmade69 said:

query = sys.argv[1]

 

To avoid encoding errors, change this line to:

query = sys.argv[1].decode('utf-8')

Unfortunately, this code is nonsense: you haven't understood what's going on:

r = requests.post('https://cracked.to/search.php', data=payload)
print(r.url)

The URL you're printing doesn't contain the payload because it's a POST request. POST means, by definition, that the data are sent in the request body, not the URL.

 

The request is almost certainly also failing because you need to be logged in to perform a search, and your workflow isn't logged in to the site.

 

Edited by deanishe
Link to post
45 minutes ago, deanishe said:

The URL you're printing doesn't contain the payload because it's a POST request. POST means, by definition, that the data are sent in the request body, not the URL.

 

Weirdly enough, the code does work to an extent. After the POST request something is returned, including the status code (200), a cookie with an sid, and a URL. Said URL does not contain the search query but it does contain the sid. And if you follow that URL, you’re directed to a page with results relevant to your search, though it looks like it’s just one of them.


The forums of this type I’ve encountered severely limit the amount of searches you can do in a row. Presumably that sid is being saved and associated with that search, so that when you follow the link so soon after the POST request you still get relevant results.

 

1 hour ago, deanishe said:

because you need to be logged in to perform a search

 

I’m being able to do logged out searches just fine.

Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...