Jump to content

Recommended Posts

This workflow allows you to quickly look up bible passages and copy them to your clipboard. It submits passage reference queries to the wonderful ESV API (https://api.esv.org/) and parses the response. The API is smart enough to handle many variations and abbreviations!

 

Download and use it here: ESV Online Bible (on Packal)

 

GitHub repo: https://github.com/emmanueljl/ESV-Bible-Alfred-workflow

 

esv-bible-workflow-ss.png

Edited by emmanuel
Add GitHub repo

Share this post


Link to post

You're welcome :) Let me know of any bugs you may happen across, or areas to improve! The underlying API powering this workflow is from a third-party. If there are similarly excellent APIs out there for other versions, I could potentially whip something up depending on demand.

Share this post


Link to post

I love this workflow, and people are amazed when they see how I so quickly pull up passages in the blink of an eye.

 

However, I notice that the API v. 2 has since deprecated, and this workflow no longer works.  Is it possible to update this?  Thank you so very much for this workflow

Share this post


Link to post

Hey all, thank you for your feedback! I have a fix ready using the new version of the API. However, my auth key is still undergoing the approval process. As soon as it's ready I'll update this post and notify everyone! :)

Share this post


Link to post

I'm getting the following debug messages whenever I run it:

 

[2017-11-20 00:21:08][ERROR: input.scriptfilter] Code 1: Traceback (most recent call last):
  File "ESVPassageFilter.py", line 37, in <module>
    resp = urllib2.urlopen(req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1240, in https_open
    context=self._context)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1197, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 54] Connection reset by peer>

Am I running the wrong version of python or a library? When I type just `python` it says I'm running v2.7.10.

Share this post


Link to post

Hey JGC,

 

So I assume running the workflow through Alfred isn't working for you either? Python v2.7.10 should be fine. I won't be able to debug from only the log messages provided. Is this an ephemeral issue, or are you constantly seeing this? What was the specific query or bible passage input that you used to get this error?

Share this post


Link to post
7 hours ago, emmanuel said:

Python v2.7.10 should be fine

 

It's not.

 

7 hours ago, emmanuel said:

I won't be able to debug from only the log messages provided

 

I can!

 

It is indeed a problem with the old version of Python (most likely in the TLS/SNI handling, which has always been crap on Python). Python 2.7.13 works just fine.

 

There are two realistic options to fix it, imo: use curl or use requests to make the HTTP requests.

 

Here's the requests version:

# encoding: utf-8

from __future__ import print_function

import json
import sys

import requests

rawInput = sys.argv[1]
key = "<redacted>"

options = {
    'include-passage-references': 'false',
    'include-first-verse-numbers': 'false',
    'include-verse-numbers': 'false',
    'include-footnotes': 'false',
    'include-footnote-body': 'false',
    'include-short-copyright': 'false',
    'include-passage-horizontal-lines': 'false',
    'include-heading-horizontal-lines': 'false',
    'include-headings': 'false',
    'include-selahs': 'false',
    'indent-paragraphs': '0',
    'indent-poetry': 'false',
    'indent-poetry-lines': '0',
    'indent-declares': '0',
    'indent-psalm-doxology': '0'
}

headers = {
    'Accept': 'application/json',
    'Authorization': 'Token ' + key,
}

baseUrl = 'https://api.esv.org/v3/passage/text/'


def exit_with_error(query, errMsg):
    """Show an error message in Alfred and exit script."""
    errorOutput = {
        "items": [
            {"title": query, "subtitle": errMsg}
        ]
    }
    print(json.dumps(errorOutput))
    sys.exit(0)


# Combine query and options. Requests will take care of URL-encoding
params = dict(q=rawInput)
params.update(options)

# Execute request
r = requests.get(baseUrl, params, headers=headers)
try:
    # Raise exception if request failed
    r.raise_for_status()
    # Parse response
    content = r.json()
except Exception as err:
    exit_with_error(rawInput, str(err))

if content['canonical'] == '' or len(content['passages']) > 1:
    exit_with_error(rawInput, 'No passage found')


passageRef = content['canonical']
pageSplit = content['passages'][0].split('\n\n')
passage = ' '.join(pageSplit).rstrip()
# print passage
passageWithRef = passage + ' (' + passageRef + ' ESV)'
data = {
    'items': [{
        'arg': passageWithRef,
        'valid': 'YES', 'autocomplete': passageRef, 'title': passageRef,
        'text': {'largetype': passageWithRef},  # Show whole passage on CMD+L
        'subtitle': passage
    }]
}
print(json.dumps(data))

 

Share this post


Link to post

Thanks.  I first tried substituting @deanishe's updated script. As soon as I typed the '1' of "esv Eph1" in Alfred, the debug log showed:

[2017-11-20 09:37:58][ERROR: input.scriptfilter] Code 1: Traceback (most recent call last):
  File "ESVPassageFilter.py", line 8, in <module>
    import requests
ImportError: No module named requests

This sounded like my python install was incomplete, and as I know very little about python and its libraries, I decided it was then easier to attempt to install 2.7.13.  Well, I found 2.7.14 on https://www.python.org/downloads/mac-osx/ so have installed that.

Unfortunately I then get the same errors as posted when I run either version.

@emmanuel I was attempting to run "Eph 1.15-23" but it wasn't working with the screen shot text of "pr31:30".

Share this post


Link to post
1 hour ago, JGC said:

This sounded like my python install was incomplete

 

requests isn't part of Python: you have to install it.


Assuming you have pip installed, right-click on the workflow in Alfred Preferences and choose Open in Terminal. Then enter pip install --target=. requests to install requests in the workflow.

 

1 hour ago, JGC said:

Unfortunately I then get the same errors as posted when I run either version.

 

What do you mean "run either version"?

 

The workflow will always run with the built-in Python unless you edit it to point at the newer version you installed.

Share this post


Link to post
6 hours ago, deanishe said:

requests isn't part of Python: you have to install it.


Assuming you have pip installed, right-click on the workflow in Alfred Preferences and choose Open in Terminal. Then enter pip install --target=. requests to install requests in the workflow.

 

OK done `pip install requests --target=.` (which Is what I think you meant) in the workflow subdirectory, and `pip install requests` for the general python install (I hope).

 

6 hours ago, deanishe said:

What do you mean "run either version"?

 

The workflow will always run with the built-in Python unless you edit it to point at the newer version you installed.

 

When I run either your version or @emmanuel's I still hit errors, the last of which (for your version) is:

user.workflow.748EA499-B00B-45DF-800C-498A7F2CC500 jonathan$ python ESVPassageFilter-newer.py Jude
Traceback (most recent call last):
  File "ESVPassageFilter-newer.py", line 55, in <module>
    r = requests.get(baseUrl, params, headers=headers)
  File "/Users/jonathan/Dropbox/Alfred.alfredpreferences/workflows/user.workflow.748EA499-B00B-45DF-800C-498A7F2CC500/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/jonathan/Dropbox/Alfred.alfredpreferences/workflows/user.workflow.748EA499-B00B-45DF-800C-498A7F2CC500/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/jonathan/Dropbox/Alfred.alfredpreferences/workflows/user.workflow.748EA499-B00B-45DF-800C-498A7F2CC500/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/jonathan/Dropbox/Alfred.alfredpreferences/workflows/user.workflow.748EA499-B00B-45DF-800C-498A7F2CC500/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/Users/jonathan/Dropbox/Alfred.alfredpreferences/workflows/user.workflow.748EA499-B00B-45DF-800C-498A7F2CC500/requests/adapters.py", line 490, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(54, 'Connection reset by peer'))

I get this when I run the code outside Alfred as well (`python ESVPassageFilter-newer.py Jude`). Feels more like a problem with the web service itself, rather than the code?  I confirm I've tweaked your code to include the `key` from @emmanuel; could it be that I need my own API key?

Thanks for your help on this.

Share this post


Link to post

Right. My bad: I was using a newer Python.

 

In the workflow directory again, install pyOpenSSL with pip install --target=. pyOpenSSL.


requests should then use that library instead of Python's built-in one, and all should be well.

 

Share this post


Link to post

Thank you both for contributing to this workflow!

I've been meaning to publish this to GitHub but never got around to it. Now that it's evident there's a larger community surrounding this (both here on the forum and others e-mailing me), I'll put it up asap! The ESV API requires an API key that has been approved for use. Feel free to use the key that is present. It's been approved as this workflow's key. I think publishing the API key should be fine. I'll add some comments about proper/ethical use of the key. Let me know if any of you have suggestions on this though!

 

@deanishe Is there a way to "bundle" these packages into the Alfred workflow so that users won't need to follow extra instructions during installation?

 

@JGC You're totally right that the input "pr 31:30" behaves incorrectly, oops! Looks like the test verses I used didn't account for the returned output structure of these specially-formatted, multi-line passages from the API. You're more than welcome to try to tackle this and submit a PR once I publish the code :)!

Share this post


Link to post
Just now, emmanuel said:

Is there a way to "bundle" these packages into the Alfred workflow so that users won't need to follow extra instructions during installation?

 

Sure. Just follow the pip instructions I gave, install the libraries in the workflow. That way, when you export it, the libraries will be included.


TBH, I think curl would be a better fit, though. Adding all the SSL libraries makes the workflow 13MB…


Let me see about using curl instead.

Share this post


Link to post

Try this script instead. It uses /usr/bin/curl for HTTP requests. No need for megabytes of additional libraries.

# encoding: utf-8

from __future__ import print_function

import json
import sys
import subprocess
from urllib import urlencode

rawInput = sys.argv[1]
key = "5974948d3baa3d1cabc4eb00e4099e3d785d43df"

options = {
    'include-passage-references': 'false',
    'include-first-verse-numbers': 'false',
    'include-verse-numbers': 'false',
    'include-footnotes': 'false',
    'include-footnote-body': 'false',
    'include-short-copyright': 'false',
    'include-passage-horizontal-lines': 'false',
    'include-heading-horizontal-lines': 'false',
    'include-headings': 'false',
    'include-selahs': 'false',
    'indent-paragraphs': '0',
    'indent-poetry': 'false',
    'indent-poetry-lines': '0',
    'indent-declares': '0',
    'indent-psalm-doxology': '0'
}

headers = {
    'Accept': 'application/json',
    'Authorization': 'Token ' + key,
}

baseUrl = 'https://api.esv.org/v3/passage/text/'


def fetch_url(url, params, headers):
    """Fetch a URL using cURL and parse response as JSON."""
    # Encode GET parameters and add to URL
    qs = urlencode(params)
    url = url + '?' + qs

    # Build cURL command
    cmd = ['/usr/bin/curl', '-sSL', url]
    for k, v in headers.items():
        cmd.extend(['-H', '{}: {}'.format(k, v)])

    # Run command and parse response
    output = subprocess.check_output(cmd)
    return json.loads(output)


def exit_with_error(query, errMsg):
    """Show an error message in Alfred and exit script."""
    errorOutput = {
        "items": [
            {"title": query, "subtitle": errMsg}
        ]
    }
    json.dump(errorOutput, sys.stdout)
    sys.exit(0)


# Combine query and options into GET parameters
params = dict(q=rawInput)
params.update(options)

# Execute request
try:
    content = fetch_url(baseUrl, params, headers)
except Exception as err:
    exit_with_error(rawInput, str(err))

if content['canonical'] == '' or len(content['passages']) > 1:
    exit_with_error(rawInput, 'No passage found')


passageRef = content['canonical']
pageSplit = content['passages'][0].split('\n\n')
passage = ' '.join(pageSplit).rstrip()

# print passage
passageWithRef = passage + ' (' + passageRef + ' ESV)'
data = {
    'items': [{
        'arg': passageWithRef,
        'valid': 'YES', 'autocomplete': passageRef, 'title': passageRef,
        'text': {'largetype': passageWithRef},  # Show whole passage on CMD+L
        'subtitle': passage
    }]
}

json.dump(data, sys.stdout)

 

Edited by deanishe
Tidy up code a wee bit

Share this post


Link to post
41 minutes ago, emmanuel said:

Let me know if any of you have suggestions on this though!

 

One feature I'd consider, and it's suggested in the API docs, is caching the results of queries, at least for a few days.

 

The verses/passages returned by the API aren't going to change that often, and fetching something from the network is a lot slower than fetching it from disk. Also, it could potentially reduce the usage of the API key quite a lot.

 

(Though I've no idea how often people are likely to re-enter the same query. I've just been repeatedly mashing in the one from the screenshot because Bible verses aren't really my thing.)

 

Edited by deanishe

Share this post


Link to post

I noticed an encoding error in my script when I entered "job" as the query (probably something to do with "smart" quotes).

 

If you don't mind, I "Pythonified" the script and implemented the caching I mentioned. The script below catches and displays API errors (apparently the API returns a detail field if something went wrong), handles non-ASCII text, and caches the results for queries for 40 days (by default), which takes the response time from ~0.8 seconds to ~0.02 seconds for responses that are in the cache.


Hitting ⌘L on the result will show the passage in Alfred's Large Type window, and ⌘C will copy the same to the clipboard.

 

So if you'd like to, try this:

# encoding: utf-8

"""Alfred Script Filter to search the ESV Bible."""

from __future__ import print_function

from hashlib import md5
import json
import os
import re
import subprocess
import sys
from time import time
from urllib import urlencode

API_KEY = '<redacted>'
API_URL = 'https://api.esv.org/v3/passage/text/'
API_OPTIONS = {
    'include-passage-references': 'false',
    'include-first-verse-numbers': 'false',
    'include-verse-numbers': 'false',
    'include-footnotes': 'false',
    'include-footnote-body': 'false',
    'include-short-copyright': 'false',
    'include-passage-horizontal-lines': 'false',
    'include-heading-horizontal-lines': 'false',
    'include-headings': 'false',
    'include-selahs': 'false',
    'indent-paragraphs': '0',
    'indent-poetry': 'false',
    'indent-poetry-lines': '0',
    'indent-declares': '0',
    'indent-psalm-doxology': '0'
}

# HTTP request headers
API_HEADERS = {
    'Accept': 'application/json',
    'Authorization': 'Token ' + API_KEY,
}


# Directory for this workflow's cache data
CACHEDIR = os.getenv('alfred_workflow_cache')
CACHE_MAXAGE = 86400 * 40  # 40 days


def log(s, *args):
    """Write message to Alfred's debugger.

    Args:
        s (basestring): Simple string or sprintf-style format.
        *args: Arguments to format string.

    """
    if args:
        s = s % args
    print(s, file=sys.stderr)


class ESVError(Exception):
    """Base error class."""


class NotFound(ESVError):
    """Raised if no passage was found."""

    def __str__(self):
        """Error message."""
        return 'No passage found'


class APIError(ESVError):
    """Raised if API call fails."""


class Cache(object):
    """Cache results of API queries.

    Attributes:
        dirpath (str): Path to cache directory.

    """

    def __init__(self, dirpath):
        """Create a new cache.

        Args:
            dirpath (str): Directory to store cache files.

        """
        log('cache directory=%s', dirpath)
        self.dirpath = dirpath
        if dirpath is None:  # not being run from Alfred
            return

        # Alfred doesn't create the directory for you...
        if not os.path.exists(dirpath):
            os.makedirs(dirpath)
        else:  # remove old cache files
            self.clean()

    def search(self, query):
        """Perform API query, using cached results if not expired.

        Args:
            query (unicode): Search string.

        Returns:
            Passage: Passage from API or cache.

        """
        cachepath = None
        if self.dirpath:
            cachepath = os.path.join(
                self.dirpath,
                md5(query.encode('utf-8')).hexdigest() + '.json'
            )

        # ------------------------------------------------------
        # Try to load data from cache
        if cachepath and os.path.exists(cachepath):
            # Expired cache files were deleted when `Cache` was created
            log('[cache] loading passage for "%s" from cache ...', query)
            with open(cachepath) as fp:
                data = json.load(fp)
                return Passage.from_response(data)

        # ------------------------------------------------------
        # Fetch data from API

        # Combine query and options into GET parameters
        params = dict(q=query.encode('utf-8'))
        params.update(API_OPTIONS)

        # Execute request
        data = fetch_url(API_URL, params, API_HEADERS)
        passage = Passage.from_response(data)

        if cachepath:  # Cache response
            with open(cachepath, 'wb') as fp:
                json.dump(data, fp)

        return passage

    def clean(self):
        """Remove expired cache files."""
        i = 0
        for fn in os.listdir(self.dirpath):
            p = os.path.join(self.dirpath, fn)
            if time() - os.stat(p).st_mtime > CACHE_MAXAGE:
                os.unlink(p)
                i += 1
        if i:
            log('[cache] deleted %d stale cache file(s)', i)


class Passage(object):
    """A Bible passage.

    Attributes:
        fulltext (unicode): Passage text as paragraphs
        ref (unicode): Canonical passage reference
        summary (unicode): Passage text on one line
        with_ref (unicode): Passage as paragraphs + reference

    """

    @classmethod
    def from_response(cls, data):
        """Create a `Passage` from API response.

        Args:
            data (dict): Decoded JSON API response.

        Returns:
            Passage: Passage parsed from API response.

        Raises:
            NotFound: Raised if ``data`` contains no passage(s).

        """
        if not data.get('canonical') or not data.get('passages'):
            raise NotFound()

        ref = data['canonical']
        s = data['passages'][0]
        summary = re.sub(r'\s+', ' ', s).strip()
        p = cls(ref, summary, s)
        log('---------- passage -----------')
        log('%s', p)
        log('---------- /passage ----------')
        return p

    def __init__(self, ref=u'', summary=u'', fulltext=u''):
        """Create new `Passage`."""
        self.ref = ref
        self.summary = summary
        self.fulltext = fulltext
        self.with_ref = u'{}\n\n({} ESV)'.format(fulltext.rstrip(), ref)

    def __str__(self):
        """Passage as formatted bytestring.

        Returns:
            str: Full text of passage with reference.
        """
        return self.__unicode__().encode('utf-8')

    def __unicode__(self):
        """Passage as formatted Unicode string.

        Returns:
            unicode: Full text of passage with reference.
        """
        return self.with_ref

    @property
    def item(self):
        """Alfred item `dict`.

        Returns:
            dict: Alfred item for JSON serialisation.

        """
        return {
            'title': self.ref,
            'subtitle': self.summary,
            'autocomplete': self.ref,
            'arg': self.with_ref,
            'valid': True,
            'text': {
                'largetype': self.with_ref,
                'copytext': self.with_ref,
            },
        }


def fetch_url(url, params, headers):
    """Fetch a URL using cURL and parse response as JSON.

    Args:
        url (str): Base URL without GET parameters.
        params (dict): GET parameters.
        headers (dict): HTTP headers.

    Returns:
        object: Deserialised HTTP JSON response.

    Raises:
        APIError: Raised if API returns an error.

    """
    # Encode GET parameters and add to URL
    qs = urlencode(params)
    url = url + '?' + qs

    # Build cURL command
    cmd = ['/usr/bin/curl', '-sSL', url]
    for k, v in headers.items():
        cmd.extend(['-H', '{}: {}'.format(k, v)])

    # Run command and parse response
    output = subprocess.check_output(cmd)
    log('---------- response -----------')
    log('%r', output)
    log('---------- /response ----------')
    data = json.loads(output)
    if 'detail' in data:  # 'detail' contains any API error message
        raise APIError(data['detail'])

    return data


def exit_with_error(title, err, tb=False):
    """Show an error message in Alfred and exit script.

    Args:
        title (unicode): Title of Alfred item.
        err (Exception): Error whose message to show as item subtitle.
        tb (bool, optional): If `True`, show a full traceback in Alfred's
            debugger.

    """
    # Log to debugger
    if tb:
        import traceback
        log(traceback.format_exc())
    else:
        log('ERROR: %s', err)

    # Send error message to Alfred
    output = {
        'items': [{'title': title, 'subtitle': str(err)}]
    }
    json.dump(output, sys.stdout)

    sys.exit(1)  # 1 indicates something went wrong


def main():
    """Run Script Filter."""
    log('.')  # Ensure real log starts on a new line

    # Fetch user query and decode to Unicode
    query = sys.argv[1].decode('utf-8')

    cache = Cache(CACHEDIR)
    try:
        passage = cache.search(query)
    except ESVError as err:
        exit_with_error(query, err, False)
    except Exception as err:
        exit_with_error(query, err, True)

    # Show passage in Alfred
    json.dump({'items': [passage.item]}, sys.stdout)


if __name__ == '__main__':
    start = time()
    main()
    log('------ %0.3fs ------', time() - start)

 

Edited by deanishe
Simplified code

Share this post


Link to post

Wow. Simply amazing!! You have my heart(s). When I get home today I'll publish a GitHub repo and it will be open for pull requests. I fear that it won't be enough credit to you though @deanishe. I will be adding a shout-out to you and any links (personal GitHub/website/etc.) you provide me to all the public places. Please tell me what else I can do to give you credit for your awesome contributions!!

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...