ESV Online Bible

emmanuel · October 28, 2016

This workflow allows you to quickly look up bible passages and copy them to your clipboard. It submits passage reference queries to the wonderful ESV API (https://api.esv.org/) and parses the response. The API is smart enough to handle many variations and abbreviations!

Download and use it here: ESV Online Bible (on Packal)

GitHub repo: https://github.com/emmanueljl/ESV-Bible-Alfred-workflow

Edited November 21, 2017 by emmanuel
Add GitHub repo

dustying · December 9, 2016

Awesome! Thanks for this! Any plans for different versions (NASB) in the future?

emmanuel · December 15, 2016

You're welcome Let me know of any bugs you may happen across, or areas to improve! The underlying API powering this workflow is from a third-party. If there are similarly excellent APIs out there for other versions, I could potentially whip something up depending on demand.

milopus · November 5, 2017

I love this workflow, and people are amazed when they see how I so quickly pull up passages in the blink of an eye.

However, I notice that the API v. 2 has since deprecated, and this workflow no longer works. Is it possible to update this? Thank you so very much for this workflow

milopus · November 5, 2017

Actually, I just updated from your link above for the workflow which was updated October 2017, but I get an error that says "No Passage Found"

fly4him17 · November 5, 2017

I've got the same question. Does anyone know how to update this to the 3rd version of the API. This script was written for version 2 which as of Nov. 1 has been depreciated.

http://www.esvapi.org/api

emmanuel · November 6, 2017

Hey all, thank you for your feedback! I have a fix ready using the new version of the API. However, my auth key is still undergoing the approval process. As soon as it's ready I'll update this post and notify everyone!

emmanuel · November 18, 2017

Thank you for your patience! My auth key was finally approved, and the workflow on Packal has been updated with the fix.

milopus · November 19, 2017

thank you sooooo much. I've been waiting for this update. This is a huge time saver for me and my whole family.

JGC · November 20, 2017

I'm getting the following debug messages whenever I run it:

[2017-11-20 00:21:08][ERROR: input.scriptfilter] Code 1: Traceback (most recent call last):
  File "ESVPassageFilter.py", line 37, in <module>
    resp = urllib2.urlopen(req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1240, in https_open
    context=self._context)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1197, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 54] Connection reset by peer>

Am I running the wrong version of python or a library? When I type just `python` it says I'm running v2.7.10.

emmanuel · November 20, 2017

Hey JGC,

So I assume running the workflow through Alfred isn't working for you either? Python v2.7.10 should be fine. I won't be able to debug from only the log messages provided. Is this an ephemeral issue, or are you constantly seeing this? What was the specific query or bible passage input that you used to get this error?

deanishe · November 20, 2017

7 hours ago, emmanuel said:

Python v2.7.10 should be fine

It's not.

7 hours ago, emmanuel said:

I won't be able to debug from only the log messages provided

I can!

It is indeed a problem with the old version of Python (most likely in the TLS/SNI handling, which has always been crap on Python). Python 2.7.13 works just fine.

There are two realistic options to fix it, imo: use curl or use requests to make the HTTP requests.

Here's the requests version:

# encoding: utf-8

from __future__ import print_function

import json
import sys

import requests

rawInput = sys.argv[1]
key = "<redacted>"

options = {
    'include-passage-references': 'false',
    'include-first-verse-numbers': 'false',
    'include-verse-numbers': 'false',
    'include-footnotes': 'false',
    'include-footnote-body': 'false',
    'include-short-copyright': 'false',
    'include-passage-horizontal-lines': 'false',
    'include-heading-horizontal-lines': 'false',
    'include-headings': 'false',
    'include-selahs': 'false',
    'indent-paragraphs': '0',
    'indent-poetry': 'false',
    'indent-poetry-lines': '0',
    'indent-declares': '0',
    'indent-psalm-doxology': '0'
}

headers = {
    'Accept': 'application/json',
    'Authorization': 'Token ' + key,
}

baseUrl = 'https://api.esv.org/v3/passage/text/'


def exit_with_error(query, errMsg):
    """Show an error message in Alfred and exit script."""
    errorOutput = {
        "items": [
            {"title": query, "subtitle": errMsg}
        ]
    }
    print(json.dumps(errorOutput))
    sys.exit(0)


# Combine query and options. Requests will take care of URL-encoding
params = dict(q=rawInput)
params.update(options)

# Execute request
r = requests.get(baseUrl, params, headers=headers)
try:
    # Raise exception if request failed
    r.raise_for_status()
    # Parse response
    content = r.json()
except Exception as err:
    exit_with_error(rawInput, str(err))

if content['canonical'] == '' or len(content['passages']) > 1:
    exit_with_error(rawInput, 'No passage found')


passageRef = content['canonical']
pageSplit = content['passages'][0].split('\n\n')
passage = ' '.join(pageSplit).rstrip()
# print passage
passageWithRef = passage + ' (' + passageRef + ' ESV)'
data = {
    'items': [{
        'arg': passageWithRef,
        'valid': 'YES', 'autocomplete': passageRef, 'title': passageRef,
        'text': {'largetype': passageWithRef},  # Show whole passage on CMD+L
        'subtitle': passage
    }]
}
print(json.dumps(data))

JGC · November 20, 2017

Thanks. I first tried substituting @deanishe's updated script. As soon as I typed the '1' of "esv Eph1" in Alfred, the debug log showed:

[2017-11-20 09:37:58][ERROR: input.scriptfilter] Code 1: Traceback (most recent call last):
  File "ESVPassageFilter.py", line 8, in <module>
    import requests
ImportError: No module named requests

This sounded like my python install was incomplete, and as I know very little about python and its libraries, I decided it was then easier to attempt to install 2.7.13. Well, I found 2.7.14 on https://www.python.org/downloads/mac-osx/ so have installed that.

Unfortunately I then get the same errors as posted when I run either version.

@emmanuel I was attempting to run "Eph 1.15-23" but it wasn't working with the screen shot text of "pr31:30".

deanishe · November 20, 2017

1 hour ago, JGC said:

This sounded like my python install was incomplete

requests isn't part of Python: you have to install it.

Assuming you have pip installed, right-click on the workflow in Alfred Preferences and choose Open in Terminal. Then enter pip install --target=. requests to install requests in the workflow.

1 hour ago, JGC said:

Unfortunately I then get the same errors as posted when I run either version.

What do you mean "run either version"?

The workflow will always run with the built-in Python unless you edit it to point at the newer version you installed.

JGC · November 20, 2017

6 hours ago, deanishe said:

requests isn't part of Python: you have to install it.

Assuming you have pip installed, right-click on the workflow in Alfred Preferences and choose Open in Terminal. Then enter pip install --target=. requests to install requests in the workflow.

OK done `pip install requests --target=.` (which Is what I think you meant) in the workflow subdirectory, and `pip install requests` for the general python install (I hope).

6 hours ago, deanishe said:

What do you mean "run either version"?

The workflow will always run with the built-in Python unless you edit it to point at the newer version you installed.

When I run either your version or @emmanuel's I still hit errors, the last of which (for your version) is:

user.workflow.748EA499-B00B-45DF-800C-498A7F2CC500 jonathan$ python ESVPassageFilter-newer.py Jude
Traceback (most recent call last):
  File "ESVPassageFilter-newer.py", line 55, in <module>
    r = requests.get(baseUrl, params, headers=headers)
  File "/Users/jonathan/Dropbox/Alfred.alfredpreferences/workflows/user.workflow.748EA499-B00B-45DF-800C-498A7F2CC500/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/jonathan/Dropbox/Alfred.alfredpreferences/workflows/user.workflow.748EA499-B00B-45DF-800C-498A7F2CC500/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/jonathan/Dropbox/Alfred.alfredpreferences/workflows/user.workflow.748EA499-B00B-45DF-800C-498A7F2CC500/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/jonathan/Dropbox/Alfred.alfredpreferences/workflows/user.workflow.748EA499-B00B-45DF-800C-498A7F2CC500/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/Users/jonathan/Dropbox/Alfred.alfredpreferences/workflows/user.workflow.748EA499-B00B-45DF-800C-498A7F2CC500/requests/adapters.py", line 490, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(54, 'Connection reset by peer'))

I get this when I run the code outside Alfred as well (`python ESVPassageFilter-newer.py Jude`). Feels more like a problem with the web service itself, rather than the code? I confirm I've tweaked your code to include the `key` from @emmanuel; could it be that I need my own API key?

Thanks for your help on this.

deanishe · November 20, 2017

Right. My bad: I was using a newer Python.

In the workflow directory again, install pyOpenSSL with pip install --target=. pyOpenSSL.

requests should then use that library instead of Python's built-in one, and all should be well.

emmanuel · November 20, 2017

Thank you both for contributing to this workflow!

I've been meaning to publish this to GitHub but never got around to it. Now that it's evident there's a larger community surrounding this (both here on the forum and others e-mailing me), I'll put it up asap! The ESV API requires an API key that has been approved for use. Feel free to use the key that is present. It's been approved as this workflow's key. I think publishing the API key should be fine. I'll add some comments about proper/ethical use of the key. Let me know if any of you have suggestions on this though!

@deanishe Is there a way to "bundle" these packages into the Alfred workflow so that users won't need to follow extra instructions during installation?

@JGC You're totally right that the input "pr 31:30" behaves incorrectly, oops! Looks like the test verses I used didn't account for the returned output structure of these specially-formatted, multi-line passages from the API. You're more than welcome to try to tackle this and submit a PR once I publish the code !

deanishe · November 20, 2017

Just now, emmanuel said:

Is there a way to "bundle" these packages into the Alfred workflow so that users won't need to follow extra instructions during installation?

Sure. Just follow the pip instructions I gave, install the libraries in the workflow. That way, when you export it, the libraries will be included.

TBH, I think curl would be a better fit, though. Adding all the SSL libraries makes the workflow 13MB…

Let me see about using curl instead.

deanishe · November 20, 2017

Try this script instead. It uses /usr/bin/curl for HTTP requests. No need for megabytes of additional libraries.

# encoding: utf-8

from __future__ import print_function

import json
import sys
import subprocess
from urllib import urlencode

rawInput = sys.argv[1]
key = "5974948d3baa3d1cabc4eb00e4099e3d785d43df"

options = {
    'include-passage-references': 'false',
    'include-first-verse-numbers': 'false',
    'include-verse-numbers': 'false',
    'include-footnotes': 'false',
    'include-footnote-body': 'false',
    'include-short-copyright': 'false',
    'include-passage-horizontal-lines': 'false',
    'include-heading-horizontal-lines': 'false',
    'include-headings': 'false',
    'include-selahs': 'false',
    'indent-paragraphs': '0',
    'indent-poetry': 'false',
    'indent-poetry-lines': '0',
    'indent-declares': '0',
    'indent-psalm-doxology': '0'
}

headers = {
    'Accept': 'application/json',
    'Authorization': 'Token ' + key,
}

baseUrl = 'https://api.esv.org/v3/passage/text/'


def fetch_url(url, params, headers):
    """Fetch a URL using cURL and parse response as JSON."""
    # Encode GET parameters and add to URL
    qs = urlencode(params)
    url = url + '?' + qs

    # Build cURL command
    cmd = ['/usr/bin/curl', '-sSL', url]
    for k, v in headers.items():
        cmd.extend(['-H', '{}: {}'.format(k, v)])

    # Run command and parse response
    output = subprocess.check_output(cmd)
    return json.loads(output)


def exit_with_error(query, errMsg):
    """Show an error message in Alfred and exit script."""
    errorOutput = {
        "items": [
            {"title": query, "subtitle": errMsg}
        ]
    }
    json.dump(errorOutput, sys.stdout)
    sys.exit(0)


# Combine query and options into GET parameters
params = dict(q=rawInput)
params.update(options)

# Execute request
try:
    content = fetch_url(baseUrl, params, headers)
except Exception as err:
    exit_with_error(rawInput, str(err))

if content['canonical'] == '' or len(content['passages']) > 1:
    exit_with_error(rawInput, 'No passage found')


passageRef = content['canonical']
pageSplit = content['passages'][0].split('\n\n')
passage = ' '.join(pageSplit).rstrip()

# print passage
passageWithRef = passage + ' (' + passageRef + ' ESV)'
data = {
    'items': [{
        'arg': passageWithRef,
        'valid': 'YES', 'autocomplete': passageRef, 'title': passageRef,
        'text': {'largetype': passageWithRef},  # Show whole passage on CMD+L
        'subtitle': passage
    }]
}

json.dump(data, sys.stdout)

Edited November 20, 2017 by deanishe
Tidy up code a wee bit

deanishe · November 20, 2017

41 minutes ago, emmanuel said:

Let me know if any of you have suggestions on this though!

One feature I'd consider, and it's suggested in the API docs, is caching the results of queries, at least for a few days.

The verses/passages returned by the API aren't going to change that often, and fetching something from the network is a lot slower than fetching it from disk. Also, it could potentially reduce the usage of the API key quite a lot.

(Though I've no idea how often people are likely to re-enter the same query. I've just been repeatedly mashing in the one from the screenshot because Bible verses aren't really my thing.)

Edited November 20, 2017 by deanishe

JGC · November 20, 2017

@deanishe very many thanks for your help on this. It's now working fine, without needing the OpenSSL libraries. Community Hero indeed

emmanuel · November 20, 2017

Couldn't agree more! You rock, @deanishe!!

deanishe · November 20, 2017

I noticed an encoding error in my script when I entered "job" as the query (probably something to do with "smart" quotes).

If you don't mind, I "Pythonified" the script and implemented the caching I mentioned. The script below catches and displays API errors (apparently the API returns a detail field if something went wrong), handles non-ASCII text, and caches the results for queries for 40 days (by default), which takes the response time from ~0.8 seconds to ~0.02 seconds for responses that are in the cache.

Hitting ⌘L on the result will show the passage in Alfred's Large Type window, and ⌘C will copy the same to the clipboard.

So if you'd like to, try this:

# encoding: utf-8

"""Alfred Script Filter to search the ESV Bible."""

from __future__ import print_function

from hashlib import md5
import json
import os
import re
import subprocess
import sys
from time import time
from urllib import urlencode

API_KEY = '<redacted>'
API_URL = 'https://api.esv.org/v3/passage/text/'
API_OPTIONS = {
    'include-passage-references': 'false',
    'include-first-verse-numbers': 'false',
    'include-verse-numbers': 'false',
    'include-footnotes': 'false',
    'include-footnote-body': 'false',
    'include-short-copyright': 'false',
    'include-passage-horizontal-lines': 'false',
    'include-heading-horizontal-lines': 'false',
    'include-headings': 'false',
    'include-selahs': 'false',
    'indent-paragraphs': '0',
    'indent-poetry': 'false',
    'indent-poetry-lines': '0',
    'indent-declares': '0',
    'indent-psalm-doxology': '0'
}

# HTTP request headers
API_HEADERS = {
    'Accept': 'application/json',
    'Authorization': 'Token ' + API_KEY,
}


# Directory for this workflow's cache data
CACHEDIR = os.getenv('alfred_workflow_cache')
CACHE_MAXAGE = 86400 * 40  # 40 days


def log(s, *args):
    """Write message to Alfred's debugger.

    Args:
        s (basestring): Simple string or sprintf-style format.
        *args: Arguments to format string.

    """
    if args:
        s = s % args
    print(s, file=sys.stderr)


class ESVError(Exception):
    """Base error class."""


class NotFound(ESVError):
    """Raised if no passage was found."""

    def __str__(self):
        """Error message."""
        return 'No passage found'


class APIError(ESVError):
    """Raised if API call fails."""


class Cache(object):
    """Cache results of API queries.

    Attributes:
        dirpath (str): Path to cache directory.

    """

    def __init__(self, dirpath):
        """Create a new cache.

        Args:
            dirpath (str): Directory to store cache files.

        """
        log('cache directory=%s', dirpath)
        self.dirpath = dirpath
        if dirpath is None:  # not being run from Alfred
            return

        # Alfred doesn't create the directory for you...
        if not os.path.exists(dirpath):
            os.makedirs(dirpath)
        else:  # remove old cache files
            self.clean()

    def search(self, query):
        """Perform API query, using cached results if not expired.

        Args:
            query (unicode): Search string.

        Returns:
            Passage: Passage from API or cache.

        """
        cachepath = None
        if self.dirpath:
            cachepath = os.path.join(
                self.dirpath,
                md5(query.encode('utf-8')).hexdigest() + '.json'
            )

        # ------------------------------------------------------
        # Try to load data from cache
        if cachepath and os.path.exists(cachepath):
            # Expired cache files were deleted when `Cache` was created
            log('[cache] loading passage for "%s" from cache ...', query)
            with open(cachepath) as fp:
                data = json.load(fp)
                return Passage.from_response(data)

        # ------------------------------------------------------
        # Fetch data from API

        # Combine query and options into GET parameters
        params = dict(q=query.encode('utf-8'))
        params.update(API_OPTIONS)

        # Execute request
        data = fetch_url(API_URL, params, API_HEADERS)
        passage = Passage.from_response(data)

        if cachepath:  # Cache response
            with open(cachepath, 'wb') as fp:
                json.dump(data, fp)

        return passage

    def clean(self):
        """Remove expired cache files."""
        i = 0
        for fn in os.listdir(self.dirpath):
            p = os.path.join(self.dirpath, fn)
            if time() - os.stat(p).st_mtime > CACHE_MAXAGE:
                os.unlink(p)
                i += 1
        if i:
            log('[cache] deleted %d stale cache file(s)', i)


class Passage(object):
    """A Bible passage.

    Attributes:
        fulltext (unicode): Passage text as paragraphs
        ref (unicode): Canonical passage reference
        summary (unicode): Passage text on one line
        with_ref (unicode): Passage as paragraphs + reference

    """

    @classmethod
    def from_response(cls, data):
        """Create a `Passage` from API response.

        Args:
            data (dict): Decoded JSON API response.

        Returns:
            Passage: Passage parsed from API response.

        Raises:
            NotFound: Raised if ``data`` contains no passage(s).

        """
        if not data.get('canonical') or not data.get('passages'):
            raise NotFound()

        ref = data['canonical']
        s = data['passages'][0]
        summary = re.sub(r'\s+', ' ', s).strip()
        p = cls(ref, summary, s)
        log('---------- passage -----------')
        log('%s', p)
        log('---------- /passage ----------')
        return p

    def __init__(self, ref=u'', summary=u'', fulltext=u''):
        """Create new `Passage`."""
        self.ref = ref
        self.summary = summary
        self.fulltext = fulltext
        self.with_ref = u'{}\n\n({} ESV)'.format(fulltext.rstrip(), ref)

    def __str__(self):
        """Passage as formatted bytestring.

        Returns:
            str: Full text of passage with reference.
        """
        return self.__unicode__().encode('utf-8')

    def __unicode__(self):
        """Passage as formatted Unicode string.

        Returns:
            unicode: Full text of passage with reference.
        """
        return self.with_ref

    @property
    def item(self):
        """Alfred item `dict`.

        Returns:
            dict: Alfred item for JSON serialisation.

        """
        return {
            'title': self.ref,
            'subtitle': self.summary,
            'autocomplete': self.ref,
            'arg': self.with_ref,
            'valid': True,
            'text': {
                'largetype': self.with_ref,
                'copytext': self.with_ref,
            },
        }


def fetch_url(url, params, headers):
    """Fetch a URL using cURL and parse response as JSON.

    Args:
        url (str): Base URL without GET parameters.
        params (dict): GET parameters.
        headers (dict): HTTP headers.

    Returns:
        object: Deserialised HTTP JSON response.

    Raises:
        APIError: Raised if API returns an error.

    """
    # Encode GET parameters and add to URL
    qs = urlencode(params)
    url = url + '?' + qs

    # Build cURL command
    cmd = ['/usr/bin/curl', '-sSL', url]
    for k, v in headers.items():
        cmd.extend(['-H', '{}: {}'.format(k, v)])

    # Run command and parse response
    output = subprocess.check_output(cmd)
    log('---------- response -----------')
    log('%r', output)
    log('---------- /response ----------')
    data = json.loads(output)
    if 'detail' in data:  # 'detail' contains any API error message
        raise APIError(data['detail'])

    return data


def exit_with_error(title, err, tb=False):
    """Show an error message in Alfred and exit script.

    Args:
        title (unicode): Title of Alfred item.
        err (Exception): Error whose message to show as item subtitle.
        tb (bool, optional): If `True`, show a full traceback in Alfred's
            debugger.

    """
    # Log to debugger
    if tb:
        import traceback
        log(traceback.format_exc())
    else:
        log('ERROR: %s', err)

    # Send error message to Alfred
    output = {
        'items': [{'title': title, 'subtitle': str(err)}]
    }
    json.dump(output, sys.stdout)

    sys.exit(1)  # 1 indicates something went wrong


def main():
    """Run Script Filter."""
    log('.')  # Ensure real log starts on a new line

    # Fetch user query and decode to Unicode
    query = sys.argv[1].decode('utf-8')

    cache = Cache(CACHEDIR)
    try:
        passage = cache.search(query)
    except ESVError as err:
        exit_with_error(query, err, False)
    except Exception as err:
        exit_with_error(query, err, True)

    # Show passage in Alfred
    json.dump({'items': [passage.item]}, sys.stdout)


if __name__ == '__main__':
    start = time()
    main()
    log('------ %0.3fs ------', time() - start)

Edited November 20, 2017 by deanishe
Simplified code

deanishe · November 20, 2017

28 minutes ago, emmanuel said:

You rock

Then get clicking on that heart-type thing on my posts. I've got days to win

emmanuel · November 20, 2017

Wow. Simply amazing!! You have my heart(s). When I get home today I'll publish a GitHub repo and it will be open for pull requests. I fear that it won't be enough credit to you though @deanishe. I will be adding a shout-out to you and any links (personal GitHub/website/etc.) you provide me to all the public places. Please tell me what else I can do to give you credit for your awesome contributions!!

ESV Online Bible

Recommended Posts

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Create an account or sign in to comment

Create an account

Sign in