is it possible to reverse engineer scripture quotations in books?

When Spurgeon, for example writes "On the temple’s pinnacle he will tempt thee, saying, “Cast thyself down;” and on the mountain’s highest peak he will attack thee with. “Bow down and worship me,"

would it be possible to make “Cast thyself down;” and “Bow down and worship me," as tags to Scripture text, similar to what Mt 4:6 reference would become.

potential issues:

not all quotes are scripture quotes. nothing can be done about that.
Scripture translations. most authors consistently use one translation in their book or specifically mark the quote with translation used. Setting a reference token for a book would produce consistent results.
multiple hits for Gospel quotes. "Cast thyself down" would have returned Mt 4:6 and Lk 4:9. However Logos could display both
author deliberately misquotes Scripture (whether for clarity, as substituting a pronoun for example or other reasons).

Find more posts tagged with

Comments

TCBlack

You mean that by hovering over the phrase "cast thyself down" MT 4.6 and LK 4.9 would pop up in the dialog?

Hmm. That might be good, especially in sermons and some of the other monographs.

Matthew C Jones

https://community.logos.com/discussion/comment/590878#Comment_590878

You mean that by hovering over the phrase "cast thyself down" MT 4.6 and LK 4.9 would pop up in the dialog?

Hmm. That might be good, especially in sermons and some of the other monographs.

I think it is a great idea. Many preachers have a habit of not giving scripture references. It may be tedious to accomplish on a global scale

William Gabriel

That is a phenomenal idea--that would be hot stuff.

Rich DeRuiter

https://community.logos.com/discussion/comment/590884#Comment_590884

You mean that by hovering over the phrase "cast thyself down" MT 4.6 and LK 4.9 would pop up in the dialog?

Hmm. That might be good, especially in sermons and some of the other monographs.

I think it is a great idea. Many preachers have a habit of not giving scripture references. It may be tedious to accomplish on a global scale

It is quite tedious.

I did this once for "Distinguishing Marks of a Work of the Spirit of God" by Edwards. It took a very long time, especially since Edwards (as Spurgeon above) does not always quote verbatim, so some allusions to Scripture can be a bit obscure. I remember this took me hours to do, and this was only a 43pg document (in Word).

Given the gamut of Bibles used by authors, the variations in synonyms and figures of speech over the centuries and across continents, I'm not sure there's a computer program yet developed that could reliably identify these.

toughski

https://community.logos.com/discussion/comment/590899#Comment_590899

It is quite tedious.

...

Given the gamut of Bibles used by authors, the variations in synonyms and figures of speech over the centuries and across continents, I'm not sure there's a computer program yet developed that could reliably identify these.

But this is the stuff computers excel at. Throw some Fuzzy search in the mix and it is a piece of cake [;)]

Ok, it is hard. But think of how humans do it:

Step 1 - I copy the phrase in a Bible search. Chances are I get immediate results. Especially if my assumptions about the Bible version are correct
Step 2 - if the previous step did not produce desired results, I start thinking about what the author could have "messed up" - pronouns, added words, skipped words, changed "loveth" to "loves", ...
Step 3 - Again, if the previous steps went nowhere, I try to identify a unique word or phrase, that I am sure would yield some results
Step 4 - if the Bible search proved fruitless, I can try Full library search
Step 5 - if everything else fails, I would search the internet.

As you can see, most of these steps quite doable to program [8-|]

William Gabriel

https://community.logos.com/discussion/comment/590899#Comment_590899

It would be tough, but:

1) Logos is already going through resources in a tedious manner for all kinds of other tagging (e.g. speaker labels)

2) A computer would get you most of the way

-This is how they do a lot of their work currently, right? It's not like their OCR or other reference pick ups are perfect

-They could employ a labor force to research some of the tougher quotes (and sure, in the end some may remain ambiguous)

-There is a huge pool of willing users who would help round out the final problems. With something like "Report Typo" -- which should probably be expanded to some other categories by this point -- this project would get trounced

3) This feature would be enormously useful for research and be another thing that sets Logos apart from the pack

Rich DeRuiter

https://community.logos.com/discussion/comment/590905#Comment_590905

As you can see, most of these steps quite doable to program

The hard part is not getting results, the hard part is getting accurate and reliable results. You use the phrase "desired results" in step 2. How would you design a computer program to evaluate the appropriateness of a Scripture reference given the context of the resource. Humans do this quite easily (and not totally reliably either!), but for a program to do this requires evaluating not only the words & phrases but also the context in which a Scripture is being quoted.

Let me give an example. Suppose I was writing about the importance of being "in the world but not of it." There is a Scriptural allusion there, but it is not a direct quote (it's actually a popular way of summarizing of an actual Scriptural quote). If I start dialing down, eliminating one word at a time from the end of the phrase, searching all Bibles, I eventually get results, but none of them are accurate. Interestingly it gets close (with John 17:11 coming up in several versions), but it doesn't get to John 17:15-17 (which the phrase alludes to).

What if I said something in that context like "come out and be different." That would be an allusion to a text as well, but it would be difficult for a computer program to sort through what would be appropriate or not, especially since "different" and "separate" are not synonyms, but rather "different" would be an interpretive result of what it looks like to "come out" [of worldliness, e.g.].

We humans can see the connections, interpret appropriateness, filter out spurious results (e.g. Matt. 13:49 (ESV)), and quickly go to 2Cor. 6:17, in fact we may have a hunch where to look before we even begin our search, because of our experience with "church speak" or "Christianese." If we're clever enough we might even know when something is not in the Bible at all, such as "God helps those who help themselves" (from Poor Richard's Almanac) or "God moves in mysterious ways" (from William Cowper's famous hymn).

But the most difficult part is that not every phrase would be a candidate for searching, only those that have some allusion to Scripture. When is a phrase not an allusion to Scripture, or not intended to be? We humans have a sense when someone is making an allusion because we are reading meaning, not just words. Computers can't do that (at least not yet). If I say we should be thankful for Logos and all it can do, is "be thankful" an allusion to Colossians 3:15 (the very end of that verse). No. If I say "Gratitude should characterize our lives; we should be thankful," is "be thankful" an allusion to that verse? Maybe, the larger context might help me decide.

All that said, I wish it were possible to do this with a computer program, and some day it could be, but we're not there yet.

MJ. Smith

https://community.logos.com/discussion/comment/590899#Comment_590899

I'm not sure there's a computer program yet developed that could reliably identify these.

Some fuzzy search algorithms do quite well. However, there is an additional problem of searching only translations available at the time the text was written - Logos normally carries only the latest or a couple of major editions of a particular translation. But it does add to my case that the metadata should include the translation(s) used by default.

MJ. Smith

https://community.logos.com/discussion/comment/590925#Comment_590925

But the most difficult part is that not every phrase would be a candidate for searching, only those that have some allusion to Scripture.

This is a problem that is unmanageable in such a wide ranging corpus - especially since we don't agree on canon and many would like to include early church fathers and pseudepigraphia. I would suggest a right click function rather than a hover.

JT (alabama24)

https://community.logos.com/discussion/comment/590996#Comment_590996

the metadata should include the translation(s) used by default.

good idea MJ. [:)] [Y]

Rosie Perera

https://community.logos.com/discussion/comment/590925#Comment_590925

But the most difficult part is that not every phrase would be a candidate for searching, only those that have some allusion to Scripture.

This in my opinion is the main thing that makes an automated solution to this unworkable at this stage. We as humans when reading a book will be easily able to read over three or four paragraphs that have no specific Scriptural allusions in them and go straight for the next sentence that looks like it might have one and start searching there. The computer program doesn't know what any of the text actually means, so it would have to plod along searching for every potential phrase. And where are the phrase boundaries? It would have to try words 1-5, say, in the first sentence, and try all the steps toughski listed above, and if that came up with nothing, then move ahead to words 2-6, and maybe 2-7, and all the other potential overlapping phrase units in the first sentence. And then on to the next, etc. It quickly grows exponentially in the number of times through the loop, which in computer lingo becomes an "N-squared" (or even more, maybe "N-factorial") algorithm. That is, the longer the input text, the slower the process becomes, not just as a geometric multiple, but as the square of the length grows, which is huge. It's not something you can "optimize" away; it's inherent in the nature of the algorithm that it can't be fast enough to be practical in real time.

MJ. Smith

https://community.logos.com/discussion/comment/591022#Comment_591022

re: feasibility http://www.accordancebible.com/Amplify-To-Perform-A-Fuzzy-Search

Rich DeRuiter

https://community.logos.com/discussion/comment/591057#Comment_591057

re: feasibility http://www.accordancebible.com/Amplify-To-Perform-A-Fuzzy-Search

It's one thing to start from a known phrase and find a fuzzy fit (which probably still has to be deemed correct). It's not that hard to do it with a little imagination and patience with a hard search either (usually).

It's a very different thing to start with a 500 page work, and find all the Scriptural allusions in it. I believe it's this second thing that the OP is asking for, for the reasons I stated, and those that Rosie also stated.

MJ. Smith

https://community.logos.com/discussion/comment/591073#Comment_591073

It's a very different thing to start with a 500 page work, and find all the Scriptural allusions in it. I believe it's this second thing that the OP is asking for, for the reasons I stated, and those that Rosie also stated.

I recognized this when I suggested a right click function. Logos has not implemented some of the basic corpus analysis tools that normally precede any attempt to scan for quotes and allusions within a fixed corpus. Bob and I had a discussion on this topic a couple of years ago. Logos made their development choice based on the corpus being only Scripture. Under that assumption their choice makes perfect sense.

MJ. Smith

https://community.logos.com/discussion/comment/591078#Comment_591078

My warped sense of humor is showing: isn't that what plagiarism software does?[:D]

David Paul

https://community.logos.com/discussion/comment/590998#Comment_590998

the metadata should include the translation(s) used by default.

good idea MJ.

I think I supported this suggestion from MJ in a long-ago thread. I still like it. There are many places (the NET Bible, for instance) where a specific Bible is being referenced--it is practically pointless to get a default "favorite Bible" hyperlink from such references. This should be an obvious and automatic conclusion when creating these links. Even if it takes forever to re-engineer these links, all links from this point forward OUGHT to have this metadata included. Creating them in this way is apt. Failing to do so is inept.

Mark Barnes

https://community.logos.com/discussion/comment/590899#Comment_590899

Given the gamut of Bibles used by authors, the variations in synonyms and figures of speech over the centuries and across continents, I'm not sure there's a computer program yet developed that could reliably identify these.

A computer algorithm could definitely do it, but not on demand. It would take considerable processing power.

In fact, such algorithms already exist and power anti-plagiarism software for University lecturers, like TurnItIn. It would certainly be possible (though not easy) for Logos to create visual filters powered by the cloud to add this information to popular resources.

MJ. Smith

https://community.logos.com/discussion/comment/591110#Comment_591110

It would take considerable processing power.

What I don't know is the effect of the much more limited corpus that would need searched. TurnItIn limits the source rather than the corpus to which it is compared. I would think that with the ability to further constrain the corpus by probable/available translations/sources and looking only for allusions/quotations in scripture/psuedepigraphia/early church fathers it could be on demand. The alternative is to do the processing once and store the results. This should move Logos towards recognition that the user needs to be able to tag and annotate Logos generated/curated data - again a feature I have strongly supported.

Rosie Perera

https://community.logos.com/discussion/comment/591112#Comment_591112

Here's another thing that would make this a difficult task. Often authors will interpose other words that are not part of the biblical quote in the midst of their text. Take for example this sentence I came across in a book in my Library:

Man, saith Job, is born to labour.

Of course any of us would immediately recognize that as probably a quote from the book of Job, and we'd go search not for the whole sentence but just for "Man is born to labour." But how would a computer know to leave out the "saith Job" part in its search? And one can imagine even more complicated interpositions of words in the midst of a quotation or allusion which would make finding them in Scripture difficult for an algorithm.

As it turns out, even that exact text is only found in Douay-Rheims and the Septuagint Version of the Old Testament: English Translation. Not in any mainstream translations. KJV has "man is born unto trouble" and all other English translations have something similar, with trouble or distress as the noun there, not labour/labor.

MJ. Smith

https://community.logos.com/discussion/comment/591630#Comment_591630

But how would a computer know to leave out the "saith Job" part in its search?

Speech and Language Processing, 2nd Edition by Daniel Jurafsky and James H. Martin combined with Probabilistic Graphical Models: Principles and Techniques (Adaptive Computation and Machine Learning series) by Daphne Koller and Nir Friedman will answer this question.

is only found in Douay-Rheims .... Not in any mainstream translations.

Douay-Rheims is mainstream with some people still wanting to hold on to it much as to the KJV.

toughski

https://community.logos.com/discussion/comment/591633#Comment_591633

Quit saying "how would computer recognize this" or "how can it know that..."

Computers WILL know and recognize because smart people program it to. Case in point:

I needed to remind myself to call and order milk for my family at 7pm, so when I said to Siri in the morning "remind me to call milk at 7," she entered "Reminder: call Milk at 7pm"

Question: why did it capitalize Milk?
How did it know that I meant 7PM and not AM (next morning)

in the end, I would welcome a lot of false positives over no result at all.

LOGOS, are you up to the challenge?