Bible Book Explorer Manual?

Mattillo said:

I could be mistaken but I thought when I was in the canon comparison there was no about/help option.

There's information in the right side-bar, but no About button. 

Mattillo said:

Maybe in the future, one big interactives manual could be made where everything can be found for those like me who couldn't find the about button

If only we had one place where you could start your study, with a page for each dataset and interactive ...[;)] We've discussed adding documentation pages to Factbook, but that's still on the long list of information we'd like to include. 

Faithlife Feedback link if you want to vote for this suggestion: Add dataset documentation to Factbook | Faithlife Feedback

Mattillo said:

On a related note, could you explain the stats menu to me?  I get the other options but I was trying to figure out how to use the Stats menu. There are options for Lemmas in pasage, used one in passage, roots, etc but I'm not quite sure what they are trying to show.  Could the help/about option explain those better?

I made this interactive. Howdy!

First off, you may not get value out of the Stats page, and that's okay.

That said, what it's trying to show is quantifiable relationships between the books in terms of their general (not specific) language usage. This is in the same spirit as the Charts tool, but instead of charting and comparing search results, it's charting and comparing counts of various phenomena that are difficult or impossible to construct a search for. 

I chose an X/Y "scatter" plot instead of a regular bar or column chart because I didn't just want to show frequency or counts, I wanted to show how the books are related to one another. The most important relationship (in a statistical sense) is length. Any countable thing in a book is going to be in relation to its length, and the expected frequency of that thing occurring in that book is going be a function of its length.

For example, if you compare "Lemmas" to "Words", you are doing what is known as a type-token ratio (TTR), which is one way to measure the amount of lexical density in a text. Here's a pretty succinct explanation I found (here): 

[quote]

> In analysis of text, token refers to individual words, and type to the amount of unique words.

> Type-token ratio is the division of those two, a crude measure of the lexical complexity in text. For example, if a 1000-word text has 250 unique words, it has a type/token ratio of 0.25, which is relatively low and suggests a simple writing style, or possibly legalese with a lot of repeated phrases.

> Type-token ratios of different texts are not directly comparable as words in text usually follows a power law type of distribution, implying that longer texts will almost implicitly show a larger increase of tokens than of types.

> One semi-brute way around this is to calculate type-token figures for same-sized chunks of the text, then average those for a single figure (which doesn't work too well if some documents are much shorter than the chunk size). [This is what the bar chart at the bottom of the Stats page underneath the scatter plot is showing, and why it's there. — Eli]

> Type-token ratio is often one among various statistical figures/factors in text/corpus summaries. Consider plagiarism detection: if text was copied and reworded only slightly, it will have a comparable type-token ratio.

> You may wish to apply stemming or lemmatization before calculating a type-token ratio, when you want to avoid counting simple inflected variations as separate tokens.

A text with low lexical density uses the same words over and over, and one with high density shows more variation in its terminology. Lexical density is sometimes used as a crude measure of linguistic "sophistication" but that's not the full story. For example, proper names get counted, so a list of people with no repeats has a lexical density of 1:1, but the preamble to John's gospel—a much more sophisticated text that displays a remarkable economy of language—is much lower. (I like John. He uses small words to tell about big ideas.)

It should be unsurprising to find John uses a smaller inventory of lemmas than, say, Luke does. Or that poetry generally speaking is more dense than prose. We could just put the number of lemmas on a bar chart, but that would obscure the length factor, how say, 2 Jn is much shorter than Luke. Putting it on the scatter plot lets you see which books are:

• longer and have more lemmas (higher relative lexical density)
• longer and have fewer lemmas (lower relative lexical density)
• shorter and have more lemmas (higher relative lexical density)
• shorter and and have fewer lemmas (lower relative lexical density)

If you choose "Roots" you are examining the same type-token relationship, just with a more primitive "type", the root.

You can see this kind of analysis being done on the pastoral epistles in Carson & Porter's Linguistics and the New Testament: Critical Junctures, and in other books. Search for "type token" in your library and you may find some real-world applications.

If you choose chapters or verses from the second drop down (I almost never would, to be honest) you are simply choosing a different measure for the length of a book.

So choosing "Lemma" or "Roots" is a measure of how diverse the book's vocabulary is in relation to its length.

The other options in the drop down have to do with how unique the book's vocabulary is. Hapax legomena are words that occur only once within a given corpus, eg, ἀγαθοποιός, agathopoios occurs in 1 Pet 2:14 and nowhere else in the NT. Because the meaning of a word is established by how it's used, and these words are used infrequently, they attract a lot of examination and generate a lot of debate.

If lexical density is a rough and ready way to compare overall complexity of a text, hapax are a rough approximation of its obscurity. Job has a lot of hapax, and that (among other things) makes it relatively weird. Note that some (many) hapax are just proper names, unique but not particularly obscure.

Anyway, we went one step further than the typical hapax definition of words that are used only once in the corpus. What about lemmas/roots that are used only once in the book? (We wanted to do lemmas/roots per author, eg, this word used once in Paul, but that got dicey outside of the NT letters. Might revisit.) 

How about lemmas/roots that are used in this book, perhaps many times, but in no other book in the corpus? That's a measure of the uniqueness of the book's vocabulary relative to others. Again, versus the length.

One thing that I wanted to do was to include word lists of the unique lemmas/roots per book, but time was not my ally. You can get to those lists for a book by using the Concordance tool. Make a concordance of, say, SBL Greek New Testament, and then choose Lemma and narrow it down to a single book. 

Happy to take more questions, and suggestions. As always, you can correct me where I'm wrong, and quote me where I'm right.