Smart search suggestion prioritize primary sources.

John Goodman
John Goodman Member Posts: 1,706 ✭✭✭
edited November 21 in English Forum

e.g. 'What did the council of trent teach about justification?' great search results except that the primary source is not found.

Buckley, Theodore Alois. The Canons and Decrees of the Council of Trent. London: George Routledge and Co., 1851. Session 6 - Decree on Justification.

Great to have reference articles but can it be tuned to prioritize key / primary sources too?

גַּם־חֹשֶׁךְ֮ לֹֽא־יַחְשִׁ֪יךְ מִ֫מֶּ֥ךָ וְ֭לַיְלָה כַּיּ֣וֹם יָאִ֑יר כַּ֝חֲשֵׁיכָ֗ה כָּאוֹרָֽה

Tagged:

Comments

  • Dave Hooton
    Dave Hooton MVP Posts: 35,776

    great search results except that the primary source is not found.

    I could not find it in the results, but it is a matter of opinion that it is the primary source as there are many books/articles that explore the subject and many are not at the top of the list. Prioritization is a matter of tuning the AI.

    Dave
    ===

    Windows 11 & Android 13

  • MJ. Smith
    MJ. Smith MVP Posts: 53,436

    In my library there are two related source documents:

    • Buckley, Theodore Alois. The Canons and Decrees of the Council of Trent. London: George Routledge and Co., 1851.
    • Buckley, Theodore Alois. The Canons and Decrees of the Council of Trent. London: George Routledge and Co., 1851.

    The latter is qualified by (text only) in the title but not in the citation. The primary source would be the actual canons of the Council not commentaries or discussions of the canons. To a Catholic, the Canons and Decrees of a Council are by definition the source document.

    I would suggest the confessional documents always have priority over monographs at least in Verbum.

    Vote at Confessional documents prioritized above monographs and commentaries | Logos

    Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."

  • John Goodman
    John Goodman Member Posts: 1,706 ✭✭✭

    The records and publications of the council are the primary source for its teaching on the subject that is not a matter of opinion. The only thing more primary would be a recording of the discussions or minutes of meetings. These are the published outcome of the council.

    Obviously there are many other places to read about justification but that isn't my point.

    I hope the AI could understand what is a primary source given the question. I might easily have asked the same question about what did Calvin teach about it? Then I'd prefer a reference to the Institutes over an article about him in a journal etc.

    גַּם־חֹשֶׁךְ֮ לֹֽא־יַחְשִׁ֪יךְ מִ֫מֶּ֥ךָ וְ֭לַיְלָה כַּיּ֣וֹם יָאִ֑יר כַּ֝חֲשֵׁיכָ֗ה כָּאוֹרָֽה

  • Dave Hooton
    Dave Hooton MVP Posts: 35,776

    The records and publications of the council are the primary source for its teaching on the subject that is not a matter of opinion

    This would require the AI to know about primary sources:

    1. "Council of Trent" (and other events) need to be tagged with a list of source documents.
    2. The same would have to be done for Calvin and other authors!
      1. When restricted to Systematic Theology (Books Search), Calvin's book was well down the list of results for "What does Calvin teach..."

    But it would introduce a significant bias to the parsing as books that mention both "Council of Trent"  (or "Calvin") and "justification" will not be preferred (as is currently the case).
    If not source documents, Confessional/Catechism books as proposed by Martha are similarly biased to the subject alone e.g. "The Westminster..." books (except for an uncomplimentary reference to the Council).

    I think it would be better to perform a Smart Books Search on the (tagged) books you prefer.

    Dave
    ===

    Windows 11 & Android 13

  • MJ. Smith
    MJ. Smith MVP Posts: 53,436

    This would require the AI to know about primary sources:

    My suggestion is simply to give priority to Confessional documents (and possibly Catechisms) which I know from years of searching on collections yields good results for confessional and ACELO churches. Yes, it might create some pressure to convert some resources to Confessional documents, but I don't see that as a drawback. And it preserves the advantages of an All smart search. It may be that it should be limited to Verbum but that would add more pressure for a number of Protestant users to move to Verbum to meet their needs.

    Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."

  • Dave Hooton
    Dave Hooton MVP Posts: 35,776

    MJ. Smith said:

    My suggestion is simply to give priority to Confessional documents (and possibly Catechisms) which I know from years of searching on collections yields good results for confessional and ACELO churches. Yes, it might create some pressure to convert some resources to Confessional documents, but I don't see that as a drawback.

    It is not "simple" and there are massive drawbacks to converting resources to those types purely because they may be considered "source documents". The mere mention of "justification" would attract them as primary results when other types are more suitable and present a balanced view of the subject i.e. for and against. As is, the top Verbum results of the this query are more favorable to the Catholic view, contrasting it to the Lutheran and Protestant view.  From my point of view, I do not desire the Westminster confessionals as primary results when I am also querying the Council of Trent.

    Dave
    ===

    Windows 11 & Android 13

  • John Goodman
    John Goodman Member Posts: 1,706 ✭✭✭

    Here is an example which I think shows how such a feature could develop. Working this way combines the concept of search with summarization but goes further to contrasting and explaining. In Logos, this answer could include links and citations to the actual books in my library.

    Can you briefly contrast Thomas Aquinas and Calvin on justification?

    ChatGPT

    Certainly! Thomas Aquinas and John Calvin hold distinct views on the doctrine of justification, reflecting their respective theological traditions: Catholicism and Protestantism.

    Thomas Aquinas on Justification
    Theological Tradition: Catholic

    Nature of Justification: Aquinas sees justification as a transformative process that involves both the forgiveness of sins and the sanctification of the individual.
    Role of Grace: For Aquinas, grace is both prevenient and infused. It precedes human effort and is infused into the soul, transforming it and enabling cooperation with God's will.
    Faith and Works: Aquinas maintains that faith, hope, and charity (love) are infused together at justification. Good works, prompted by grace, are essential in the process of maintaining and increasing justification.
    Sacraments: The sacraments, especially baptism, play a crucial role in the justification process. Baptism cleanses original sin and infuses grace.
    John Calvin on Justification
    Theological Tradition: Protestant (Reformed)

    Nature of Justification: Calvin understands justification as a forensic act, where God declares a sinner righteous based solely on the righteousness of Christ imputed to them.
    Role of Grace: For Calvin, grace is imputed rather than infused. This imputation involves Christ’s righteousness being credited to the believer's account.
    Faith Alone: Calvin emphasizes sola fide (faith alone). Justification is through faith alone, apart from works, which are the fruit and evidence of faith but not the basis of justification.
    Sanctification Distinction: While sanctification is a separate process that follows justification, it is inevitably linked to it. The justified person will produce good works, but these works are the result, not the cause, of justification.
    Key Differences
    Transformative vs. Forensic: Aquinas views justification as a transformative process involving the infusion of grace, while Calvin views it as a forensic declaration by God.
    Role of Works: Aquinas incorporates good works as necessary to maintain and grow in justification, whereas Calvin sees good works as a result of justification, not a means to achieve or maintain it.
    Nature of Grace: For Aquinas, grace is infused into the believer, changing their nature. For Calvin, grace is imputed, with Christ's righteousness covering the believer.
    These differences highlight the broader theological divergences between Catholic and Reformed thought during the medieval and Reformation periods.

    What are the primary sources for further reading?

    ChatGPT
    For a deeper understanding of Thomas Aquinas and John Calvin's views on justification, the primary sources are their own writings, as well as authoritative works and commentaries on their theology. Here are some key texts:

    Thomas Aquinas
    Summa Theologica
    Relevant Sections: Part I-II (The First Part of the Second Part), Question 113 ("Of the Effects of Grace")
    Description: This section of Aquinas's magnum opus deals extensively with grace and justification, outlining his views on the transformative nature of justification and the role of the sacraments.
    Access: Available in various translations, both in print and online.
    Summa Contra Gentiles
    Relevant Sections: Book 4, Chapters 53-55
    Description: Provides additional insights into Aquinas's understanding of grace, salvation, and the nature of divine justice.
    Access: Also available in multiple translations and formats.
    John Calvin
    Institutes of the Christian Religion
    Relevant Sections: Book III, Chapters 11-18
    Description: These chapters cover Calvin’s doctrine of justification by faith, imputation of Christ’s righteousness, and the relationship between faith and works.
    Access: Widely available in various translations, with the 1559 edition being the most comprehensive.
    Commentaries on the Pauline Epistles
    Relevant Sections: Particularly on Romans and Galatians
    Description: Calvin’s exegesis of Pauline texts provides practical insights into his theological stance on justification.
    Access: Available in various translations and often found online.
    Secondary Sources
    "Aquinas: An Introduction to the Life and Work of the Great Medieval Thinker" by F.C. Copleston
    Description: Offers an accessible overview of Aquinas's life, context, and key theological contributions, including his doctrine of justification.
    "Calvin: Institutes of the Christian Religion" edited by John T. McNeill
    Description: A comprehensive edition of Calvin’s Institutes with detailed introductions and notes, providing historical and theological context.
    "The Theology of John Calvin" by Charles Partee
    Description: An in-depth analysis of Calvin’s theological system, with substantial focus on his views on justification.
    These primary and secondary sources provide a thorough foundation for studying the contrasting views of Thomas Aquinas and John Calvin on justification.

    גַּם־חֹשֶׁךְ֮ לֹֽא־יַחְשִׁ֪יךְ מִ֫מֶּ֥ךָ וְ֭לַיְלָה כַּיּ֣וֹם יָאִ֑יר כַּ֝חֲשֵׁיכָ֗ה כָּאוֹרָֽה

  • MJ. Smith
    MJ. Smith MVP Posts: 53,436

    It is not "simple" and there are massive drawbacks to converting resources to those types purely because they may be considered "source documents". The mere mention of "justification" would attract them as primary results when other types are more suitable and present a balanced view of the subject i.e. for and against. As is, the top Verbum results of the this query are more favorable to the Catholic view, contrasting it to the Lutheran and Protestant view.  From my point of view, I do not desire the Westminster confessionals as primary results when I am also querying the Council of Trent.

    You are attributing very different effects than I do when making my suggesting. Giving them priority does NOT imply making them the primary results (I assume you mean top 10 or such). Giving them priority simply gives them a better chance of landing in the first page (top 50) of results. This would especially be true when a confessional document is specifically mentioned in the request as in the example of the OP. If no confessional document is mentioned, the increased likelihood of appearing in the top 50 should be small as the resource priority is only one piece of information affecting the ranking of results. If I anticipated the effects you are anticipating, I would not have made the suggestion.

    Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."

  • John Goodman
    John Goodman Member Posts: 1,706 ✭✭✭

    I agree MJ except that from my perspective I don't mind whether the source is a confessional or not, I'd like the AI to be able to understand better what I'm looking for from the question itself.

    I agree with many times you have rightly pointed out, Logos / Verbum need a lot of tuning to bring out things like confessional documents, liturgy, awareness of lectionaries etc. 

    גַּם־חֹשֶׁךְ֮ לֹֽא־יַחְשִׁ֪יךְ מִ֫מֶּ֥ךָ וְ֭לַיְלָה כַּיּ֣וֹם יָאִ֑יר כַּ֝חֲשֵׁיכָ֗ה כָּאוֹרָֽה

  • MJ. Smith
    MJ. Smith MVP Posts: 53,436

    I agree MJ except that from my perspective I don't mind whether the source is a confessional or not,

    What do you propose AI consider "source" material? That is the key to any implementation or evaluation of the quality of the results. Remember that despite all the hoopla computers still are binary yes-no and understand nothing more. Everything else is calculated probabilities, implied or explicit paths, ...we are a long way from AI understanding the concept of source documents as a general concept although it can build probabilities that some types of documents are source documents.

    Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."

  • John Goodman
    John Goodman Member Posts: 1,706 ✭✭✭

    Source material is contextual to the question. If my question is, what did the Council of Trent teach about justification, then the conciliar documents are source material. If I ask about a particular theologians opinion then their own writings etc. If I ask simply for resources on the theology of justification then I think it is already a good search result. I just think it is odd for articles and commentary to be prioritised above these kinds of resources.

    ChatGPT didn't do too badly although it had the Institutes as both a primary and secondary source for Calvin's thinking above.

    גַּם־חֹשֶׁךְ֮ לֹֽא־יַחְשִׁ֪יךְ מִ֫מֶּ֥ךָ וְ֭לַיְלָה כַּיּ֣וֹם יָאִ֑יר כַּ֝חֲשֵׁיכָ֗ה כָּאוֹרָֽה

  • John Goodman
    John Goodman Member Posts: 1,706 ✭✭✭









    It does appear ChatGPT can distinguish.


    I asked: Are Calvin's institutes a primary or secondary source when considering a question about Calvin's theological views?


    Answer:

















    Calvin's Institutes of the Christian Religion is a primary source when considering a question about Calvin's theological views.









    Why Institutes of the Christian Religion is a Primary Source:

    • Authorship: It was written by John Calvin himself, making it a direct account of his theological positions.
    • Content: The work systematically outlines Calvin's own theological ideas, doctrines, and interpretations, providing first-hand insight into his thought.
    • Historical Context: It is a foundational text in Reformed theology, directly reflecting the theological debates and issues of Calvin's time as he understood and responded to them.

    In contrast, secondary sources would include books and articles written by other scholars analyzing, interpreting, or critiquing Calvin's Institutes and other works.

    I asked: Why did you list "Calvin: Institutes of the Christian Religion" edited by John T. McNeill" as a secondary source then?



















    Answer: You're correct to point out the distinction. The listing of "Calvin: Institutes of the Christian Religion" edited by John T. McNeill could be seen as a bit confusing because it appears to blur the lines between primary and secondary sources. Let me clarify:


















    The Dual Nature of Edited Works

    • Primary Source Component: The actual text of Calvin’s Institutes of the Christian Religion remains a primary source because it contains Calvin's original writings and theological views.
    • Secondary Source Component: The introduction, footnotes, commentary, and analysis provided by the editor (John T. McNeill in this case) constitute secondary material. These elements interpret, contextualize, and critique Calvin's work.

    Clarifying the Listing

    The book "Calvin: Institutes of the Christian Religion" edited by John T. McNeill includes both Calvin’s primary text and secondary analysis by McNeill. It is commonly used because:

    • Primary Source: It provides access to Calvin’s original writings.
    • Secondary Source: McNeill’s contributions help readers understand the historical context, theological nuances, and scholarly interpretations of Calvin’s work.

    Therefore, the Institutes itself is a primary source, but when referring to the specific edition edited by McNeill, one is also referring to the secondary scholarly apparatus that accompanies the primary text.

    To avoid confusion, it’s often helpful to specify that you’re referring to Calvin’s original text when discussing his theological views, while recognizing that edited versions may include valuable secondary commentary.









    גַּם־חֹשֶׁךְ֮ לֹֽא־יַחְשִׁ֪יךְ מִ֫מֶּ֥ךָ וְ֭לַיְלָה כַּיּ֣וֹם יָאִ֑יר כַּ֝חֲשֵׁיכָ֗ה כָּאוֹרָֽה

  • MJ. Smith
    MJ. Smith MVP Posts: 53,436

    I was thinking primary sources in a broader context. I offered a solution that would meet my needs and your needs as I misunderstood them. Given that Logos is using pre-trained 3rd party sources. I'll offer no suggestion for your particular case.

    Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."

  • John Goodman
    John Goodman Member Posts: 1,706 ✭✭✭

    but do you think it has merit if its possible? Even if you don't know how it could be done?

    גַּם־חֹשֶׁךְ֮ לֹֽא־יַחְשִׁ֪יךְ מִ֫מֶּ֥ךָ וְ֭לַיְלָה כַּיּ֣וֹם יָאִ֑יר כַּ֝חֲשֵׁיכָ֗ה כָּאוֹרָֽה

  • MJ. Smith
    MJ. Smith MVP Posts: 53,436

    I prefer to withdraw from the discussion because I frame the question differently than you do. To me, theological source documents consist foremost as Scripture and confessional documents. In this context it is not realistic to assume that the person writing the query will know the name(s) of the relevant source documents. For the cases you appear to be referencing, I would limit the search to the source document, optionally adding commentaries upon that source document. I have no objection to your request; it simply doesn't fill the entire need as I see it.

    Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."

  • John Goodman
    John Goodman Member Posts: 1,706 ✭✭✭

    גַּם־חֹשֶׁךְ֮ לֹֽא־יַחְשִׁ֪יךְ מִ֫מֶּ֥ךָ וְ֭לַיְלָה כַּיּ֣וֹם יָאִ֑יר כַּ֝חֲשֵׁיכָ֗ה כָּאוֹרָֽה

  • Dave Hooton
    Dave Hooton MVP Posts: 35,776

    MJ. Smith said:

    I prefer to withdraw from the discussion because I frame the question differently than you do. 

    That is a good place for me as well.

    Dave
    ===

    Windows 11 & Android 13

  • Roger Pitot
    Roger Pitot Member Posts: 195 ✭✭

    It's clear that the words used to frame the question are critical in the Smart Search.

    I tried " Are Calvin's institutes a primary or secondary source for Calvin's theological views" and got some interesting results, many from Journals. When I removed "or secondary" I got some of the same references but some different.

    What I learned from this is that it's worth asking Smart Search differently-worded questions, so thanks for that lesson MJ!