Logos and AI transparency

I have some misgivings about the use of AI in Logos. I want it, but I'd also like transparency about how it is used.

My own experience with GPT 4 and Copilot reveal increasing, rather than decreasing errors, and sometimes what appear to be intentionally misleading conclusions. (Ever try to persuade Copilot to change its view on a woke issue?) Such AI may rate its own answer confidence high when it is simply wrong. I explored this at length yesterday.

I am not going into detail here, because anyone can test this for themselves. My concern is when deception, unintentional or intentional, is introduced by LLM AI and could be extended to Logos users.

Rather than write an encyclopedia about what all this means, just see this:

https://www.pnas.org/doi/full/10.1073/pnas.2317967121

Summary (from Schneier)

GPT-4, for instance, exhibits deceptive behavior in simple test scenarios 99.16% of the time (P < 0.001). In complex second-order deception test scenarios where the aim is to mislead someone who expects to be deceived, GPT-4 resorts to deceptive behavior 71.46% of the time (P < 0.001)

For transparency, what usage or filters may mitigate such errors in Logos?

Find more posts tagged with

Comments

Bradley Grainger (Logos)

For transparency, what usage or filters may mitigate such errors in Logos?

That's an interesting paper, but I don't understand the relevance to Logos. We do not use prompts instructing an LLM to deceive a human (see Appendix 01 from that paper for what they were actually testing).

Also, if you haven't seen it already, please read Mark's blog post, particularly this section:

3. We’re transparent and open about our use of AI

We never use AI in the app without telling the user we’re doing so. This allows Christians to choose whether and how to use those tools for themselves. AI-generated content is always clearly indicated, so no student or pastor can unknowingly use AI-generated content in their papers or sermons.

GaoLu

For example, I can readily imagine a slant affecting:

Sermon illustrations
Outlines
Summaries

MJ. Smith

I can readily imagine a slant affecting:

Why bother with imagining. The material used to train ChatGPT was broad rather than curated and the results show a distinct bias that is not compatible with my theology. I get distinctly better results from Gemini which is less prone to hallucinations and more willing to state it cannot answer a question. But I have not found the bias in ChatGPT to be particularly worse than the bias in Logos itself. Because different theologies use the same words to mean different things, I do not expect any tool dealing with Christianity to be unbiased, nor do I expect any AI/Search tool to distinguish between the multivalent theological terms. What I do expect of myself is due diligence in rooting out the bias (and, unfortunately, as I am human inserting my own bias.)

Note: the reading list Hermeneutics master list (logos.com) has a list of seminal books on hermeneutics from ChatGPT and Gemini that illustrates how differently they response to the same prompt.

DMB

Using AI in Logos (that seems as best as can be described so far, since GauLu isn't having much luck) ...

Is like mindlessly pulling books from the library, saying I'll have to check each one to be sure. By and large, I follow real people's judgments (eg references, bibliographies, etc).

If this sounds like criticism, it's not. I guess I'm wondering if our Tennessee wood-worker's curated library makes more sense?

John

I have some misgivings about the use of AI in Logos

I have been following Apples announcements as to how it will implement AI.

Apples own presentation">
A cornerstone of Apple Intelligence is on-device processing, which delivers personal intelligence without collecting users’ data. Private Cloud Compute sets a new standard for privacy in AI, with the ability to flex and scale computational capacity between on-device processing, and larger, server-based models that run on dedicated Apple silicon servers.

All the big tech companies are rushing into this. Google AI has already been shown to be completely biased in its results. Not surprising, given that they have been specializing in manipulating public opinion by controlling search results and advertising for decades now.

Microsofts "Recall" feature that records everything you do on your computer has not even been released yet, and it has been hacked into and the "safe encrypted data" all stolen: This Hacker Tool Extracts All the Data Collected by Windows’ New Recall AI

Apple appears to be the only company that has a well thought out plan to protect users privacy.

Apples plan is to have a 3 level system.

Level 1 is based completely on local data, and processing takes place on the users device.

Level 2 goes to a higher level cloud based server, which is completely controlled by Apple and will be transparent in its data handling and security practices, with Apples promise to protect your privacy.

Level 3 goes out to public cloud services, in which privacy would otherwise be compromised.

This is covered in more detail in this summary of the WWDC conference: Apple WWDC 2024 keynote in 18 minutes

I also found Leo Laporte's podcast to be very helpful.

I have two concerns with what I know so far about Logos plans for AI. The first is that there is no level 1 that is localized to the users machine.

The second is that the AI being utilized apparently works on the surface text of Logos resources just like it would any other text. It does not take advantage of the tagging and indexing that users have paid for and which was intended to provide precise and accurate search results.

Does this not mean that any competitor could do the same thing and get the same or comparable results, without the large investment in tagging the resources?

I hope Logos realizes that Microsoft and Google (and unfortunately, OpenAI too now that Microsoft has a controlling interest) are not promoting this technology for the good of humanity.

Elon Musk considers OpenAI to be a threat to humanity now that it is controlled by Microsoft, and he is also very unhappy about Apple's deal to use ChatGPT as the third level. Apple is promising to protect individual users by channeling requests and responses through its own cloud servers.

As this article explains, Musk considers any integration of AI into an operating system to be a security risk.

Elon Musk threatens to ban iPhones and Macs at his companies">
"Apple’s AI approach is actually more privacy conscious than others, with its aim to process as much on the device as possible and with a fallback that explicitly asks users before sending information to OpenAI,” said Catherine Flick, professor of ethics and games technology at Staffordshire University in England. "

Until WWDC, many in the media were claiming Apple was "behind in AI". I do not believe that. As the presentation showed, they are integrating AI capability into Apple products in ways that produce practical and useful results. And they are not compromising their promise to the customer to protect their privacy. Those other companies never made any promise of privacy.