Logos 4.0d suggestion: Optimise phrase searching

2

Comments

  • Dave Hooton
    Dave Hooton MVP Posts: 35,770

    I don't get it. Either there's an incompatibility between OR and NEAR, or Bradley can make that type of search work as the user intended. I don't see how it can be both.

    Sorry, Rosie - that was a poor "explanation".

    This might have been better "Bradley has explained the
    incompatibility between the output of a Boolean expression (OR, AND) and a proximity term (BEFORE, NEAR)  but it is still possible to make that type of search work as the user intended."
    , however Todd has provided a more complete explanation.

    Dave
    ===

    Windows 11 & Android 13

  • Matthew C Jones
    Matthew C Jones Member Posts: 10,295

    In the US Summer is June, July, and August.

     

    Not really. It is June 21st through September 20th.  Astronomically based, of course. [S] [*]

    And since God told us the heavenly lights were for signs, I'd defer to Genesis over my local school board's schedule . [C]

    Logos 7 Collectors Edition

  • Kevin Becker
    Kevin Becker Member Posts: 5,604 ✭✭✭

    Not really. It is June 21st through September 20th.  Astronomically based, of course. SleepStar

    And since God told us the heavenly lights were for signs, I'd defer to Genesis over my local school board's schedule . Coffee

    I knew someone would go and look up the specifics! [:D]

  • Matthew C Jones
    Matthew C Jones Member Posts: 10,295

    I knew someone would go and look up the specifics! Big Smile

    I didn't need to look it up.

    My father bought me a telescope for my 12th birthday and took me to the top of Mt Zao to show me the rings of Saturn. It sparked a love of astronomy in my young heart. The telescope is long gone but the memory is treasured forever.

    Logos 7 Collectors Edition

  • JimTowler
    JimTowler Member Posts: 1,383 ✭✭✭

    Thanks for the answers re summer.

    I never get why mags come out titled "Fall 2010 Issue" or something. I never know when that is or when to expect the next one.

    Its Winter here, a massive strorm outside, very wet and windy, and 4:38 am Tuesday. See my problem.

  • Mark Barnes
    Mark Barnes Member Posts: 15,432 ✭✭✭

    I don't get it. Either there's an incompatibility between OR and NEAR, or Bradley can make that type of search work as the user intended. I don't see how it can be both. I do remember a thread explaining why (a,b) NEAR c works while (a OR b) NEAR c doesn't, but I don't remember why and can't find the thread.

    OR and comma are not the same, though they give the same results in some situations. This is from the Wiki:


    Using lists

    Lists are a very useful feature which provide shortcuts in a number
    of searches. A list is written like this: (term1, term2, term3,
    etc.)
    . When Logos encounters a list, it performs the search
    using just term1. Then it repeats the search using just term2, then with
    just term3, etc. Once it has finished, it then ORs the
    results. Here are some examples:

    • (Jesus, Christ) is equivalent to Jesus
      OR Christ
    • (Jesus, Christ) AND love is equivalent to (Jesus
      AND love) OR (Christ AND love)

    Lists are most useful when used with fields (see
    below), or when trying to ensure proximity operators are only used in
    the outer terms of your search. For example:

    • The search described earlier (master NEAR love) OR
      (master NEAR serve) OR (neighbor NEAR love) OR (neighbor NEAR serve)

      can be simplified to (master, neighbor) NEAR (love, serve).
      Logos treats the two lists separately, iterating through them until
      every combination as been reached, like this:
      • master NEAR love
      • master NEAR
        serve
      • neighbor NEAR love
      • neighbor
        NEAR serve

    Please note: Some people get confused as they equate the list
    with the OR command. They are not the same, even though in a very basic
    search they will perform in the same way. Remember, Logos iterates
    through lists, then ORs the results
    .

    This is my personal Faithlife account. On 1 March 2022, I started working for Faithlife, and have a new 'official' user account. Posts on this account shouldn't be taken as official Faithlife views!

  • Ron
    Ron Member Posts: 1,229

    OR and comma are not the same, though they give the same results in some situations. This is from the Wiki:

    So the short version is that 99% of the time we should be using commas instead of ORs in our searches?

    Can anyone explain to me a case where using OR would be correct and appropriate?

  • Mark Barnes
    Mark Barnes Member Posts: 15,432 ✭✭✭

    I can't think of any situation where you mean OR, but where a comma wouldn't produce the result you intend - so the 'safest' course of action would be to always use commas. Commas will always give you the results you expect. (But that doesn't mean you should be using commas. OR is not incorrect most of the time.)

    The reason why it's wrong some of the time is that OR is a boolean logical operator. In logic, a OR b either produces TRUE or FALSE. It can have no other result. In Logos, searching for Jesus OR Christ will return TRUE if the article contains one of those words, FALSE if the article contains neither of those words. After a search, Logos lists all the articles that are TRUE. This is quite satisfactory and quite correct. It's exactly what we want.

    But what if you type (Jesus OR Christ) NEAR (love OR compassion)? The left-hand side is evaluated, and let's imagine returns the result TRUE. Then the right-hand side is evaluated. Let's imagine that returns TRUE as well. So we're left with TRUE NEAR TRUE. Logically, that makes no sense, hence Logos returns no results for that time of search.

     

     

    This is my personal Faithlife account. On 1 March 2022, I started working for Faithlife, and have a new 'official' user account. Posts on this account shouldn't be taken as official Faithlife views!

  • Mark Barnes
    Mark Barnes Member Posts: 15,432 ✭✭✭

    JimT said:

    Its Winter here, a massive strorm outside, very wet and windy, and 4:38 am Tuesday. See my problem.

    Sounds to me like your problem is insomnia! [:)]

    This is my personal Faithlife account. On 1 March 2022, I started working for Faithlife, and have a new 'official' user account. Posts on this account shouldn't be taken as official Faithlife views!

  • Ron
    Ron Member Posts: 1,229

    I can't think of any situation where you mean OR, but where a comma wouldn't produce the result you intend - so the 'safest' course of action would be to always use commas. Commas will always give you the results you expect. (But that doesn't mean you should be using commas. OR is not incorrect most of the time.)

    The reason why it's wrong some of the time is that OR is a boolean logical operator. In logic, a OR b either produces TRUE or FALSE. It can have no other result. In Logos, searching for Jesus OR Christ will return TRUE if the article contains one of those words, FALSE if the article contains neither of those words. After a search, Logos lists all the articles that are TRUE. This is quite satisfactory and quite correct. It's exactly what we want.

    But what if you type (Jesus OR Christ) NEAR (love OR compassion)? The left-hand side is evaluated, and let's imagine returns the result TRUE. Then the right-hand side is evaluated. Let's imagine that returns TRUE as well. So we're left with TRUE NEAR TRUE. Logically, that makes no sense, hence Logos returns no results for that time of search.

    Ahhh...that makes sense now.  Thanks for the great explanation Mark. [;)]

  • Bradley Grainger (Logos)
    Bradley Grainger (Logos) Administrator, Logos Employee Posts: 11,969

    Logos 4 is superfast for most searching, but very slow with phrase searching. Take the following example:

    • "look out for the dogs, look out for the evildoers" which takes about 30s in my library. That's too long.
    • but dogs NEAR look NEAR evildoers takes 0.55s - that's one sixtieth of the time!

    Now it seems to be that if you can find the latter search in such a short time, it surely must be possible to optimise the former one some how.

    image

  • Rosie Perera
    Rosie Perera Member Posts: 26,202 ✭✭✭✭✭


    Logos 4 is superfast for most searching, but very slow with phrase searching. Take the following example:

    • "look out for the dogs, look out for the evildoers" which takes about 30s in my library. That's too long.
    • but dogs NEAR look NEAR evildoers takes 0.55s - that's one sixtieth of the time!

    Now it seems to be that if you can find the latter search in such a short time, it surely must be possible to optimise the former one some how.

    image


    Hmm, Bradley you must be using a very small library and an extremely fast machine. I know my machine is slower than most, but I'd be happy with 10 or 20 or even 30 seconds! Here's my result for the same search (EDIT: This is in 4.2 RC 2):

    image

  • Dave Hooton
    Dave Hooton MVP Posts: 35,770

    7.5s in 4.2 vs. 10.4s in 4.1, both on the Desktop [:|]

    Dave
    ===

    Windows 11 & Android 13

  • Robert Pavich
    Robert Pavich Member Posts: 5,685 ✭✭✭

    12 seconds for my aging 1.9 Core Duo with a 1 GB video card

    Robert Pavich

    For help go to the Wiki: http://wiki.logos.com/Table_of_Contents__

  • Milford Charles Murray
    Milford Charles Murray Member Posts: 5,004 ✭✭✭

    Peace to all!       *smile*

    Mine took 96.4 seconds.  Have almost 3,000 resources.

    OS Name Microsoft Windows XP Professional
    Version 5.1.2600 Service Pack 3 Build 2600
    OS Manufacturer Microsoft Corporation
    System Name THEFUTUREISNOW
    System Manufacturer Dell Inc.
    System Model Dell DXG051
    System Type X86-based PC
    Processor x86 Family 15 Model 4 Stepping 3 GenuineIntel ~3392 Mhz
    BIOS Version/Date Dell Inc. A06, 10/02/2006
    SMBIOS Version 2.3
    Windows Directory C:\WINDOWS
    System Directory C:\WINDOWS\system32
    Boot Device \Device\HarddiskVolume2
    Locale Canada
    Hardware Abstraction Layer Version = "5.1.2600.5512 (xpsp.080413-2111)"
    User Name THEFUTUREISNOW
    Time Zone Eastern Standard Time
    Total Physical Memory 4,096.00 MB
    Available Physical Memory 795.04 MB
    Total Virtual Memory 2.00 GB
    Available Virtual Memory 1.96 GB
    Page File Space 7.84 GB
    Page File C:\pagefile.sys

    Philippians 4:  4 Rejoice in the Lord always; again I will say, Rejoice. 5 Let your reasonableness be known to everyone. The Lord is at hand..........

  • Rosie Perera
    Rosie Perera Member Posts: 26,202 ✭✭✭✭✭

    7.5s in 4.2 vs. 10.4s in 4.1, both on the Desktop Indifferent

     


    12 seconds for my aging 1.9 Core Duo with a 1 GB video card


    Mine took 96.4 seconds.  Have almost 3,000 resources.

     

    Hmm, I wonder if it has something to do with how many resources we have, not just how fast our machines are? Dave, Robert how many resources do you have? I have over 4,000. So my timing would make some sense proportionally with Milford's.

  • Dave Hooton
    Dave Hooton MVP Posts: 35,770

    Hmm, I wonder if it has something to do with how many resources we have, not just how fast our machines are? Dave, Robert how many resources do you have?

    The second figure for v4.1 on the same computer should indicate whether Bradley was boasting about an improvement in search speed for 4.2. But my times today were 7.0s (4.2) and 7.2s (4.1, achieved during startup!), so it grows increasingly unlikely.

    I have 972 resources.

    Dave
    ===

    Windows 11 & Android 13

  • Rosie Perera
    Rosie Perera Member Posts: 26,202 ✭✭✭✭✭


    Hmm, I wonder if it has something to do with how many resources we have, not just how fast our machines are? Dave, Robert how many resources do you have?

    The second figure for v4.1 on the same computer should indicate whether Bradley was boasting about an improvement in search speed for 4.2. But my times today were 7.0s (4.2) and 7.2s (4.1, achieved during startup!), so it grows increasingly unlikely.

    I have 972 resources.


    Wow! I would give almost anything for 7 second phrase search times! Except I wouldn't give away over 3000 of my Logos resources. :-)

  • Mark Barnes
    Mark Barnes Member Posts: 15,432 ✭✭✭

    Remember that caching can help. I got 38.3s in a first run, 29s on subsequent runs. My library size is similar to Rosie's. I do wonder whether the relatively slow speeds we're suffering from are partly due to the relevant part of our large library indexes not being able to be held in memory.

    This is my personal Faithlife account. On 1 March 2022, I started working for Faithlife, and have a new 'official' user account. Posts on this account shouldn't be taken as official Faithlife views!

  • Dave Hooton
    Dave Hooton MVP Posts: 35,770

    Remember that caching can help.

    I usually take the first attempt after a startup. Another go with 4.1/4.2 shows that my 7s times vary by no more than +0.5s.

    EDIT: to simplify that, my times are in the range 7s to 7.5s

    Dave
    ===

    Windows 11 & Android 13

  • Mark Barnes
    Mark Barnes Member Posts: 15,432 ✭✭✭

    Remember that caching can help. I got 38.3s in a first run, 29s on subsequent runs. My library size is similar to Rosie's.

    Just as a comparison, I ran the search on my other installation. The times above were on a 15,000rpm drive running 4.2. The times below are on a 7,200rpm drive running 4.1.

    At 7,200rpm I got 48s on the first run, and 40s on the second run, and 27s on subsequent runs.

    This is my personal Faithlife account. On 1 March 2022, I started working for Faithlife, and have a new 'official' user account. Posts on this account shouldn't be taken as official Faithlife views!

  • Robert Pavich
    Robert Pavich Member Posts: 5,685 ✭✭✭

    Hmm, I wonder if it has something to do with how many resources we have, not just how fast our machines are? Dave, Robert how many resources do you have? I have over 4,000. So my timing would make some sense proportionally with Milford's.

    Rosie,

    I have 1075 resources (I've hidden probably 300 or 400)

    Robert Pavich

    For help go to the Wiki: http://wiki.logos.com/Table_of_Contents__

  • Ted Hans
    Ted Hans MVP Posts: 3,171

    I ran a search for "look out for the dogs, look out for the evildoers" and it took 36.30s. I have almost 9000 resources.

    Ted

    Dell, studio XPS 7100, Ram 8GB, 64 - bit Operating System, AMD Phenom(mt) IIX6 1055T Processor 2.80 GHZ

  • Dave Hooton
    Dave Hooton MVP Posts: 35,770

    Ted Hans said:

    I have almost 9000 resources.

    One zero too many?

    Dave
    ===

    Windows 11 & Android 13

  • Ted Hans
    Ted Hans MVP Posts: 3,171

    Ted Hans said:

    I have almost 9000 resources.

    One zero too many?

    he,he,he

    Dell, studio XPS 7100, Ram 8GB, 64 - bit Operating System, AMD Phenom(mt) IIX6 1055T Processor 2.80 GHZ

  • Jack Caviness
    Jack Caviness MVP Posts: 13,514

    At 7,200rpm I got 48s on the first run, and 40s on the second run, and 27s on subsequent runs.

    Running L4 Mac 4.0b, I got 43.65s first run and 43.34s second run. Guess we do not have caching yet.

  • Mark Barnes
    Mark Barnes Member Posts: 15,432 ✭✭✭

    Guess we do not have caching yet.

    The caching is provided automatically by Windows. Perhaps you should... [no, I'll resist [;)] ]

    This is my personal Faithlife account. On 1 March 2022, I started working for Faithlife, and have a new 'official' user account. Posts on this account shouldn't be taken as official Faithlife views!

  • Bob Pritchett
    Bob Pritchett Member, Logos Employee Posts: 2,280

    Hmm, Bradley you must be using a very small library and an extremely fast machine.

    Or maybe he has a newer version of the code that hasn't been released yet...

    :-)

  • Robert Pavich
    Robert Pavich Member Posts: 5,685 ✭✭✭

    Or maybe he has a newer version of the code that hasn't been released yet...

    :-)

     

    My heart just palpitated.

     

    Robert Pavich

    For help go to the Wiki: http://wiki.logos.com/Table_of_Contents__

This discussion has been closed.