Logos 6 Best Feature: NO INDEXING !!!!!!!!!!

Page 7 of 14 (263 items) « First ... < Previous 5 6 7 8 9 Next > ... Last »
This post has 262 Replies | 5 Followers

Posts 1132
Sean | Forum Activity | Replied: Fri, Oct 4 2013 8:16 PM

Yeah, I hate to jump on the bandwagon, but this is just getting to be too much.

Today I say I had an update available. It was an 80KB (yes, that's a K, not an M) update. I figure, hey, it must just be a little fix to a dataset or something. It downloads, then I restart.

Logos says: updated resource: La Bible Louis Segond 1910. Indexing started about 10 minutes ago. It's 11% complete.

I couldn't care less about that resource. Surely some people do, but couldn't it have been held off till a larger update, or done with the last one?

Posts 3938
abondservant | Forum Activity | Replied: Fri, Oct 4 2013 8:21 PM

Bob Pritchett:

Wow... this is a fascinating discussion....

you might feel justified in assuming we're incompetent... [smile]

We have. (I have been a guest speaker to the graduate-level class of one of the authors.) :-)

I feel weird saying "trust me, we're professionals", but in a sense that is the answer. It's hard to explain when your experience is unpleasant or undesirable, but at some level this is like arguing why a flight from Seattle to New York takes 5 hours when it's 2013. "That's crazy! We can fly faster than that!" Yes, by burning insane amounts of fuel. By taking the SR-71 Blackbird instead of a 737. By using rockets. But not really if you want scheduled daily service and a Coke on board.

If you want Logos to do what it does, indexing is going to take A) a fast, dedicated web connection, B) a massive, massive hard drive with a large file that's updated nightly, or C) some time when new books or new data are released.

We definitely hear your concern, though, and will continue to work on improvements. We can probably re-engineer around certain specific problems (we're looking at a different way of integrating reverse interlinears and the detailed data tables for referent, sense, lemma, morph, etc.), and we can tweak process priority, scheduling, etc. But -- short of adding rocket engines (or SSD hard drives to your computer) -- we can't fly from Seattle to New York in 1 hour. It's math and physics, not engineering competency.

-- Bob

P.S. There are possibly some 'completely re-engineered' solutions that would be faster, but they involve trade-offs. Index merging with ranked results, etc. could lead to some speed improvements, but can bring massive costs in disk space, and remove other options, like the ability to tweak or improving the term breaking, stemming, and at the cost of lower-quality ranking. We've made what we think are the right choices in these areas, but will continue to revisit them in light of user feedback.



Frankly, I dont notice a performance issue... But I'm running an octocore, 32gb of ddr3, and an ssd (with multiple terabytes of mechanical drive space). This is obviously not the typical end user set up.

I apologize if my replies infered you guys didn't know what you're doing. Clearly you do... What Logos does is pretty amazing, and comparable software doesn't really exist in other sectors that I'm aware of, and for its level of complexity it does remarkably well.

Forcing a rebuild of the index (4012 resources) while watching TV on same computer, reading my email, surfing the forums etc. Started at 11:30. Will post back the complete time.

L2 lvl4, L3 Scholars, L4 Scholars, L5 Platinum,  L6 Collectors. L7 Baptist Portfolio. L8 Baptist Platinum.

Posts 31
Doug Witmer | Forum Activity | Replied: Fri, Oct 4 2013 8:24 PM

Hi George:

George Somsel:

Can't we simply assume that Logos knows what it's doing and let it go at that?

I'm afraid Pandora's box is open now. There's no way we're going to get all the snakes back inside! Wink

Doug

Posts 31
Doug Witmer | Forum Activity | Replied: Fri, Oct 4 2013 9:17 PM

Okay as long as I'm stirring up trouble, let me give a simple, concrete, non-technical example of what I'm talking about. It's still Friday, and every Friday WordSearch gives a free book. So just a few minutes ago I ordered my free book. When I opened WordSearch it automatically downloaded and installed the book. That took about 10 seconds. Then I watched for the indexing. There was none. "So", I thought, "let's try a search." I told the search facility to search for "second blessing", and to search across the entire library. In about three seconds it had retrieved 183 hits, and six of them were from the new free resource, "The Second Crisis in Christian Experience by Christian Wismer Ruth".

You can try this for yourself. Just download a free copy of WordSearch Basic, and then "purchase" the free book and search on it.

Posts 3674
BillS | Forum Activity | Replied: Fri, Oct 4 2013 9:19 PM

Bob Pritchett:
Wow... this is a fascinating discussion....

Bob Pritchett:
I feel weird saying "trust me, we're professionals", but in a sense that is the answer.

Bob Pritchett:
We definitely hear your concern, though, and will continue to work on improvements.

Bob,

You're listening! And I'm hearing that it's more complicated than we may think. I'll choose to trust... Thanks for hearing our concerns.

Grace & Peace,
Bill


Asus GF63 8RD, I-7 8850H, 32GB RAM, 1TB SSD, 2TB HDD, NVIDIA GTX 1050Max
Samsung S9+, 64GB
Fire 10HD 64GB 7th Gen

Posts 285
Luigi Sam | Forum Activity | Replied: Fri, Oct 4 2013 9:42 PM

mike:
I love updates & I WANT THEM RIGHT AWAY (I'm sure everyone is on the same boat with me) .. but the indexing is just killing me.

hi Mike,

I was just thinking when I read your post: do the updates get listed for users to see exactly what each update is?  ( ie in windows 7, the updates can list each update, and describe them breifly)

does logos 5 do this? 

it would be funny if we were reindexing just because one book corrected one word (and we never actaully even use that book anyway)

if this happened 10 times a day: we are wearing out our hardware reindexing.

thanks if anyone can answer my query.

Posts 2435
David Ames | Forum Activity | Replied: Sat, Oct 5 2013 4:04 AM

Sean:

Logos says: updated resource: La Bible Louis Segond 1910. Indexing started about 10 minutes ago. It's 11% complete.

I couldn't care less about that resource. Surely some people do, but couldn't it have been held off till a larger update, or done with the last one?

I use it to prepare the lesson study for my wife who teaches a French language weekly class.  That is the download that I tried to skip and got into indexing anyway.

Posts 3938
abondservant | Forum Activity | Replied: Sat, Oct 5 2013 4:59 AM

Having other questions Bob, what about Multi-thread and x64 support for those computers that can handle it? Most everything has multiple cores these days. Is this something that Logos does already? If not can it be something that is considered for L6?

it took 15 minutes for the first 10%. Was at 25% by 29 minutes.

I was typing out the mile stones above as they happened, but fell asleep before 50%. When I woke up 90 minutes later it was complete.

L2 lvl4, L3 Scholars, L4 Scholars, L5 Platinum,  L6 Collectors. L7 Baptist Portfolio. L8 Baptist Platinum.

Posts 10632
Forum MVP
Jack Caviness | Forum Activity | Replied: Sat, Oct 5 2013 5:01 AM

George Somsel:
Can't we simply assume that Logos knows what it's doing and let it go at that?

It is just too much fun trying to teach Logos basic programming Devil

If WS does all you want Bible Software to do, then use it. Frankly, getting tired of those who come to the Logos forums to sell WS and Accordance.

Posts 1875
Paul-C | Forum Activity | Replied: Sat, Oct 5 2013 5:54 AM

Jack Caviness:

It is just too much fun trying to teach Logos basic programming Devil

...and how to run a business. Big Smile

Posts 9947
George Somsel | Forum Activity | Replied: Sat, Oct 5 2013 5:56 AM

Jack Caviness:

George Somsel:
Can't we simply assume that Logos knows what it's doing and let it go at that?

It is just too much fun trying to teach Logos basic programming Devil

If WS does all you want Bible Software to do, then use it. Frankly, getting tired of those who come to the Logos forums to sell WS and Accordance.

Amen !!

george
gfsomsel

יְמֵי־שְׁנוֹתֵינוּ בָהֶם שִׁבְעִים שָׁנָה וְאִם בִּגְבוּרֹת שְׁמוֹנִים שָׁנָה וְרָהְבָּם עָמָל וָאָוֶן

Posts 2230
Forum MVP
John Fidel | Forum Activity | Replied: Sat, Oct 5 2013 7:15 AM

First of all, comparing the various bible software programs is difficult and in most cases unfair to all the developers. This is because each program is unique in the program's main focus. Since this is a Logos forum, I will primarily remark on Logos. Logos is of course Bible focused, but the programmers have created reports that provide pertinent information for the user. The program integrates the library for the user through the Passage Guide, Exegetical Guide, Bible Word Study Guide, Cited By tool etc. Other programs require the user to run searches to get the same information, if the program can provide the same information at all. Getting information to provide these reports specific to a related pericope requires different indexing than a program that does not provide that information. You can compare the user experience, given the strengths and limitations of each program, but not really whether indexing is required or how it is accomplished.

Some other programs while Bible focused are more search focused. As such they can run some really complex searches quickly. That is their focus, but they may not have the guides and reports that are available in Logos. Or they may only be designed by default to search one resource at a time, but do so really fast without indexing. Or perhaps they do not provide resources that can be added daily, so the index is more static. Perhaps the program is not focused for those working with the original languages and as such the amount of information required to be indexed will be less complex. 

My point is that comparing Logos to other programs is not an apple to apple comparison. Perhaps those other programs meet one's needs better because of one's focus of study. If you don't require original language support other than Strong's based study then there are other programs that may meet your needs without the need to index or index more quickly. If you require the ability to run graphical or complex original language searches then there may be programs that do that more quickly. There is give and take in design and related function.

Logos has improved the indexing function greatly. I remember downloading the first alpha for L4 and it really took a long time. I am confident they will continue to improve this process and make it less cumbersome on their customers. In the meantime, I think discussions such as these are helpful to all who participate and give the programmers at Logos an opportunity to evaluate opinions and suggestions made by their customers.

On a final note, some may want to discuss the other programs and challenge my opinion, and I will gladly discuss such as long as it does not violate the intent of this forum.

Posts 10428
Denise | Forum Activity | Replied: Sat, Oct 5 2013 7:28 AM

No offense Jack and George. But the last two comments either have to be joking or on their way to meanness. You both know the Bible software companies prevent using purchased books on each others software. People aren't rich, so they're at mercy of profit interests. You both know the bulk of users use Bible software visa viz their future with the Divine. And you both know, absent Dave and the Marks, that the issue is sufficiently complex, that competitor experience is the only evidence one can validly point to.  Especially when those same competitors have customers ALSO studying for their future withe the Divine.  

Don't like it, leave?

So, I'll assume we're all just joking!


Posts 13379
Forum MVP
Mark Barnes | Forum Activity | Replied: Sat, Oct 5 2013 8:34 AM

Doug Witmer:
Actually WordSearch does have reverse interlinears. I own two of them. The difference is that the WordSearch reverse interlinears are fixed. That is, the index for that individual book should not need to be rebuilt unless the book is revised.

Sorry, I didn't realise that. Logos indexes are fixed, too, however. The index won't be rebuilt unless the book is revised.

Doug Witmer:
The beauty of a binary tree index is that the index for each book can be generated at the source (Logos) and then the individual indexes can simply be downloaded to a user's computer, and properly placed in the master index for that computer. It takes no time at all!

As William Gabriel points out, that's not how b-tree indexes work. By definition, b-tree indexes have to be merged (or re-created). If WordSearch isn't using indexes that require merging, then searches across the library are going to be slower. You either have the slowness when you index, or the slowness when you search, it's as simple as that.

Doug Witmer:
Now I seemed to have stirred up some emotions. Perhaps I should have just suffered in silence.

Stirred up some passion, yes. That's not a bad thing!

Doug Witmer:

When I opened WordSearch it automatically downloaded and installed the book. That took about 10 seconds. Then I watched for the indexing. There was none.… You can try this for yourself. Just download a free copy of WordSearch Basic, and then "purchase" the free book and search on it.

OK, I'm now running WordSearch basic. There is indexing when you install books (check your status bar), but it's very quick, and doesn't use much CPU. 12 resources took 60 seconds to index.

An index is stored at C:\ProgramData\WORDsearch\Library\ws10searchindex.dat

Search is also very quick (almost instant for the 13 resources that came by default). But each resource appears to have it's own index file (which is downloaded with the resource).

It's impossible to tell without detailed analysis, but it seems that the WORDsearch index isn't a true index, but more of a word list - listing which words are stored in which resources. If I'm right, that means the index is only used to choose which resources are searched (i.e. those that contain the word being searched for). Then, every individual index file is searched independently.

This is a perfectly reasonable design choice. But it's a basic design choice. It's worth bearing in mind that WORDsearch searching can't do any of the following:

  • Do a case-sensitive search.
  • Rank searches.
  • Search for any datafield at all (i.e. you can't search for Louw-Nida numbers, Bible references, links to the church fathers, etc.)
  • Do word-stemming (search for all forms of the word)
  • Specify specific proximity (you can use NEAR, but not WITHIN x WORDS)
  • Search for words in a specific language
  • Search for special fields (e.g. heading text, etc.)

So, as someone pointed out above, this is not a like-for-like comparison. WORDsearch is faster - there's no doubt about that - but there's a lot less going on, and a lot less flexibility. That's good - we can all make a choice as to what type of Bible software we want.

But if Logos indexing became became more like WORDsearch indexing, there are very, very, very few Logos users who would see that as an improvement.

Posts 468
BKMitchell | Forum Activity | Replied: Sat, Oct 5 2013 8:35 AM

John Fidel:

First of all, comparing the various bible software programs is difficult and in most cases unfair to all the developers. This is because each program is unique in the program's main focus...My point is that comparing Logos to other programs is not an apple to apple comparison.

Excellent pointYes and I'll take your word for it because I know in past how hard you've worked to review Bible software while trying to remain objective.   

חַפְּשׂוּ בַּתּוֹרָה הֵיטֵב וְאַל תִּסְתַּמְּכוּ עַל דְּבָרַי

Posts 9947
George Somsel | Forum Activity | Replied: Sat, Oct 5 2013 9:33 AM

Denise:

No offense Jack and George. But the last two comments either have to be joking or on their way to meanness. You both know the Bible software companies prevent using purchased books on each others software. People aren't rich, so they're at mercy of profit interests. You both know the bulk of users use Bible software visa viz their future with the Divine. And you both know, absent Dave and the Marks, that the issue is sufficiently complex, that competitor experience is the only evidence one can validly point to.  Especially when those same competitors have customers ALSO studying for their future withe the Divine.  

Don't like it, leave?

So, I'll assume we're all just joking!

No, I'm perfectly serious.  It is acceptable to mention a problem one is having with the software, but it is not acceptable to treat the company and its employees as though they are pure neophytes.  They are in a thriving business so they must be doing something right.  I wouldn't walk into a doctor's office and refer him to a Wikipedia article on appendicitis or inserting a stent.  I would assume that Logos is fully acquainted with b-trees to the point where a Wikipedia article is like reading a child's story book.  I have never had problems with indexing so I must conclude that there is something wrong with the hardware or setup of those who do.  I wonder how much free space they have on their hard drives since it's difficult to reorganize something on a computer if you don't have the room to do it.  Then they talk about how program X or Y does it better in their opinion which makes me wonder whether they really are interested in Logos or whether they are propagandizing for another company.

george
gfsomsel

יְמֵי־שְׁנוֹתֵינוּ בָהֶם שִׁבְעִים שָׁנָה וְאִם בִּגְבוּרֹת שְׁמוֹנִים שָׁנָה וְרָהְבָּם עָמָל וָאָוֶן

Posts 2927
Forum MVP
Jacob Hantla | Forum Activity | Replied: Sat, Oct 5 2013 9:40 AM

Bradley Grainger (Logos):

One possibility is to move the index from your computer to our servers (similar to what is done for the mobile apps right now). Would it be an acceptable solution to require an Internet connection for searching to work?

I personally do not find indexing a problem, which may be because I have a pretty fast computer. Indexing is rare, and when it occurs is pretty quick and doesn't really slow down the computer (I can pause it if I have to). At this time I don't mind having my index on my local computer. 

On my desktop and laptop, I would prefer to have my index on my computer because I have the space and the processing power for it. If you were to force us to be connected to the internet for searching to work, I think this would introduce a lot more problems into my use of Logos. Please don't go this way as the sole means of searching.

However, for those who might have a laptop that is slow (netbook) or with limited hard drive space (ssd only) maybe giving them an option to use web for search would be acceptable. Web based search would also be good for those who want to get up and functioning immediately while indexes are being built.

But if a computer has the hard drive space for it, I think that having the local index is part of the power of Logos (not an impediment that needs to be overcome). 

Jacob Hantla
Pastor/Elder, Grace Bible Church
gbcaz.org

Posts 31
Doug Witmer | Forum Activity | Replied: Sat, Oct 5 2013 11:36 AM

I have a challenge for you. I'm going to give you two binary trees, and you can pretend that they're two indexes. Can you show the simplicity of merging? I do not believe you can do it by simply connecting the entire new index at the appropriate place in the original/master index.

Tree 1 (Master Index):

                               50

                            /       \

                        10          80

                     /      \       /      \

                  1        30  70         90

Tree 2 (New book index):

                              45

                            /     \

                         15       51

                                      \

                                        75

Hello William:

I thought about your challenge over night, and I came up with a solution. You can quickly turn these two indexes into a single index (without merging), and the combined index will be perfectly useable (and fast). But I'm not going to give the solution here. If I did, it would just give rise to more questions and more challenges. It's your challenge. Lets see if you can solve it.

Doug Witmer

Posts 31
Doug Witmer | Forum Activity | Replied: Sat, Oct 5 2013 11:58 AM

Mark Barnes:

OK, I'm now running WordSearch basic. There is indexing when you install books (check your status bar), but it's very quick, and doesn't use much CPU. 12 resources took 60 seconds to index.

I haven't actually  used WordSearch Basic. I'm running WordSearch 10.6. I suspect WordSearch Basic is just an older version of WordSearch. If so, it would be rebuilding the index every time you added a book. That's what WordSearch 10.5 did.

I don't see any indexing when adding a book to WordSearch 10.6. But perhaps my computer is so fast the status bar never has a chance to appear. On the other hand, with the older versions of WordSearch I got to see plenty of the status bar! It used to be really annoying because you had to click "Ok" from time to time so you couldn't just start it up and go to bed.

I have over 1700 resources.

Mark Barnes:

An index is stored at C:\ProgramData\WORDsearch\Library\ws10searchindex.dat

Search is also very quick (almost instant for the 13 resources that came by default). But each resource appears to have it's own index file (which is downloaded with the resource).

Yes, there is an index. The file ws10searchindex.dat is the combination of all the little indices that come with each book. The question is not the existence of an index. The question is what happens to the index when a new book is added.

Doug Witmer

Posts 13379
Forum MVP
Mark Barnes | Forum Activity | Replied: Sat, Oct 5 2013 12:40 PM

Doug Witmer:
The question is what happens to the index when a new book is added.

Correct. And as I stated in my post, Logos is doing a lot more than WordSearch. That's what matters, because it explains that the 'slow' indexing isn't down to sloppy programming, but to a difference in specification.

Do you really think Logos customers would be happy if Bob or Bradley said, "We'll reduce the time it takes to index to almost nothing, but you won't be able to do a case-sensitive search, rank searches, search for any reference types (e.g. Louw-Nida numbers), do word-stemming, specify specific proximity, search for words in a specific language or search for special fields". There is almost no-one on this forum who would think that was a good idea.

Page 7 of 14 (263 items) « First ... < Previous 5 6 7 8 9 Next > ... Last » | RSS