Huge index files and fragmentation

So I have around 5700 resources in my library, which runs the index file just south of 5gb. Because I do a lot on community pricing, I get files added every other week or so (not to mention Logos constantly updating resources). Needless to say my index gets rebuilt rather often, which doesn't bother me a whole lot. After each rebuild though I'll run a check on my defrag and see my sizeable index file is now split into a ton of fragments (usually around 3000), which slows a lot of things down in Logos. I regularly defrag my computer, and most everything stays pretty clean, but I can always count on the Logos index file being scattered after each rebuild.

So my question is: has anyone tried setting up a separate drive for their index file? Like a virtual drive? I'd imagine it'd have to be big enough to handle any future resources added, but if that file was confined to its own space wouldn't it theoretically suffer less fragmentation, and therefore be a bit speedier? I'm not sure exactly how to get Logos to point to an index file on a different drive.

I suppose another option would be to see if the devs can rig a way to portion off space on a HD for index file (kinda like the system page file) so it could have its merry way without being bothered to scatter.

Any ideas?

Find more posts tagged with

Comments

David Wilson

I run Logos on an entirely seperate drive containing no other applications, and I give it about 60% free space in which to do its indexing. This seems to keep fragmentation down to a dull roar, requiring only periodic attention to deal with the rebuilt/updated resource files which download quite frequently !!

Dave Hooton

So I have around 5700 resources in my library, which runs the index file just south of 5gb.

If that is just the Library index folder then don't forget the Bible and Personal indexes.

I regularly defrag my computer, and most everything stays pretty clean, but I can always count on the Logos index file being scattered after each rebuild.

That happens on any HDD due to Windows' storage allocation strategy (can't speak for Mac), even when I install Logos onto a (not quite dedicated) data partition with 50% free space.

I suppose another option would be to see if the devs can rig a way to portion off space on a HD for index file

An SSD is the best option - it guarantees fragmentation but is so fast that it doesn't matter (my laptop data partition was 67% fragmented the last time I bothered to check). With a HDD use a product like Defraggler that can defrag individual files.

Eric

https://community.logos.com/discussion/comment/455469#Comment_455469

It's not the folder, it's one specific file that gets hammered. I'm not as concerned about the other files because they don't suffer as much. Defraggler is what I use to defrag, but I guess I'm looking for something a little more long-term than having to run it once a week for a particular file. My main computer is a laptop, and I considered getting a SSD when I bought it but it was an extra $400 and the speed difference was minimal according to their numbers - not sure if it's because of the drive itself or if the laptop's motherboard can only handle so much, either way I didn't want to pay an extra $400 for 1/3 the size at the same speed (future mechanical failure notwithstanding).

More or less that's why I was thinking of a dedicated drive just for that file. Hard to fragment a drive if only one file is on it.

David, what does the fragmentation on that drive look like after a rebuild?

Dave Hooton

https://community.logos.com/discussion/comment/455475#Comment_455475

David, what does the fragmentation on that drive look like after a rebuild?

I'm not David but my almost dedicated desktop (HD) drive/partition can get 40% fragmented. However, the percentage is almost meaningless because one large index file with about 500 fragments will be the main contributor!

Eric

https://community.logos.com/discussion/comment/455484#Comment_455484

I'm not David

agreed, but the first reply is David, to whom I was asking the question. I was hoping it wasn't going to get too confusing [*-)]

David Wilson

https://community.logos.com/discussion/comment/455490#Comment_455490

Hi Eric, Dave is our resident expert when it comes to such things, so he will probaby be able to give you better guidance that I could.

I try to avoid doing a "rebuild index" as it does not seem to be any advantage to do that and tends to make the system unavailable for too long (so I have no data re fragmentation related to a rebuild.) Mine is an older system with no SSD running Windows Vista 64 on a pair of Western Digital Black HDs (no raid applied).

My LibraryIndex index.idx file is 9.89 GB and LibraryIndex index.lxn is 1.53 GB, BibleIndex index.idx is another 1.3 GB, PersonalBookIndex is small by comparison.

Logos 5 resides on my U drive with currently just over 67 GB free space. I use Norton Utilities Real-time defragmentation on the drive

Eric

https://community.logos.com/discussion/comment/455499#Comment_455499

Gotcha. Purchasing an SSD is more or less off the table for me, so I'll have to find a different route (or just put up with insane fragmentation).

Eric

https://community.logos.com/discussion/comment/455484#Comment_455484

Dave, after rereading what I wrote I feel like I may have been a little curt. I didn't mean to be abrasive, so if you got offended I apologize [:$]

Dave Hooton

https://community.logos.com/discussion/comment/455514#Comment_455514

Dave, after rereading what I wrote I feel like I may have been a little curt. I didn't mean to be abrasive, so if you got offended I apologize

No offence was taken, so please don't be embarrassed.