Logos 5.0a SR-2 CRASH (again!): Too many open files

DMB said:

My experience with crashing at step #6 in your sequence didn't have any reference to 'too many files open'.  I haven't seen that error message since maybe Windows 3.1 or so back when you had to worry about that in the startup files. On my crashes, they are always (1) in the switch from a used 'Bible' search to a 'Basics' search (2) that is done by a right-click search (3) into an existing search panel (I assume it's not properly initializing the internal variables).

When it has crashed for you (on Windows I am assuming) did you look at / submit the log files. Just curious. The same/similar sequence can trigger a different bug on different platforms.

On the Mac there is the consideration of the extra Mono cross-platform layer that Windows users don't have to put up with.

As I have been helping a number of users who have reported crashes I have, again, got a bit frustrated because the logs often are obtuse/incomplete in what they report. For example they will say something like 'problem opening [SQLite] database file' — but they don't say which database. Given that the code knows which one it is opening when the problem happens, and the log file is written out by the system after a crash the system knows which database it was trying to open — why doesn't it say which one?

Also if there are, what are called in programming terms, 'exceptions' (unforeseen things due to programming errors or other things outside the program's control) then the system could/should deal with them more 'gracefully' doing things like saving user's work before crashing (if it can). But I see many crashes (more now in Logos 5) where the system just dies straight to the OS, we don't see the OS X dialog that says there was an issue.

Anyway.... tomorrow is a new day.

Jack Caviness said:

Patrick S. said:

But I see many crashes (more now in Logos 5)

For me, L5 is superior to L4 in ever way, and I have experienced few crashes—none in the past two weeks. Makes me curious as to the differences between my system and those of you who are experiencing these crashes.

Using a 5-6 year old Mac Pro with 11GB RAM and 2TB HD on Mac OS 10.7.5. Are these crashes more frequent with 10.8?

Oh Logos 5 is definitely superior to Logos 4, in functionality, usability and performance — no arguments from me on that score. However I have, on exactly the same hardware I was running Logos 4 on, experienced more crashes with Logos 5 then Logos 4. Not saying it crashes every minute.... and I can understand that because Logos (and I thank them for this) moved to new platforms (.NET 4.5, Mono whatever latest) there will be 'teething' problems.

This however has put me in touch with looking at log files (for my own installation and flowing on from that helping others) to see what is crashing. And I get frustrated and annoyed at what I see in log files, and... that crashes are not (IMHO) being dealt with effectively and efficiently. Then on top of that I read long drawn out discussions about 'social / community features' and that aggravates me further. I'll say it again — in my opinion crash problems with the software should have (they do in my organisation and team) absolutely the highest developer priority. Nice to haves like social features should definitely take a back seat.

I would really like (I'm not up to demanding yet, but getting close) to have some formal confirmation from Logos that there is an issue, a "sorry about that", and confirmation that the issue is being dealt with as a matter of priority.

Bradley Grainger (Logos) said:

Patrick S. said:

Yep — you may divine I am a little annoyed, I would appreciate some response — from someone.

We have developed a fix for this issue in 5.0b. We are currently backporting it to 5.0a so we can ship it in a service release.

Thanks Bradley — appreciate your follow up. Now you can print out the previous photo of Peter Finch, with this one...

next to it and put it up in your cubicle.

Martin Potter said:

Patrick S. said:

C'mon Logos - you don't even uninstall the old version of Logos 4 on the Mac when you do a new install of Logos 5! I'm sorry, and you can dispute me on this, but as far as I am concerned that is just slack programming.

If the program is upgraded to Logos 5 through the automatic update process the Logos 4 application bundle will be replaced with the Logos 5 application bundle. If you choose to manually download the disk image and copy the application to your hard drive, Logos 5 will not delete Logos 4 from your computer.

Thanks for replying Martin. As I'm sure, being a developer, you will know there are a few ways to install Mac applications, the two main ways being, 'Application Bundles' used usually for simpler apps, and executable package installation files which (quoting from a website I visited) for "more complicated and involved applications such as XXXXXXX will use an installer to simplify the installation process for you".

I believe you will see where I am going with this... the key take-away phrase from the above is "simplify the installation process for you", 'you' being the Logos customer.

In my opinion, and I think most of us would agree, Logos 5 is not a simple app. I believe the number of messages on the forum from reasonably intelligent people having problems post install of Logos 5 would confirm that.

Anyway even if the Logos 5 install uses the simplest method, just dragging an app bundle into /Applications, as we know there could be startup routines to check to see if there was an old application version, other software companies making even the simplest application seem to manage it. There are, after all, startup routines in the Logos 5 bundle app to create data folders and so on.

Anyway I believe Logos should take this experience of a less than optimal experience for Logos Mac customers and move to the more professional method, used by serious applications, for delivering installs (not updates which — hey I just remembered, screenshot below — are done by a dedicated app in the Logos app bundle, but which users don't have... until after the initial install) via an executable package installation file.

This would have a number of benefits, as there would be a number of tasks the installation (first time/upgrade) could do, for example:

• Check to see if there is an old version of Logos 4, move it somewhere in preparation for deletion
• Check to see that there is a valid Logos data folder
• Delete all the old Logos 4 logging files if there are any - they are no longer of any use and users have to be told each time to get rid of them manually
• Do a health check of SQLite databases before install — always a good idea!
• Check computer's Internet connection to make sure the machine can get a good link to the Logos servers to download resources
• etc.
• etc.
• After a successful install last step do final deletion of Logos 4 application; if Logos 5 application installation fails maybe even restore the previous Logos 4 installation.
• Kick off downloads, install, and indexing for any new resources — don't let user use system until it is stable and all pre-requisite steps are complete.

All in just think  — how much a more pleasant experience that would be for Logos customers; how many less calls there would be to Logos support; how many less forum postings there would be.

(Ergo, what's the very next posting I read on the forum.... http://community.logos.com/forums/t/62034.aspx )

Everyone would be happier, and have a positive first experience with the software.

I think a good aim to have.

Martin Potter said:

These programs generally require installing files in several different place, the installation of several different programs or installation of items that require system modification. 

Didn't you just describe the Logos 5 install above?

Martin Potter said:

For applications [...] such as Logos it is much easier to for the user to be able to drag and drop the bundle into their /Applications folder.

Martin Potter said:

Having to use an install program is viewed as the less than optimal experience for Mac customers.

Based on what experience, of whom? Logos has never delivered Logos 4 / 5 as an install program (as I know) so how can you say it is viewed as less than optimal experience for Mac customers? There is no (prior) experience. Anyway (using experience of many other applications) what's hard about download & double click to install?

Martin Potter said:

[...] even Apple is moving away from the use of installers with their shift to the App Store, which downloads application bundles and places them in your /Applications folder.

Not a valid comparison, the App Store does so using a (for want of a better word) 'meta-installer' platform so to speak. As you say, "[it] places them", not the user. What does the user do? Point and click. All the App Store applications fully manage all aspects of their own installation whether they be application bundles or package installs. Basically it is invisible to the user, the Logos installation is, obviously, completely different.

Martin Potter said:

This is a light-weight application which provides the prompts to the user and allows the relaunch of Logos, which has to be initiated from an external process since it can not occur if it is currently running.

Be that as it may — the end result for the user is 'point and click' for updates. Not so for the initial install.

Martin Potter said:

Patrick S. said:

Ergo, what's the very next posting I read on the forum.... http://community.logos.com/forums/t/62034.aspx

I am unsure what in particular you are referring to in the above post.

The experience of the user, the mixed up, messed up installation with both Logos 4 and 5 on the machine.

And here's another case http://community.logos.com/forums/p/61808/437835.aspx#437835

where a user managed to, easily, get into hot water resulting in this!

in the end no doubt ending up looking like Peter Finch v1 above.

David Mitchell said:

Based on Apple's guidelines. [...]

Hi Dave — long time. Can we approach this from different angle rather than saying Apple said "A," someone else said "B" and so on.

I believe we both would have the intention of wanting to see the best possible experience for users of Logos, I'm sure you do. On that basis of understanding I believe we would be in agreement that, given the number of persons experiencing problems and the extent of problems, it would be of benefit for Logos users to have a more robust installation.

Now we both know (and I mentioned earlier) that in one sense it does not matter if the installer is a bundle, or package installer, because really they are basically the same thing, usually a DMG container with one or more .app bundles in them. In fact (but obviously you know this) an .app bundle, even in place in the /Applications folder, is not an application in the same sense as a Windows application but rather a special folder. This folder can contain folders, files even other .app bundles (folders) which can themselves contain folders, files etc. etc. The LogosIndexer & LogosInstaller apps are .app bundles inside the Logos.app bundle.

The Logos.app bundle could contain an .app called, for example, 'PreFlight', that checked the computer environment on first run, post install, to make sure that it was clean and to do some housekeeping (like I listed in earlier post). This PreFlight.app application bundle could be embedded in the Logos.app bundle, or it could be part of an install program — doesn't matter (well in my opinion it would be cleaner to only run it once during install, but again doesn't really matter).

Because what's more important is what is done, not how it is done.

And that is my point — given that we agree that, if at all possible, it is a good goal to ensure that Logos customers have a good first experience with the software. And I would contend that, for Mac customers, that experience could/should/can be improved. The results of customers' experiences we see in the forums bear this out.

Thanks for your consideration.

David Mitchell said:

On this point, we can agree. In fact, we do numerous checks on every app startup to verify the integrity of the user's environment and fix what we can (some users will do the strangest things to databases…). When an issue is widespread enough to merit extra code, we add it.

That's great, thanks for sharing. At the risk of overstaying my welcome... can I just ask, some time in the future will the Logos 5 app (on install/first run) do the task of removing the old Logos 4 installation and/or check for other installations of Logos software? Ref. the screenshot above with five copies of Logos on a user's machine? That would be a beneficial thing to do, and in line with your criteria.

Also, just as I have your ear, you may have seen that there has been a number of posts with log reported database issues. Is there any Logos 'health check' app for doing a simple OK/Not OK check of all the SQLite databases? Not to try to fix, but simply to confirm (for exclusion purposes) whether all the databases are OK or not?

" rel="nofollow">Keep Smiling 4 Jesus said:

Patrick S. said:

Whoa boy..... when you start playing with low level system files like sysctl.conf you are really playing with the kernel — literally.

Literally concur plus observe that Apple's default max file limit is 256 and maximum files is 12288.  With seven applications open on OS X 10.8.2 for one user, noticed lsof | wc -l shows 2441 open files.  Prior to adjusting launchd.conf and sysctl.conf a bit more and restarting OS X, Logos 5.0b Beta 1 had 1025 files open with 2960 files overall for all applications. Currently have 1 line in /etc/launchd.conf  (effectively change default 256 file limit to 4096)

limit maxfiles 4096 12288

along with 2 lines in /etc/sysctl.conf

kern.maxfiles = 12288

kern.maxfilesperproc = 4096

Caution and caveat: be careful if choose to mess with OS X system files, which needs to be done as root (superuser) and followed by restart to take effect.  My initial /etc file modifications caused Logos 5 to crash quicker.   Couple commands to show current file limits:

launchctl limit

sysctl -a

Personally tweaking OS X system settings to work around too many open file crashes.  Logos 5 is a resource intensive application, which opens and uses lots of files.

Keep Smiling

Where do you get the "kern.maxfilesperproc" parameter from? My "man sysctl" page only shows these...

Name                            Type          Changeable
kern.ostype                     string        no
kern.osrelease                  string        no
kern.osrevision                 integer       no
kern.version                    string        no
kern.maxvnodes                  integer       yes
kern.maxproc                    integer       yes
kern.maxfiles                   integer       yes
kern.argmax                     integer       no
kern.securelevel                integer       raise only
kern.hostname                   string        yes
kern.hostid                     integer       yes
kern.clockrate                  struct        no
kern.posix1version              integer       no
kern.ngroups                    integer       no
kern.job_control                integer       no
kern.saved_ids                  integer       no
kern.link_max                   integer       no
kern.max_canon                  integer       no
kern.max_input                  integer       no
kern.name_max                   integer       no
kern.path_max                   integer       no
kern.pipe_buf                   integer       no
kern.chown_restricted           integer       no
kern.no_trunc                   integer       no
kern.vdisable                   integer       no
kern.boottime                   struct        no

Also regardless of all this above, and statements like "Logos 5 is a resource intensive application, which opens and uses lots of files.", which I am obviously aware of, my question is — why are we doing this?? Why are we as users having to look at and change OS system kernel tuning parameters? It is one thing, and bad enough, that they (may) have to be changed to make Logos 5 run effectively, but it is not right in my book that users have to do this. The software developers should be figuring this out and fixing it during software installation.

Logos?