Typos in BHS

It's not a typo. The typographic distinction between holem-waw and waw+holem, where the latter has the holem dot shifted slightly to the left, is an optional convention. There was no support for it until Unicode 5, and as such it has some compatibility issues (not just at the level of some fonts not supporting the new holem code point - SBL Hebrew does, but also issues relating to how you search the new mark - is the user required to type the correct holem in order to find the word they are looking for or are the marks treated identically for the purposes of indexing/searching? It seems obvious that for 99% of users, 99% of the time, you'd want to treat these marks as the same for everything but display, but do we need to encode a new match syntax to make it possible to find just the one or the other for the people who care? Does '[match all]' really match all, or is this left as an exception to the rule without another flag like '[match all, holem]'? How about for KeyLinking and lexicon navigation - do we need to write additional code to support working around the change? Does the text comparison tool need code to ignore these cosmetic differences when comparing two texts? It might turn out that none of that is terribly difficult to handle, but it would all have to be tested.)

A bigger issue for us is that none of our databases come from Unicode source files and they don't distinguish between the two holem characters, so it would be an exercise for us (OK, me) to determine which character to use where. Now I might be wrong about that - some of the source files might encode the difference, but in a fairly non-obvious, non-Unicode way, where the trick is to look at each holem and ask 'is this on a waw (checking for intervening characters, like dagesh)?' and then treat it differently if the answer is 'yes' because the combination holem-waw is consistently handled with a completely different mark. Or perhaps the source file encodes the different in order (OW vs. WO) and our code carefully obliterates this difference to get the characters in proper Unicode order, which would mean rewriting the code we use to convert the source files to intercept 'WO' differently without breaking the proper conversion of 'OW'. In the absence of a consistent code in the source files, I could easily write a script that could catch 95% or more of these, but then you run into sequences like hireq+yodh+waw+holem and there isn't a programmatic way to know if this is i/yo or iy/wo. If you're lucky, there might be the right type of accent directly on the yodh or the waw which clears the matter up, but if not, I'd have to appeal to some other authority to look up which way to read it. Which, of course, I could do. My point isn't to make this seem unsolvable, just time consuming and I haven't decided if the pain is worth it for a non-standard typographic feature that is only partially supported. In your Genesis example, there is only one way to read that word - it's really only in the tricky sequences where the effort pays off, so an automated solution that only gets it right say 98% of the time isn't really very compelling to me.

But I'll think about it some more. I'm supposed to be getting new SESB files sometime soon, and that'll give me an excuse to look at those source files again and see if they have actually made all the editorial decisions in a way that we simply ignored because before Unicode 5 there was only one proper way to handle this. If so, maybe I'll try building it with two different holem code points and see how well it works in our system.

Sorry, Lee. I mistakenly thought you'd find the details interesting. I didn't mean to give the impression that I was minimizing your request. I did say I'd look into it.

Lee said:

But some of the thinking, I'll just say I'm not in agreement with.

OK, I'll try to make my thinking a bit clearer.

Flip to Google and enter the following searches:

מִצְוֹת

מִצְוֺת

The first one gets 'about 81,400' hits, the second gets 1,230 hits. Both mark orders (U+05D5, U+05B9 and U+05D5, U+05BA) are technically supported by the Unicode standard (as of version 5), but the first (the way we have it now) is far more common and despite Google's massive development team, they obviously haven't solved the 'problem' that these are two valid ways of encoding the same word (both searches should return the same count: 'about 82,600'). That's what I meant by 'partially supported'. In most areas of computer development, 8 years is an eternity, but support for complex, minority scripts moves at a different pace. Of course, as a bible software company, we care about biblical Hebrew more than Google does, and we could fix the things under our control whether or not the resulting text plays well with other software. But the point of my first paragraph was 1) the way we have it is actually standard (in all three senses: it adheres to the Unicode standard - as does the alternative, it's the way most people encode these letters and it's what most print books do - grab a print copy of BDB, for example, and you won't find any orthographic variation between holem-waw and waw+holem), and 2) implementing the less common orthography would require some testing and possibly some coding to make sure we weren't wrecking other parts of the user experience by adopting this convention.

I guess I can understand a little why you misunderstood my comment about time; you don't know me. If you did, you'd realize the irony of thinking that I wouldn't do something just because it would take a long time. I'm the guy who spent 9 months lining up different bibles verse by verse so that Logos 4 could do a bang-up job of syncing them. However, I'm sure you can appreciate that there are trade-offs that have to be made, both for my time and for development time. Maybe it was bad form for me to mention it? You may have a point that I'm no good at marketing.

Leaving aside that I could find fifty contemporary language references on my shelf that don't follow the BHS convention, let me ask you this:

Would you still advise making the change in the BHS SESB if as a result you could no longer right-click the affected words, run a search on your entire library, and expect to see ANY hits outside of that Bible? If the answer is 'yes', then we do have a difference of opinion, because I don't think it's worth breaking basic functionality over a minor orthographic difference. If you answer 'no', then we're actually on the same page - it's a desirable change as long as it doesn't break basic functionality.

The Google example just illustrates that this is something that has to be tested with any off-the-shelf components we use in our development (on five different platforms) and so at that level, it doesn't matter that the Unicode Standard specifies either encoding if the tools for processing Unicode don't play well with the newer code point.

Lee said:

Vincent Setterholm said:

Would you still advise making the change in the BHS SESB if as a result you could no longer right-click the affected words, run a search on your entire library, and expect to see ANY hits outside of that Bible? If the answer is 'yes', then we do have a difference of opinion, because I don't think it's worth breaking basic functionality over a minor orthographic difference. If you answer 'no', then we're actually on the same page - it's a desirable change as long as it doesn't break basic functionality.

That's a relevant consideration. It used to be, for example, that in early days Adobe Reader had problems with perfectly kosher Unicode content, even Latin character ligatures.

However, they've got it fixed a few years ago. Somehow or other, it's no longer an issue, as long is there is Unicode compliance.

I also see that the issue doesn't affect a particular piece of software that we both don't want to talk about.

Now, let's look at it through the customer's eyes. He runs a search, and it does not distinguish vav-holem, holem-vav? Alright, that may not be so important. And the actual event that led up to this. He exports the verse, and it gets printed. And later, embarassingly, the typo is seen by some people. Where did I get this? People are interested to know.

Now put yourself in my shoes, and tell me how I can convince experts that Logos is a serious piece of software if this orthographic representation stays despite the company being aware of it. It is NOT a typo? BDB? Google? Search compatibility? 50 contemporary language references say you're wrong?

And how do I answer people who advise: don't rely too much on the software if you're into serious work?

My advice to you is to actually LOOK at the text and determine the order.  It shouldn't take more than a moment to determine which is correct.  If what you copied doesn't appear to be correct, change it.  It has taken more time to discuss the matter than it would take to correct the text.

Lee said:

If you want to offer real advice, George, stick to the issues or offer some substantiation please.

The discussion taking place between myself and V.S. is whether this typo is a typo at all, or whether it is something that we can all live with.

If you do not have substantial help or goodwill to offer, please take your own advice that you rudely threw at me in another thread. It was so rude, I won't even repeat it.

But not nearly so rude as you.

Could some form of visual filter be a work around for Lee?

Lee said:

Vincent seems to be saying that if the database is rectified it could interfere with the Logos search engine convention. I think that's putting the cart before the horse, design-wise.

That's not a fair summary of what I've written. You're making it sound as if I'm making excuses for not looking into this when I've repeatedly written that I will. Furthermore, the very definition of 'putting the cart before the horse, design-wise' would be to make this, frankly cosmetic, change to the resources without first making sure that our architecture supports it.

I was quite happy to give you the last word on this thread. I don't know what this new post was supposed to contribute to the discussion. In hindsight, I should have kept it simple: Thank you for your suggestion. We appreciate it.

Interesting. I checked a few other Hebrew texts and they were correct, including BHS/WIVU and LHB but I do like the morphology in BHS/WHM 4.2. Its also wrong in BHS/WHM 3.5.