CrowdSourcing: Is it time for a revival...

Page 3 of 3 (44 items) < Previous 1 2 3
This post has 43 Replies | 4 Followers

Posts 1638
Forum MVP
Reuben Helmuth | Forum Activity | Replied: Wed, May 13 2015 6:35 AM

I've been working on a proposal/ideas document to submit to Logos and would welcome any feedback.

Crowdsourcing Proposal

This is an informal (and developing) proposal for a system that would exploit crowdsourcing for the correction of typos and the input/correction of hyperlinks throughout the Logos ecosystem of resources. Suggestions and feedback are welcome and can be directed to reubedu AT gmail

Pre-requisite: User requests permission to join the “typo/links” Faithlife group.

  1. Typos
    1. Group member submits a typo 
      1. This is immediately visible to other group members as a visual filter.
    2. Group members vote on submissions.
      1. The initial filter would be red and hovering would display both the original and proposed spelling with the option to submit a yes/no vote directly from the tag. Each user could only vote once for a given submission. Voting will be anonymous to group users to prevent biased voting).
      2. After receiving a minimum of 10 votes with a 90+% acceptance rate, a tag is changed from red to green and the option to vote is removed. The submission is now “ripe for harvesting.”
      3. In the event of receiving multiple negative votes, a submission could accept up to 20 votes in an effort to reach the required 90%, before being rejected.
    3. A system of points will be put into place for several purposes. Firstly, it will prevent indiscriminate voting and promote caution and carefulness throughout (think QC). Secondly, it will provide a means for “compensation” (more on that later).
      1. If a submission gets accepted, the original user who submitted the correction as well as those who voted in favor, will receive 1 point, while those who voted against will be docked a point.
      2. If a submission gets rejected (doesn’t reach 90% approval by 20 votes), the original poster as well as those who voted in favor, will get docked 1 point and those who voted against will gain a point.
    4. Approved/accepted submissions get harvested at the time of the six week release cycle.
      1. In the case of typos, “harvesting” would be slightly more time consuming on the Faithlife employee end of things since each submission would need to be assessed against the original document because of permission issues. Bear in mind, however, that these submissions would not need to be weeded through for non-sense, British/American spellings, or other erroneous reports that are currently plaguing the typo-report database.
      2. Typos which prove to be present in the original document (and aren't permitted to be corrected) would be moved to a separate visual filter document that would insert the correction immediately after the typo. The filter could also cause the misspelled word to be struck through or grayed out. THIS visual filter should be available to ALL users (like propositional outlines). It could obviously be turned off and thus should not violate publisher prohibitions for typo correction. 
      3. Searches should then support the text in the visual filter as though it were surface text (or whatever the filter was applied to). 

  1. Hyperlinks to Bible References (only minor differences from typos above)
    1. Group member enters a link to a Bible reference.
      1. This is immediately visible AND clickable for other group members.
    2. Group members vote on accuracy of links.
      1. Same steps as Typos section
    3. System of Points
      1. Same as typos section, but that each submission (and vote) would be worth 2 points instead of only 1.
    4. Approved/accepted submissions get harvested at the time of the six week release cycle.
      1. These could be harvested “wholesale” without manual intervention!!

  1. Hyperlinks to Non-Bible References
    1. Same as the Bible reference section except that each submission (and vote) would be worth 5 points.

  1. Super-Users
    1. Group members will accumulate “points” as they submit or vote on corrections.
      1. Members who acquire (and retain) a total of 500 points will receive “super-user” status
    2. Super-users could function much like regular users, but their votes would be worth half the total required votes for a submission. In other words, where it would take 10 regular users to approve a submission, 2 super-users could accomplish the same thing. Also, 1 super-user combined with 5 regular members would provide approval. In light of the ration required for approval, a super-users negative vote may not be able to carry as much weight as his/her positive vote.
      1. Super-users could effectively use their track-record of reliability to “fast-track” submissions into the harvest-ready stage.

  1. Compensation
    1. Users will be compensated on a monthly basis.
      1. Much like Faithlife is offering Logos credit as compensation to competent writers, group members will be compensated for their help on a per point basis each month. I propose the nominal fee of a penny per point per period. Each month a user’s new points will be converted to Logos credit at a rate of $0.01/point and applied to the users Logos account.
        1. These credits will be non-expiring and can be allowed to accumulate in order to buy any Logos resources available on
Posts 2948
David Taylor Jr | Forum Activity | Replied: Wed, May 13 2015 9:20 AM

David Paul:


David Paul:

Eli Evans:
Typo fixing: Unfortunately, there's just no way around the legalities involved in letting users fix typos. Sad but true: Sometimes WE don't get permission to fix typos in the print.

Understood, but to reduce user frustration (at least to some degree), I suggest that Logos implement some sort of margin icon similar to the speaker icon that would indicate that a given typo has previously been reported and/or acknowledged but cannot currently be fixed. That will alert users to the situation and keep them from wasting time and help them know that the situation has been addressed even if not remedied.

 Eli, I like David's suggestion. Some means of identifying known errors that doesn't violate copyright, or publisher contract, must be attainable.

This would be the equivalent of an ebook sic. When quoting books with errors, sic is used to indicated the error is in the book and is being quoted "as is". It makes perfect sense that ebooks would indicate the same thing.

I see your point but it isn't exactly equivalent. You don't see people who re-publish a public domain work putting [sic] within the text.

At the end of the day publishers need to catch up to the digital age where it doesn't cost thousands of dollars to redo a typeset to fix an error or running a new edition. It is a couple of keystrokes combined with the submit button.

Teacher, Ministry Leader, Student, Author, Husband

How to upload logs

Visit My Site: Reformed Truths

Posts 73
Brian Wilson | Forum Activity | Replied: Thu, May 14 2015 10:34 AM

Mark Barnes:

Yes, Reuben, I think a hybrid system like that would be an ideal one if it wasn't too complicated to implement. Something like this:

  1. "Ordinary" users can submit typos and vote on other people's typos.
  2. If a typo has 90% acceptance after 10 votes, it gets automatically accepted.
  3. When a typo is accepted, the user who submitted it gets a point added to their profile, if it's rejected they lose a point.
  4. Users with more than 100 points become superusers, whose can fast-track other people's typos to acceptance, without needing 10 votes.

and those with a certain amount of points get logos credit for future purchases.

The spirit of a pilgrim greatly facilitates praying. An earth-bound, earth-satisfied spirit cannot pray.--E. M. Bounds

Posts 790
Eli Evans (Faithlife) | Forum Activity | Replied: Thu, May 14 2015 3:19 PM

Thank you everyone for the enthusiasm and willingness to pitch in.

Crowdsourcing hyperlink tagging might be a good idea, or it might not. Either way, it's definitely "before its time."

We've discussed similar ideas for allowing data collection for hyperlinks by users a number of times over the years. The hold up on our end isn't really the "How would we manage the collection?" (front-end) part of it so much as the "Do we have all the infrastructure and workflows necessary to do what we need to with the resulting data?" (back-end) part. We'd hate to build a hyperlink collection system that encouraged users to waste time tagging references that went into a black hole. We'd also hate to build a data integration system that would require rebuilding, re-downloading, or re-indexing resources too frequently, or en masse. We'd hate to build a system where you couldn't get remunerated for tags, or couldn't freely self-manage your level of contribution.

In the meantime, we've been building most of the infrastructure components that would be necessary for it to work for other reasons -- for example, Community Tags and a points system for remuneration, and lot of other plumbing stuff that you don't see. I still think we're a ways off, though. Some of the necessary pieces were just shipped in L6 and others are still on the drawing board.

We are definitely thinking creatively about the bigger problem (namely: how to tag more stuff more fast but not less good).

Page 3 of 3 (44 items) < Previous 1 2 3 | RSS