Straight Talk about Arbitration

June 8, 2016  - by 

On nearly every indexing-related article on this blog, regardless of the subject, one or more people make an “arbitrator done-me-wrong” sort of comment. It is the most commonly expressed concern of indexers. But if you judge the overall state of indexing and arbitration by these comments or even by your own negative experiences, you may be in need of a little more straight talk.

Here’s What’s Really Going On

For starters, arbitrators are to be commended for working tirelessly on the hard and sometimes undesirable work that has resulted in the consistently high-quality indexes now on FamilySearch.org. The work of these volunteers is highly valued, and they can’t be recognized enough for their contribution. The fact that they occasionally make mistakes is to be expected. And it’s OK.

Remember, indexing and arbitration are volunteer activities. No one who does this work went to school to learn how to do it, and no one has a college degree or special certificate indicating that he or she is particularly qualified to do the work. If FamilySearch were to require that sort of certification of volunteers, we would all still be using microfilm to find our ancestors.

Indexing is a cause that people care about, and most people take it very seriously and do the very best work they know how. So let’s not assume that the isolated negative experiences we all have had with arbitration are somehow reflective of the whole effort. They clearly are not. That said, there are things we all can and should do to improve the quality of published indexes.  

Indexers can read the instructions more carefully and take the time to make sure they are being methodical and precise. If they can’t read a name or find the instruction they need, they can share their batch with a more experienced indexer, or they can share it online in one of the indexing communities where lots of dedicated experts hang out. Indexers should take responsibility for doing their best work and not rely on arbitrators to correct their mistakes.

Arbitrators can improve by taking the arbitration self-assessment, studying and following the recommendations that have been given, and taking the time to be deliberate and careful. Despite their experience, they should read and fully understand the instructions for every project they arbitrate. Arbitrators who don’t index are generally also the ones who receive the most feedback about mistakes. Before arbitrating batches in any project, an arbitrator should first index enough batches of the project to understand its unique quirks. 

FamilySearch can improve by striving to write clearer, noncontradictory instructions; continuing to encourage stake indexing directors, group administrators, and arbitrators to focus on quality rather than quantity; and working on solutions that will allow indexers and arbitrators to be more collaborative.

A (Last) Word about the Arbitration Results Number

You may have recently heard that the arbitration results number has been discontinued. After surveying many indexers, we learned that although a number of indexers gain personal satisfaction from the number and use it for positive reinforcement, a much larger percentage of volunteers feel the number is more punishing than encouraging. Indexers seeking to improve will still be able to review the work of arbitrators and see the arbitration results of a specific batch, but those numbers will no longer be combined into an overall percentage.

When FamilySearch began publishing this number for the first time in 2011, it was to answer requests from indexers who wanted a way to know how well they were doing. While the number gives quick feedback, too many people let it determine how they feel about indexing and even about themselves. Far too many have let this number persuade them that indexing is either too hard and beyond their capabilities or too frustrating and not worth continuing.

This number led one very accomplished indexer to write a letter expressing her frustration and the reasons she was giving up indexing. She indicated that indexing was, for her, a form of relaxation, but the stress from her arbitration results number made it no longer enjoyable. This feeling is tragic. Here was a person who was finding fulfillment and who was making a substantial contribution, and she gave it all up because of the arbitration number.

Now, in fairness, some bad arbitration may have been done on some of her batches, but that would have happened with or without the number. And since there was nothing she could do about the bad arbitration, it probably would have been better if she had never known about it.

By removing the arbitration results number, we lose something of value, but in our view and in the view of most indexers who were surveyed, we gain something more important and substantial. This comment from one survey respondent seems to summarize things fairly well.

“It is good to know if mistakes were made, but grading is a discouragement to me. There are so many factors involved with each individual document, so if one person feels it is one way and another person sees it differently, it does not necessarily mean the indexer is doing poor work. You spend lots of time trying your best to decipher the writing, and I always felt I did my best and was sure of what I put down. It was a bit of a blow to see the arbitration results, knowing how much time and effort I put in.”

What Happens When Something Is Arbitrated Incorrectly?

Many have asked about the effectiveness of the feedback button. We track the amount of feedback individual arbitrators get. When there is excessive feedback, FamilySearch representatives reach out to those arbitrators individually to provide them with additional help and training.

Beyond this, the FamilySearch.org search engine compensates for many common transcription errors (including missing diacritics, although that is no excuse not to index them properly). Future improvements will include capabilities such as making both A and B indexed values searchable, enabling users to make corrections in published collections, and potentially even matching volunteers’ skill level and personal research interests to specific projects. We intend to do everything we can to continue to find ways to avoid indexing errors and to correct them when they happen.

As always, grateful researchers the world over thank you for your hard work and your concern with the quality of the work you do. As you continue in your remarkable service, please do your best to understand the projects you work on, and then be patient with yourself and others as we all try to accomplish this magnificent task together.

 

You may also like:

God Bless the Indexer

God Bless the Indexer

“Lazy Fisherman” Grateful for Volunteers

Lazy Fisherman Family Historian So Grateful for Online Volunteers and Indexes

What it Means to “Fuel the Find”

What it Means to “Fuel the Find”

 

 

 

 

 

 

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *

Comments

  1. I just wish that arbitrators would read the instructions. I indexed many of the Australia-Cemetery Inscriptions and News Paper Clippings 1802 to 2005, and on the cemetery names I have indexed them every way I can think of…I have followed the instructions to the letter (and also did them every other way I could think of) and even e-mailed and called the help line. The people on the help line told me that I was doing it right…and in almost every case it was arbitrated WRONG!!! I feel this problem would be alleviated if the arbitrators would just READ THE
    INSTRUCTIONS… I am also an arbitrator and in arbitrating a batch I often read the instructions 2 or 3 times to make sure I have it right. Come on people, it is very frustrating when you say everything I do is wrong!!! I know I make mistakes, sometimes silly ones, and I accept my mistakes, but accepting YOUR mistakes is something else.

    1. Amen!! The only time i index anymore is when I move to a new type of batch.. then index at least 50 or more to learn how I am doing.. make sure I understand it. even then when I get dinged, and I question it, i will re read the field helps and project instructions again to see if I miss understood something before I ever arb a batch. When I mostly get 100’s then I will arb them. But when I index again, there is sloppy arbitrating.

    2. I was recently dinged repeatedly because the arbitrator Added the Mr, Mrs, word county, the Ft and Inches on height, even adding parenthasis.. totally burned up the feed back button.

    3. Your comment is right on. Feedback needs a short comment field to explain or refer to a certain project instruction.

  2. Like many others who have commented before me, I am an indexer and arbitrator. As an indexer, I know the frustration of poor arbitration; as an arbitrator I know the frustration of poor indexing. 9 times out of 10 it comes from not reading the instructions (in particular the field helps). For myself, when I am uncertain, I try to glean clarity from the project examples. In almost every case however the examples are snap easy. It would help tremendously to see examples showing how to index a challenging record – you know one with missing letters, smudged letters, tears on the page, county/township names instead of city/town names, etc.

    There are project managers and it seems reasonable to me (with limited information about those who oversee these projects) that they could provide some more complicated examples and how to index those.

    Lastly, it would also help if the indexing program could open a project to have the entire scan “fit to the page.” My reasoning here is that then an indexer would be able to see instantly that there are 2 pages/multiple records that need to be included. Too often I’ve arbitrated pages where only the information they saw in the initial window was included because they didn’t know to check for more to the page (either by using the move tool or reducing the image size to show more).

    In any event, I love both aspects in spite of the frustrations and I continue to index and arbitrate in the hopes that I’m providing more information for those who are seeking those in the past.

  3. THIS IS A FIRST!!!! I have just received argbitration on a Boston Crew List. there was only ONE name on the list, which was arbitrated correctly. I was arbitrated wrong on the sex of the second name (a line that was BLANK??!!) It brought me from 100% down to 61% COME ON PEOPLE… If you can’t do it right… DON’T DO IT…GIVE US A BREAK… QUIT NOW AND IMPROVE ALL OUR AVERAGES!!!

    1. I haven’t indexed the Boston Crew List, but why did you have a second line if there was only one record?
      I would have thought you would have used the “Records per image” setting to only have one record to index.

    2. Was the sex indicated on the record or did you go by the name. If it was not indicated i think you make it blank

  4. I use to arbitrate and my percentages were all very good but due to comments like the last one, I’ve stopped arbitrating, what if I screw up like I’ve read so many others have. I would be devastated to be talked about like he did. I guess some of us are just ‘too human.’

    1. I am sorry to talk about some people badly, but it gets SO frustrating to try and be correct and have someone come in and mess it all up by either not reading the instructions or just doing it completely WRONG!!! another example… today I got a batch back that had 3 names on the paper. Name 1 was a sailor that was being left in the hospital listed on the top of the page on the left, name 2 was the master of the ship listed on bottom of the page on the left, and name 3 was a sailor discharged in the port listed in the middle of the page on the right. I thought we were suppose to index from top to bottom and left to right… In indexing I put name 1 on the 1st line, name 2 on the 2nd line, and name 3 on the 3rd line. The arbitrator said the first line was correct, he then put the name on the 2nd line on the 3rd line and counted mine wrong, he then put the name on the 3rd line on the 2nd line and counted mine wrong. ALL OF MY NAMES WERE SPELLED CORRECTLY

      1. But although all my information was correct I got a score of 40% I don’t mind being corrected if I am wrong, but when you are correct and are counted wrong anyway, it gets kind of frustrating. I just hope those that are arbitrating incorrectly read these comments and try to improve their work…. I am an arbitrator and if I am doing wrong I want to be told. And if I cannot learn to do it right I need to stop arbitrating. It just defeats the purpose of indexing and makes it very hard for people to look up their ancestors….
        That’s it… rant over …g’night all….

      2. The arbitrator can move names up or down in either batch A or B to make them match up, no matter the order. That way both indexers get 100%, and everyone is happy! They’ve just got to do it, though.

  5. This is not really a complaint about arbitration, but for some weeks now when indexing Australia, Victorian shipping, there are too many instances where crew, as in ships master, are being deleted or marked incorrect. I was wondering if it would be possible for someone to have a look at the following and let me know why this is happening. Its very discouraging to have all fields marked incorrect because of the arbitrator re-entering the information.
    Australia, Victoria—Outward Passenger and Crew Lists, 1852–1924/007782373[655]
    This is just one of a number I have had this happen to. Thank you in advance.
    Jim

  6. I AM arbitrating and indexing the census records from Finland from 1810-1915.I have been arbitrating for over 5 years. My question is: Have the rules for surnames been changed? Should the surname be noted with every given name in the family? I learned that the surname went with head of household, but not the rest of the family.
    Also, are we now to guess as to the sex if we recognize a female name? I see more and more of that.

    Just checking!!

    1. The rules are still to write what you see. Unless there are special instructions in the project instructions or field helps, do as you always have done. The basic indexing guidelines are always up to date so feel free to revisit those when you have questions.

  7. Can we please get it to where arbitrators need to show knowledge of the project? I’ve been working on the WWII discharge forms for Missouri. I’ve had a couple of batches that the arbitrator expanded the abbreviations, which we are instructed not to do. This wouldn’t bother me so much except now these men and women are from Saint Louis, Montana?!? As I understand it, there is nothing that can be done about it now. It took me forever to find my great grandmother’s birth record because it was indexed as Kansas City Montana also.

    1. Arbitrators do not follow the rules… but that is not the thing that is so stupid to me. It is totally incomprehensible to me that The Church Of Jesus Christ Of Latter-Day Saints, the organization that is responsible for all this indexing, and, according to the e-mails I have gotten from them, value accuracy VERY highly, has NO means for the indexer to say WHY he indexed in a certain manner. A method of putting a comment on the feedback button would be wonderful, but I have been told that the higher up’s have absolutely NO intention of doing this, when it would take probably a line of text in the indexing program to do so… I find this inexcusable, but then it is not my ballgame, I am only a player… I will just keep on playing.

    2. Are arbitrators specially trained?
      Some things I can see them changing, but when it comes to names, is it recorder vs. arbitrator?

      1. Arbitrators are supposed to be very experienced indexers. The guidelines say that no matter how experienced an arbitrator should read through the project guidelines and index enough batches on that project to become familiar with the guidelines. I don’t know about others, but I do know in my group, I am the only arbitrator that does any indexing at all any more. I would really like to see the arbitrators have to take a short quiz on a project to qualify them to arbitrate on that project. Guidelines vary from project to project. Some projects want you to expand abbreviated place names and some don’t. Some want you to correct misspelled place names and some don’t. It is very important to read the project guidelines for every project before beginning. There is also information in the field help that is not included in the project guidelines. This is why it is so important for arbitrators to index a few batches before arbitrating any project. If I am working on a particular project for a long time, I go back and reread the project instructions each week or so, just to be sure I am following the proper guidelines. I take pride in my work and I want the indexes available for us to search are accurate. I am so disappointed that there is no way to correct a mistake by an arbitrator.

        1. I have been indexing since the request for new indexers in the spring of 2016, and have now totalled over 11000 records done.
          I have had quite a few records changed where the arbitrator has obviously not read the instructions, and it annoys me that there is no way to tell the arbitrator what he/she has done wrongly.
          In the last two weeks I have had marriage records changed where the guidelines say that the condition of the bride and groom should be entered as seen, and gives an example as “Bachelor”. My arbitrator changed all my entries to “S”. This is not really going to cause a problem for researchers, but consider the next instance of arbitrator error I encountered recently.
          I indexed baptism records for Surrey in the UK, and at the top of the page where the main details of location and year are recorded, the year was very clearly shown as 1830. All 16 records that I entered as 1830 were changed to 1820 because the first record had a year written in very small numbers with the number 3 unclear as to what it was.
          The indexed records now indicate that 16 persons were born ten years earlier than they actually were, which could cause problems for researchers. I asked for all 16 records to be reviewed, but I will never know if it is actually done.

          1. Who has the final say really. Is an arbitrator more trained than an indexer.
            They are shooting themselves in the foot at this rate. It might take longer but it should go back to the LDS to interpret which one is the better of the two.

            1. Currently, the arbitrator has the final say.Most often this is no problem since there are thousands of highly qualified arbitrators that pay attention to details. However, there are a few who either have not read or did not understand the guidelines or maybe do not understand abbreviations of the time or area. There are two ways this can be fixed, first if the “please review” button actually did anything and secondly maybe have a place where an indexer can make comments or email someone to look at the results from a specific arbitrator. By far and away, most of my batches are arbitrated correctly, but it is very frustrating knowing some misinformation is being published.

          2. I totally understand your frustration. There have been so many times an arbitrator has changed my correct entry with one that does not follow the guidelines. If they are minor ones that do not affect the research, I just deal with it. But there really does need to be a way to correct the arbi
            trators. I can not count the number of times MO was indexed as Montana so people were born in Saint Louis Montana. The latest one, was I have been indexing Cook County death certificates in Illinois. Apparently, one individual died in Chicago, Italy and one died on June 80th 1928. I know this is pressing work that needs to get done as quickly as possible, but we all need to take enough time to make sure the information is correct. Does anyone know if they are still working on a way for other people to correct what was indexed? If this becomes a reality, then I could go back and change the ones that will affect research. We are all doing this so others can find their family members. Let’s make it as easy as possible for them to search accurate indexing.

            1. This is a first for me!! An arbitrator check a batch that I indexed for the Marriage documents: UK, England, Surrey—Parish Registers, 1754–1992 [Part B]/007569148[679], the image is complete on one side and about 1/3 of the image on the other side so you can only index the 2 complete entries on the one side. The arbitrator said the part that I did was 100% correct THEN he went on and ADDED information from the other side of the image…. HE INCLUDED INFORMATION THAT COULD NOT BE SEEN BECAUSE IT WAS CUT OFF IN THE IMAGE. Like the names of the grooms father and the brides father… I have no idea where he got the information. THESE ARE THE IDIOTS THAT WE ARE DEALING WITH. I do a perfect indexing and get a 50% for my troubles…

              1. I haven’t read the project instruction on that particular project, but several US marriage projects were changed at the end of the year so they now state that if a record spans two images to index all of the information from both images on the first record and then mark the second record no extractable data. Could this be the reason? Like I said I have not read the instructions on that particular project.

        2. Well I don’t know if this will do any good, but, maybe this will get at least one arbitrator thinking… while indexing Australian Cemetery records, in the place of burial site, having no place listed, I put instead… ( ARBITRATORS: Please read the rules and get it right!!!)
          Think it will do any good…I doubt it, but then just maybe.. nothing ventured nothing gained,

          1. I have been indexing marriage records for both Surrey and Lincolnshire over the past eight months, and most of the images have been the two complete and two partial records. I have always only indexed the two whole records, and have never had an arbitrator do what happened to Larry.
            Usually, the partial two records appear as whole ones in the next image in sequence, which could be where the arbitrator was getting the details of the groom and bride’s parents.