Documentation:How to use the Fixer

Let me tell you about the most insidious mistake you are going to make when creating layers. It usually happens when you work with a layer that contains a large number of instantaneous events. Take for example the layer containing events of the French Revolution. I’ve made (and then fixed) a couple of such mistakes when creating it.

Example

To explain this mistake let’s use some fake data. Imagine that you have some source text and, using the Parser, you have marked it up like this:

Here is the text version that you can copy and experiment with:

March 12, 1808*Some event that happened on March 12, 1808
March 15, 1808*Some event that happened on March 15, 1808
March 17, 1808*Some event that happened on March 17, 1808
This date is selected accidentally: March 18, 1808. We don't want it on the Timeline
March 20, 1808*Some event that happened on March 20, 1808
March 21, 1808*Some event that happened on March 21, 1808
March 25, 1808*Some event that happened on March 25, 1808
March 30, 1808*Some event that happened on March 30, 1808
Notice how we forgot to select the date below!
April 3, 1808*Some event that happened on April 3, 1808
April 6, 1808*Some event that happened on April 6, 1808
April 10, 1808*Some event that happened on April 10, 1808

Notice, how you’ve made two mistakes in the markup.

If there was only one mistake, the Parser would have detected it, because it knows that for each red text there should be one blue text. But because there are two mistakes, the second one sort of compensates for the first one, and the Parser thinks that everything is OK.

When the Parser moves data to the right text field it couples texts with dates the wrong way. Here is a visualisation of the couplings that the Parser makes.

As you can see, the first 3 and the last 2 events are OK. But the 5 events in the middle get the wrong dates. This is what it would look like on the Timeline if you used the resulting data (I underlined the wrong dates).

This mistake is very hard to detect. I’ve made the mistakes super obvious in this fake example, but in real life scenarios the event name doesn’t contain the date itself. And so all you see is, for example, “Battle of Thermopylae” and some date on top of it that may be right or wrong. Plus, most of the events get the correct dates and so everything looks fine at first glance. But somewhere in the middle there may be a few dozen events with wrong dates. Or just one wrong date. You never know.

This mistake is totally unacceptable

We want people to trust the data on the Timeline. ‘Trust but verify’ is a good principle to follow in many circumstances. And we have sources specified for each layer. So, if users want to verify the data they can certainly do so. However, if the users start finding events with wrong dates in multiple places on the Timeline, eventually they will feel like they can’t trust any data on this website unless they verify each and every event themselves. This will make the Timeline virtually unusable. We don’t want that. So you must do everything humanly possible to make sure you haven’t made this mistake before uploading data to the server.

How to detect this mistake

There is only one surefire method that I know of. When you’ve done creating the instructions for the layer, you put them into the Editor and make the events show up on the Timeline. Then you open the source page, from which you’ve taken the information in the first place, and put it next to the Timeline. And then you just go through the list of events and compare each and every event in the source to the corresponding event on the Timeline, one by one. However long it takes, you have to do it. Otherwise you don’t know if what you are uploading to the server is correct information or a pile of garbage.

This is very important. I think in the future we may have some peer review process in place. For example, there may be a rule that a layer can’t leave the Sandbox until, let’s say, 2 or 3 admins have checked that the dates in it are correct.

How to fix it

If you’ve found this mistake while parsing the data, you can just fix the markup in the Parser and parse the data anew. But what if you’ve found the mistake much later and you don’t have the source text any more?

Don’t try to fix it manually. If there are more than few incorrect dates, it will be very tedious and error prone to fix them manually. I have created a special program that helps you fix this mistake. I call it the Fixer. And you can find a link to it in the bottom right corner of the Editor.

This is how it works. Step 1: Put the data into the input field.

Step 2: Click the Parse button. The table will be created on top of the input field.

Step 3: Find the dates that are incorrect. To select a range of dates select the first one with a mouse, then, holding down the Shift key, select the last date.

Step 4: Use up or down arrow key on the keyboard to move the stack of dates up or down. In our example, we need to move them up.

Step 5: Click the Serialize button.

Now the dates are in the correct places, and all you need to do is manually insert the date for the event that happened on April 3rd.