Ancestry.com’s new “Genetic Communities”

The ethnicity calculations used by ancestry.com and many other commercial DNA testers are toxic hokum. Wonderful marketing tools precisely because they appeal to the lizard back-brain in all of us, they gloss over the fact that there is no such thing as “ethnicity”. Peoples and differences and communities there are aplenty, but ahistorical essences that define groups as this ethnicity or that? Puh-lease.

Ancestry’s assertion that “the ethnicity estimate provides a distant picture of a customer’s genetic origins, perhaps hundreds or thousands of years ago”, is just plain wrong. What you get is a comparison of your test results with a (pretty paltry) reference panel, reasonably accurate to four generations, less accurate to five, sometimes useful to six and almost always worthless before then. A distant picture it is not. (For more on the flaws of this stuff, see UCL’s “Debunking Genetic Astrology“.)

138 on the Irish reference panel? There are more on any 16A Dublin bus.

So when ancestry.com announced “Genetic Communities” as a new feature of its DNA service, I was sceptical, to say the least.  Then I saw the map produced by their analysis of my own test, and I was blown away. None of my family tree is on ancestry, so it was produced purely by DNA analysis. And they hit the bulls-eye on the detailed North Connacht and Galway origins of all 16 of my 3 X great-grandparents.

That’s my mother’s people where all the communities overlap. 

How could they do this, working purely from the DNA? According to the white paper accompanying the new service, a “genetic community” is simply a group of people from more or less the same place who married each other over multiple generations, a nice, loose target, and much more sensible than “ethnicity”.  They arrived at their more-than-300 communities by detailed meta-analysis of the DNA matches in more than 2 million samples. Instead of just comparing my test with all the others and seeing to whom I was most related, they took all those to whom I was related and examined who they were related to. And so on and so on.

It was then possible with the aid of an algorithm for detecting densely connected sub-networks within large datasets (the “Louvain Method“, if you must know) to identify the groups most closely related to each other. They then went on to use their own online trees to associate these groups with particular locations, and then ran the whole process again and again to zero in on sub-sub-groups. The granularity of the results is truly extraordinary. In Ireland alone, there are (so far) seventeen different subgroups, ranging from East Donegal to West Cork to North Connacht to Connemara. Each group is presented alongside a series of good short histories explaining the history of the area over the past two centuries and its outmigrations to the US.

Ancestry has used the critical mass of its huge collection of DNA test results to provide a genuine, scientifically-grounded genetic atlas of the past 200 years, no less.

I still have my quibbles (to misquote Charlton Heston, “They’ll prise the quibbles from my cold, dead hands”.) There are unexplained sciency-looking variations in the size of the location circles on the map: what do they represent? Are they unique to each test analysis, or generic? How were the precise-looking boundaries of the communities arrived at? Above all, why is it not possible to see the data underlying the maps by clicking through?

But quibbles they remain. The whole thing is nothing short of deadly.

Dormant, Abeyant, Forfeited and Extinct

Only one thing is certain about absolutely every ancestor you have: all of them had at least one child. Obviously.  Otherwise you wouldn’t exist.

Does this mean that everyone alive today is a winner in an evolutionary competition to reproduce?

Not quite seven billion

With a world population of 7 billion, we are a spectacularly successful species, but self-congratulation is a bit premature. Before we start clapping each other on the back and congratulating ourselves as champions bred of the loins of champions, it’s worth examining some details.

First, the genes of even the most fecund of our ancestors eventually cease to exist. A child receives exactly half of their genetic makeup from each parent, meaning that the original genome is diluted further and further with each generation. So it doesn’t matter if Niall of the Nine Hostages was your 35 times great-grandfather. There’s almost none of him left in you.

Go on. Have a good gloat.

And what about all those who have no living descendants? Were they all spinster aunts and bachelor uncles? Not at all. Entire multiple-generation dynasties of the rich and powerful, spawning dozens of rich and powerful children who had dozens of children in their turn, have simply vanished from the face of the earth. Burke’s Dormant, Abeyant, Forfeited and Extinct Peerages of the British Empire provides plenty of object lessons (and is always good for a little schadenfreude).

In fact, the iron laws of statistics show that there was a point, somewhere between 5 and 15 millennia ago, where each individual then alive was either the ancestor of every individual alive today, or has no living descendants at all. Genetic genealogy calls it the “Identical ancestors point”, because, logically, earlier than this point everyone now alive shares precisely the same set of ancestors.

Like a lot of genealogical musing, this stuff can seem very trivial or very profound, but it’s hard to stop thinking about it once you start.

And yes, I do suffer from insomnia

I just don’t want to pay that price any more


When people complain to me about transcription errors in our shiny new databases, I try to reassure them: “Notch your scepticism up to 11, come at the records from as many angles as possible and always remember that these errors are the price you pay for having the databases in the first place.”

Going through the National Archives online Tithe Books collection over the past week, I repeated that advice to myself over and over, but it didn’t work. I just don’t want to pay that price any more.

I ain’t gonna pay that price no more

As a source, the Books’ main virtue is that they exist. If we had the censuses destroyed in 1922, they’d be a quirky footnote, a textbook example of blinkered Irish sectarianism shooting itself in the foot. Supporting your clergy by imposing a tax on near-destitute members of a rival church was hardly a recipe for inter-faith harmony.

But they are now virtually the only census substitutes for most places in the 1820s and 1830s, and the NAI online collection is the only route of access. And it’s god-awful.

First, the quality of the personal name and place name transcriptions is only wojus. In a single parish, Knock in Mayo, I’ve counted at least ten mistakes for the surname Flatley: Flattey, Hattley, Halley, Hattely, Hatley, Huttley, Slatterly, Slattery, Thally, and Harley.

Flattey, Hattley, Halley, Hattely, Hatley, Huttley, Slatterly, Slattery, Thally, Harley. And Flatley

OK, but at least we can use the online images as if we were at a microfilm reader and just ignore the database?

No. The parish names are jumbled up so badly, it’s impossible to be sure what images you’re looking at. To take Carlow alone, the link for Aghade takes you to Aghada (Kerry), Clonmelsh to Clonmult (Cork), Lorum to Loughbraccan (Meath), Painestown (Carlow) to Painstown (Meath) … And the online correction facility provides no way of pointing out gross navigational problems like these.

Then last week, while wandering idly through the FamilySearch online catalogue, I came across the listing for the original microfilm collection on which the NAI site is based.  The layout makes it clear that the originals were organised into 140 bundles alphabetised by parish name. So bundle #1 ran from Abbey to Aghaboy and bundle #140 from Wallstown to Youghalarra. (and incidentally bundle #45 Drung to Duncormick was never microfilmed at all and so is just not digitised).

All the mistaken parish identifications are there on FamilySearch too, but seeing the original microfilms laid out like this makes it possible to burrow down to the actual start point of each tithe book. I’ve done it for Carlow (here), but with the results linked to FamilySearch rather than NAI. For one thing, the FamilySearch online microfilm reader has previous/next links, praised be the Saints. For another, where there are database transcripts, they appear below the images, providing (limited) help in deciphering.

There’s a lot more work to be done to make the other 25 counties actually useable. I don’t think anyone else is going to do it.

Why are Irish records so weird?


Answer 1:

They’re not. All the other records in the world are weird. Irish records are the only normal ones.

Did you see my little Jimmy marching
With the soldiers up the avenue?
There was Jimmy just as stiff as starch
Just like his father on the seventeenth of March.
Did you notice all the lovely ladies
Casting their eyes on him?
Away he went to live in a tent
Over in France with his regiment.
Were you there, and tell me, did you notice?
They were all out of step but Jim’

 

Answer 2:

OK, maybe a little weird, but with good reason.

  • 1922

In April 1922, in the series of events that began the short but vicious Irish Civil War, a group of Republican guerrilla fighters took over The Four Courts, the complex of buildings in the centre of Dublin that coincidentally housed the Public Record Office of Ireland.  They used the most solid structure in the complex to house their large hoard of munitions. This was the Treasury store-room of the PRO.

When the phoney war ended on June 29th 1922, their opponents began to shell The Four Courts and on the morning of June 30th, a series of enormous explosions took place. The munitions dump inside the Treasury had detonated. As a result every single document in the Treasury was destroyed.

Those documents included:

  • The census records of 1821, 1831, 1841 and 1851;
  • Church of Ireland parish records dating back to the 17th century;
  • Deeds going back as far as 1174;
  • Court records dating back to the 13th century;
  • Military records with details of local yeomanry from the 18th century;
  • Transportation records;
  • Wills going as far back as 1500;
  • Records of the huge land transfers of the 17th century.

Before June 30th 1922, Ireland had one of the richest sets of historical documents on the planet. What happened on that day is the reason so much Irish research on periods before the 1850s is now focussed on bizarre, otherwise unimportant sources: tithe books, rent rolls, sectarian head-counts …

But what wasn’t in the Treasury is still there: fragments on their way back from the Reading Room or out for rebinding, finding aids, published copies … And of course many records were never in the PRO in the first place – other church records, civil registration records, local tax records, the 1901 and 1911 censuses and much more.

  • One-and-a-half languages

If you’re researching German ancestors, you have to deal with records written in the German language. If you’re researching Russians, you’ll come up against Russian. If you have Irish, you’ll be looking at records apparently in English, but distorted by the huge background presence of a completely different language, Gaelic.

Take surnames, for example.

Well into the nineteenth century – especially in the poorest areas of the West and North from which there was most migration –  the Irish language, Gaelic, was the language of everyday life. So when a baptism or marriage or burial was recorded, most of the people being recorded supplied their names in Gaelic. But no written records of these events were ever kept in Gaelic. So the record-keeper somehow had to import those Gaelic surnames into English.

The result was an extraordinary range of variation, with names mangled and distorted out of all recognition. First the venerable Gaelic prefixes Ó and Mac (meaning “grandson of” and “son of”) were treated as nuisances to be got rid of. Then the English-speaking record-keeper wrote down what he heard. Or what he thought he heard. Or what he thought the meaning of the surname was. Or a completely different English surname that just sounded a bit similar.

So Ó Maoildeirg (meaning “grandson of the red monk”) became Mulderrig. But it also became Reddington: I’ve seen members of the same family baptised as both. Mac Giolla Bhríde (“son of the follower of St. Bridget”) became Bride and McBride and Gilbride and Kilbride. Mac an Bhreithiún (“son of the judge”) is in the records as Breheny and Brehon and Judge and Abraham. Ó hIongardáil became the stout, bully-beef English surname Harrington.

In sum, one of the biggest obstacles to successful research in Irish records is the lack of appreciation of the extraordinary variation in the written records of Irish surnames. They are unimaginably slippery.

There are so many of you and so few of us

The relationship between Ireland and the descendants of those who left Ireland is unique. There are over ten times more people claiming Irish descent in the US alone than there are in the Old Country itself.

For a long time, this was a source of shame in Ireland – what kind of a country forces so many of its people to leave? Genealogy, for the Irish overwhelmingly a process of re-knitting family connections broken by emigration, was bizarre and unrespectable.

“We spent a long time sweeping all that under the carpet, don’t be bringing it out now”.

In the 1990s, that began to change. For whatever reason, we began to come to terms with Ireland’s history and official attitudes to what was now our “disapora” began to shift. The Irish nation became looser and baggier, to include Irish-America, Irish-Australia, Irish-Canada. (Irish-Britain is still a bit of a stretch, but we’re getting there.)

One practical result was that official assistance for people researching their Irish ancestry mushroomed. It became government policy to make as many records as possible freely available online. The extraordinary success of the digitised 1901 and 1911 censuses showed what could be done. Since then Griffith’s went free, the Catholic register microfilms went free, and, most extraordinary of all, last September almost all of the state records of births, marriages and deaths went free.

The effect has been that it is now straighforward to take any Irish family back to the mid-nineteenth century, a revolutionary change completely unforeseeable even two or three years ago. And Irish research is the least commercialised of any English-speaking country.

So yes, weird, but mostly in a good way.

 

How to find other Mother-and-Baby home deaths

Like everyone else in Ireland, I’ve been aghast at the revelations of the treatment of the bodies of babies and infants who died in the Tuam Mother and Baby Home between the 1920s and the 1960s. Once again, we’re reminded how little value independent Ireland placed on its own children, with particular loathing and cruelty reserved for the children of the poorest and most vulnerable.

And it is Ireland that bears responsibility, not just the Catholic Church. The men and women who ran these institutions were our parents, grandparents, aunts and uncles, not alien occupiers dispatched from the Vatican, as some recent anti-Catholic commentary seems to imply.  I remember my father jokingly threatening me with Salthill Industrial School for some misdemeanour in the early 1960s. He knew, and I knew, that there was horror behind its doors.

Bystanders most of us may have been, but innocent? No.

St. Joseph’s Industrial School, Salthill, Galway c. 1960

As a researcher, the aspect of the Tuam story that struck me most strongly was how local historian Catherine Corless went about retrieving the memory of the children who died in the Home. She examined all the local death records in order to identify the deaths of those who had died in the Home and then bought individual General Register Office print-outs at €4 each.

And of course all of these records are now free online and easily searchable up to 1964. And there are many other Mother and Baby Homes whose infant deaths have not been retrieved, where the memory of those children is still obscured.

So it’s now possible and simple to extend what Catherine did to those other children. This is a list of the fourteen homes being investigated by the Mother and Baby Homes Commission.

Pick a home, go to the Advanced Search section of the IrishGenealogy civil records site and confine the search to that home’s Registration District, with the age at death 0, 1 or 2 . Then just work your way, year by year, through the death records, as Catherine did. It quickly becomes clear just how appalling the child death rates were in those institutions.

The deaths of Paul Dunne (7 weeks) and Malachy Byrne (6 months) in Sean Ross Abbey, Roscrea in 1939. This is one of the Homes under investigation

Where are they all buried? Almost none in marked graves, that’s for sure. At least the act of retrieving their names might begin the process of ensuring they are not completely forgotten.

My light is no longer under a bushel

Almost all the genealogy writing and coding I do is the product of advanced laziness. Tracing Your Irish Ancestors came about because I needed handy pigeonholes for all those source references I couldn’t be bothered to remember. The databases and websites arose out frustration at having to repeatedly check the same reference works in the same order – find a townland, identify the civil parish, work out the Catholic parish, check the diocese, check the dates, order the microfilm. Argh. Shortcuts, give me shortcuts!

Hard at work on them databases

So most of this website is the result of me building tools to make research easier for myself.

One of the problems with this approach is that it tends to favour just getting things done over deciding what to do or telling other people about what you’ve done. As Bill Gates once said, small organizations spend almost all their resources doing things, and large organizations spend almost all of theirs talking about what to do. And organizations don’t come any smaller than me.

All of this is a roundabout way of saying that after ten months, I’ve finally got around to having a proper “What’s new?” page, with dated details of all the record references and new features I add, as I add them, and a proper “email me when you add something” service.

My light is no longer under a bushel.

Good deeds

The historic memorial books.

Many veteran researchers have a soft spot for the Registry of Deeds in Dublin. Research here is research as it should be, with much climbing up and down ladders hefting giant hand-written tomes, much poring over 200-year-old legal abbreviations and inhaling 200-year old dust. All that’s missing are the powdered wigs.

But an irksome aspect of recent research there has been the Registry’s official policy on photography.  It is utterly bananas.

A picture I took of the Registry’s ‘No cameras’ sign.

Every archive balances the contradiction between making records accessible to the public in the present and keeping them safe for the future. And every archive except the Registry has long recognized that encouraging readers to take digital images is a near-perfect answer: it decreases wear and tear on the originals and gives readers a chance to chew over complex records at their leisure. But the Registry bans cameras completely and polices the ban with CC-TV in every research room.

Land index Dublin city 1739-1810

Now FamilySearch.org has rendered the ban moot. So moot, in fact, it couldn’t possibly be mooter. In 1950, the Mormons made a microfilm copy of all of the Registry’s records, Lands Indexes, Grantors’ Indexes, Memorial Books, the lot, all the way from its opening in 1708 to 1929, comprising a massive 2686 microfilms. And they are now digitising the microfilm and making it freely available online. So far, all the Grantors’ and Lands Indexes up to 1929 are complete, all of the eighteenth-century memorial books are complete and about 90% of 1800-1850 memorials are there.  The intention appears to be to complete the set.

Memorial 383002 March 29 1805. Click to see the quality

Hurrah. Almost everything of interest to genealogy is now online, and imaged very well indeed. So much for the camera ban.

Don’t get me wrong. Research on these records remains as cumbersome as it ever was: identify deeds of interest from the indexes; find the matching volume number, then the right page number, then the right memorial. But now, instead of humping 50-pound books up and down ladders, you’re downloading 500 MB microfilm files.

A couple of spin-off implications come to mind. First the heroic volunteer transcription site “Registry of Deeds Index Project Ireland” has depended up to now on its transcribers having physical access to the Mormon microfilms. With direct online access, it should gain hordes of new transcribers and gather serious speed. Hurrah again.

Second, I don’t think I’ll ever breathe that centuries-old dust again. Or maybe ever get out of my dressing-gown.

Down the 1911 rabbit-hole

Every single human intervention in a record-set leaves its own layer of errors and omissions. Even the originals have mistakes. My own birth cert records my father as a farmer, something that irritated him immensely when he found out – he was proud he couldn’t tell one end of a cow from another. I presume the registrar in Portiuncula hospital in 1954 didn’t know Da’s occupation and made a reasonable guess. In 1950s rural Ireland he would have been right most of the time.

The cow is of the bovine ilk; one end is moo, the other milk.

Add to that the errors made when the records are catalogued. And then the omissions when they’re microfilmed. And the ones that are overlooked when the microfilms are digitised. Not to mention the mistranscriptions.

It’s a wonder we can find anything at all.

What set me off on this was last week’s post about 1911 census returns  imaged online but not transcribed. It produced an itch that had to be scratched: what about all the other 1911 returns that are missing? Some fell down the back of a desk early on and never made it to the National Archives. Some were missed by the Mormon microfilm team, but exist in hard copy in NAI. And some were microfilmed but never made it online, for reasons only known to the digitisers, Library and Archives Canada.

Down the 1911 rabbit-hole. That’s Valencia DED just below me.

So I’ve scratched that itch and put together a master list of

  1. online but untranscribed,
  2. microfilmed but not online,
  3. not microfilmed but in hard copy
  4. gone, God knows where.

The sources are the Rootschat forum on the topic, NAI’s own list of what’s missing (don’t ask) and my own fevered scratchings. I hope it will provide a home for any other refugees.

Time to break out the Calomine lotion.

The strange afterlife of the census microfilms

The method used by the National Archives of Ireland to digitise its genealogical records was sensible and straightforward. It took the existing microfilms (all created by the LDS Church) as the starting point and used digital images created from the microfilms as the basis of transcription and to provide an online copy of the original.

So far so good.

The first result is that the online collections include any flaws in the microfilms. For instance, the 1901 microfilms omitted the reverse of all Form As, which contain useful place-name identifiers, and so these are missing from the online image collection. Or areas missed by the microfilm team  – Ramelton Road in Letterkenny – are still offline only.

The second result is that it is possible to use the online image collections as if they were microfilm, scrolling forward and back through them in sequence.

The famous census.nationalarchives.ie/reels/nai002500860/

How do you do this? In your browser address-bar, you’ll see something like “census.nationalarchives.ie/reels/nai002500860/”. Just add 1 to that number to go forward a frame and subtract 1 to go back.

But the real question is why in the name of all that’s holy would anyone want to treat these image collections as if they were microfilm? Haven’t we spent half our lives praying for an escape from microfilm?

We don’t get away that easy. Just one example: the 1911 finding aid used to oversee the transcriptions included a category “Townland”. Many enumerators’ returns for small, single-street villages left the townland name blank and entered the name under “City, Urban District, Town or Village”. But where the finding aid left the townland blank, the transcribers presumed there were no returns and left it untranscribed. So around forty smallish villages are imaged online, but not findable by searching the 1911 database of transcripts.

The only way to research them is by going to the initial image and scrolling through them by hand. Precisely as if you were at one of those blasted microfilm machines.

So now you know why you couldn’t find Ballaghdereen and Moate and Kinvara in 1911.

Below is a table of the omissions I know of, with a link to the initial file. Let me know if you come across more.

Street-villages untranscribed in 1911, but imaged online

County Start file DED Name Comment
CORK nai002025430 Myross
CORK nai002026380 Crookhaven
DONEGAL nai002096139 Moville
DOWN nai002248323 Rosstrevor
GALWAY nai002405276 Kinvarra Kinvara town
GALWAY nai002456942 Headford Headford town
GALWAY nai002441621 Portumna Portumna town
GALWAY nai002370022 Sillerna
GALWAY nai002423570 Killeroran Ballygar town
GALWAY nai002423611 Killeroran Ballygar town
GALWAY nai002426479 Mount Bellew Mount Bellew Demesne
KERRY nai002499945 Tarbert
KILDARE nai002561153 Graney Castledermot town
KILDARE nai002562371 Donaghcumper Clonoghlis
KILDARE nai002560339 Ballitore Ballitore Town
KILDARE nai002570290 Rathangan Rathangan village
KILDARE nai002561729 Celbridge Entire DED
MAYO nai002951079 Bunaveela
QUEEN’S CO. nai003161018 Vicarstown
ROSCOMMON nai003185940 Ballaghadereen Ballaghadereen town
ROSCOMMON nai003226133 Cloontuskert Lanesboro town
ROSCOMMON nai003218186 Croghan Croghan village
TIPPERARY nai003368519 Mullinahone Mullinahone town
TIPPERARY nai003360703 Kilbarron Ballinderry town
TIPPERARY nai003365169 Terryglass Terryglass town
TIPPERARY nai003316315 Ballina
TIPPERARY nai003381603 Killenaule Killenaule town
TYRONE nai003434634 Stewartstown Stewartstown village, West Street
TYRONE nai003436854 Moy Moy village
WATERFORD nai003957523 Kilwatermoy, West Janeville
WATERFORD nai003479421 Courmaraglin
WATERFORD nai003510696 Faithlegg Cheekpoint village
WATERFORD nai003481043 Dromana Villierstown
WATERFORD nai003511636 Killea Dunmore village
WATERFORD nai003475372 Dungarvan No. 1 Urban Part of Mitchel Street
WESTMEATH nai003525104 Moate Moate town
WESTMEATH nai003525164 Moate Moate town
WESTMEATH nai003525203 Moate Moate town
WESTMEATH nai003525665 Moate Moate town
WESTMEATH nai003554911 Kilbeggan Kilbeggan town
WEXFORD nai003574228 Castle Talbot
WICKLOW nai003641691 Glendalough All of Glendalough DED missing
[This post was originally published in March 2016, but got washed away in the Great Delete of June 2016. Joe Buggy wanted it back, so it's his fault. Paddy Waldron continues to unearth other missing returns at Rootschat.]

How to date Griffith’s Valuation precisely

Griffith’s Valuation is an astounding achievement, a masterpiece of Victorian social quantification that measures every property on the island of Ireland with painstaking, pinpoint precision. But it is not a census, and to use it as a census substitute, you have to understand how it works.

Sir Richard Griffith in 1854

Griffith was charged with producing a scientific basis for property taxation in Ireland, and that is exactly what he did. Every building and every field in the country was assessed in meticulous detail to produce a monetary figure that represented the income that property should produce every year. The results were published between 1847 and 1864 in a series of 301 volumes.

These volumes were a public statement of the property tax liabilities of the inhabitants of the areas they cover, and were open to challenge. So accuracy was paramount. And part of this accuracy was precision about the date of publication – property, then as now, was a moving target.

For researchers, those precise dates of publication can be very important: if your William Burke was in Boston in March 1856, that can’t be him in Castlebar on January 26th 1857.

Title page of the Griffith’s volume for Castlebar Poor Law Union

So how do you get that precise date? Every volume has the date on its title page. And you can get to the title page by going through Askaboutireland.

Run a search (e.g. http://goo.gl/JFSkd8 ), then open up a page image in a new tab or browser window. If you want, you can then just click the “previous page” link until you get to the volume title page.

But each volume can have up to 500 pages, making that process mind-numbingly tedious. Here’s a shortcut: in the browser address bar, you’ll see something like “griffiths.askaboutireland.ie/gv4/z/zoomifyDynamicViewer.php?file=210173[…]”

Griffith’s browser address bar

The”file” in that address is made up of two parts, a three-digit volume number and a three-digit page number. The example above therefore refers to volume 210, page 173. If you want to go to page 1 of volume 210, just change that 173 to 001 in the address bar, hit “enter” and there you are. The precise publication date is usually about two-thirds down on the left. In the example, it’s January 26th 1857.

Et voila.

[This post was originally published in March 2016, but got washed away in the Great Delete of June 2016. It's useful enough to republish, I think]