About The Project

Teaching data science with ‘real-world’ datasets

Since 2017 teams at the University of Edinburgh have been supporting students to learn about data science and develop their skills through working with a ‘real-world’ datasets. The Survey of Scottish Witchcraft database is one of those datasets.

The Survey of Scottish Witchcraft Database

We appreciate all feedback and will incorporate it into future iterations of the map and the projects delivered by students.

Professor Julian Goodare is a historian and project director for the original Survey of Scottish Witchcraft, so his input and suggestions continue to be extremely valuable to us. The technical work on this project has been led by colleagues in our Digital Learning and Media Teams and in Wikimedia UK . Most notably Karen Howie, Ewan McAndrew, Anne-Marie Scott, Richard Lawson, Stewart Cromar, Andrew Middleton, Navino Evans, Martin Poulter and Simon Cobb. We are indebted to them for the work they have done and the time they spend supporting students who work on data skills curriculum projects and internships.

Richard Lawson

Richard was the person most responsible for creating this website and for always giving his time generously to this project; to support student learning within the University and to support understanding about the Scottish witch trials globally. He taught himself within two weeks how to work with an unfamiliar platform and achieved the remarkable in always using his considerable technical expertise to solve any issue we had in creating the site and helping put these persecuted women on the map. Richard supported all subsequent ‘Witchfinder General’: Data Visualisation student interns and always with kindness, humility and a gentle humour. He died suddenly on 10 October 2023 and this site is a tribute to his brilliance and commitment to the project and we dedicate it to his memory.

Work done by University of Edinburgh students

Data Science for Design MSc Projects 2017-19

During the 2017/2018 and 2018/2019 course programme, the University of Edinburgh’s Wikimedian in Residence, Ewan McAndrew, supported students to surface selected data from the original MS Access database to Wikipedia’s sister project, Wikidata, as structured, machine-readable, linked open data.

Students on the Data Science for Design MSc created engaging visualisations from these real world datasets. One group chose create an interactive physical map of accused witches in Scotland laser cut and engraved from an A1 block of wood and creating bespoke pieces to visualise the gender distribution of accused witches in each region of Scotland and the different torture techniques employed in each region.

Data Science for Design MSc students with their project. CC-BY-SA 4.0

You can see a short film of the results: An interactive laser cut map of accused witches in Scotland

The success of this project, and the engaging visualisations created as a result, made the case that an internship dedicated to geographically locating the places recorded in the database as linked open data would be the next logical step.

The ‘Witchfinder General’ Internship

Emma Carroll (June to September 2019)

Emma Carroll, a Geology and Physical Geography undergraduate student, was recruited in May 2019 as the new Data and Visualisation intern, or ‘Witchfinder General’, as part of a three month long Equate Scotland internship. Emma was asked to do the detective work and data reconciliation required in order to geolocate the places mentioned in the Survey so that each accused witch in Scotland could be plotted on an interactive map as linked open data through Wikipedia's sister project, Wikidata

Emma Carroll locating the accused witches residencies. CC-BY-SA 4.0

As a result of Emma’s work, there was a place of residence recorded onto each accused witches’ Wikidata page which could then be queried for. Geographical information such as trial location, place of detainment and place of death locations were also added onto the accused witches’ Wikidata pages using the open source software, OpenRefine. There were also other properties added for the accused witches from their charge to inhumane treatment type to build a strong Wikidata page for each accused witch which is displayed for each accused witch on the website maps.

Emma Carroll exploring collections at the National Library of Scotland. CC-BY-SA 4.0

The uploaded data could then be directly queried for using the Wikidata Query Service and geographically projected onto a map using Leaflet or downloaded into a CSV file and uploaded onto ArcGIS Online. The geographical data was used to create a variety of different visualisations.

You can learn about her experiences, the digital skills she learned working on the project:

The Wicked Findings of a Witchfinder General #OEAwards2021. CC-BY-SA 4.0

Since the site went live in September 2019, work has continued on the project. Several more student interns have worked on new features, conducted user testing, and imported more data to create an upcoming version of the site which we hope will go live in Autumn 2024 (to coincide with 5th anniversary of the original site launching).

Maggie Lin (June to September 2022)

4th year Computer Science student, Maggie Lin, was employed as our second ‘Witchfinder General’: Data Visualisation intern and worked incredibly hard to help illuminate what happened in the Scottish witchhunts of 1563 to 1736 by focusing on opening up the rich historical data in the Survey of Scottish Witchcraft database and turning it into linked open data, creating user-friendly web visualisations with her developer colleague Josep Garcia-Reyero (also an undergrad student).

Maggie’s focus was on the hitherto unexplored temporal data and the all the rich information on the witchcraft investigations, which encompassed the initial or supposed denunciation, the arrest, the investigation, the trial, and its aftermath. She focused especially on all the different accusations: who named/denounced who, what occurred in the supposed witches’ meetings, the ritual objects used, accusations of shapechanging and much more. By working round the clock to turn this data into something parseable she created timelines, network analyses of who named who, helped show how the hysteria spread in space and time during panic and non panic periods and even investigated accusations of demonology and attended a symposium on such. She presented on her work at the Edinburgh Book Festival 2022 before the Jenni Fagan book event, Hex, and has contributed blog articles documenting her work and recorded a podcast interview.

Maggie Lin at the University of Edinburgh's Gender Ed 5th anniversary event on 29 May 2023. CC-BY-SA 4.0

Watch Maggie Lin's tutorial for a guide to bulk data item creation using OpenRefine:

Bulk data item creation using OpenRefine -tutorial by Maggie LinS. CC-BY-SA 4.0

Josep Garcia-Reyero (June to September 2023)

Josep worked as our Open Source Web Developer intern and had a close collaboration with Maggie Lin to embed her new visualisations in the website by the end of the Summer.

His goals were:

  • To update the software packages and re-factor the site code.
  • Improve the site functionality, accessibility and user interface.
  • Added a group clusters on/off button on the map.
  • Allow filtering on multiple filter options at a time and move the filters to the left hand side of the screen to allow more space for the map.
  • Add a new customizable and more responsive timeline of witchcraft investigations which can be filtered to search for ‘witchcraft panic’ periods and ‘non-panic’ periods.
  • Add Maggie’s new visualisations including: filters for accusations of shape-shifting, demonic pacts, property damage, ritual objects used, witches’ meetings, a network analysis of who named/denounced who and an interactive Histropedia timeline that is searchable by name of accused.

Josep Garcia-Reyero working on the website. CC-BY-SA 4.0

Read all about his work here: Josep's blog on the Witch Visualisation site

Claire Panella (February to May 2023)

Claire Panella worked as the Scottish Witchhunter intern for 1 day a week from February to May 2023. During this time she created and documented a robust and reusable quality assurance process (using RStudio) for the data behind our website so that we could compare the original 2003 Survey of Scottish Witchcraft data with the data shared to Wikidata since 2017 to check for any data anomalies.

Read all about her work here: Claire's blog on the Data Quality Assurance process

Claire Panella developing a Data Quality Assurance process. CC-BY-SA 4.0

Claire Panella describing the Data Quality Assurance process. CC-BY-SA 4.0

Ruby Imrie (June 2023 to October 2024)

Ruby has been quality checking and consistency checking the data using newly developed quality assurance methods in R Studio created by another student intern, Claire Panella, earlier this year that can be reused for many years to come. Ruby’s other focus has been on taking the extremely rich) data Maggie Lin added on all the 3,816 full witchcraft investigations recorded in Scotland from 1563-1736 and embedding those new interactive visualisations on our Map of Accused Witches in Scotland website using a Javascript and the Vue.js framework. By also identifying and addressing bug fixes, conducting rigorous user testing sessions and using the feedback received to address areas for site improvement and action planning for developing new features she has helped to show how the data, and the individual human stories behind the data, can be better visualised, explored and interrogated as never before.

Ruby Imrie with Professor Julian Goodare. CC-BY-SA 4.0

Ruby Imrie on the Map of Accused Witches in Scotland project. CC-BY-SA 4.0

Map of Accused Witches in Scotland internship - Summer 2023 Summary. CC-BY-SA 4.0

Media Responses