My sister Loren is a PhD candidate at UVA studying medieval French literature, integrating digital humanities methods into her work. Because I’m an older sister incapable of minding her own business, I asked too many questions and ended up presenting with her at two conferences on a project exploring the use of generative AI to fill in missing manuscript fragments. The slides from one of these presentations are embedded below, and here is the link to the associated GitHub repo.
I ended up having the most success using an out-of-the-box Stable Diffusion model for doing the inpainting and using the Segment Anything Model 2 (SAM2) to create the masks. You can see the results in the slides linked to above.
Before landing on that approach, I tried (and sort of failed) at several different things, which you can see evidence of in the repo. Initially I was hoping I could train my own model, but quickly realized that was still a bit above my level for the moment, and would also require more raw data and compute power than I had available.
I also wanted to try performing inpainting on the images that Loren had taken of her specific texts at libraries all over Europe. I learned about some of the pre-processing steps that are often necessary to get good results with these sorts of images, such as optimizing the size, adjusting the color, binarizing, and correcting for skew. It was valuable to learn about these steps, but I still wasn’t able to get good results on these “homemade” images and had to use professionally digitized ones.
With more time and practice, I’d like to keep trying to train my own model and work with homemade images, and to add more automation to some of these workflows so it’s easier for other humanities researchers to apply the same methods to their own work. Next installment coming soon to a medieval studies conference near you.