Finishing War Diary Data Cleanup and an ...

Finishing War Diary Data Cleanup and an Updated Base Map

Feb 22, 2026
405

The Data Cleanup — What It Actually Involved

Most of the past few weeks have been spent on work that's nearly invisible from the outside, but which matters enormously to the accuracy of everything on the site. I've been doing a deep clean of the war diary geocoded locations data, and it was one of those tasks that kept expanding the more I looked at it.

The early geocoded references, some of them done over 5 years ago by manually comparing War Office maps to Google Maps, were often slightly off. Not badly, but enough to matter.

I've been going back through thousands of records and updating them through a semi-automated process, using data from the true-to-map coordinate converter, which is based on properly georeferenced map overlays, to recalculate the positions far more accurately than the original manual method ever could. The improvement is noticeable.

imageAbove is the utility tool created for the semi-automated updating

At the same time, I've standardised the grid reference format across the whole dataset. Previously the way the grid references were recorded in the database was inconsistent complicating the process of looking up existing location when geocoding new war diaries. Everything now follows the same format consistently: with the grid reference system and two-character prefix throughout. It sounds like a small thing, but it makes the data much easier to search and removes a whole category of errors that were quietly causing problems.

This also builds on the place name cleanup and duplicate location removal I mentioned in the last update, these things compound. The data is in significantly better shape than it was a month ago.

New Tools, With a Public Release Coming Soon

A big part of getting through this cleanup efficiently was building better validation tools. A new one I've put together is a lat/lng to WW2 grid reference converter, essentially the reverse of the existing coordinate converter. Being able to take a modern coordinate and instantly verify which WW2 grid reference it corresponds to has made it dramatically faster to validate and correct records in bulk. I'm planning to release a public version alongside the existing coordinate converter in the next few weeks.

imageDoesn't look too exciting as a command line tool, but with the addition of a web interface and maps will prove to be another useful tool for researchers

I also caught and fixed a bug in the coordinate converter while I was at it. References sitting right on the edge of a 100km square like vT991834, weren't triggering the map-to-true conversion instead always falling back to the projection based conversion. It was an edge case in both senses of the word, and it's now fixed.

The Updated Base Map

imageI've also published an updated base map for the war diaries. The coverage has expanded significantly, the image quality is better, and the map loads faster than before. It's one of those updates that's hard to articulate in words but immediately noticeable if you use the map regularly.

The main changes were switching the tile format from PNG to WebP and nearly tripling the number of maps in the mosaic. The original mosaic consisted of 12 maps, 2.1GB across 65,729 tiles. The new version consists of 32 maps, 241,381 tiles, and comes in at 2.4GB despite the much greater coverage, because WebP achieves better compression at equivalent or higher quality. In practice that means faster load times and lower hosting costs, which is essential for keeping the project sustainable as the plan is to keep adding maps.

I've already started scanning more War Office maps to expand the coverage further eastwards, and those should start appearing on the site in the coming days.

New Memorials

Alongside all of this, I've added four new memorials to the site:

Park Radley Walters, Buron: Named after Brigadier General Sydney Radley-Walters of the Sherbrooke Fusiliers, who commanded A Squadron during the fighting at Buron on 8 July 1944, earning the Military Cross and Distinguished Service Order.

Lt Chaplain John Anderson memorial, Buron: A plaque within Parc Radley-Walters honouring the Padre of the Highland Light Infantry, 1913–2004.

Sergeant H.A. Newman RAF grave, Sainte-Honorine-des-Pertes: The only Commonwealth burial in the village churchyard. Newman died 15 November 1941, service number 912090.

95th Infantry Division memorial, Mandeville-en-Bessin: A plaque on the town hall recording the division's presence in the village from 14 September to 9 October 1944.

All four are now on the site with the usual write-up and photos.



Thanks again for your support. The last few updates have been slow, behind-the-scenes work, not the most exciting reading I know, but with the data cleanup now finished the focus is shifting back to new maps, war diaries, and some more interesting features I've been wanting to add for a while. More on that soon.

Vous aimez cette publication ?

Achetez un café à WW2 Normandy War Guide

Plus de WW2 Normandy War Guide