Unveiling the Mystery of Compressed Regions
Sub-micron scanning reveals why some papyrus layers appear blurred
Last month, both the community and the Vesuvius Challenge team continued working hard to make our dream of reading the ancient collection a reality.
In particular, on beamline ID11 at ESRF, we scanned a tiny 3D region of interest from fragment PHerc. 500P2 at an unprecedented pixel size of 0.55 µm!
Handling the amount of data generated is no trivial task: the reconstructed volume for the same region is about 64 times larger than a scan captured at a 2 µm pixel size, and 4,096 times larger than one captured at 8 µm — the pixel size used to scan PHerc. Paris 4 (Scroll 1) and PHerc. 172 (Scroll 5).
We hope to be able to detect some feature of ink directly in this scan with little aid of Machine Learning, but the analysis will take time.
Meanwhile, the preliminary results have already unveiled a fascinating mystery!
For years, our unwrapping efforts have been limited by the so-called “compressed” or “mushy” regions — some parts of the scan where papyrus sheets coming close seem to generate a sort of “fog”, or artificial blur, which makes segmentation a real nightmare.
We hypothesized that this blur might be caused by microscopic periodic structures in the fibers that make up the papyrus sheets.
These structures should correspond to the lumen (a cavity) in the cell walls of the fibers. Periodic structures just a few micrometres across can make X-rays scatter — or slightly deviate. The longer an X-ray travels through a region with this periodicity before reaching the detector, the stronger the effect becomes.
As a result of this effect, 3D regions with a higher density of periodic structures (fibers, hence sheets) can appear blurred.
The previous picture is a cross section of the scan acquired with a pixel size of 0.55 µm. Each fiber is surrounded by a thick wall of cells, and the cavity inside these cells is clearly visible. When fibers are densely packed — as in closely spaced sheets — these cavities are likely the source of blur in lower resolution scans.
How can we turn this knowledge into better scans?
Ideally, one should scan with a pixel size lower than the wavelength of the periodicity ( < 1 µm ).
Since sub-micron scanning is impractical for large volumes, we ran several tests and developed a recipe to reduce blur by balancing three factors:
Pixel size as low as feasible (meaning higher resolution);
Sample–detector distance short enough to enhance phase contrast without letting scattering dominate (generally <1 m, and less than half the near-field limit);
Incident beam energy as low as possible to boost absorption, within the constraints of pixel size and sample–detector distance.
The three factors are coupled in a non-linear way.
What worked the best for us, for several small pixel sizes:
pixel size 0.55 µm, sample-detector distance 7 cm (could not go lower), energy 65 keV;
pixel size 2.4 µm, sample-detector distance 22 cm (could not go lower), energy 77 keV.
These configurations aim to balance the trade-off between high resolution, low blur from scattering, and strong contrast from both phase effects and absorption.
Phase retrieval trades a little of effective resolution in exchange of more contrast. This seems to be beneficial for segmentation and unwrapping, but its effect on ink detection is yet to be explored.
Behind the scenes
The Vesuvius Challenge Team has been on fire, developing internal pipelines that let us handle the massive amount of data generated during recent scan sessions far more efficiently.
In total, we have scanned more than 30 new scrolls — and we needed tools to process and analyze them at scale.
Johannes built several Argo Workflows using Kubernetes and AWS to parallelize data handling and processing across hundreds of spot instances.
Forrest deployed multiple updates to VC3D, fixing stability issues and significantly improving its user interface. Check out the list of merged pull requests!
Anton is developing a tool to speed up iterative ink labeling in Webknossos. The goal is to make fine-tuning and running ink detection models accessible even to non-technical users, such as papyrologists. You can find a draft pull request here.
Stephen is working on a tool to make it easier to register volumes (scrolls) scanned with different pixel sizes and parameters, enabling more straightforward comparisons of analyses and results.
July progress prizes
This month, we are awarding a Papyrus ($1,000) prize to multi-award-winning community member Will Stevens.
Will has made major strides on his automatic “unwrapping” pipeline. In July, he developed more efficient ways to store his data — making it smaller and faster to handle — and improved the algorithms that stitch small flat patches into larger, continuous surfaces.
One of the toughest problems occurs when a sheet splits into two paths during reconstruction and later needs to rejoin without gaps or overlaps. Will is experimenting with a clever “ball and spring” model to gently realign those pieces, producing a smooth and accurate final surface. Please read his report here.
It would be exciting to see these advances tested in VC3D or compared directly with VC3D’s own surface tracer — both to benchmark performance and to explore hybrid methods that combine the strengths of each approach. Such integration could bring us even closer to fully unwrapping entire scrolls quickly and accurately.
sick volume registration tool, Stephen 🔥