Introduction
As part of a larger project within a London-based NHS hospital, we needed a tool to parse complex binary files containing hundreds of images and extensive metadata. The tool had to be able to save these images in more widely used formats at the highest quality possible. Furthermore, the tool had to have a mode mimicking the image post-processing of existing converters.
There were four main expected use cases:
- Integration into an existing Rails-based portal through AWS lambda. Within the lambda, the tool would be run on binary files uploaded to AWS S3 through the portal.
- Integration into the browser using WASM, so that the files are converted (and anonymized, if required) before uploading to the cloud.
- Integration with an Electron application, where the files can be converted and viewed on Windows computers.
- Usage on servers with tens of thousands of files as a part of research projects at the hospital.
As such, the requirements for the project can be summarized as follows:
- Possible to use as command-line tool as well as a library.
- Performant and compute-resource efficient. At the very least, it should be faster than our existing Python and JavaScript equivalents.
- Static linking in order to improve portability. As a comparison, the existing Python implementations required over 40 development libraries to be pre-installed.
- Support the following targets: Linux, Windows, and WASM.
- Work with local files as well as files in the cloud (AWS S3).
At the time, none of these were possible with Ruby (although it has received WASM support since then), so it brought us to a few alternatives:
It was a difficult choice, and even though Crystal didn’t have proper Windows support at the time, we decided to go with it because it offered greater productivity than Rust and was more familiar to Ruby developers (we have several in our company) than Go.
The path
Extracting images and metadata from binary files using Crystal turned out to be a relatively easy process. We did it with a mix of BinData and IO.
The first challenge we met is that in the binary files we are working with, images are stored in either the JPEG 2000 format or as raw data with dimensions attached (something akin to PPM).
We considered a few options:
ImageMagick is slow. As much as we wanted to use libvips, it wasn’t possible due to the license. Indeed, LGPL 2.1 requires applications to be licensed under the same license when statically linking. So, GraphicsMagick seemed to be a natural alternative.
We decided to get Linux support done first, so for now, we created a statically-linked version of
GraphicsMagick using StaticX, but the resulting binary
required a file named delegates.mgk to be present and had additional performance overhead. The
solution to both problems was to build GraphicsMagick manually on Alpine Linux (using Docker). It
worked well.
However, we realized that even this is not perfect because:
- Calling the CLI is error-prone, inconvenient, and has an additional performance overhead. This is because we must first save temporary images on disk, then pass them to the CLI, which will read and save them again, and then delete the old files.
- Creating bindings is difficult due to the heavy use of macros.
- JPEG 2000 to JPEG conversions are slow.
For JPEG 2000, we contacted the author of Grok, an open-source JPEG 2000 codec, to buy a commercial license. He helped us a lot by providing support and adding features that we needed (like WASM compilation, extensive examples, etc.). The Crystal bindings we created, called grok.cr, are open source.
For everything else, as an experimental solution, we decided to try making our own image-processing library in Crystal. We went with adding support for PPM (probably the most simple image format available), then implemented algorithms for resizing, blurring, and other image processing operations. After that, we added support for JPEG, using libjpeg-turbo for the best performance.
This is how Pluto, an open-source image processing library in Crystal, was born. We hope it to be suitable for a wide range of users, especially when using complicated C libraries is undesirable. PRs and suggestions are welcome!
Benefits
Overall, Crystal is a very respectable language with numerous benefits:
- Ability to experiment and prototype quickly
- Awesome documentation, including the reference and the API
- Awesome performance for a managed language
- Community that (including the core developers themselves) answered all asked questions
- Standard library that is complete, well-built, and allows for minimum external dependencies
- And more
Measurements with an exemplar file (39.0 MB):
The developed project can potentially benefit thousands of users. We enjoyed Crystal and hope that you will too!