songflower

reflow bitmap sheet music to a different paper format
git clone https://a3nm.net/git/songflower/
Log | Files | Refs | README | LICENSE

commit 4345fbad5ae882bb6f5fd644ef8c05c7eb0b20d8
Author: Antoine Amarilli <a3nm@a3nm.net>
Date:   Thu,  5 Sep 2019 23:45:02 +0200

start documenting

Diffstat:
LICENSE | 19+++++++++++++++++++
README | 192+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 211 insertions(+), 0 deletions(-)

diff --git a/LICENSE b/LICENSE @@ -0,0 +1,19 @@ +Permission is hereby granted, free of charge, to any person obtaining a +copy of this software and associated documentation files (the +"Software"), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice shall be included +in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS +OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY +CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, +TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + diff --git a/README b/README @@ -0,0 +1,192 @@ +Songflow is a collection of scripts to reflow sheet music (e.g., from a PDF) to +a new page size, e.g., to fit it on a mobile phone or tablet. It does not +require additional information about the music, and works with bitmap +renderings. + +In particular, I have used Songflow to reformat the public-domain sight-reading +course "Melodia" +<https://ia800203.us.archive.org/17/items/cu31924021781434/cu31924021781434.pdf> +to a more convenient format. + +== 1. What it does == + +Songflow does the following: + +- Splitting a PDF file into multiple pages + +- Splitting pages into systems, separated by sufficient consecutive lines of + white or near-white space. For this to work, your file must have sufficient + contrast, and must not be skewed (the separation between systems should be + horizontal) + + The systems are also trimmed of near-white content at the left and right + +- Splitting systems into blocks of measures of the right width, and resizing + them to the desired width. This is the most fragile step. + + Empty spaces in the system are detected as having near-minimal height of + non-white content, and near-minimal total weight. Bars are identified as two + consecutive empty spaces that are sufficiently close and such that the space + between them has a significiantly higher density of non-white content. This + step requires the bars to be sufficiently vertical and the scan to be + sufficiently crisp. It works more reliably on systems where measure bars take + the whole system. For this step to work, you will probably need to adjust + threshold and distances. The program may fail by refusing to split (more + accurately doing aggressive splits at random points), or may misdetect some + patterns as bars (especially the stems of half-notes). The program will also + cut at bars that break a slur. + + Once bars are detected, the program splits them in blocks of the right width + (by a bruteforce algorithm minimizing the length of the shortest segment), and + each block is stretched to the required width by stretching empty space only + to avoid distorting the picture. This step may fail by stretching things + (e.g., notes) that should not be stretched, or by failing to detect some empty + space and stretching too much the places that it detects (especially when + constrained because not all bars were correctly detected). + +- Combining the blocks back into pages of the right size (greedily fitting them + and arranging them on the page) + +== 2. What it requires == + +You need imagemagick, Python 3, and some Python libraries (numpy, imageio). + +== 3. How to use it == + +A script, master.sh, is provided to automate all of the conversion. Basic usage +would be: + + ./master.sh INFILE.pdf WIDTH HEIGHT OUTFILE.pdf + +The process can take several hours for large PDF files (e.g., for Melodia). + +However, it is likely that you will need to peer into the internals, so read on. + +== 4. The scripts == + +The interesting scripts are: + +=== 4.1. Splitting pages into systems === + +splith.py splits pages into systems. The way to use it is: + + ./splith.py file.png out/ + +It will write files out/file_0001.png, out/file_0002.png, etc., one for each +system, covering disjoint regions of the page. + +- The parameter --whitethreshold indicates the sensitivity to consider things + as white space (this is a grayscale value, i.e., between 0 and 255). Setting + it to a higher value will make the program cut more agressively. + +- The parameter --maxheight can be used to ensure that the extracted "systems" + will not be higher than the specified height (in pixels). This can be useful + in case you want to be sure the extraction won't fail later (e.g., combine.py + will ignore stuff which is too high to fit on the requested page height). + +- The parameter --mincontentheight (in pixels) indicates the height of content + that is "too small to matter". If some small junk on the page gets extracted + as a system, or decorations or lyrics are attached to the wrong system, try + increasing this parameter. + +- The parameter --minheight indicates the minimal height of an extracted system + (in pixels). If small junk gets extracted as a system, you can increase this + +- The parameter --distthreshold (in pixels) can be lowered to stop cutting when + the empty space between two "systems" is too low compared to the largest empty + space. You can lower this parameter if the program cuts too agressively, but + it may then fail to cut, e.g., on pages with large space because of a title + +=== 4.2. Splitting lines into chunks == + +splitw.py splits lines into chunks. The way to use it is: + + ./splitw.py file.png out/ WIDTH + +It will write files out/file_0001.png, out/file_0002.png, etc., one for each +system, having exactly the requested width WIDTH. + +Parameters to detect the height: + +- The parameter --whitethreshold (grayscale value between 0 and 255) indicates + what counts as "white" when measuring the height of a line + +- The parameter --minlength controls the minimum consecutive number of pixels at + which something can be "low-height" (i.e., the minimal bar height) + +- The parameter --margin indicates the number of pixels to be excluded at the + left and right of the screen when finding the minimal height + +- The parameter --outlierquantile in a percentage indicating the proportion of + minimal height values to discard (consider as outliers) + +- The parameter --heightthreshold indicates the tolerance (in pixels) up to + which something is considered "low-height" + +Parameters to detect the weight: + +- The parameter --outlierquantile mentioned above is also used to eliminate + outlier weights + +- The parameter --weightthreshold (grayscale value between 0 and 255) indicates + the weight threshold up to which something is still considered as minimal + weight. + +- The parameter --weightwindow (in pixels) indicates the width of the window + over which weight is computed (for smoothing) + +Parameters to detect bars: + +- The parameter --maxbardistance (in pixels) indicates the maximal distance + between two low-height, low-weight parts of the staff on each end of a bar + +- The parameter --minbarweight (float) indicates how much more weight a bar + should have relative to the minimal weight + +- The parameter --minchunk (in pixels) indicates, where we cannot find a bar + where to cut, what is the smallest admissible width for doing a cut at an + "empty point" + +Parameters to debug: + +- With --debug, the program will write out/debug.png with the input file colored + to indicate low-height and low-weight parts as well as detected and added bars + and the bars chosen to cut + +== 4.3 Combining chunk into pages == + +combine.py combines chunks into a page. The way to use it is: + + ./combine.py infolder/ outfolder/ HEIGHT + +All files in infolder/ should have the same width, and they will be considered +in alphabetical order. They will be grouped in files in outfolder/out_0001.png, +etc., having the prescribed HEIGHT and the common width. Input files whose +height is too large to fit will be ignored with a warning. + +- The parameter --hmargin indicates the space in pixels left at the left and at + the right of the produced images + +- The parameter --vmargin indicates the space in pixels left at the top and at + the bottom of the produced images + +- The parameter --separator indicates the minimal vertical space in pixels + between two chunks + +== 5. Limitations == + +- Songflow will rasterize vector PDFs. On raster PDFs, you need to specify by + hand a new rasterization density which will re-scale the content during the + export. It would be conceivable to change splith.py to extract regions + vectorially, but splitv.py which stretches the partition in a complicated way + it would be trickier. + +- When splitting pages into systems, sometimes lyrics are lost or not put at the + right place + +- Splitting lines is pretty error-prone, with some inadequate cuts (not at bars) + and some weird stretching. The accuracy could be improved by better improved, + or by genuine learning techniques instead of crude heuristiques. + +- If you call Songflow on something that is not music, it will not notice and + will happily botch the content instead of leaving it alone.