Regexes are Wow

I've just spent the last little bit playing around with adding more regexes, and I think I'm in love. It feels both easier to use than Perl 5 regular expressions and vastly more powerful. Here's the latest batch I just added:

regex abc_basenote { <[a..g]+[A..G]> }
regex abc_octave { \'+ | \,+ }
regex abc_accidental { '^' | '^^' | '_' | '__' | '=' }
regex abc_pitch { <abc_accidental>? <abc_basenote> <abc_octave>? }

regex abc_tie { '-' }
regex abc_note_length { [\d* ['/' \d*] ] | '/' }
regex abc_note { <abc_pitch> <abc_note_length>? <abc_tie>? }

The only difficulty I had here was not realizing at first that abc_basenote needed angle brackets around the character class specification. That caused me a few minutes of confusion, because I was trying to jump to (informally) testing more complex things built up around it, and so it wasn't immediately obvious that the base note regex was simply not working.

But hey, I know how that works now, and wow! The rest of it was dead easy and expressive to boot. And using .perl to dump the match structure is awesome.

There are two things to ponder here, however. First, having abc_ at the beginning of every regex is an obvious wart. I think I can get around that by putting these in a grammar? Must investigate.

The second is that at this point, I've got enough structure to what I'm doing that I really need a test suite. I suppose if I package the regexes in a grammar, then I can use it from another file, and test that? Must experiment.

Getting Started with ABC

So, in classic style, I've started off this project by just flailing around a bit. My first thought was to try TDD. But I very quickly realized I didn't have the first clue what the interface being tested should look like. So I dashed off the roughest idea for some classes. But I don't know how they are best filled. So I am reversing course, and will start by playing around with regexes a bit, trying to learn how the new facilities in Perl 6 work.

So my real starting point is pulling up The Book and reading up on regexes. No pretense I understand what's going on, let me just document what happens as I go along. BTW, this project is at github.

So, my first attempt looked like this:

use v6;

my $abc = q¬´X:64
T:Cuckold Come Out o' the Amrey
S:Northumbrian Minstrelsy
M:4/4
L:1/8
K:D
A/B/c/A/ +trill+c>d e>deg | GG +trill+B>c d/B/A/G/ B/c/d/B/ |
A/B/c/A/ c>d e>deg | dB/A/ gB +trill+A2 +trill+e2 ::
g>ecg ec e/f/g/e/ | d/c/B/A/ Gd BG B/c/d/B/ | 
g/f/e/d/ c/d/e/f/ gc e/f/g/e/ | dB/A/ gB +trill+A2 +trill+e2 :|»;

regex abc-header-field { ^^ \w ':' .* $$ }
regex abc-header { <abc-header-field>+ }

if $abc ~~ m/ <abc-header-field> /
{
    say $<abc-header-field>;

Note how I'm trying to get with the program and use dashes in my Perl 6 identifiers! Unfortunately, this doesn't actually work in this case:

Confused at line 15, near "abc-header"
in Main (file , line )

I played around a bit, and finally ended up changing the dashes to underscores. Changed that way it runs, but say $<abc_header_field>; prints out the entire string rather than just the first header field I had expected.

Aha! Apparently the .* $$ sequence in abc_header_field doesn't give you every up to the end of the line, as I'd expected. Instead it matches all characters (including newlines) up until it hits the end of the file, then backtracks until it finds the last end-of-line. Good to know. Changing it to \N* $$ works nicely.

Now I tried making it match abc_header instead of abc_header_field. No joy, we still just get the first header line. Why? Because we aren't matching the newlines between header lines! So tweak abc_header to account for the newlines, and bingo!

regex abc_header_field { ^^ \w ':' \N* $$ }
regex abc_header { [<abc_header_field> \n]+ }

if $abc ~~ m/ <abc_header> /
{
    for $<abc_header><abc_header_field> -> $line
    {
        say "header: $line";
    }

And here are the new results:

header: X:64
header: T:Cuckold Come Out o' the Amrey
header: S:Northumbrian Minstrelsy
header: M:4/4
header: L:1/8
header: K:D

Wow. After a couple of false starts, I've got some simple, powerful code there. I think I like these regexes...

Updated: Ugh, some weird formatting issues with the Perl 6 syntax highlighter. Too tired to mess around with now. Rest assured that there is a closing curly bracket in each example code.

Intro to ABC

So, the ABC format is a simple ASCII format designed primarily to make it easy to work with single line musical notation. It's great for, say, traditional dance tunes, like this tune from Northumberland.

X:64
T:Cuckold Come Out o' the Amrey
S:Northumbrian Minstrelsy
M:4/4
L:1/8
K:D
A/B/c/A/ +trill+c>d e>deg | GG +trill+B>c d/B/A/G/ B/c/d/B/ |
A/B/c/A/ c>d e>deg | dB/A/ gB +trill+A2 +trill+e2 ::
g>ecg ec e/f/g/e/ | d/c/B/A/ Gd BG B/c/d/B/ | 
g/f/e/d/ c/d/e/f/ gc e/f/g/e/ | dB/A/ gB +trill+A2 +trill+e2 :|

My favorite ABC program renders that like this (click on it for a high resolution version):


So, my goal here is to write a Perl 6 module that can read and understand this format, maybe do some light processing, and output it again. I'm hoping a grammar will make this pretty easy and straightforward, but I'm fairly ignorant of them so far. It will be an adventure!

I guess my first step is to set up a class to represent a tune, and then write some tests for it. Next time...

NUBS and Polynomials, with graphic

As has happened before, while putting the finishing touches on the code for this post, I suddenly realized I needed to rewrite the code before I could be happy with it. So this post will be devoid of Perl 6 code, and just discuss what is going on in the (now-improved) graphic.

The basic idea I'm illustrating here is that a NUBS curve is basically a convenient way of merging a series of partial polynomial curves. In this graphic, the NUBS curve is in black. The polynomials are in red, green, and blue.



If you start where the red doesn't overlap the black, and then follow it to the black and follow the black beyond that, you can trace the curve and get an idea of what is going on. Basically, the black NUBS curve follows one section of the red polynomial, then smoothly switches to a section of the green polynomial, and finally switches to a section of the blue polynomial. It is piecewise polynomial, in other words, with smooth transitions. (It is possible to generate unsmooth transitions too with a NUBS, or even out-and-out discontinuities, but generally this is not wanted.)

The other great thing about NUBS is how natural they are to specify, because their control points conform roughly to the shape of the curve, and modifying them changes the curve in a fairly natural fashion. I don't have time to go into great depth here, but here's a quick comparison. First, here's the code to specify the curve in the picture:

my @control_points = (Vector.new(-1, -2),
                      Vector.new(1, 0),
                      Vector.new(1, 1),
                      Vector.new(0, 1),
                      Vector.new(1, 2),
                      Vector.new(1, 2),
                      Vector.new(1, 2));
my @knots = (-1, -1, -1, -1, 1, 2, 2, 3, 3, 3, 3);
my Nubs $nubs = Nubs.new(3, KnotVector.new(@knots), @control_points);

Then here's the red polynomial:

(0.194444444444444, 0.111111111111111) x^3 
+ (-0.916666666666667, -0.666666666666667) x^2 
+ (0.583333333333333, 1.33333333333333) x^1 
+ (0.694444444444444, 0.111111111111111) x^0

Actually, that's a fairly clean polynomial in this case, but even so, it's hard to work with. The bit we're interested in is parameterized from -1 to 1, so for instance, to find the starting point we need to evaluate the polynomial at x = -1:

- (0.194444444444444, 0.111111111111111)
+ (-0.916666666666667, -0.666666666666667) 
- (0.583333333333333, 1.33333333333333) 
+ (0.694444444444444, 0.111111111111111)
= (-1, -2)

Whereas the starting point of the NUBS version is just the first control point.

Ack. I could go on, but I fear I've exhausted the patience of the Perl readers out there. And I feel like I'm constantly butting up against the limits of Rakudo with this project, so I think I'll put it aside for a few months and tackle something else. Maybe try to whip up a grammar for ABC files, or something like that. Then return to Vector when after Rakudo has had a chance to mature a bit more...

Ack in TextMate

For a while now I've known about ack, the slick grep substitute written in Perl. And I've known that the multi-file search function in TextMate was lousy, slow and memory-eating.

What I didn't know was that there was an Ack bundle for TextMate. It is a dream come true as far as I'm concerned. Okay, I've only briefly played with it, but it's screamingly fast and produces good-looking results. If there's a better add-on for TextMate, I don't know about it.

Good Thing Perl is Dead

PerlPilot comments that the "Perl is dead" meme annoys him. Personally, considering the apparent exponential growth in Perl-related Advent calendars this year, I think it's a good thing Perl is dead. If this is dead, a living Perl community would probably overload the Internet with Advent calendar traffic...

Wiktory!

Rakudo has been fixed, and the code I've been trying to get work for a month works beautifully now! If I understand the fix properly, the problem was in Rat addition's call to Rat.new. The code was very dumb, so it always tried set the denominator of the new Rat to the product of the denominators of the two numbers being added. Rakudo's Ints are really Int32 right now (more or less), so if that product was equal to or greater than 2**31, it was autoconverted to Num. But the Rat.new which takes positional arguments takes Ints, so it wouldn't dispatch to that. Instead it would try to dispatch to the default autogenerated named argument form of Rat.new. But that only takes the implicit self parameter, and we were sending it self, Int, Num -- thus the "too many positional arguments: 3 passed, 1 expected" error!

Rakudo now autoconverts this case (424/61731 + 832/61731) to Num to avoid the overflow in the denominator. Obviously this is less than ideal -- a denominator of 61731 would work fine for this sum -- but it does work. And how! Gone are the mysterious crashes and errors. With the number of samples cranked up, the curves look beautiful.

Now I just need to do a bit of polishing to the output and figure out how to post it to the blog. I'm definitely feeling I've accomplished something cool in Perl 6....

PS Aha! Preview graphic, I'll explain what it means next time.