Category Archives: projects

Visualizing Seasons

Several months ago, I made a rough tool to visualize flight seasons for dragonflies (or anything else). I finally cleaned it up this week and put the builder online. The end result is that it’s very easy to make something like this:

Screenshot of spreadsheet

Darkness indicates the number of years observed for each segment (so the light ones are seen occasionally and the darker ones all the time more or less). Originally, this was embedded here but since it’s slow to load, there’s a screenshot and a link. Here’s another example: birds in Middlesex county by half months.

To make your own:

  1. Create a csv file with your data. You need name and date, but other fields are ignored so exports from AviSys or eBird or something are good.
    Here’s an AviSys sample line:
    "Spatterdock Darner","Rhionaeschna","mutata","Prospect Hill","06/08/10",    1,"/m netted at base","MA","US","  ",  544  

    And an eBird one:

    S6645808,Common Chaffinch,Fringilla coelebs,30694,5,CH-ZH,,Switzerland,47.3681293,8.5372353,07/25/1997,,eBird - Casual Observation,0,,,1,, Zurich Zoo
  2. Go to the seasonality charter and run your file through it. There are instructions there and it should be fairly obvious other than counting the fields.
  3. I’m using Google Docs but Excel or other spreadsheets should be similar. You can either upload the file to the web somewhere or paste it into the document (or create the document around it, but that requires more work every time you update).
    1. If you upload to the web, create the spreadsheet. In cell A2 enter the formula =importdata(“url-to-file”)
    2. If you copy and paste, paste into A2. In B2, use =split(A2,”,”). Select from B2 down to the B(whatever the last row is) and hit Ctrl-D to fill down. Now right click on column A and select Hide Column (click the little right arrow to unhide when you want to paste new data in).
  4. Add the date headings.
  5. Now to format and turn the pile of numbers into a nice chart. Select the entire range, the pick Conditional formatting from the Format menu.
  6. Create a bunch of rules like the screenshot below matching the text and background color. I used a breakdown of 1, 2-3, 4-5, 6-7, and 7+ for five levels of shading but you can use whatever.
  7. Add one last rule for Is equal to 0 and select white for text and background.Conditional Formating
  8. You’re done! You can use the Publish to the Web option on the file menu to share or embed it like the samples above.

If anything isn’t clear, leave a comment or send an email. If you have large amounts of data, it’s probably better to ask me for the scripts and run them locally.

Flickr Plugin

I decided it’s easier to use flickr for hosting pictures with the new camera. Flickr has a requirement that all photos link back, so it would get ugly pretty quickly if I had to do the link and image tag every time (although I could just cut and paste their code).

Instead, I wrote a Blosxom plugin that takes the url to the picture itself and a description and builds the link. It was all of 20 lines, most of which was blosxom required empty methods. The actual linking consisted of one regex match, a small bit of parsing that and then a replace with the url and image link.

Unfortunately, flickr urls include underscores and the Markdown plugin tried to make those into emphasis. Instead of just renumbering/renaming, I added a Markdown fixer that does a search and replace for <em> tags in the src links and reverts them to underscores.

Roman Numeral Fun

In my waiting-for-people-to-leave-so-I-can-get-into-classrooms time at work I wrote a roman numeral translator in Factor. It’s a bit different from your normal implementation as Factor’s parser actually does almost all the work:

USING: strings parser kernel words sequences math ;

: NUMERAL: CREATE dup reset-generic dup t "parsing" set-word-prop parse-definition  parsed add define-compound ; parsing

NUMERAL: I 1 ;
NUMERAL: IV 4 ;
NUMERAL: V 5 ;
NUMERAL: IX 9 ;
NUMERAL: X 10 ;
NUMERAL: XL 40 ;
NUMERAL: L 50 ;
NUMERAL: XC 90 ;
NUMERAL: C 100 ;
NUMERAL: CD 400 ;
NUMERAL: D 500 ;
NUMERAL: CM 900 ;
NUMERAL: M 1000 ;

: separate ( str -- str )
    "" swap [ " " append ] [ add ] interleave ;

: join-special ( str str -- str )
    dup >r split1 [ 1 r> remove-nth swap 3append ] [ r> drop ] if* ;

: merge-specials ( str -- str )
    [ "I V" "I X" "X L" "X C" "C D" "C M" ] [ join-special ] each ;

: convert-numerals ( string -- arr )
    separate merge-specials parse ;

: all-numerals? ( str -- ? )
    [ "IVXLCDM" member? ] all? ;

: roman>number ( roman -- number )
    >upper dup all-numerals? [ convert-numerals sum ] [ drop "Not a roman numeral" ] if ;

Instead of grabbing characters and keeping a running tally, I defined a bunch of parsing words using NUMERAL: to hold the values. I then took the string and split it into individual characters (“XIV” becomes “X I V”). The 4’s and 9’s are then rejoined (so we get “X IV”). I then simply parse the string, which gives a list of numbers and sum that up. It’s not perfect as it allows any pattern of numerals (“IVIVIVIV” parsed to 22), but good enough.

Checklist

The code for generating the checklist is written in Factor. Factor is rather outside the mainstream, but it works beautifully for my purposes.

I store the checklist data in a text file that looks like the following:

CHECKLIST:
    FAMILY: Wrens
        SPECIES: Carolina Wren
            STATUS: Fairly common year-round. Can be found almost anywhere. There's too many around to not be breeding, but I have yet to find a nest.
            PB
        END

        SPECIES: House Wren
            STATUS: Regular in spring and summer. Possible almost anywhere. Young at Paine Estate imply breeding.
            CB
        END

        SPECIES: Winter Wren
            STATUS: Probably a rare visitor, possible at all seasons?
            RECORD: M. Emmons, 5/14/1997, ? , 2
            RECORD: R. Haaseth/D. Finch, 2/6/2004, near Prospect Hill, 1
            RECORD: R. Haaseth/D. Finch, 10/2005, ?, 1
            PERSONAL: 11/25/2006, Met State, 1
        END

        SPECIES: Marsh Wren
            STATUS: Likely a regular migrant and possibly a breeder at Waverly Oaks Marsh.
            RECORD: M. Rines, 4/20/2006, Waverly Oaks Marsh, 1
        END
    END

    HYPOTHETICAL: White-winged Crossbill | irruptive species
    HYPOTHETICAL: Hoary Redpoll | irruptive species

    HISTORICAL: Black Vulture | coming soon
    HISTORICAL: Boreal Chickadee | coming soon
    HISTORICAL: Louisiana Waterthrush | coming soon

This is a nice clean structure that is easy to parse. In fact, the code to parse it is "USE: checklistn" swap dup file-length swap <file-reader> stream-read append parse call ; Yep, the checklist is actually code that parses itself.

To do this, I defined a few tuples:

  • TUPLE: checklist confirmed hypotheticals historicals
  • TUPLE: family name species
  • TUPLE: species name status breeding records
  • TUPLE: record observer date place quantity
  • TUPLE: historical name details
  • TUPLE: hypothetical name reason

Each one simply contains the data and stores anything below it in a vector. I then defined a bunch of words that handle the parsing.

  • CHECKLIST: creates a new checklist
  • FAMILY: creates a new family and gives it the name of whatever is on the line
  • SPECIES: does the same for species
  • STATUS: stores the status
  • PB and CB store the breeding
  • RECORD: and PERSONAL: create and store records (PERSONAL: creates a record with observer “me”)
  • END is a generic word that adds the species or family to the family or checklist respectively (I’d like to define it for the checklist as well, but the stack effects don’t match so it doesn’t work)
  • HISTORICAL: and HYPOTHETICAL: do the obvious

For the most part, defining those was straightforward. The trickiest part was dealing with status, as I had to be able to call a word for the next object on the stack after reading the status in. I ended up with the ugly

: STATUS: rest-of-line swap ?push  swap swap ?push  set-species-status V{ } clone [ push ] keep >quotation swap ?push  keep swap ?push ; parsing

mess. I’m sure there’s a better way but that took a couple days to get and I haven’t fiddled with it since.

Getting from the checklist tuple to the output was pretty easy, just doing lots of formatting with make and working down.

Blosxom Articles

The following is a quick patch for Blosxom that lets me hide articles from the daily updates. I’m sure it would better as a plugin, but this was way faster.

    @@ -180,6 +182,7 @@
               # not an index, .file, and is readable
               and $2 ne 'index' and $2 !~ /^./ and (-r $File::Find::name)
             ) {
    +
                 # to show or not to show future entries
                 (
                   $show_future_entries
    @@ -326,6 +329,13 @@
         # Allow for the first encountered plugin::sort subroutine to override the
         # default built-in sort subroutine
         my $tmp; foreach my $plugin ( @plugins ) { $plugins{$plugin} > 0 and $plugin->can('sort') and defined($tmp = $plugin->sort()) and $sort = $tmp and last; }
    +    if(keys %f > 1) { # jason change
    +        foreach my $path_file( &$sort(%f, %others)) {
    +            if(-e "$path_file.article") {
    +                delete $f{$path_file};
    +            }
    +        }
    +    }

         foreach my $path_file ( &$sort(%f, %others) ) {
           last if $ne < = 0 && $date !~ /d/;