St Hallett Old Block Shiraz

Three of us visited Craig's place last night. It was supposed to be night for board games, but since it had been a fair while since the last one, we had a bit of chat to get through. And the tour of his house. The tour ended with a visit to the wine fridge. Then we had to pore over the wine list and choose something.

Craig was fairly confident that the 1991 St Hallett Old Block shiraz (acquired at auction 6 years ago) would be past its best. That was just the excuse I needed to encourage him to choose it - no point waiting longer!

The cork was very soft, and less than half came out with the corkscrew. After some digging, the only option was to push the cork in, and to sieve and decant. The bottle was fairly free of lees, the wine was on the brickish side of red, and the scent was a cause for optimism. After decanting, we left it for about 30 mins.

The first tastes revealed a rather solid depth with central smoothness, although there were still some aromatics and a hint of acetic acid. But after it had warmed slightly in the glass, and perhaps with slightly recalibrated palates thanks to the Mersey Valley cheese, the tannic complexities began to reveal themselves, and worries about the wine being too old seemed to evaporate. No unripe peppery flavours here; it was just plums with a hint of barnyard. The only things I could have asked for more of would have been some touches of tobacco and a bit more length.

We concluded in the end that the wine may well have been better a few years ago, but it was definitely still on the delicious side of good last night.

Thanks Craig!

Binary Polictical Prisoners

I get jealous when I see that other governments are making all their data freely available. I can understand why, in the old economy, it could make sense to have user-pays access to all that data. But these days, it's so much less appropriate to keep that data locked up - political prisoner. I've written to my state Member of Parliament and the relevant minister to ask LNP and Labor positions.

Dear ...
I believe that as much state government information as possible (such as property sales data) should be free. While the government raises some revenue from selling access to such information, I think that making such information publicly available would reduce the bureaucracy.  Most importantly though, it would create an environment that would allow people to create useful innovative services for the public. There are precedents:

• In 2005 Peter Costello made all the Australian Bureau of Statistics information freely available. http://www.abs.gov.au/websitedbs/d3310114.nsf/4a256353001af3ed4b2562bb00121564/83b66e9ffafd6dabca257140007575b5!OpenDocument
• The UK has recently decided to take this path on an even bigger scale. http://www.cabinetoffice.gov.uk/news/public-data-corporation-free-public-data-and-drive-innovation

Does (your party) support such a position?

Best regards,

John.

If I get any answers, I'll put them here.

Monad tutorials

I studied some computer science when I was at uni. It was pretty interesting. Perhaps I didn't do enough, though. I didn't learn about compilers, nor much about functional languages. I've been redressing both of these deficits a little, recently. I've been reacquainting myself with the Antlr compiler compiler (20 years ago it was yacc and lex!), and looking at monads - not because I have an urgent need, or because I've had a Haskell conversion, but because it seems like an interesting intellectual challenge. I've even started reading SICP.

I'm not tempted to tell you that a Monad is like a Burrito, nor to write my own monad tutorial. I can't say I've put in the effort to write a historical analysis of a whole bunch of them. Instead, all I can do is tell you that the three best descriptions I've found are:

• Brian Beckman's video
• Mike Vanier's tutorial
• Jeff Newbern's guide

Using Apache httpclient through an NTLM authenticating proxy with ftp

I needed to (programmatically) retrieve a file from an FTP server out in the internet. In this example, the URL is ftp://site.com/dir/file.txt. My computer can only access the Internet through proxies. There is an HTTP proxy called web-proxy.local, and an FTP proxy called ftp-proxy.local.

I noticed that I could retrieve the file using my browser, but not using command-line ftp. I determined that the ftp-proxy was slightly mis-configured, and didn't believe that my host was a legitimate user. But how did the browser fetch the file, using that URL above? A little work with wireshark showed that the browser makes an HTTP connection to the proxy, and passes the HTTP command:

GET ftp://site.com/dir/file.txt HTTP/1.1

When I tried to use the java.net.URLConnection with this URL, it wouldn't connect to the web-proxy. That seemed reasonable - it was probably trying to connect to it with FTP. but somehow I needed to create an HTTP connection to a URL starting with FTP.

I decided to try Apache HttpComponents HttpClient - Apache code is always great. After a little difficulty getting the right version (eventually 4.1), I found that I was getting

java.lang.IllegalStateException: Scheme 'ftp' not registered.
at org.apache.http.conn.scheme.SchemeRegistry.getScheme(SchemeRegistry.java:71)
at org.apache.http.impl.conn.DefaultHttpRoutePlanner.determineRoute(DefaultHttpRoutePlanner.java:111)
at org.apache.http.impl.client.DefaultRequestDirector.determineRoute(DefaultRequestDirector.java:710)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:356)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
...

I peered into the source and javadoc of the DefaultHttpRoutePlanner and SchemeRegistry, and discovered that I could add the 'ftp' scheme like this:

HttpClient hc = new DefaultHttpClient()

// Register a scheme so that we can ask the proxy to use ftp
Scheme ftp = new Scheme("ftp", 80, new PlainSocketFactory())
hc.getConnectionManager().getSchemeRegistry().register(ftp)

In this case, I don't think it matters what port number (80) I use, or even which type of socket factory, since the connection will be sent via the proxy anyway - the system doesn't really need to know how to create an ftp socket, or which port to use.

The next issue I had was making it work with the proxy. I had copied the example HttpClient code for using authenticating proxies, but it didn't work. Again, wireshark helped. When the browser fetched the file, I could see the 3-phase NTLM negotiation. But not when my software ran. A spot of googling showed me that instead of using the UsernamePasswordCredentials, it would be better to use NTCredentials. And now it works. The final code looks like this:

HttpClient hc = new DefaultHttpClient()

// Register a scheme so that we can ask the proxy to use ftp
Scheme ftp = new Scheme("ftp", 80, new PlainSocketFactory())
hc.getConnectionManager().getSchemeRegistry().register(ftp)

// Set up NT(LM)Credentials for use with the proxy.
hc.getCredentialsProvider().setCredentials(AuthScope.ANY, new NTCredentials("myUsername", "myPassword", "", ""));

// Set up the proxy
HttpHost proxy = new HttpHost("web-proxy.local", 8080);
hc.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy)

// Set up the URL to fetch
HttpGet hg = new HttpGet(ftp://site.com/dir/file.txt)
HttpResponse hr = hc.execute(hg)

HttpEntity entity = hr.getEntity()
InputStream instream = entity.getContent()
...

Grails proxy settings

I always forget these.

1. When setting the details from the (Windows) command line, use quotes everywhere:
> grails add-proxy someProxyName "--host=the.host.name" "--port=8080" "--username=myUserName" "--password=myPassword"

2. Don't forget to set-proxy:
> grails set-proxy someProxyName

3. The ProxySettings.groovy file is stored in the top level of .grails - not in the 1.3.5 subdirectory etc.

Online book shopping

For some professional development, and to see what Uncle Bob is on about, I decided to buy Structure and Interpretation of Computer Programs (SICP).

I was impressed that QBD Online had a price (including delivery) of $53.95. This was comparable to Amazon ($53.18), and Book Depository ($57.54). (It's available for free online, but I think I'd rather pay for a hard copy.) I ordered through QBD. Eleven days later (admittedly some were public holidays), they informed me that they were "temporarily out of stock", and the publisher couldn't advise when more stock would be available. I wasn't happy, and asked them to cancel the order. I was pleasantly surprised to see that they responded very quickly (cancel requested 9pm, response at 6am). It means that I'm more likely to go back to them in future.

It was simple to order the book at Amazon. They said that the book was in stock. Twelve hours later I have an email saying that the book has been shipped. Nice contrast.

Setting up Scholastic "I Spy Fantasy" for non-administrative users

My children have non-administrator accounts on their computers, one of which uses Windows XP. Maybe that doesn't encourage them to learn, but that's how it is at the moment. They wanted to install (again) I Spy Fantasy, an old game, but it required administrative access. That was ok. But then we found that administrative access was required not just to install, but also to play the game. That's not ok.

A short web search didn't yield any answers, so I delved. The error message reported that it couldn't save game details to the disk. Probably because it stores the saved games in C:\Program Files\Scholastic...

I tried to move the saved games folder, and make a shortcut, but that seemed fruitless. So I moved the whole installation folder from Program Files to daughter's Documents and Settings folder. But when running the application, it still complained about the same problem. I wondered if the registry was pointing to the old location. Sure enough - Local Computer/Software/Scholastic inc/I Spy Fantasy/install-dir pointed to Program Files... . So I changed that, switched users back to the non-administrative account, and tried again. This time, the error message was new: the program was trying to modify the registry! Well, it looks like in regedit, you can set permissions for different entries. So I gave the non-privileged user full control for just the I Spy Fantasy folder. That's better.

Oops. One last problem. Don't forget to adjust where the start menu shortcut points to!

That seems to have solved the problem. I wonder if anyone else cares about such things. Maybe you'll leave me  a comment if this helps.

Prime factors coding kata using transformations.

Inspired by Uncle Bob's Transformation Priority Premise, Glennn and I tried a Groovy kata.

  @Test
  void shouldReturnPrimeFactors(){
    assertThat getPrimeFactors(1), equalTo([])
    assertThat getPrimeFactors(2), equalTo([2])
    assertThat getPrimeFactors(3), equalTo([3])
    assertThat getPrimeFactors(4), equalTo([2,2])
    assertThat getPrimeFactors(5), equalTo([5])
    assertThat getPrimeFactors(6), equalTo([2,3])
    assertThat getPrimeFactors(7), equalTo([7])
    assertThat getPrimeFactors(8), equalTo([2,2,2])
    assertThat getPrimeFactors(9), equalTo([3,3])
    assertThat getPrimeFactors(10), equalTo([2,5])
    assertThat getPrimeFactors(11), equalTo([11])
    assertThat getPrimeFactors(12), equalTo([2,2,3])
    assertThat getPrimeFactors(13), equalTo([13])
    assertThat getPrimeFactors(14), equalTo([2,7])
    assertThat getPrimeFactors(15), equalTo([3,5])
    assertThat getPrimeFactors(16), equalTo([2,2,2,2])
    assertThat getPrimeFactors(17), equalTo([17])
    assertThat getPrimeFactors(18), equalTo([2,3,3])

    assertThat getPrimeFactors(25), equalTo([5,5])

    assertThat getPrimeFactors(64), equalTo([2,2,2,2,2,2])

    assertThat getPrimeFactors(74), equalTo([2,37])
}


We didn't record everything as we went. It would have been good to look back at our mistakes. But here's the steps I took when re-doing it later.

Step 1. "{} -> nil" transformation. Suceeds for 1, fails at 2.
  static def getPrimeFactors(int number) {
    return []
  }


Step 2. "unconditional -> if" transformation. Succeeds up to 3, fails at 4.
  static def getPrimeFactors(int number) {
    if (number == 1) return []
    return [number]
  }


Step 3. "unconditional -> if" transformation and "statement-> recursion" transformation. Fails at 9.
  static def getPrimeFactors(int number) {
    if (number == 1) return []
    if (number%2 == 0) {
      return [2]+getPrimeFactors(number.intdiv(2))
    }
    return [number]
  }


Step 4. "unconditional -> if" transformation. Fails at 25.
  static def getPrimeFactors(int number) {
    if (number == 1) return []
    if (number%2 == 0) {
      return [2]+getPrimeFactors(number.intdiv(2))
    }
    if (number%3 == 0) {
      return [3]+getPrimeFactors(number.intdiv(3))
    }
    return [number]
  }


Step 5. Refactor to remove duplication. No change to behaviour.
  static def getPrimeFactors(int number) {
    if (number == 1) return []
    for (int divisor=2; divisor<=3; divisor++) {
      if (number % divisor == 0) {
        return [divisor] + getPrimeFactors(number.intdiv(divisor))
      }
    }
    return [number]
  }


Step 6. "constant -> scalar" transformation. Success for all cases.
  static def getPrimeFactors(int number) {
    if (number == 1) return []
    for (int divisor=2; divisor*divisor<=number; divisor++) {
      if (number % divisor == 0) {
        return [divisor] + getPrimeFactors(number.intdiv(divisor))
      }
    }
    return [number]
  }

One slightly undesirable characteristic is that this method attempts to use composite divisors, although these can never succeed. Some may consider it better to construct a list of primes as potential divisors. I'll look into this another time, but I suspect that the additional work to construct the list of primes will add complexity, and not improve performance too much (unless the primes are calculated just once, and there are many calls to getPrimeFactors).