Morton Fox's journal

Yes, it's already the second week of the new year but I was busy the first week and 2011 was such a busy year that it was hard to distill the important things from the year. The biggest change is after 16 months being out of work, I returned to work for a small startup (literally a 6-person office when I joined) in Wilmington. My previous job wasn't at a big firm but this one is microscopic in comparison and it's a lot different. There is no bureaucracy, no office politics, no cubicles, and very little formality. I was given full access and was trusted to do the right thing from day one. I learnt a lot as I went along and within a week, I had started contributing important code to the system. Best of all, I was never left with the feeling that anything I did was futile or a waste. Any software I wrote was put to good use pretty much right away and I deployed it myself most of the time. So all in all, it's been a promising start. Then again, I thought the same thing back when my previous job began, so let's check back on this one in 14 years and see if it has withstood the test of time. :)

Of course, with the new job, I had less time for geocaching but I think I did as much of that as I wanted to on weekends and after work in the summer. I ended the year with more geocache finds than year 2009 but not 2010. The year was dominated by Lancaster with two big geocache series, the battleship caches and the rail trail caches, although Delaware was no slouch either with the DGT caches, the C&D power trail, and the First State Challenge. I tried to get out to more geocaching events as well, especially the ones in Bucks and Montgomery County that would've been a 2-hour drive from my old home in North Jersey. Now that these are only an hour from home, it hardly feels like an arduous journey getting there. I also started going to a few Where's George events locally and in Maryland. And of course, I went to local furmeets too whenever I could and attended 3 furry conventions, FA: United, Anthrocon and Furfright. The great thing about this area is there's an event or gathering going on pretty much every weekend. On the other hand, the downside is I haven't had as many mascot gigs now that I'm distant from Hi-4. I'm also losing touch with NYC-area folks but that is to be expected.

I kept up the pace on personal projects as usual. In its 4th year, TwitVim had another round of incremental improvements, although I thought the multiple login feature was big enough to bump the version number. My Foursquare webapp, 4sqNoGPS, became PlainSquare when I rewrote it for the Foursquare V2 API. I also changed the CSS so it looks more like a mobile app on small screens while keeping the desktop interface simple. I've found this setup to be good enough for most day-to-day use and really only hit the official app to upload photos and claim specials. Remember the beta Google Chromebook I received in December 2010? Well, I used it a lot more throughout 2011 and that led to my participation in the Cr-48 binaries project. I probably wouldn't use the Cr-48 for serious development work but now that we have a text editor, a few scripting languages, and some dev tools ported to it, light work is possible. And finally, I did get around to scrapping my old website and reworking it into a Drupal-based site. This doesn't mean that I'll update it any more frequently but when I do, it'll be less of a pain. :)

The stock Chrome OS installation on a Google Chrome notebook doesn't make it easy to install additional software. So all this while, in order to use TwitVim, I'd been opening an ssh session to my webhost, Joyent, and running Vim and TwitVim in the shell account there. This is an adequate solution to get up and running with TwitVim quickly but it has a few problems. The worst problem is ssh will disconnect if the network connection is bad or if you leave it idle for too long. Also, the ssh session is not preserved if I close the lid of the Cr-48 (to put it in sleep mode) because the network connection goes away then too. Those problems are lessened somewhat if I run Vim and TwitVim in a screen virtual terminal, but still I thought there had to be a better way.

Recently, I came across a blog post about installing Nano on the Cr-48. The basic idea is the stock Cr-48 setup can use binary packages from Arch Linux. That completely eliminates the need to set up a Chrome OS development system and build the software yourself. I figured if he can do that with Nano, then I can install Vim the same way and after that, running TwitVim would be a snap.

Here are the steps I used:

( Read more... )

Busy morning. Seems like all the good job leads came in at once and all shortly before Anthrocon. What I was really looking for is somewhere to use my Perl (or Python, or Ruby, if it came to that) scripting skills. I'd been doing the C and C++ gig for over 14 years and I would prefer not to go back to that if possible. I came across a few openings that fit the bill, including one that I was told allows telecommuting. So I spent all morning applying and filling out questionnaires.

In the afternoon, I went to Harbor Freight to return a faulty cell phone holder that ripped apart after only a few days. Harbor Freight has cheap stuff but quality is suspect in some of their wares. Fortunately, they have a good return policy.

Started to pack for Anthrocon in the late afternoon. My packing list is more or less the same from convention to convention, so I was done in a bit over an hour. The only last-minute items are things that I'm still using, like this netbook and toiletries. Also, there is the possibility of bringing more than two fursuits this time if I pack a bit better. Not going to bring the husky suit this time because the tail is about ready to fall off, but there was one that didn't make the cut last time that may be coming along this time.

Since the packing went so quickly, I worked on some TwitVim features this evening. I'd been holding off on adding user and list management features until after the OAuth work was done but there's no reason to delay any further. So I added commands for following, unfollowing, blocking, unblocking, reporting spam, and adding/removing from lists. Also improved the error messages for many commands so we won't have to diagnose problems by HTTP status codes any more. (r117 thru r124)

A few days ago, I wrote a proof-of-concept Twitter OAuth login and posting script in Perl. Today, with no interruptions from anything or anyone (even the mail carrier was extra quiet), I added the code to TwitVim. (revisions 88 to 92) The only part of the OAuth handshake and signing code that I couldn't yet write in Vim script is the HMAC-SHA1 message digest code. I'm using Perl to do that but I'll look for other ways.

The impression I got while working with Twitter OAuth is the API itself is an unfinished work with some rough edges and inconsistencies. For example, the documentation for statuses/home_timeline states that this call supports only the GET request method. This is true for basic authentication but once you switch to OAuth, it requires the POST method if you add any optional parameters. statuses/user_timeline, on the other hand, disallows POST even if you're using the optional parameters. I suspect the documentation is out of date and statuses/user_timeline will eventually be modified to be consistent with the other timeline calls but with Twitter, it's anyone's guess.

Twitter's OAuth implementation itself is not fully documented. I figured out a few things about what it does and does not allow by experimenting with my demo Perl script. I tried not to make too big a change at a time because it's tough to debug. Any error simply results in a "401 Unauthorized". It doesn't matter if you misspelled the signature method, if your nonce is not unique enough, or if your timestamp is not fresh enough. You'll get the same error code. Also, Twitter can and will throw a fail whale in the middle of an OAuth handshake, which is a consideration if your app is expected to handle this problem gracefully.

So what's left to do? TwitVim needs to save the access token after doing an OAuth handshake so it won't have to re-authenticate every time it starts up. Once it does that, it also needs to offer a way for the user to switch to another login. And of course, I need to document all of this so the user will know how to set it up.

Bram recently added a bunch of if_ruby (Ruby Interface) patches to Vim to add support for Ruby 1.9, improve overall support for Ruby, and fix a few problems. Of course, two of those patches affected TwitVim and I'll discuss them here. First, the good stuff: Patch 7.2.360 finally fixes the if_ruby Windows sockets problem. I used to recommend this unofficial patch but that won't be needed any more if the user has patch 7.2.360 or (the upcoming) Vim 7.3 or later.

Patch 7.2.374 allows VIM::evaluate() to return Ruby arrays or hashes instead of strings if the Vim expression results in a list or dictionary. This, unfortunately, did break TwitVim because TwitVim does the following:

    keys = VIM.evaluate('keys(a:parms)')
    keys = keys.split(/\n/)

If the parms dictionary has keys "user", "text", and "status", prior to patch 7.2.374, VIM.evaluate() will return "user\ntext\nstatus" and the next line will split it into the array ["user", "text", "status"]. After patch 7.2.374, VIM.evaluate() returns ["user", "text", "status"], causing the next line to hit a Ruby error because a Ruby array doesn't have a split() method.

How do we solve this problem? At first, I was thinking of checking for this Vim patch using a conditional like this:

    v:version > 702 || v:version == 702 && has("patch374")

In other words, Vim 7.3 and later are guaranteed to have patch 7.2.374 and we only check for the patch itself if the Vim version is 7.2. If I stick this expression into VIM.evaluate(), it'll return "1" if that patch is present and "0" otherwise. Then it occurred to me that there are two better ways to deal with this:

1. Make the Vim expression return a string regardless by doing an explicit join on the array:

    keys = VIM.evaluate('join(keys(a:parms), "\n")')
    keys = keys.split(/\n/)

2. Use Ruby's type introspection to see if a split is needed:

    keys = VIM.evaluate('keys(a:parms)')
    keys = keys.split(/\n/) if keys.is_a? String

I prefer the second approach because it is more efficient and a few years into the future, once everyone has upgraded their Vim installations past this patch, we can clean up the code easily by removing the second line. Thus, this fix is in TwitVim revision 82.

There was a snowstorm forecast for Friday evening and Saturday. The snow didn't start until after dusk here but I stayed home anyway to do a couple of things. First, I worked on photos from the last few days. Then I did a bit more of my tax return, which I started a few days ago. I won't be able to finish it until mid-March because some of the forms typically don't get to me until early March but I'll do as much as I can now. Also worked on a new TwitVim feature, which was suggested by a user.

( Friday Five )

Well, I didn't get the job for which I went to the on-site interview on Tuesday. The verdict was I had good skills and knowledge but just not specifically what their project was looking for. It could be a blessing. I had some concerns about the software firm that I didn't share on that day but now that it's over, I guess I'm clear to write a little more. The first was the salary discussion, which I was surprised actually happened during the interview. I thought it was supposed to be done only after they make a hiring decision. They made a few attempts to bargain me down, saying, for example, that even PhDs at that company only made $95K or so and that some people started really low (below $60K) and got raises afterwards. So I was wondering if they even had the budget for an additional software engineer. The second thing I noticed was even though the company doesn't force anyone to work long hours (i.e. significantly beyond the usual 9 to 5), it looks bad if you don't. Again, it was somewhat odd that they felt the need to impress that on me during the interview. Overall though, the place was eerily similar to my old job. Same suburban-type office building. I think they even have the same office furniture, carpet, and cubicle suppliers. I would've been right at home arguing with management there. :)

Looking back at the last 14 years of my life, I came up with an analogy. A job is like an apartment. On day one, it's a blank slate. Full of promise and ready to be filled with your presence. Then after a while, the plaster starts peeling, the faucet starts leaking, the toilet becomes unreliable, and dealing with building maintenance becomes somewhat unpleasant. Yet, you stick on year after year because staying put is less of a pain than leaving, until you wake up one day and find that you absolutely loathe the place and would do anything to get out. Yes, the apartment was getting like that too. :) Which reminds me, I still need to ask the landlord at the end of the 21-day period if he'll be returning any of the security deposit.

Rainy day today so I stayed home and worked on code. I added support to TwitVim for the new Retweet API. (r68 thru r71) It's not complete yet because of a few things I noticed in the API documentation. First: retweets_of_me does not appear to return info on who did the retweeting. There'll be a significant cost in time and API quota if the client makes a separate query for that info for each retweet. And second: Retweets are stripped out of user_timeline and unlike friends_timeline/home_timeline, there is no alternative API method that returns the full timeline with retweets included. I checked the latest version of Tweetdeck and, even though it does support new-style retweets, it's missing retweets when you bring up a user's profile, confirming that issue. No matter. I just made do with what the API provides and I hope there'll eventually be API revisions or workarounds to address those two issues.

Over the last two weeks, I've done a lot of work (r35 thru r50) on TwitVim to get it from version 0.3.5 to 0.4, so I think it would be good to stop for a while and explain some of the enhancements I've put in. This article will be the 0.4 release notes when it comes out. Of course, since I know of only about a dozen or so users of this Twitter client, almost everyone can safely skip this post. :)

( Read more... )

36°F this afternoon. Snowing/raining. Got a little tired of dealing with insurance and police all morning, so I went out to hit two geocaches at Garret Mountain Reservation in West Paterson. It actually stopped snowing for the duration of my trip out there, although it was still quite windy. What's in a name? was the first of the two. It was by a rocky cliff with a view of Paterson. While climbing towards the cache site, I slipped on some leaves and landed on the rock. Already, that's more injury than I got in the auto accident. :) In order to retrieve the cache, I had to walk down a rock crevice so it's a good thing I'm not too fat. (The cache owner himself almost didn't fit.) The second cache, Can you see me now? was a lot easier. It was just a quick grab off a residential road at the edge of the park.

Back to insurance matters: I called Encompass Insurance this morning. I had trouble reaching the insurance agent handling this claim but eventually she did call me. It sounds like they'll pay for the repair since their insured was at fault. My next step is to get an estimate from an auto body shop to replace the bumper and whatever else they find damaged. I also called River Edge PD but no one picked up the phone in the records department. It's possible that the records clerk had the day off. However, I don't expect the police report to be ready so soon anyway.

Did some work on TwitVim. In r36, I added a command to send a DM via the Twitter API. Previously, the only way you could do that in TwitVim was by using the "d username ..." special form. I added this enhancement mostly for identi.ca because their server did not support "d username ...". However, I found out today that they have recently added the "d username ..." functionality. Oh well. It'll still be useful for Laconica servers that haven't been upgraded.

This afternoon, I went to Indian Rock Shopping Center in Suffern for the Indian Rock geocache. The strip mall is built around Indian Rock, a large glacial erratic that got its name because local tribes used to hold tribal meetings there. There used to be a cache right at the rock but that one didn't even last a day; the cache owner archived and removed it as soon as he found out that the spot had historic significance. This new cache is somewhat different. The first part now involves using a number at the cache site to calculate the final coordinates and the actual cache container is on the other side of the parking area, which in my opinion is a much better cache location.

( TwitVim update )