Bruno Lopes' Lack of imagination

Error 2 when trying to start windows event log service

2011-04-15T16:24:00.000+01:00

Today my Windows 7 machine couldn't start the windows event log service, and so I couldn't open the event log viewed.

After a bit of troubleshooting I found out that the issue seemed with a particular registry key.

Removing the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\eventlog\parameters registry key allowed event log to start without problems.

Zero-friction deployment: from local to github in one powershell command

2010-10-04T21:35:00.000+01:00

Short version:

Code is here, depends on a dll here and here. Allows for a powershell script to upload a file to github.

Long version:

There are one core belief powering this post:

- There should be zero friction in getting a software release out in the open. In the age of build scripts and APIs and everything over http there is no reason a human should be involved in cutting a version except in the decision to make it so.

The main side effect of having very little to no friction in cutting a release is that it happens way more often, since there is no reason not to do it if the code is good to go.

In this particular case, all I lacked was a way to upload a file to github. The packaging was already done, courtesy of psake+write-zip. Having found nothing in the powershell world to do it, I ended up “porting” part of a ruby script.

Github stores files in Amazon’s S3, so there were two steps to uploading a file:
-Telling github that the file exists and getting a token to pass to S3.

-Uploading the file itself to S3 using the token gotten from step 1.

The biggest issue ended up being that WebClient doesn’t handle POST’ing to an url with MIME values, which is what S3 expects in this scenario. Using the underlying *thingie* directly from powershell would bit a bit harder and more prone to errors, so I just used an existing helper to upload the file correctly.

The powershell code that uses it is available here, with another extra dependency on the Html Agility Pack to scrape the downloads page for info on existing downloads.

The function itself (not iet a full commandlet due to lack of time) is upload_file_to_github:

function upload_file_to_github($login, $repo, $api_key, $file, $filename, $description){
    [void][System.Reflection.Assembly]::LoadFrom((get-item "lib\Krystalware.UploadHelper.dll"))

    $full_repo = $login+"/"+$repo
    $downloads_path = "http://github.com/"+$full_repo+"/downloads"

    $post = new-object System.Collections.Specialized.NameValueCollection
    $post.Add('login',$login)
    $post.Add('token',$api_key)
    $post.Add('file_size',$file.Length)
    $post.Add('content_type',"application/octet-stream")
    $post.Add('file_name',$filename)
    $post.Add('description',$description)
    $wc = new-object net.webclient
    $upload_info = [xml][System.Text.Encoding]::ASCII.GetString($wc.UploadValues($downloads_path, $post))

    $post = new-object System.Collections.Specialized.NameValueCollection
    $post.Add('FileName',$filename)
    $post.Add('policy',$upload_info.hash.policy)
    $post.Add('success_action_status',"201")
    $post.Add('key',$upload_info.hash.prefix+$file.Name)
    $post.Add('AWSAccessKeyId',$upload_info.hash.accesskeyid)
    $post.Add('signature',$upload_info.hash.signature)
    $post.Add('acl',$upload_info.hash.acl)

    $upload_file = new-object Krystalware.UploadHelper.UploadFile $file.FullName, "file", "application/octet-stream"
    [void][Krystalware.UploadHelper.HttpUploadHelper]::Upload("http://github.s3.amazonaws.com/", $upload_file, $post)
}

As you can see, it’s just a normal POST to the web to notify github of a new file, interpreting the response as xml to extract needed information for the S3 POST. That one uses the helper to pass the parameters as MIME-encoded values.

I have to say, using xml in powershell was a lot easier than I thought it would be. I wonder if JSON is equally well supported…

Blaze-IPP – Design choices and implementation

2010-09-27T10:25:00.000+01:00

When I set out to create the support to code commands in IronPython, there were two main guidelines in my mind: simple commands and fast feedback.

Simple commands

Commands should only need to be a name and the definition of execute for the simplest of cases. No imports, just the code:

This is the simplest way to define a command as of version 1.2. As you can see, there’s nothing that isn’t required, and simple shortcuts or expanders could be defined by a simple method

The side effect is that even short or small tasks can be easily automated since there’s no ceremony.

There are two main features to accomplish this guideline:

Incrementing the scope with the extra classes and namespaces.
Some “smarts” to allow for both methods and classes to be commands.

Adding definitions to a scope is simple, since all we need to do is associate a value to a symbol:

ScriptScope scope = _engine.CreateScope();

scope.SetVariable("BaseIronPythonCommand", ClrModule.GetPythonType(typeof(BaseIronPythonCommand)));

scope.SetVariable("UserContext", UserContext.Instance);

Allowing for both classes and methods is also trivial, since we can filter the values on the scope by type after executing code.

Anything that’s a PythonType for which the CLR type can be assigned to an IIronPythonCommand is a command class.

Anything that is callable and a PythonFunction that doesn’t start with “_” is a command method.

Fast feedback

I shouldn’t need to reload the application just to pick up a new script or a change to an existing one. Blaze should pick up changes from the file system and reload the files as needed.

Here the big issue was locked files when the “file changed” was triggered.

This meant that changed files were placed in a queue, and a timer running in the background pulled files from the queue and reloaded them. If a file is locked, then put it again in the queue.

Forked dbdeploy.net

2010-09-20T10:30:00.000+01:00

Dbdeploy.net is a database change management library, used by us at weListen to track the changes made to a database in the scope of a project and apply them as needed. It takes a folder full of ordered files containing the change scripts in sql and applies them as needed to a given database. It tracks which changes have been applied in a special table.

We’ve used for quite some time, and were mostly happy with it, except when using it from the command line on the servers. The msbuild task worked well to obtain and apply the changes, but the command line application added extra information to the output that needed to be trimmed before applying it to the server.

Well, it annoyed me enough to fork it from its home at sourceforge to a new place at github.

I refactored to projects a bit, to exclude a direct dependency on NAnt, and added a Dbdeploy.Powershell assembly with 3 commands:

Export-DbUpdate, which outputs the scripts that need to be applied. Can write the scripts directly to a file;
Push-DbUpdate, which applies the changes to the database (still has a bug when using with SQL Server, as I need to split the resulting script into batches);
Select-DbUpdate outputs information about which sets need to be applied;

All of these commands take a path to a config file describing the database type, connection string and table name to store the changelog, and the directory where the actual changesets exist.

Also, instead of mucking about with NAnt as the build script host, I’m using psake. I really can’t stand using xml to describe build steps anymore.

To use it just download the release, and in powershell “import-module Dbdeploy.Powershell.dll”.

Anatomy of a command

2010-09-06T19:37:00.000+01:00

Creating an IronPython command to launch the command prompt isn’t much, since you can have a shortcut to do just that.
You might have noticed that the command subclasses “BaseIronPythonCommand”. This is an abstract class that implements the basics and just leaves the Name and the Execute method for you to implement:

There’s also an interface you can implement directly (IIronPythonCommand) if you want to assume control of everything.
BaseIronPythonCommand’s implementation is quite simple. Most of it is boilerplate for a Blaze plugin, leaving only the name and execution to be done in the script:

GetName is the name used to invoke the command. It should be something unique and descriptive. A long name is good, and you can teach Blaze a shortcut to it by typing the shortcut and selecting the command on the dropdown box. This teaches blaze about the shortcut.
GetDescription returns a description for the command, to be shown on the line below it on the interface.
AutoComplete returns a completion for the given string. This means you can transform “off” into “office” inside your command.
Execute tells the command the user has selected it. The command returns either a program to execute in the form of a path concatenated with the arguments, null if the command has executed itself.
Any one of these can be implemented in ironpython, although for basic commands you only need to implement GetName (or a property called Name) and Execute.

Tutorial: How to create your own plugins for Blaze-IronPythonPlugins (BPP)

2010-09-01T10:27:00.000+01:00

On the first post in this series I presented Blaze IPP, which allows for scripting of a launcher using IronPython. Here I tell you how you can write your own plugins from scratch. It will be a short post.

First of all, I assume you’re already running Blaze with the plugin installed.

Let’s create a new directory under “Plugins” to store our new plugins. This way if any future upgrade on IPP brings an ironpython file with the same name, your plugin will not be overwritten. We’ll call it LocalIronPythonPlugins:

Now, configure IPP to also monitor that directory. Call Blaze, right click on it and in the “settings” open the “Plugins” tab. Select “IronPythonHostPlugin”, “Configure” and add the newly created directory (you can either browse to it with “b” or you can paste the path into the text box):

Now that we have a clean directory to work with, let’s create a file called “OpenCommandLine.ipy”. Open it and add the following code:

As you can probably guess, this will create a new command named OpenCommandLine, and when you select it it will open a new command prompt. IPP is clever enough to know that if a command returns a string, it is a program to be executed.

This is just a simple example to get you started. Since this is IronPython, you can use the entire .Net framework and fetch things from the web, open the registry, connect to a remote host or start a service, to give a couple of examples.

Creating plugins in IronPython for Blaze, an application launcher.

2010-08-30T10:25:00.001+01:00

Everyone loves the start menu from Windows Vista/7, but compared to Launchy, Quicksilver or any other application launchers it’s a bit anemic. No wonder, since it’s just a search over your start menu and indexed locations.

But even any one of those is a bit lacking in support for runtime extensibility. You can create plugins, but it involves C (or Objective-C in the case of Quicksilver) and restarting the app. Not very friendly for those cases where you just want to pick up the currently selected object and throw it over an scp connection autocompleted from your favorites.

Enter Blaze + IronPythonPlugins. Blaze is a .Net application launcher for Windows that includes a learning facility that tries to complete repetitive tasks for you. IronPythonPlugins is a plugin itself for Blaze that hosts simple plugins written in IronPython.

If you want to try it just download the latest version from here. It includes a version all ready to use. Start the app, call it with Ctrl+Alt+Space and try a couple of plugins:

Open windows explorer, select a text file, call blaze and type “edit with notepad2”. If you have notepad2 in your path, it will be launched with the selected file opened.
If you use putty (the ssh terminal), “putty <name of session>” will launch putty with that session. Here session is autocompleted, so if you have a session called “my.home.box”, typing “putty home<tab>” will autocomplete to “putty my.home.box”.
Type “random-text <number>” and your clipboard will have up to <number> random letters. Useful to generate test data.

There are a couple more plugins in the “Plugins\IronPythonPlugins” directory, so if you’re interested in how it’s done, open them in your editor of choice. In the next few days I’ll do a couple of posts on how you can create your own plugins without having to guess from the already existing ones.

If you want to change anything edit the file, save it and Blaze will automatically pick it up, together with any other .ipy file in that directory that has changed. If you open Blaze’s configuration page for the plugin, you can add other directories to search for .ipy files. Useful to keep your personal plugins separated from the “official” ones.

Sadly errors are still only sent to a System.Diagnostics.Debug, so if you edit a file and can no longer use it, open DbgView, edit the file again to force blaze to reload it and be on the lookout for output starting with “Error with file”. There will be a stacktrace telling you what you did wrong.

So, try it, have fun and comment on it. I’d love to hear what other people think of it.

Toggling with an IEnumerator and yield.

2010-04-09T17:25:00.001+01:00

Instead of using a state variable to jog around and check what is the state of a window, why not just use an infinite collection?

public partial class Admin : Window
    {
        private IEnumerator _toggler;

        public Admin()
        {
            InitializeComponent();
            _toggler = Toggler().GetEnumerator();
        }

        public void Toggle()
        {
            _toggler.MoveNext();
        }

        private IEnumerable Toggler()
        {
            while (true)
            {
                Show();
                yield return new object();
                Hide();
                yield return new object();
            }
        }
    }

I wonder if this is a good pattern, or just me being "clever".

Using ironpython to generate nhibernate mapping files from a fluent configuration

2010-03-17T10:00:00.000Z

I admit, this is a bit of a goldberg machine, but here's the context:

In my current project we need to store a bunch of events related to some 20 entities. Those events are the same for all entities, and they all need to be stored in the database. We use NHibernate to handle the "talk to the database" part, and fluent-nhibernate to configure the mappings via conventions for the most part. There are some entities which are either manually mapped or have specific overrides for particular cases (manually mapping some entities and automapping others is a mistake I think I'll avoid in the future).

For the most part this setup does wonders. We can add and change our entities and the mappings are generated without we thinking much of it. Since development speed has a larger priority than runtime speed in this project, we don't have much problems with the default mappings that are generated.

Except in the case of those events. Since the events are the same for all the entities, the event classes are open generics. Now, nhibernate can't map open generics, but can map closed generics, which is how we do it. We just iterate through all the types in the assembly that correspond to our entities and map those with fluent. The problem then becomes one of performance. As of now there are about 500 generated classes, and those take a while to discover and map. Since we're using fluent nhibernate to configure it in runtime, this means that each time the app is restarted there's a small downtime (on the scale of about 1 minute). While that's not much for an app in production, while developing one tends to restart the app a bit often. That's a problem in the day to day work.

I used our integration test suite to time the mapping, and on my pc it took about a minute to run. On the other hand, if we already have an .hbm file to feed to nhibernate, we can shave about 30 to 40 seconds. That's less than half the time for a simple trade-off of having to know when to generate the mappings.

To generate the mapping we could have created a new console project to just call the correct methods on our session provider factory, which would be a bit overkill. All we needed was to setup the environment and ask fluent for the mapping file.

And here is something that IronPython shines in.

Just create an .ipy file, add references to the correct assemblies, and since there's a clear separation of concerns (or so we hope), it's easy to configure fluent and ask it to generate the mappings for us. Most of the time you'll use .net code like you do in c# or vb.

There are only a couple of issues here:

- We need to remember to regenerate the hbm files when classes/mapping changes

- We need to remember to recompile both before and after regenerating hbms. Since we're loading the assemblies with ironpython outside of visual studio, the script doesn't know if the assemblies are stale or not.

Most of this can be avoided by having a build server rebuild the hbm files on its own and fail the build if they don't match with the latest in the source control. Of course, we could just have it commit the files, but that's one step I'm not fully comfortable with.

Notes and reflections about the second annual scrum meeting in Portugal

2010-03-10T10:00:00.000Z

A month ago I attended the second annual Scrum Meeting in Portugal, and ended up taking some notes about two of the presentations.

Here they are, both for the team at weListen, and for the world at large.

Notes on "A Practical Roadmap to Great Scrum:A Systematic Guide to Hyperproductivity" presented by Jeff Sutherland

This was the presentation that changed the way I perceive scrum. Where before I saw scrum as an agile software development methodology like xp, now I see it as a management and process template. This means a team should use scrum and complement it with technical practices. Apart from that, the largest focus to me was on 3 velocity multipliers for teams and 2 required practices for hyper-productive teams.

The 3 velocity multipliers are ready, done and self-organization. Ready and done shield an iteration from churn. A story only enters development when it is ready, and by the end of the sprint it should be done. Ready means that the story is understood and sized by the team with input from the client, and done means the same story is tested and approved to be deployed. Self-organization ties nicely into the two required practices mentioned: talk about problems and fix the same problems.

Two classes of problems mentioned: bugs that take more than 8 hours to fix, and stories that take more than twice the calendar time than estimated ideal time. For each bug or story in this situation, do a root cause analysis to figure out the underlying problem. This can be multitasking (a dev assigned to several projects), final inspection or testing done too late or interruptions and form the basis of an impediment list for the team together with the product owner and scrum-master to solve.

There were several interesting data points mentioned. In a given company, the peak of productivity was measured on 60 hour work weeks in waterfall projects, and on 30 hour work weeks with double story points delivered. This means an productivity increase of about 3 to 4 times. Part of the presentation used Systematic, a dutch software company as the source for several data points for scrum teams. They noticed a linear scale in developer productivity for project sizes, going against Brook's Law, although I think the Law mentioned adding people to an already late project, and didn't mention scale. They apparently implemented scrum using Mary Poppendieck's lean tools as described in her book.

Some "required truths" about hyper-productive teams were discussed, and I found them interesting insights. The ones that stuck to my mind the most were:

Everyone must be trained in scrum. This is so that all the players follow the same playbook and the concepts and practices are well understood.
Backlog must be ready to implement before the sprint, and done by the end (tying into the concepts of ready and done mentioned earlier) .
Pair immediately on a task if there's only one person capable of handling it, to avoid bottlenecks in the throughput.
Short sprints (often just one week)
Servant leadership, where the product owner and scrum-master are a resource of the team, instead of using the team as a resource.
No multitasking. This one I'm a bit curious as to how realistic it is, considering maintenance work, and teams with many past projects.

Overall it was a very interesting presentation. There are several videos on youtube with Jeff Sutherland about scrum. This one is very similar to the one I saw.

Notes on "Scrum and TFS 2010" by Mitch Lacey

This was mostly a storytelling and demo about new features in TFS. I did end up asking Mitch about the importance of accounting for time originally estimated, time in task and time remaining for small and recent to scrum teams, as to me it seemed a bit too much bureaucracy for a small team. The answer was enlightening, since those numbers are particularly important for those teams as a way to surface problems (either with estimation, delays, interruptions or other causes). This means I might start looking into accounting for such numbers. "You can't manage what you don't measure". When a team gels and is proficient, the numbers may not matter as the team starts getting into the groove and noticing impediments becomes intuitive.

OCD and filenames, as pertaining to TV series

2010-01-24T01:19:00.000Z

I like all my file names to be similar, and this applies to both mp3 and series. For mp3 I use the wonderful foobar2k to tag and rename files, but for series I ended up coding my own formatter. All this assumes a linux system (as is my home server, flatline)
First of all, to extract the video files from the rars:

find -name "*rar" -type f -exec rar e {} \;

This extracts all rars from under the current directory (and subdirectory) into the working directory.
If the files already come uncompressed, but are in subdirectories:

find -name "*mkv" -type f -exec mv {} . \;

Replace mkv with avi if that's what you've got.
Now, sometimes the files come in different formats. Some have [season][episode], others [season]x[episode], and I'd like for them all to be in the same [title]S[season]E[episode] format.
For that all you need is the following python script:

It can take 2 arguments: the directory where the file names are, and a title that overrides the one found in the file names. Hope this is helpful for someone else :)

Just replaced the fan on my laptop

2010-01-03T04:01:00.000Z

For a couple of months my macbook's fan has been a bit on the fritz. Sometimes when the CPU got a bit hot it started making some noises and once in a while stopped working completely.

The repair would most likely cost me about 100€ (50 for the fan, 50 for replacing it), so I decided to look up if anyone sold those. Thankfully a store in Hong-Kong does, and they ship international (they're called eeshop). It took a while to get here (a month and a half, perhaps) but for 10€ that's not a big issue. I did ask after a while if it was supposed to take so long, and answer was prompt (apparently the postal office was swamped).

Opening the laptop was another issue. First attempt failed when some of the screws were a bit too small for the screwdrivers I was using, so today I went and bought a new set with a better grip. After removing the screws, I just had to take care to remove the connection for the trackpad and slide the keyboard left to remove it. Opening it gave me access to the internals as you can see on the left. Replacing the fan was a matter of just removing the connector, two screws and taking it off. The right screw was hidden under some cables, but gently pulling them off gave me access to remove it.

I do have to say, putting it all back again was a bit difficult, since on the right hand there are some "hooks" that you need to slide the keyboard plate into, and that wasn't trivial. Other than that it was easy.

And now my laptop doesn't scream anymore when hot! And it cost me just about 15€ plus an hour and a half. Yay.

My presentation for netponto about Lucene.net

2009-12-26T18:08:00.001Z

A couple of weeks ago I did a presentation for a .net user group (netponto) here in Portugal about Lucene.net, an open source textual search engine. You can see the slide deck here, and I've uploaded the source code used in the demo to my github page.

What went well

Using git (with the git-extensions gui) to bookmark the various steps I wanted for the code demo. This is something that surprised me on how well it worked and actually ended up being remarked upon. It also generated a few questions about git that were answered mid-demo in a bit of a detour.

The pattern here is to build the demo as a series of linear git branches, each a small step in the demo, and instead of writing the code in-loco at the demo or storing snippets to be used, I just move up along the branches. Each step is small and self contained, and I spend more time just highlighting the important parts and the decisions I made and less time typing.

Also, using git gives me the advantage to rewrite history, so that if I figure out that the demo app needs to have a foo somewhere from the start, I can switch to the root branch, make the edit, and rebase the branches to update them with the change. With some care, this propagates the change upwards with little to no conflicts.

Short deck of slides, demo with ongoing q&a. While the initial presentation didn't take long, the demo ended up going into overtime. If there's no hard and fast timebox, this is good, otherwise answering questions in the middle of the demo can push a well timed demo out of its allotted time. But the audience seemed to be enjoying it, and I think this helps spread knowledge a bit better than just giving a presentation and showing code.

What could have been better

The "oh dear, a crash" gimmick had an interesting effect, but I think it was wasted on a slightly minor point. A better and more important part to emphasize would have been the impact of the analyzer on both indexed fields and search query. I should have had an unexpected result for a search be the "crash" and not the lack of values for fields.

I used a transition I saw on slide:ology in 3 slides, which while cool, might have been more distracting than useful. On the other hand, I think it emphasized the fact that the index was shared between the indexer and the searcher, which part of the point I was trying to make.

What I'll improve

Relax. Although the feedback I gathered from some of the audience was that it didn't show, I felt like I had zipped through the initial presentation. Ended up taking a bit of a breather while answering questions.

Might be interesting to consider dropping even more slides in detriment of more code. Although I think there's a minimum needed to intro the subject matter. Otherwise people will look at the code but the concepts will not gel.

Feedback

If you were there, I'd love to hear from you. I've received the report from Caio with the flattering comments, but I'd also love to hear your opinion on what could have been improved.

I wish I could put 4gb of ram on my macbook...

2009-08-30T17:24:00.001+01:00

According to an article on cnet Snow Leopard boots by default into a 32-bit kernel and mentions that macbooks with an older, 32-bit EFI chipset (which handles the early boot process) cannot boot the 64-bit version.

Although having a 64-bit kernel would be good, I'm more interested in the side effect of the 32-bit chipset memory limitation. The same article mentions that a 32-bit EFI chipset cannot address more than 3Gb of ram. Since most likely this is the chipset installed on my mac, I think that's the reason why 4 Gb are not supported. I wonder if the chipset could be replaced. Doubt it, since it must be soldered into the board.

Sadly, that's pretty much the largest reason I'm envious of the new macs. 2 gb isn't much, specially if you're like me and have tons of apps loaded all the time.

Man, it would be awesome to go...

2009-08-19T20:07:00.002+01:00

Roy Osherove is giving an hands-on TDD Masterclass in the UK, September 21-25. Roy is author of "The Art of Unit Testing" (http://www.artofunittesting.com/), a leading tdd & unit testing book; he maintains a blog at http://iserializable.com (which amoung other things has critiqued tests written by Microsoft for asp.net MVC - check out the testreviews category) and has recently been on the Scott Hanselman podcast (http://bit.ly/psgYO) where he educated Scott on best practices in Unit Testing techniques. For a further insight into Roy's style, be sure to also check out Roy's talk at the recent Norwegian Developer's Conference (http://bit.ly/NuJVa).

Full Details here: http://bbits.co.uk/tddmasterclass

Update:

bbits are holding a raffle for a free ticket for the event. To be eligible to win the ticket (worth £2395!) you MUST paste this text, including all links, into your blog and email Ian@bbits.co.uk with the url to the blog entry. The draw will be made on September 1st and the winner informed by email and on bbits.co.uk/blog

Windows 7 blues

2009-08-08T21:10:00.001+01:00

why haven't I learned yet not to go for first releases?

With Windows 7 reaching RTM status and the general low working demands of August, I decided it was time to kick Vista to the curb on my dev machine and get myself a brand new shiny OS.

Triple-headed Blue-screens

Because it can never be two easy, the upgrade experience ended up being rather painful, with both attemps resulting in blue screens before the setup was finished. Deciding that perhaps it was some conflict with device drivers, one of the many programs I had installed or just bad luck, I decided to cut my losses and just repave the system partition to do a clean install.

Still no joy. Same bluescreen at the end of the setup.

The dev machine's most unusual part is the two NVIDIA display adapters installed on the PCI-E slots, which are used to drive 3 monitors. Remembering that vista also had some troubles with the secondary adapter, I removed it and gave it another shot.

Yap, that worked. Setup finished and gave me a brand new Windows 7 installation to play with.

You no like restart?

With a folder full of drivers for the display adapters, chipset, SATA controllers and audio, I spent some time updating them before heading home. Somewhere around the end ,after a restart, another bluescreen.

That's no good.

After some fiddling I found out that it only happened on a restart, and never on a cold boot.

Oh well, that's a bother but at least it's work-around-able. Off I go to install sql server, visual studio and a miriad of essential software for a dev machine. Thankfully with portable apps living in a dropbox folder most of it takes little to no time at all.

System Interrupts should not interrupt all the time.

Some time afterwards, while installing some software I decided to look at resource monitor as a source of "oh, look at all that data" entertainment.

And it showed "System Interrupts" using about 40% of the cpu, all the time. What. The. Hell. At first I thought it would have something to do with the troublesome display adapters, and reverting to the standard VGA adapter driver reduced the usage, but it still was too high (about 20, 25%)

Again, after some fiddling around, uninstalling drivers, disabling devices I found out that the problem was with the "Marver 88SE611 SATA V12069" drivers. Reverting the drivers to the default drivers makes the cpu usage go back to normal.

Sadly, the reboot bluescreen still appears from time to time. I think it always happened when the second display adapter was installed, but I'm not sure.

In a nutshell.

When upgrading/installing, remove the secondary display adapter.
With two display adapters, don't reboot (away from the machine)
Don't install the Marvell SATA driver
Get a rabbit's foot.

I do hope the reboot bluescreen gets fixed soon, but for now I'm happy with having a usable machine.

In the case someone has the same problem, this machine has the following specs:

Core2Duo E8400
4 x 2Gb Kingston ValueRam
Asus P5Q Pro
NVIDIA 7300 LE
NVIDIA 7200 GS
Western Digital 150Gb Velociraptor
Seagate 200Gb

Dr. Googlelove or: How I Learned to Stop Worrying and Love the Cloud

2009-06-11T00:15:00.001+01:00

written in full blown hipster <censured> mode on a macbook, listening to an ipod and sipping an iced mocca at a starbucks.

Having three different machines that I work with daily tends to cause me some pain whenever I try to send an email from the two that are work/fun related. I've got 3 pcs that I use on a daily basis:

My personal laptop is a black Macbook called mac-sendai. It handles my email, IM, photos, videos and contacts. My social life is stored in it, being it work or personal and I carry it around for most of the time.
My home machine is armitage. It's a Windows XP machine that I use mostly as a gaming/movie machine.
My work machine is 2finn. It runs Windows Vista 64 and is my main development machine at the office.

From each of the 3 pcs I usually need to send off emails. I use Gmail on any of the desktops and Mail.app on the laptop. Since mac-sendai handles the social part, my address book there is where I aggregate and collate most of the information about contacts and sync both the cell phone and an ipod touch to it.

Up until now the only connection to the cloud was from the laptop to Plaxo, to get auto-updates for contact information of friends and "business associates". It works rather well and the best way to prove it is that most of the time I don't even notice it.

But lately I've noticed that OS X is able to sync remotely to google contacts, and since I use gmail on the desktops, having it synchronized with the address books is the last step in having a always available and up to date contact store on all machines, with the added benefits of remote backups (besides the time machine I store at home).

So nowadays here's the replication flow for my "personal network":

Dropbox handles the "utils", which are about 50 or 100 megs of portable apps including notepad2, gnu utils and Launchy's preferences, current working set documents, emacs site-lisp and .emacs. Some time ago I used foldershare for the utils, but it was too finicky, and with hard links from the dropbox to the correct places, it all works much better.
Gmail handles the email. I've used IMAP even before gmail on all desktop programs and now on the ipod touch, so I haven't even thought about that particular issue for a while.
Google Calendar keeps some of my appointments, with a shared calendar for weListen.
And now, Google Contacts stores the emails and phone numbers for my little social network.

The migration of contacts ended up costing about half an hour of deduplication and merging, but thankfully the address book includes a shortcut to merge contacts, so most of the time it was spent cleaning up emails that I don't need and just checking the contacts to see that nothing was mangled.

My biggest worry now is that the address book will become bloated. GMail adds any address that you send email to to the contacts list, and most of the time if it's a one time only reply I won't want it to be replicated, so the jury's still out on how I'll handle that.

But for now, I'm a happy camper. No more copy pasting of emails, and my inner software engineer feels more confortable at not having duplicated information spread around.

Edit: thanks to Edgar Gonçalves for the title fix :) In my defence Starbucks had no wifi, and I was unsure of the correct title :)

My delicious tag wordle

2009-04-22T12:33:00.001+01:00

Does it show that I am a .net developer with a python deviancy?

Typemock Isolator Plug (to get a free license and try out something other than Moq)

2009-01-13T19:57:00.000Z

Programming Visual Basic applications?

Typemock have released a new version of their unit testing tool, Typemock Isolator 5.2. This version includes a new friendly VB.NET API which makes Isolator the best Isolation tool for unit testing A Visual Basic (VB) .NET application.

Isolator now allows unit testing in VB or C# for many ‘hard to test’ technologies such as SharePoint, ASP.NET MVC, partial support for Silverlight, WPF, LINQ, WF, Entity Framework, WCF unit testing and more.

Note that the first 25 bloggers who blog this text in their blog and tell us about it, will get a Free Full Isolator license (worth $139). If you post this in a VB.NET dedicated blog, you'll get a license automatically (even if more than 25 submit) during the first week of this announcement.

Go ahead, click the following link for more information on how to get your free license.

Evil Extension Method, or just handy?

2009-01-07T16:25:00.001Z

I just realized I can have the following extension method:

        1: public static class StringExtensions

       2: {

       3:     public static bool HasContent(this string self)

       4:     {

       5:         return !string.IsNullOrEmpty(self);

       6:     }

       7: }

And then use it like this:

       1: [TestFixture]

       2: public class StringExtensionMethodsTest

       3: {

       4:     [Test]

       5:     [Category("Unit")]

       6:     public void TestStringHasContentWorksForNullStrings()

       7:     {

       8:         string s = null;

       9:         Assert.IsFalse(s.HasContent());

      10:     }

      11:  

      12:     [Test]

      13:     [Category("Unit")]

      14:     public void TestStringHasContentWorksForEmptyStrings()

      15:     {

      16:         string s = "";

      17:         Assert.IsFalse(s.HasContent());

      18:     }

      19:  

      20:     [Test]

      21:     [Category("Unit")]

      22:     public void TestStringHasContentWorksForStringsWithContent()

      23:     {

      24:         string s = "content";

      25:         Assert.IsTrue(s.HasContent());

      26:     }

      27: }

The interesting part is that a method call on a null object for an extension method works, which allows for the s.HasContent() idiom, instead of string.HasContent(s).

My question is the following: is relying on such “detail” good, in the way that it allows for less line noise and cleaner code, or bad for violating the idea that you can call a method on a null object and not get an exception?

I’d love to hear opinions on this.

2 more apps I end up installing on all computers I use.

2008-12-07T00:47:00.001Z

Foldershare, set to sync my utils folder, which includes putty, teracopy, some gnu command line utils, a couple of scripts to start and stop services, .net reflector, keepass binary, notepad2, and some other portable apps that I end up collecting over time.

Dropbox, mostly for the public folder. The explorer/finder integration is a wonderful piece of magic. Dropping a file on the public folder and using the "Copy the public link" item on the context menu ended up being my favorite way to share files with the world.

First shot at a couchdb installer on windows

2008-11-26T01:39:00.001Z

Disclaimer: this is for the brave folks who wish to try things out. So far I've only tested the installer on my own machine and on a clean Windows XP vmware image.

Now for the running bits: here's a first try at getting a workable couchdb installer on Windows.

There were some instructions down at the wiki on how to get all the bits and bobs running together, but parts of it required a C compiler and some rather tedious setup on part of the user. It was also a bit frail, as noted by me trying for a couple of afternoons to get the unit tests to pass.

The biggest problem was that Spidermonkey must be compiled in a specific way (one that wasn't apparent to me at first), which I dismissed at first because the blog containing the steps was down. Also, one must use lowercase paths on the config file, otherwise erlang will choke when trying to open the database files.

The next step would be to create a shortcut on the start menu, get some sensical text on the installer, do some error checking to find out if the prerequisites are installed, and create an uninstaller.

For now, here's the installer for the world to try. Leave a comment if you find a problem or have a suggestion.

Thank you guys!

2008-10-28T23:12:00.001Z

My great friends and colleagues at weListen just offered my these two wonderful sets of DVDs for my birthday:

These are Q&A sessions Kevin Smith does at packed theaters with thousands of people, and where he talks about everything and anything. It includes some inside jokes for his Askewniverse fans, and each disk is about an hour and a half of him just having fun with the audience. And that's the brilliance of it. It really is just a dude with some great stories having fun with people who dig his work, and he just goes with the flow, having people from the audience up on the stage for whatever reason it is at the time, or going on a binge discussing Prince's flamboyance.

For anyone who's a fan of Kevin Smith's movies, these dvds are must see. And I really thank the guys at work for the gift. :D

Unexpected change

2008-10-17T00:42:00.001+01:00

Got home at 11pm. General slouch mode before bedtime engaged. Open firefox for one last news feed round and look at a large change on iGoogle.

My iGoogle page is "designed" to fit on one 20" screen at 1650x1050 or thereabouts. I want to open it, scan it quickly for any big news and move on. Now it takes two screens with a short description for each feed item (as most of the widgets are based on newspapers and tv channels' feeds), which kind of defeats the purpose. Also, the tabs moved to the left, which may be a good option for today's wide screens, but looks a bit odd having lots of negative space on a vertical bar.

Thankfully a quick trip to the tab's settings shows me a way to remove the new feed behavior and return the page to a more sane, no scroll, quick scan mode. It also shows a manual backup for the iGoogle page, and export settings. May pay someday to add these settings to the backup jobs at home.

My first Visual Studio Plugin - CullWindows

2008-10-12T18:33:00.000+01:00

okay, that was hell...

In the .net world I usually work with the wonder tool that is Resharper. For this particular story, the main points are the hugely useful navigation shortcuts: Go to symbol, Go to file, Go to type, Navigate to implementation, Navigate to base, you get the picture.

So, the Solution Explorer window usually doesn't get much work, but the tab strip on top is cluttered beyond recognition even after just half an hour or two of work. And while most of the source navigation is done via shortcuts, sometimes it's handy to just click on the respective tab. Just try and find it amidst 20 other files.

Hence, my idea to expand Visual Studio with the simple ability to keep just the top X files I use.

Meet CullWindows. Direct download link here, google code home page here. A simple solution to a simple problem. It was the implementation that was hell.

The Plugin

The plugin's logic is quite simple. Keep track of which files are opened for editing. When the user views a file, see if we've hit the limit. If we have, close documents, from last to first, until we're back on the limit. Ignore unsaved files, and things that are not files.

The limit is configured on the Visual Studio Options panel, under Cull Windows.

The installed is "dumb", meaning it won't give you any feedback, and just install and go away. I'm sorry for that.

For now the plugin is Visual Studio 2008 only, no version for 2005, mostly because I don't use it. If there's any demand, I may give it a try.

The Implementation

Firstly, I created an add-in. Seeing an example that came with the SDK, I managed to get a reference to the Running Documents Table, which contains all currently opened documents, and registered an event sink to listen for changes in the table.

This was all good until I get home and try the same code on another computer. There, the table was nowhere to be found, and found no error. I could've tried to debug it for some time, but it seemed better to just restart it.

Next up, a VSPackage. From what I had gathered, it is a "new" and more powerful way to extend VS, and the API and samples seemed to be a bit more OO. I managed to get it to work, again, but now the debugging is a bit stranger than when it was an add-in, since it now needs to be installed on the registry hive for the IDE.

Most of the interaction is a bit nasty from someone used to clean, object oriented APIs. It seems most of the extensibility points in Visual studio are done via COM, which is a beast I've encountered few times before, and I dont' keep fond memories from those occasions. It may be that I'm too new working with it, but the code that resulted wasn't exactly... pleasant. Feel free to browse the code. It isn't pretty, specially the first iterations, but I'm open to suggestions and criticisms.

Now, the API is HUGE. There are hundreds of interfaces on each of the Visual Studio namespaces, and trying to get from one point to another is somewhat tiring. I have a document cookie from the event handler, now how to get the name of the file. Okay, have the name, now how can I find if it is modified. Where to keep the preferences, and how to access them. There were some concepts which I lacked, but mostly it was just be being a newbie at it and not knowing where to look.

The solution ended up being simply 3 files: the package, the document monitor and the options page.

The package was built mostly by the VSPackage template, and the only things I changed was to remove the menu item, add the option pane and build the document monitor. Some of the attribute values were a matter of faith on the documentation, and I didn't dare touch most of the generated code.

The document monitor does the grunt work of maintaining a document list, ordered by access date, listening to the running document table events, and closing documents when the limit is reached. The first implementation was fetching the window frames from the UI shell and iterating through it to find the one with the file to close. Not optimal by a long shot, but worked. A second (and final) iteration used the IsDocumentOpen method from VsShellUtilities to get the window frame. This was not intuitive. I expected a method starting with Is to just return true or false, and not return more information.

The option page was actually the easier part, with all of the work taken care of by the DialogPage class, including persistence. It's a bit too much magic for my tastes, but it works. I was supposed to access the options via DTE, but it kept throwing an invalid cast exception, and I gave up and just passed the option page object to the monitor.

The Next Steps

The first implementation is done, but I think I can make it a bit smarter if I have the time:

Ponder the amount of time spent on a given file when picking which one to remove. This would prevent newer files which were opened by mistake to stay opened when earlier files which had more use go away.
Keep a background timer to cull windows after some time with no visits to the document.
Have the installed say something instead of just installing and going away.

I guess that's all for now, thanks for tuning in.

More Buxfer goodies (this time, a backup script)

2008-09-29T01:14:00.001+01:00

I just realized that I'm putting a fair amount of effort (and information) on Buxfer. Been tracking rather faithfully my expenses for a month, and I'm rather happy with the service.

But what if it someday fails, or disappears? What about the data? It seems to have a data export facility implemented, but it's for Pro accounts only. Okay, it seems like either I misread the membership plans, or they changed them. Either way, the data export is good for when you want to take a look at the statements in excel, but as far as a backup goes, it's no good for me.

A backup must be something I don't have to think about for it to work. It should do its job in the background as much as possible, and only warn me if something goes wrong. Thankfully Buxfer's API has enough functionality to cover this.

This is a small script that fetches all transactions from your Buxfer accounts (or one particular account) to disk. It has some smarts implemented to "continue" a previous backup, so you can setup a scheduled task periodically to fetch the transactions, and it should just fetch the new ones after the first run.

So far it stores the info as pure json, which was the easiest way to implement it. Should I require in the future to actually act on this, I can parse the json again and convert it to csv, or any other format I need.

The source is here, and a skeleton configuration file is here.

To use it you'll need Python (I tested it with 2.5, but I think it should work with 2.4) and simplejson.

Configuration should be self-explanatory, it needs you to fill out your username, password, and filename for backup.

To run it, you can pass the path to the configuration file (this way you can have several different configurations, one for each account, or for different users).

I guess this is it. Any questions, feedback or general chat, just leave a comment.

That's all, folks!

v1.2 of BuxferSubmit

2008-09-26T16:02:00.001+01:00

Version 1.2 is up at the project's page.

Direct link

Changes:

Uses jQuery for the async request.
Stores the user's password in the user's keychain (thanks to Stan Lemon for the code and plugin)
Minor interface tweak (pressing enter when on the tags text field submits the transaction)

That's all for now. I'd love to get some tag auto-complete, but getting a good interface for it has proven difficult.

Waiting.

2008-09-10T22:25:00.001+01:00

on the topic of forced rest time

I actually like waiting for someone at the airport. Of course, this may have something to do with the fact that this only happens once in a while, when I do so I've got my trusty macbook with me, and that I'm not all that eternally busy that I can't spare some time to sit at a coffee shop with wireless.

Not that I do much work, or read all that much, or have a positive contribution to give the world while I'm sitting here. I actually spend most of the time just enjoying people going to and fro or milling about while waiting, like me, for someone else. The coffee shop I'm sitting in has about 32 tables, and it's directly in front of the arrival gate. There are 12 people here, all of them sitting the same way, having a dring, chatting on the phone or just waiting with a bit of a blank stare.

And I'm talking about a mildly peripheral airport like the Lisbon one, which I bet doesn't have one tenth of the cultural cross section of a Heathrow, or an LAX. On the other hand, because it's a smallish one, I can sit at the coffee shop and view most of the arrivals, and it doesn't sound hectic or too crowded. It's just a bunch of people. Waiting.

And you can play a small game, while waiting. Try to think of all these people, every single one of them, as an distinct individual, not just "people". With needs, thoughts, desires, passions, pains and hungers. All of them have a single shared purpose here, but that can be the only thing that they have in common. Imagine the diversity. The blonde lady, dressed in a white short sleeve which passes you? Is she coming or going? Where to? The couple (or so you assume, it's a man and a woman, they could be brothers, or just friends) with the large backpacks. Travelling the world, vacations or volunteer work? The black family, waiting for a relative? Perhaps a son, gone abroad to meet some friends. Of course, the suit, lugging his laptop luggage, blackberry on hand talking and looking just a bit lost, almost (just almost) looking like it's the first time he's here.

And then it starts thinning out. More people are leaving than arriving. There are less flights. And now it's my turn to go. My charge has arrived.

v1.1 of BuxferSubmit

2008-09-02T01:09:00.000+01:00

For the impatient:

New version. Should not freeze if Buxfer is slow to respond or if you forget to plug in the cable. Also some better handling of special characters while talking to the server.

Download here.

For the curious:

Buxfer seemed to have had a bit of trouble with site slowness, which exposed my cluelessness in developing widgets that connect to web services (as in, forgot to consider timeouts of XMLHttpRequests).

I was a bit amazed to find neither much info on Apple's site nor on the web on how to best handle these problems. Ended up just setting a timeout before the request, clearing it on success and canceling the request should the timeout happen. Perhaps I should look into a javascript library such as mochikit or jquery to help with the heavy lifting.

Also, most of the code feels to me a bit fragile. The error handling is primitive, and the fact that the request is async makes me a bit queasy. I'll chalk that up to immaturity on my part in developing in javascript, but I hope to have the time to refactor it a bit.

As I said before, the code is up on google code, so feel free to browse around, all three of you who are reading this :)

Google Code page for the widget

2008-08-30T20:01:00.001+01:00

Well, instead of just having a zip file containing the source code, I decided to put it up on google code. The page is here and I've updated the homepage to include a link to it. The releases will live there, also, and I think I'll look into getting the widget to autoupdate or something, with the files on google code I think I can get an rss feed to the releases.

I'll keep y'all posted.

Buxfer widget

2008-08-30T12:37:00.001+01:00

If you follow my twitter feed, you may have noticed that I’ve created and published a small widget to use with buxfer here. Right now it’s just two forms, one to submit and one to configure, and it isn’t all that smart at what it does.

My intent is to keep the interface simple, but to improve on three points:
- Usability-wise, I want to be faster at adding the transaction, and perhaps have some suggestion or completion for the tags. Right now it is a bit dumb, and doesn’t provide much help besides being a form.
- As far as the interface goes, I want it to occupy a bit less space, and to be prettier. As you can see on the screenshot, the widget uses the standard parts and the labels have the default size. I think I can do better than that and still keep it usable.
- The configuration is stored on the preferences cache for widgets Apple provides, which isn’t quite the best choice as far as storing sensitive data. I should put it in the keychain, and perhaps even use the same as Safari.

I was quite impressed with the ease with which you can produce widgets. Having Dashcode helps a lot, both with the layout of the forms and the debugging of the code. Widgets being based on javascript and html gives them a low entry cost if one is already used to client-side web programming, and opens up the opportunity for some rather interesting moves, such as using JQueryUI or Moo on the widget.

If you’ve tried the widget and want to leave feedback, please, feel free to contact me or to drop a comment here :)

Perhaps extra keys on a keyboard aren't that bad an idea...

2008-07-21T19:00:00.001+01:00

on the newfound use of useless keys

Whilst my work is nowadays mostly done on my macbook (running windows, much to my lament), at home I usually drop the laptop besides my desktop, boot it into mac os x, and use it as a communications central with email and IM open while working on some other thing on the desktop. Call it a strange dual-head, quad-core computing station.

Instead of switching around keyboards when I need to pass from one computer to the other, I use (and profoundly adore) synergy, which allows one keyboard+mouse combo to control more than one computer, via the network. But until now, to change from one computer to another (let's say to talk to someone on IM) my hand had to leave the home keys of the keyboard to use the mouse to move from one computer to the other (since synergy uses the edges of the monitor to cross control from one computer to another).

But no more! Remembering some button with "hotkeys" on the label on the configuration manager, I managed to bind two keys, one to move left, other to move right. But then the dilema... which keys?

I didn't want a hard, multiple key, emacs-like, binding. I was looking for a short stroke to allow for quick back-and-forth, since that was the most common scenario (coding on the desktop, and going to the laptop to spout nonsense on im and back again to serious business).

Enter the most useless keys on my current keyboard, a logitech internet navigator. A fine keyboard all in all, with a wheel on the left side (which I tend to use quite often, as a matter of fact) and two keys below it, go and back.

Bind go for left, back to right and we've got a pan-computer Alt-Tab.

Joy.

Using capistrano to deploy a python application

2008-07-05T15:46:00.001+01:00

At weListen we have a small python web application that we use as a time tracker and status keeper. Something akin to twitter, but with less social and more tracking. It lives in a Linux server together with a couple of our other services, and tends to be something I work on from time to time to improve small details or to fix small bugs.

Now, most of the time the development of this app is rather iterative. I think up of something new to try or a small improvement, code it on my local workstation, test it with live data taken from the production environment and if I'm happy with the results, I commit the change and deploy it on the production server. It's a small app, with a couple of services running, so the upgrade protocol is direct. It is still a couple of steps, and sometimes I tend to forget one of them (usually the one where I refresh the source code from subversion).

So, that looks like a good excuse to try out capistrano, of which I've heard many things, mostly coming from the ruby on rails community.

In a nutshell, what I wanted was for a library to take care of the connection and execution of commands on remote servers with minimal fuss. And that's what I got.

The Good

After installing the one click version, ruby was setup and ready to go. The gems library was already included, and installing capistrano was a matter of invoking the stanza featured on the project's home page.

Creating a script to perform the same steps I previously did manually was straightforward enough, and rather "obvious", apart from the trick to get sudo not to complain:

task :update, :hosts => "<server>" do
  default_run_options[:pty] = true # required so that sudo doesn't complain
  run "svn update ~/<directory>"
  sudo "/etc/init.d/<service 1> restart"
  sudo "/etc/init.d/<service 2> restart"
end

The server can use the standard username@host:post format, and the ssh framework plays nice with pageant, making passwordless logins updates real easy.

The Bad

No documentation, apart from a getting started tutorial, which is completely geared to deploying ruby on rails applications according to their methodology. Both seem to be a known problem, as a quick Google search for "capistrano documentation" gives us some pages with a call for help, and a mailing list post for it, and the getting started page warns us about it being devoted to ruby on rails.

Still, It leaves the rest of the world in the dark about how to use cap to deploy other kind of applications. Also, there isn't much in the way of explaining what is their methodology, and how can a user skip or customize some of the steps. I ended up just running the commands with no automatic error checking. Which works for now, but leaves me a bit unsure as to how solid it is.

The Ugly

Since I use a private/public key combo to login to the server with a non-root user (standard Linux sysadmin practice), and sudo requires a password by default, the script didn't work out as well as I wanted it to be on the first try. I could either type the password each time I wanted to deploy the app, get ssh to pass through my private key from pageant to sudo, or tell sudo that the services don't require a password.

I ended up compromising and going with the last choice. It means that an attacker which gains access to the server via the normal user can restart the services and do a bit of damage, but if he's already inside, then a couple of services going down is the least of my worries.

Conclusion

If I already had ruby installed, adding capistrano to the list of dependencies is not a big deal. The library is small and easy to install using gems. The problem was simple, and with the right tools the solution was equally direct. The lack of documentation wasn't that big a deal, but mostly because the problem was small.

Sadly, capistrano is the only reason for me to have ruby installed, so far, making the dependency a large and difficult one to explain. I might have a look in the future for python based alternatives.

Also, I'm starting to wonder if I could use the idea behind capistrano to deploy windows based applications. The biggest problem would be the remote connection, but I believe that Windows Server 2008 already has some support for console based remote connections. If this were possible, deployments could be more easily automated, which would most likely reduce the overhead of getting a new version of a web based application live.

The biggest disadvantage to a coffee machine at home...

2008-05-07T06:38:00.001+01:00

... is the period while you adapt to having a coffee machine at home.

The one where you learn when not to drink coffee.

The same one where you end up awake at 6am because you just had to get a cup of delicious, tasteful, awesomely scented java after dinner.

Live and learn...

Python, windows, sockets and Ctrl-c

2008-05-02T21:16:00.001+01:00

One of the minor annoyances of python on windows has to do with the sockets' blocking behavior.

Python's libs have several implementations of simple server loops, where the server listens for connections or specific requests and passes it on to a handler function on your code. To do so, the server blocks on an accept or read. Which, on Windows, means you can't ctrl-c out of a server process to test a code change.

To get around this little quirk I found a small workaround. We need two tools:

Powershell to launch the python script

PsKill to kill the python process

Stick an infinite loop outside the python script (like this "while (1) { server.py }"). Now, open another command prompt (I use Console for managing those) and, when you want to reload the server, just run "pskill python". Sadly this kills all python sessions currently running, which may not be what you want. But for now it works well enough.

My personal list of must-install apps

2008-04-28T15:31:00.001+01:00

For the umpteenth time I re-installed windows today on armitage (my desktop). For the first time I did it from a flash drive, since my dvd reader was on the fritz. I considered buying a new one, but after spending quite a lot upgrading the computer, I felt some effort could be spent in trying out the usb method. Much to my amazement it turned out to be quite easy. This link explained everything, and is actually quite straightforward. But that's not the point here, this is:

Synergy

Small application which allows one computer to control many via the network. A godsend for us with both laptop and desktop, an a desire to have more screens than a stock market analyst.

ClipX

Kill ring for windows. Before, I had to remember to get a notepad window, paste the clipboard I wanted to use afterwards, copy something, go back to notepad, rinse, wash, repeat. How in the 9 circles of hell did I live like that I don't know. But now that way is no more. ClipX remembers the copy operations you make, and has a pretty intuitive way to go back and forth.

Launchy

I understand the need for a start menu. I just don't understand why must I use it every single time I want to open a browser, visual studio or anything else. This one takes care of that. Not as good as Quicksilver, but it's good enough for me not to rip out my eyeballs out while trying to get stuff done.

Dexpot

The sanest virtual desktop manager for windows. Works extremely well almost all the time, and stays out of your way.

Dual core ftw!

2008-04-25T20:14:00.001+01:00

One of my small gripes with games are that you can't usually alt-tab out of them. This results in lossage of time when you happen to press a key which shows the start menu, or a modal dialog pops up (yes, I'm looking at you, windows firewall), or when you just need to pause out of a game to see some other thing.

Well, I think that's a problem no more!

Given a dual-core processor and the propensity of games to be monothreaded, I can now alt-tab out of a game, do what I must and return to it without having to spend half an hour waiting for it. On the other hand, 2 Gb of ram almost seems too little nowadays...

And for my next trick, lets Mingle

2008-04-23T17:59:00.001+01:00

Thoughtworks has recently released version 2.0 of their Agile Project Management tool, called Mingle. Apart from some new features and ways to manage the large amount of information a project usually produces and consumes, version 2.0 brings about one very interesting capability: REST APIs, accessed via http.

After spending about 1 hour before figuring out that "basic_auth_enabled: true" is different from "basic_auth_enabled:true", I was ready to at last work on a small app to use as a dashboard for a small "game" we play at weListen.

And what better way to do it than to try out the new features of .net 3.5 (and beyond), including LINQ and the ASP.Net MVC framework. Since this will be an internal project, it is an excellent chance to try them out.

For reference, here some example code to fetch the list of users from mingle:

     public static List<User> getUsers() {

        String url = "http://mingle.example.com/users.xml";

        HttpWebRequest req = (HttpWebRequest) WebRequest.CreateDefault(new Uri(url));

        String username = "john.doe";

        String password = "secret";

        byte[] credentials = Encoding.ASCII.GetBytes(username + ":" + password);

        String base64Credentials = Convert.ToBase64String(credentials);

        req.Headers.Add("Authorization",

                        string.Format("Basic {0}", base64Credentials));

        StreamReader reader = new StreamReader(req.GetResponse().GetResponseStream());

        var users = (from node in XDocument.Load(reader).Descendants("user")

                     select new User

                                {

                                    Username = node.Element("login").Value,

                                    Id = int.Parse(node.Element("is").Value),

                                    Name = node.Element("name").Value,

                                    Email = node.Element("email").Value

                                });

        return new List<User>(users);

    }

Most if it is rather straightforward, taking advantage of linq to iterate through the xml to fetch user's data and transform it to a business object. The User class here is just a container for the information, for now, and I use c#'s new property assignments in the constructor mainly to get a feel for it. Linq to me seems like a variant on python's list comprehensions, which is a good thing. It creates terse code, and abstracts away the cycle to focus on what you do with the information. To me this is a huge gain in c#'s expressiveness.

For me, the trickiest part was in getting the authorization correct. I now wonder if the problem was really with the code or mingle's not being correctly configured. I'll try a couple of other approaches to see if we can move away from having to encode manually the header and use .net's credentials' api.

Page fault accessing Mocks

2008-04-22T16:32:00.001+01:00

I realised how little I know about mocking, stubbing et. al. as soon as I read the quickstart for moq. Main problem was the domain language.

Expect? What the hell.. Does mock expect anything? Or do we expect something from mock? Perhaps I can gain some more insight by reading about other mocking frameworks...

Clicking a link about Rhino showed me some more code, and a clearer mental image started to form. Still, gained a faint image of the meaning of Expect, but had two more keywords I know nothing about: ReplayAll and VerifyAll.

Previously I had encountered and used the concept of a mock when dealing with tests and database connections. Not wanting to setup a whole database with test data just to make sure a method's implemented correctly, I ended up writing custom mocks which implement an in-memory version of the database, returning test values when certain methods are called. Useful? yes. Practical? no, not really. Different tests had different needs, and the choice became to either implement different mocks, or to have one mock support a bunch of tests. Not the most efficient way to do it, as it is now a bit clearer.

So, here's to a cup of juice, feet up and some light reading! I feel it is just the beginning...

Linux sysadmin'ing tip of the day (again)

2008-04-07T23:59:00.001+01:00

On the theme of clocks, virtual machines and just not getting it.

So you've got a spankin' new server, one with humongous disk space, lots of ram and cpus on the double digits. What's the first thing you think? "We can put all sorts of machines here and there'll still be space for a fool around linux to try new things".

Yes, the joys of virtualization. In this case, a centos machine running xen. Installation was simple enough, and after the first hour I already had a virtualized linux and was starting the process of installing a windows machine to serve as a development playground for everyone. By the second day I had already forgotten about the xen installation, since its purpose was being fully fulfilled as the host for the many (okay, just 3...) virtual machines installed on top.

Fast forward some weeks, and here I am trying to figure out why the bloody server is one hour ahead of time. NTPd is running, I can see the messages telling me that "yes, the time was a bit ahead, and we've got it right, now", but still the time was ahead one hour and it was not right. Damned time zones, I think! and so (naively, as it is clear to me now) I set the time zone from Europe/Lisbon to GMT in an attempt to make the system think he's one hour behind. And it works.

Until today.

After committing a couple of files to the server, and before packing on home, I check the integration server to see if everything is okay and the build is on its way to a green icon. Nop, still green. Last build time... yesterday. Strange. Picking through the logs I find out the server's not picking the latest change in the source. And that's when I notice that the commit emails fired from subversion come out one hour ahead. Again. Back I go, logging in to the server and trying to think up why the hell the hour kept going back.

Perhaps ntpd isn't working properly? Was it syncing to a bad server? Was the daemon not running correctly?

A peek in the logs shows ntpd trying to sync the time, time and time again without success, with no reason as to why it wasn't working. Okay, let's try and set the date by hand. Good, it works. And now it's back to normal. Hun?! Why... did the date... change by itself? Hardware clock?

Oh, wait... hardware clock... in a xen environment?

Yes, ladies and gentlemen, the answer right before our silly little noses. The hardware clock, in a xen virtualized environment, is managed by the host (dom0 in xen parlance), and unless specified otherwise in an obscure flag it stays that way, not allowing changes in the client environments.

And so, all it was needed was for the host machine to have a correct time and all was well in the land. Setting the correct date and time on the host machine sets also the correct date and time on the clients.

Linux sysadmin'ing tip of the day

2008-04-06T05:17:00.000+01:00

When you install a brand new linux machine, one of the things that is usually setup for you is a daily report of the system, sent by email to the root user.

If you're like me, you only notice it a month later when you've got a spool file with 30 emails or so.

And now, for the tip.

Two steps to solve the spool problem. Firstly, setup a .forward file on root's homedir with your email.

echo (your email) > ~/.forward

will take take of that quite nicely.

Test the config by sending a mail to root.

mail -s test root

will send the message as soon as you press Ctrl+d.

With the mail going to the correct address, the next step is to flush all those mail messages back through the mail pipe so you receive them on your email account.

formail -s /usr/lib/sendmail (your email) < /var/spool/mail/root

will do that for you. All that's missing if to clean up the queue, since you've got those messages out of the system.

cp /dev/null /var/spool/mail/root

is what you're looking for.

And that's all for today!

Useful surprise of the day

2008-04-05T20:29:00.000+01:00

With putty you can actually create new ssh tunnels on an existing connection by changing the settings of the connection and pressing apply. And here I was dropping and recreating the connection like an idiot...

Good customer service pays off

2008-01-10T02:21:00.001Z

Monday I went to a sports superstore to get some clothes as additional motivators for getting in shape. Expecting a supermarket of sports goods, I did most of the shopping by myself, getting a set of pants, some sweatshirts and eyeballing a GPS thingamajig which I set as a reward in a couple of months should I be good and reach my goals. But when I came to the shoes I actually was a bit undecided. Most of the other articles were rather straightforward. I had a clear picture of what I wanted, and the prices matched what I was willing to give. Not so much for a set of running shoes, which went from 20/30€ all the way to 120€. The adverts or informational flyers talked about how the lower priced items were for small walks or runs, once a week, in good conditions. The more expensive ones talked about competitive running, more frequently and in all conditions. So, instead of simmering over which ones I should pick, I called up on someone from the shop to ask what really was the difference and why were some marked as for once in a while jogging, and others more frequent running. To my (somewhat dumb) amazement, the clerk dropped what she was doing, and started explaining. So the whole frequency thing has to do with durability. If I'm going to run once in a while, less expensive shoes will do for a couple of years. But ramp up and those shoes last only months. Also, I should be on the look out for softer sole, which usually is better for jogging since it molds better to the motion of the foot. After fetching 3 or 4 pairs, the advantages of each were discussed, and the main points of how a set of shoes should be picked. I ended up buying the first and foremost recommendation of a brand I didn't knew about, in spite of trying both an adidas and a reebok pair. All in all, 10 minutes later and 5 pairs tried out, a client wishing to spend 30€ was convinced to spend 60€. And was happy about it. I got to try the different options, and was given good reasons for a more expensive buy from someone who was willing to spend the time to inform. The main point? Good, qualified, knowledgeable and, most of all, willing customer facing employees are worth their price, for the shop and for the client.