Friday, 15 April 2011

Error 2 when trying to start windows event log service

Today my Windows 7 machine couldn't start the windows event log service, and so I couldn't open the event log viewed.

After a bit of troubleshooting I found out that the issue seemed with a particular registry key.

Removing the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\eventlog\parameters registry key allowed event log to start without problems.

Monday, 4 October 2010

Zero-friction deployment: from local to github in one powershell command

Short version:

Code is here, depends on a dll here and here. Allows for a powershell script to upload a file to github.

Long version:

There are one core belief powering this post:

- There should be zero friction in getting a software release out in the open. In the age of build scripts and APIs and everything over http there is no reason a human should be involved in cutting a version except in the decision to make it so.

The main side effect of having very little to no friction in cutting a release is that it happens way more often, since there is no reason not to do it if the code is good to go.

In this particular case, all I lacked was a way to upload a file to github. The packaging was already done, courtesy of psake+write-zip. Having found nothing in the powershell world to do it, I ended up “porting” part of a ruby script.

Github stores files in Amazon’s S3, so there were two steps to uploading a file:
-Telling github that the file exists and getting a token to pass to S3.

-Uploading the file itself to S3 using the token gotten from step 1.

The biggest issue ended up being that WebClient doesn’t handle POST’ing to an url with MIME values, which is what S3 expects in this scenario. Using the underlying *thingie* directly from powershell would bit a bit harder and more prone to errors, so I just used an existing helper to upload the file correctly.

The powershell code that uses it is available here, with another extra dependency on the Html Agility Pack to scrape the downloads page for info on existing downloads.

The function itself (not iet a full commandlet due to lack of time) is upload_file_to_github:

function upload_file_to_github($login, $repo, $api_key, $file, $filename, $description){
    [void][System.Reflection.Assembly]::LoadFrom((get-item "lib\Krystalware.UploadHelper.dll"))
   
    $full_repo = $login+"/"+$repo
    $downloads_path = "http://github.com/"+$full_repo+"/downloads"
   
    $post = new-object System.Collections.Specialized.NameValueCollection
    $post.Add('login',$login)
    $post.Add('token',$api_key)
    $post.Add('file_size',$file.Length)
    $post.Add('content_type',"application/octet-stream")
    $post.Add('file_name',$filename)
    $post.Add('description',$description)
    $wc = new-object net.webclient
    $upload_info = [xml][System.Text.Encoding]::ASCII.GetString($wc.UploadValues($downloads_path, $post))
   
    $post = new-object System.Collections.Specialized.NameValueCollection
    $post.Add('FileName',$filename)
    $post.Add('policy',$upload_info.hash.policy)
    $post.Add('success_action_status',"201")
    $post.Add('key',$upload_info.hash.prefix+$file.Name)
    $post.Add('AWSAccessKeyId',$upload_info.hash.accesskeyid)
    $post.Add('signature',$upload_info.hash.signature)
    $post.Add('acl',$upload_info.hash.acl)
   
    $upload_file = new-object Krystalware.UploadHelper.UploadFile $file.FullName, "file", "application/octet-stream"
    [void][Krystalware.UploadHelper.HttpUploadHelper]::Upload("http://github.s3.amazonaws.com/", $upload_file, $post)
}

As you can see, it’s just a normal POST to the web to notify github of a new file, interpreting the response as xml to extract needed information for the S3 POST. That one uses the helper to pass the parameters as MIME-encoded values.

I have to say, using xml in powershell was a lot easier than I thought it would be. I wonder if JSON is equally well supported…

Monday, 27 September 2010

Blaze-IPP – Design choices and implementation

When I set out to create the support to code commands in IronPython, there were two main guidelines in my mind: simple commands and fast feedback.

Simple commands

Commands should only need to be a name and the definition of execute for the simplest of cases. No imports, just the code:

This is the simplest way to define a command as of version 1.2. As you can see, there’s nothing that isn’t required, and simple shortcuts or expanders could be defined by a simple method

The side effect is that even short or small tasks can be easily automated since there’s no ceremony.

There are two main features to accomplish this guideline:

  • Incrementing the scope with the extra classes and namespaces.
  • Some “smarts” to allow for both methods and classes to be commands.

Adding definitions to a scope is simple, since all we need to do is associate a value to a symbol:

ScriptScope scope = _engine.CreateScope();
 
scope.SetVariable("BaseIronPythonCommand", ClrModule.GetPythonType(typeof(BaseIronPythonCommand)));
scope.SetVariable("UserContext", UserContext.Instance);

Allowing for both classes and methods is also trivial, since we can filter the values on the scope by type after executing code.

Anything that’s a PythonType for which the CLR type can be assigned to an IIronPythonCommand is a command class.

Anything that is callable and a PythonFunction that doesn’t start with “_” is a command method.

Fast feedback

I shouldn’t need to reload the application just to pick up a new script or a change to an existing one. Blaze should pick up changes from the file system and reload the files as needed.

Here the big issue was locked files when the “file changed” was triggered.

This meant that changed files were placed in a queue, and a timer running in the background pulled files from the queue and reloaded them. If a file is locked, then put it again in the queue.

Monday, 20 September 2010

Forked dbdeploy.net

Dbdeploy.net is a database change management library, used by us at weListen to track the changes made to a database in the scope of a project and apply them as needed. It takes a folder full of ordered files containing the change scripts in sql and applies them as needed to a given database. It tracks which changes have been applied in a special table.

We’ve used for quite some time, and were mostly happy with it, except when using it from the command line on the servers. The msbuild task worked well to obtain and apply the changes, but the command line application added extra information to the output that needed to be trimmed before applying it to the server.

Well, it annoyed me enough to fork it from its home at sourceforge to a new place at github.

I refactored to projects a bit, to exclude a direct dependency on NAnt, and added a Dbdeploy.Powershell assembly with 3 commands:

  • Export-DbUpdate, which outputs the scripts that need to be applied. Can write the scripts directly to a file;
  • Push-DbUpdate, which applies the changes to the database (still has a bug when using with SQL Server, as I need to split the resulting script into batches);
  • Select-DbUpdate outputs information about which sets need to be applied;

All of these commands take a path to a config file describing the database type, connection string and table name to store the changelog, and the directory where the actual changesets exist.

Also, instead of mucking about with NAnt as the build script host, I’m using psake. I really can’t stand using xml to describe build steps anymore.

To use it just download the release, and in powershell “import-module Dbdeploy.Powershell.dll”.

Monday, 6 September 2010

Anatomy of a command

Creating an IronPython command to launch the command prompt isn’t much, since you can have a shortcut to do just that.
You might have noticed that the command subclasses “BaseIronPythonCommand”. This is an abstract class that implements the basics and just leaves the Name and the Execute method for you to implement:
image
There’s also an interface you can implement directly (IIronPythonCommand) if you want to assume control of everything.
BaseIronPythonCommand’s implementation is quite simple. Most of it is boilerplate for a Blaze plugin, leaving only the name and execution to be done in the script:

GetName is the name used to invoke the command. It should be something unique and descriptive. A long name is good, and you can teach Blaze a shortcut to it by typing the shortcut and selecting the command on the dropdown box. This teaches blaze about the shortcut.
GetDescription returns a description for the command, to be shown on the line below it on the interface.
AutoComplete returns a completion for the given string. This means you can transform “off” into “office” inside your command.
Execute tells the command the user has selected it. The command returns either a program to execute in the form of a path concatenated with the arguments, null if the command has executed itself.
Any one of these can be implemented in ironpython, although for basic commands you only need to implement GetName (or a property called Name) and Execute.

Wednesday, 1 September 2010

Tutorial: How to create your own plugins for Blaze-IronPythonPlugins (BPP)

On the first post in this series I presented Blaze IPP, which allows for scripting of a launcher using IronPython. Here I tell you how you can write your own plugins from scratch. It will be a short post.

First of all, I assume you’re already running Blaze with the plugin installed.

Let’s create a new directory under “Plugins” to store our new plugins. This way if any future upgrade on IPP brings an ironpython file with the same name, your plugin will not be overwritten. We’ll call it LocalIronPythonPlugins:

Untitled

Now, configure IPP to also monitor that directory. Call Blaze, right click on it and in the “settings” open the “Plugins” tab. Select “IronPythonHostPlugin”, “Configure” and add the newly created directory (you can either browse to it with “b” or you can paste the path into the text box):

Untitled2

Now that we have a clean directory to work with, let’s create a file called “OpenCommandLine.ipy”. Open it and add the following code:

As you can probably guess, this will create a new command named OpenCommandLine, and when you select it it will open a new command prompt. IPP is clever enough to know that if a command returns a string, it is a program to be executed.

This is just a simple example to get you started. Since this is IronPython, you can use the entire .Net framework and fetch things from the web, open the registry, connect to a remote host or start a service, to give a couple of examples.

Monday, 30 August 2010

Creating plugins in IronPython for Blaze, an application launcher.

Everyone loves the start menu from Windows Vista/7, but compared to Launchy, Quicksilver or any other application launchers it’s a bit anemic. No wonder, since it’s just a search over your start menu and indexed locations.

But even any one of those is a bit lacking in support for runtime extensibility. You can create plugins, but it involves C (or Objective-C in the case of Quicksilver) and restarting the app. Not very friendly for those cases where you just want to pick up the currently selected object and throw it over an scp connection autocompleted from your favorites.

Enter Blaze + IronPythonPlugins. Blaze is a .Net application launcher for Windows that includes a learning facility that tries to complete repetitive tasks for you. IronPythonPlugins is a plugin itself for Blaze that hosts simple plugins written in IronPython.

If you want to try it just download the latest version from here. It includes a version all ready to use. Start the app, call it with Ctrl+Alt+Space and try a couple of plugins:

  • Open windows explorer, select a text file, call blaze and type “edit with notepad2”. If you have notepad2 in your path, it will be launched with the selected file opened.
  • If you use putty (the ssh terminal), “putty <name of session>” will launch putty with that session. Here session is autocompleted, so if you have a session called “my.home.box”, typing “putty home<tab>” will autocomplete to “putty my.home.box”.
  • Type “random-text <number>” and your clipboard will have up to <number> random letters. Useful to generate test data.

There are a couple more plugins in the “Plugins\IronPythonPlugins” directory, so if you’re interested in how it’s done, open them in your editor of choice. In the next few days I’ll do a couple of posts on how you can create your own plugins without having to guess from the already existing ones.

If you want to change anything edit the file, save it and Blaze will automatically pick it up, together with any other .ipy file in that directory that has changed. If you open Blaze’s configuration page for the plugin, you can add other directories to search for .ipy files. Useful to keep your personal plugins separated from the “official” ones.

Sadly errors are still only sent to a System.Diagnostics.Debug, so if you edit a file and can no longer use it, open DbgView, edit the file again to force blaze to reload it and be on the lookout for output starting with “Error with file”. There will be a stacktrace telling you what you did wrong.

So, try it, have fun and comment on it. I’d love to hear what other people think of it.