The Minimalist Coder

Google Contacts Synchronisation

2011-06-29T19:43:00.024+02:00

Having recently migrated to Google Apps at the organisation in which I work, one of the glaring omissions we discovered was the concept of a Global Address List to which I'd personally become accustomed in MS Exchange. Despite this persistently being at the top of the feature request list with Google, there still seems to be no easy way of sharing contacts between users, and so I decided to write some code to implement it programmatically.

The application simply copies all contacts from one account (which might be designated the "master") to another, flagging them as it does so in order that they can be deleted/updated when the program is scheduled to run regularly. As such, the destination account's existing contacts are never modified. This works within Google Apps, and also as a means of syncing contacts between regular Gmail accounts.

To obtain the reference assemblies, you'll need to get the GData .NET Client Library from http://code.google.com/p/google-gdata/, or I've included the relevant DLLs in the source download at www.cslacey.co.uk/blog/downloads/CSL.GoogleContactsSync.zip.

The latter link also contains the compiled application - to start using it straight away, simply enter the relevant credentials into the .config file within /bin/Release, and run CSL.GoogleContactsSync.ConsoleApp.exe.

(Assembly references: System.Configuration, Google.GData.Client, Google.GData.Contacts, Google.GData.Extensions)


using System;
using System.Collections.Generic;
using System.Configuration;
using System.Linq;
using Google.Contacts;
using Google.GData.Client;
using Google.GData.Contacts;

namespace CSL.GoogleContactsSync.ConsoleApp
{
   class Program
   {
      const int MAX_BATCH_SIZE = 100;
      const string GOOGLE_APP_NAME = "CSL.GoogleContactsSync.ConsoleApp";
      const string GAL_FIELD_KEY = "Origin";
      const string GAL_FIELD_VALUE = "Global Address List";

      static void Main(string[] args)
      {
         List<Contact> sourceContacts = GetSourceContacts(
             ConfigurationManager.AppSettings["SourceGoogleUsername"],
             ConfigurationManager.AppSettings["SourceGooglePassword"]);
         DeleteExistingGlobalContacts(
             ConfigurationManager.AppSettings["DestinationGoogleUsername"],
             ConfigurationManager.AppSettings["DestinationGooglePassword"]);
         InsertContacts(sourceContacts,
             ConfigurationManager.AppSettings["DestinationGoogleUsername"],
             ConfigurationManager.AppSettings["DestinationGooglePassword"]);
      }

      /// <summary>
      /// Retrieve all contacts from the specified account in a form
      /// suitable for insertion into other destination accounts
      /// </summary>
      private static List<Contact> GetSourceContacts(
          string username, string password)
      {
         List<Contact> result = new List<Contact>();

         RequestSettings requestSettings = new RequestSettings(
             GOOGLE_APP_NAME, username, password);
         ContactsRequest contactsRequest =
            new ContactsRequest(requestSettings);
         Feed<Contact> feed = contactsRequest.GetContacts();
         feed.AutoPaging = true;

         foreach (Contact contact in feed.Entries)
         {
            // Remove all group membership, as groups may not exist in
            // destination account
            while (contact.GroupMembership.Count > 0)
            {
               contact.GroupMembership.RemoveAt(0);
            }

            // Remove all user defined fields, as they may not exist in
            // destination account
            while (contact.ContactEntry.UserDefinedFields.Count > 0)
            {
               contact.ContactEntry.UserDefinedFields.RemoveAt(0);
            }

            // Flag retrieved contact as originating from the GAL
            contact.ContactEntry.UserDefinedFields.Add(
                new UserDefinedField(GAL_FIELD_VALUE, GAL_FIELD_KEY));

            Console.WriteLine(String.Format("RETRIEVING CONTACT: {0}",
                contact.Name.FullName));
            result.Add(contact);
         }

         return result;
      }

      /// <summary>
      /// Delete all contacts which have been flagged
      /// as originating from the GAL
      /// </summary>
      private static void DeleteExistingGlobalContacts(
          string username, string password)
      {
         RequestSettings requestSettings = new RequestSettings(
             GOOGLE_APP_NAME, username, password);
         ContactsRequest contactsRequest =
            new ContactsRequest(requestSettings);
         Feed<Contact> feed = contactsRequest.GetContacts();
         feed.AutoPaging = false;

         List<Contact> contactsToDelete;

         do
         {
            contactsToDelete = new List<Contact>();

            foreach (Contact contact in feed.Entries)
            {
               for (int i = 0;
                   i < contact.ContactEntry.UserDefinedFields.Count; i++)
               {
                  if (contact.ContactEntry.UserDefinedFields[i].Key
                      == GAL_FIELD_KEY
                      && contact.ContactEntry.UserDefinedFields[i].Value
                      == GAL_FIELD_VALUE)
                  {
                     // Contact originated from GAL, so add it to the
                     // delete batch
                     Console.WriteLine(String.Format("PENDING DELETE: {0}",
                         contact.Name.FullName));
                     contactsToDelete.Add(contact);
                  }
               }
            }

            Console.WriteLine("BATCH DELETING {0} Contacts",
                contactsToDelete.Count);
            contactsRequest.Batch(contactsToDelete, feed,
                GDataBatchOperationType.delete);

            // Re-intialise batch and feed for next execution
            contactsToDelete = new List<Contact>();
            feed = contactsRequest.GetContacts();
         }
         while (feed.Entries.Count() > 0 && contactsToDelete.Count > 0);
      }

      /// <summary>
      /// Insert the specified contacts
      /// </summary>
      private static void InsertContacts(
          List<Contact> contacts, string username, string password)
      {
         RequestSettings requestSettings = new RequestSettings(
             GOOGLE_APP_NAME, username, password);
         ContactsRequest contactsRequest =
            new ContactsRequest(requestSettings);
         Feed<Contact> feed = contactsRequest.GetContacts();
         List<Contact> contactsToInsert = new List<Contact>();

         foreach (Contact contact in contacts)
         {
            Console.WriteLine(String.Format("PENDING INSERT: {0}",
                contact.Name.FullName));
            contactsToInsert.Add(contact);

            if (contactsToInsert.Count == MAX_BATCH_SIZE)
            {
               Console.WriteLine("BATCH INSERTING {0} Contacts",
                   contactsToInsert.Count);
               contactsRequest.Batch(contactsToInsert, feed,
                   GDataBatchOperationType.insert);

               // Re-initialise batch for next execution
               contactsToInsert = new List<Contact>();
            }
         }

         Console.WriteLine("BATCH INSERTING {0} Contacts",
             contactsToInsert.Count);
         contactsRequest.Batch(contactsToInsert, feed,
             GDataBatchOperationType.insert);
      }
   }
}

Migrating Classic WCF to the Service Bus

2009-11-02T10:23:00.020+01:00

The Service Bus (within Microsoft's new .NET Services product) is a nice bit of technology that allows two-way communication between nodes on the Internet, regardless of the presence of firewalls, NAT gateways or other complicated network topologies. It makes possible all sorts of useful tools, such as Rich's and Rob's SocketShifter.

What's really nice about Service Bus's implementation, however, (and often overlooked) is that it's nothing more than a new set of WCF bindings which result in the creation of publicly reachable and discoverable endpoints.

With that in mind, I wondered whether I could take my previous simple WCF example, and change it to use the Service Bus by doing nothing more than modifying the config. In theory, it seems that this should be possible once the Service Bus becomes a standard part of the .NET framework - but for now it proves difficult. This is because installing the .NET Services CTP will modify your machine.config file such that netTcpRelayBinding cannot be added again as an extension within an app.config, but HAS to be added in that way for machines without the CTP installed (i.e. all of your customers' servers, or indeed an Azure VM).

The only sensible way to achieve this, therefore, is by explicitly setting options within the code itself - but this proved pretty easy.

Messaging Model

Let's start by modifying the code of the previous example slightly so that we emphasise the delivery of a message and a reply. We'll make the GetTimestamp method take a string, which gets printed at the server and copied back within the reply:

SharedTypes.IHealthCheck
...

[OperationContract]

string GetTimestamp(string serviceName);

Server.HealthCheck
...

public string GetTimestamp(string serviceName)

    Console.WriteLine("Request received from {0}", serviceName);

    return String.Format("{0}: {1}", serviceName, DateTime.Now.ToString());

Client.Program
...

while (true)

    Console.WriteLine(healthCheck.GetTimestamp("My service"));

Migrating to Service Bus

In both the Server and the Client, we now need to configure the endpoint as follows (If you don't yet have a .NET Services account, you can create one at http://www.microsoft.com/windowsazure/developers/dotnetservices/):

Specify a Service Bus address (sb://yourAccount.servicebus.windows.net/...)
Specify an appropriate Service Bus binding (here, we'll use NetTcpRelayBinding)
Specify a new behaviour which contains your Service Bus credentials

As mentioned before, the only safe way of doing this at the moment is programatically:

Server.Program

Configure the endpoint of the ServiceHost, just before opening it:

(Assembly reference: Microsoft.ServiceBus)

using System.ServiceModel.Description;

using Microsoft.ServiceBus;

...

static void Main(string[] args)

    // Construct EndpointBehaviour for SB username and password

    TransportClientEndpointBehavior serviceBusCredential =

        new TransportClientEndpointBehavior();

    serviceBusCredential.CredentialType =

        TransportClientCredentialType.UserNamePassword;

    serviceBusCredential.Credentials.UserName.UserName = "yourAccount";

    serviceBusCredential.Credentials.UserName.Password = "yourPassword";

    ServiceHost serviceHost = new ServiceHost(typeof(HealthCheck));

    // Set the binding manually (overriding the app.config)

    ServiceEndpoint serviceEndpoint = serviceHost.Description.Endpoints[0];

    serviceEndpoint.Address = new EndpointAddress(

        "sb://yourAccount.servicebus.windows.net/test/");

    serviceEndpoint.Binding = new NetTcpRelayBinding();

    serviceEndpoint.Behaviors.Add(serviceBusCredential);

    serviceHost.Open();

    // Keep the process running (hence the service open) for an hour

    System.Threading.Thread.Sleep(3600000);

Client.HealthCheck

Configure the endpoint of the ClientBase class we're inheriting from on construction, to ensure we only do it once:

(Assembly reference: Microsoft.ServiceBus)

using Microsoft.ServiceBus;

...

public HealthCheck()

    // Construct EndpointBehaviour for SB username and password

    TransportClientEndpointBehavior serviceBusCredential =

        new TransportClientEndpointBehavior();

    serviceBusCredential.CredentialType =

        TransportClientCredentialType.UserNamePassword;

    serviceBusCredential.Credentials.UserName.UserName = "yourAccount";

    serviceBusCredential.Credentials.UserName.Password = "yourPassword";

    // Set the binding manually (overriding the app.config)

    this.Endpoint.Address = new EndpointAddress(

        "sb://yourAccount.servicebus.windows.net/test/");

    this.Endpoint.Binding = new NetTcpRelayBinding();

    this.Endpoint.Behaviors.Add(serviceBusCredential);

That's it! If you fire up the server and the client, you should now see them working exactly as before - except you can now have the server running on your home machine behind a NAT gateway, and the client running on your office machine behind a corporate firewall!

Triple Boot MacBook

2009-06-29T17:52:00.019+02:00

Having bought a shiny new MacBook a few months ago, I got a little bored of OS X and wanted to make most use of its Intel chipset - to let me boot into Windows, OS X and Linux all on the same box.

There are quite a few long-winded, complicated explanations out there that describe how to achieve this, but actually it's quite simple once you know what to avoid. The following procedure has been tested using OS X Leopard (I'm assuming it's already installed), Windows XP Professional SP2 and Ubuntu 9.04.

1. Install the rEFIt Bootloader

Bootcamp seems to have problems working with more than two parallel OS's. So inside OS X, download and install rEFIt from http://refit.sourceforge.net, which is an alternative bootloader providing support for the Extensible Firmware Interface that Macs use.

Reboot after installing to check that you get the new rEFIt bootup screen, and choose to boot into OS X again.

2. Partition your drive

You'll need at least three partitions (one for OS X, one for XP, and one for Ubuntu). Ideally, though, you might choose to have an additional swap partition for Linux, and perhaps a shared data partition which you can access from all three OS's. You'll also probably already have your first partition defined as an EFI system partition - it's best not to fiddle with that.

Limitations to bear in mind are that the Ubuntu GRUB bootloader must be installed on the third partition, and XP on the fourth (due to their dependence on PC BIOS and the MBR, both seemingly have to be within the first four partitions, and XP must be as "low down" as possible). XP also seems to be unable to see any more than the first four partitions (as it doesn't support the GUID partition scheme), so make sure that your shared data is within the first four.

I found my ideal partition map to be as follows:

EFI - 200MB (untouched, as from the factory)
OS X - 50GB (the original OS X partition, shrunk but otherwise not modified)
Shared Data (plus GRUB bootloader for Ubuntu) - 20GB
XP - 50GB
Ubuntu - 25GB
Linux Swap - 2.5GB

In OS X, enter the command:

diskutil list

...and you should see that your existing disk structure looks something like this:

/dev/disk0
#: type name size identifier
0: GUID_partition_scheme *149.1 GB disk0
1: EFI 200.0 MB disk0s1
2: Apple_HFS Macintosh HD 148.9 GB disk0s2

Given that you want to resize the large partition and make lots of other smaller ones, you should then issue the following command (without linebreaks, and assuming my desired partition map):


diskutil resizeVolume disk0s2 50G
   "HFS+" "Data" 20G
   "MS-DOS FAT32" "Windows" 50G
   "HFS+" "Linux" 25G
   "HFS+" "Linux Swap" 2.5G

In actual fact, the formats selected here don't matter all that much, because both Windows and Linux will reformat the relevant drives upon installation. Make sure, however, that for the Data partition you initially pick a format that Windows can't understand - this will ensure that the Windows partition ends up as drive C upon installation! You can then reformat Data later (as MS-DOS FAT 32, being the only format all three OSs can read and write) using Terminal or Disk Utility in OSX.

3. Install XP

Insert your XP boot CD, and restart the machine while holding down the Option (alt) key. Choose the option to boot from CD.

Install Windows as normal, selecting the fourth partition (which should be labelled C:). I also chose to reformat the drive using NTFS, as this provides better performance than FAT32.

As Windows needs to restart the machine a couple of times during this process, ensure that each time you select the "Windows" option which should appear in the rEFIt menu, to allow the installation to carry on.

Once this is complete, install the relevant Windows drivers for the Mac hardware using the "Boot Camp" installer for Windows (on the Mac OS X Install DVD).

4. Install Ubuntu

Insert a bootable Ubuntu CD, and again restart the machine while holding down the Option (alt) key. Choose the option to boot from CD.

Install Ubuntu as normal, selecting manual configuration of partitions when prompted. On the subsequent screen, you should select /dev/sda5 to be formatted with ext3 (my preference), and mounted as "/". Assuming my partition map, you should also select /dev/sda6 to be formatted as a swap partition.

Important: On the final screen of the installer, select 'Advanced...' and change the location of the GRUB bootloader to be /dev/sda3. This is important, as otherwise Ubuntu's bootloader will attempt to take over the whole disk, which can have some odd results working alongside rEFIt.

5. Celebrate

That's it! The only thing that doesn't work for me (bizarrely) is restarting from within Ubuntu... I have to make sure I only ever choose "Shut Down"!

Obfuscating QueryString Parameters

2009-01-22T09:22:00.016+01:00

I need to provide a link to a URL which contains some parameters I'd like to obfuscate. Specifically, within an automatically generated Email, instead of embedding a link /sendMessage.aspx?address=foo@bar.com&expiry=2009-01-22Z13:30 , I'd prefer /sendMessage.aspx?token=s0MEth1ngUnfath0mable!

Broken down, my requirements for obfuscation are:

Serialize an object which contains my parameters (address and expiry), so that it can be transmitted to a different server (i.e. the web server)
Encrypt the serialized bytes to prevent tampering
Make Quotable the encrypted bytes so that they can be included within a URL querystring (albeit after a suitable UrlEncode)

Key Generation

My security requirements are pretty basic, as I'm not trying to protect anything desperately important here - rather prevent basic tampering by spotting patterns in the quotable string. Therefore, I'll keep things as simple as possible and take some shortcuts, for example using a basic security algorithm (DES), and embedding the encryption key within my source code.

Firstly, we'll need to generate a key upon which the encryption and decryption can occur:

using System;

using System.IO;

using System.Security.Cryptography;

namespace ObfuscationTesting

    class Program

        static void Main(string[] args)

            // Generate a suitable key

            DESCryptoServiceProvider desCryptoServiceProvider =

                new DESCryptoServiceProvider();

            desCryptoServiceProvider.GenerateKey();

            // Represent key as a quotable string

            string base64EncodedKey =

                Convert.ToBase64String(desCryptoServiceProvider.Key);

            // Save to file

            File.WriteAllText(@"C:\key.txt", base64EncodedKey);

Obfuscator Class

Now, we'll create a static class that takes any serializable object and returns us a string which represents that object (and, naturally, allows us to re-create the original object from such a string):

using System;

using System.IO;

using System.Runtime.Serialization.Formatters.Binary;

using System.Security.Cryptography;

namespace ObfuscationTesting

    public static class Obfsucator

        // The key previously generated

        private const string KEY = "MXvUAmUobjA=";

        public static string Obfuscate(object targetObject)

            // Binary serialize

            MemoryStream memoryStream = new MemoryStream();

            BinaryFormatter binaryFormatter = new BinaryFormatter();

            binaryFormatter.Serialize(memoryStream, targetObject);

            byte[] serialized = memoryStream.GetBuffer();

            // Encrypt

            DESCryptoServiceProvider desCryptoServiceProvider =

                new DESCryptoServiceProvider();

            desCryptoServiceProvider.Key = Convert.FromBase64String(KEY);

            desCryptoServiceProvider.IV = Convert.FromBase64String(KEY);

            byte[] encrypted = desCryptoServiceProvider

                .CreateEncryptor()

                .TransformFinalBlock(serialized, 0, serialized.Length);

            // Render as string

            string quotable = Convert.ToBase64String(encrypted);

            return quotable;

        public static object Deobfuscate(string quotable)

            // Retrieve the bytes from the quotable string

            byte[] encrypted = Convert.FromBase64String(quotable);

            // Decrypt

            DESCryptoServiceProvider desCryptoServiceProvider =

                new DESCryptoServiceProvider();

            desCryptoServiceProvider.Key = Convert.FromBase64String(KEY);

            desCryptoServiceProvider.IV = Convert.FromBase64String(KEY);

            byte[] serialized = desCryptoServiceProvider

                .CreateDecryptor()

                .TransformFinalBlock(encrypted, 0, encrypted.Length);

            // Deserialize

            MemoryStream memoryStream = new MemoryStream(serialized);

            BinaryFormatter binaryFormatter = new BinaryFormatter();

            object deserialized = binaryFormatter.Deserialize(memoryStream);

            return deserialized;

Usage

We'll create a class which contains the parameters I'd like to represent, remembering to mark it as serializable:

using System;

namespace ObfuscationTesting

    [Serializable]

    public class Token

        public string EmailAddress { get; set; }

        public DateTime Expiry { get; set; }

To construct the URL with the obfuscated parameters, all we need to have is:

(Assembly reference: System.Web)

using System;

using System.Web;

...

            Token token = new Token()

                EmailAddress = "foo@bar.com",

                Expiry = DateTime.Now.AddHours(1)

};

            string url = String.Format(@"/sendMessage.aspx?token={0}",

                 HttpUtility.UrlEncode(Obfsucator.Obfuscate(token)));

And, on our web page that receives the URL, we just need to de-obfuscate again:

            string tokenString = Request.QueryString["token"];

            Token token = (Token)Obfsucator.Deobfuscate(tokenString);

            if (token.Expiry < DateTime.Now)

                // Token has expired

                Response.End();

            else

                string toAddress = token.EmailAddress;

                // etc

Self-Hosting WCF

2009-01-14T14:14:00.003+01:00

I'd like to have two separate processes (both under my control) communicate over WCF. What's the easiest way to achieve this? As an example, I'd like Process 1 (a client) to be able to check the health or presence of Process 2 (the server) by periodically requesting the server's local time.

To permit communication, we need to define a contract between the two processes. All I'd like this contract (let's call it HealthCheck) to define is an operation GetTimestamp. Given that my two separate processes will be two separate projects in Visual Studio, and that they both need to access this contract, we'll create a third class library called SharedTypes that will contain nothing but the contract.

SharedTypes (contains the contract)

Assembly references: System.ServiceModel

We'll define the contract as an interface, decorated by System.ServiceModel.ServiceContract. The interface's one method signature (GetTimestamp) should be decorated by System.ServiceModel.OperationContract.

using System;

using System.ServiceModel;

namespace SharedTypes

    [ServiceContract]

    public interface IHealthCheck

        [OperationContract]

        DateTime GetTimestamp();

Server

Assembly references: SharedTypes, System.ServiceModel

The server process, perhaps unsurprisingly, will need to implement this contract interface as follows:

using System;

using SharedTypes;

namespace Server

    class HealthCheck : IHealthCheck

        public DateTime GetTimestamp()

            return DateTime.Now;

The program's main method will simply instantiate and open a ServiceHost to expose the service, and hang around in the background:

using System;

using System.ServiceModel;

namespace Server

    class Program

        static void Main(string[] args)

            ServiceHost serviceHost = new ServiceHost(typeof(HealthCheck));

            serviceHost.Open();

            // Keep the process running (hence the service open) for an hour

            System.Threading.Thread.Sleep(3600000);

Finally, we'll need to specify the address and protocol by which the service can be called. Part of the joy of WCF is allowing this to be configurable by an end-user or sysadmin, so we'll specify a minimal app.config to contain the core settings we require:

<?xml version="1.0" encoding="utf-8" ?>

<configuration>

   <system.serviceModel>

      <services>

         <service name="Server.HealthCheck">

            <endpoint address="basic" binding="basicHttpBinding"

                      contract="SharedTypes.IHealthCheck" />

            <host>

               <baseAddresses>

                  <add baseAddress="http://localhost:8080/HealthCheck" />

               </baseAddresses>

            </host>

         </service>

      </services>

   </system.serviceModel>

</configuration>

Client

Assembly references: SharedTypes, System.ServiceModel

The client will also need to have a class that implements the service contract, but we'll make it such that calls to its methods actually result in a remote call over WCF:

using System;

using System.ServiceModel;

using SharedTypes;

namespace Client

    public class HealthCheck : ClientBase<IHealthCheck>, IHealthCheck

        public DateTime GetTimestamp()

            return base.Channel.GetTimestamp();

The program's main method will just sit in a loop calling and echoing the result from GetTimestamp:

using System;

namespace Client

    class Program

        static void Main(string[] args)

            HealthCheck healthCheck = new HealthCheck();

            while (true)

                Console.WriteLine(healthCheck.GetTimestamp());

                // Wait for a second

                System.Threading.Thread.Sleep(1000);

Again, we'll need to specify in app.config the location of the server, and the protocol we want to use:

<?xml version="1.0" encoding="utf-8" ?>

<configuration>

   <system.serviceModel>

      <client>

         <endpoint address="http://localhost:8080/HealthCheck/basic"

                   binding="basicHttpBinding"

                   contract="SharedTypes.IHealthCheck" />

      </client>

   </system.serviceModel>

</configuration>

Introduction

2009-01-14T13:47:00.001+01:00

History, or at least working within my company, has shown that the simplest, most basic implementations of code stand the most chance of being correct. As soon as code is littered with endless if-then-else scenarios or numerous configuration options, it becomes increasingly difficult to spot the core logic, and inevitably, the bugs or inconsistencies.

Working in the Microsoft space, I have become increasingly frustrated with the typical low signal-to-noise ratio of documentation, samples and articles about Microsoft technologies. Exacerbated by the fact that Microsoft developers (or more specifically Visual Studio developers) have become ever-more dependent on code generators, hidden-away partial classes, and bloated configuration files, there is a tendency for code examples to be little more than screenshots showing which buttons should be pushed within Visual Studio, and which of the configuration options should be tweaked in the sea of automatically generated XML.

The code samples I plan to show in this blog will attempt to represent the simplest implementations of particular technologies. Paring back the unnecessary bloat from Visual Studio's code generators and from the multitude of code samples on the web is often a fairly time-consuming task, and the primary purpose of these posts is to record any outcomes as notes to myself. Nonetheless, I hope they might also prove useful to others.