Russell Hay

Disclaimer: I work for Tableau Software as the Lead Software Developer on the Test Engineering Team. These are my impressions of the conference, and do not necessarily reflect the opinions, or thoughts of Tableau Software

Since I started at Tableau in April of 2011, I've heard stories about our customers, how enthusiastic they are about our software, and how great the customer conference is. As a perpetual skeptic (though some might say pessimist), I am wary when a large group of people generally say the same things, almost verbatim.

Naturally, I approached this week with my usual mix of anxiety of having to interact with strangers (something I still have to fight) and trepidation of the unknown that has been built up to be this grandiose thing.

But as I sit in the airport reflecting on my week in Vegas for TCC11, my skepticism was unwarranted. The conference was amazingly fun, and being able to interact with the customers was, hands down, the #1 reason that I enjoyed myself, thoroughly.

My job as art of the conference was as a Tableau Doctor, which is a mix between a rockstar (oh did I just say that? It is Vegas after all) and customer support. The idea was that customers and potential customers got a chance to sit down with someone and have their questions answered.

And these weren't just simple questions like "how do I make a line graph with these data points" (mainly because Tableau Desktop makes that so ridiculously easy that people don't need help with that).

The questions that I got were more along the lines of one of the following:

• I'm a data analyst who previously used Product X (many times it was Excel) and my consumers want a chart that looks just like this chart that has been taking me weeks or months to create.
• I'm an IT guy and I want to get out of the business of report writing for these analysts who keep changing their mind about what data they want! Help! (no slight there on data analysts. IT guys and data guys approach the world differently, data leads to more questions which require different data to answer, it goes with the territory!)
• Here is what I've got so far, but this ONE data set isn't right because it includes this information which I don't want here but need it here (usually pointing and switching back and forth between crosstabs and charts and their original data source

I don't claim to be an expert in anything I do because I know other people know more than me, have more skill than me, or just generally are smarter than me, and I have only used Tableau for about 6 months, but the sheer joy of working through these problems, even when I didn't know the answer directly made it all worth it for me.

The greatest thing I heard was customers thanking me and saying that their time with me made the entire conference worth it. Impact on real customers. Wow, in 15+ years in the computer industry, this was the first time I've really, truly, felt that.

Additionally, Tableau knows how to throw a party. The Conference was held at the Encore Hotel, with a private customer party held in their beach club, where we got to talk for socially with customers and just get to know the people behind the questions and data. And I even had a pole-off with one of our customers (I lost. He was much better than me.)

All in all, the experience was wonderful, and lived up to all of the stories I heard about the previous one. If I ever had doubts that Tableau was the place for me, this experience has solidified that I made the right decision, that I'm working with people I can call family, and customers I can truly appreciate.

At the end of each year, I reevaluate my photography workflow and naming convention to see if I can make improvements.  After reviewing 2010's process, I've decided I need to be a little more regimented in the process I take from Camera to Delivery.  As part of that, I'm going to create a workflow application that helps automate the workflow with prompts and some automation steps.

While Microsoft does provide Windows Workflow Foundation Classes which are designed to implement a workflow, I found it to be a bit more than I needed for my project, so I've created a simple, generic Finite State Machine that only has actions on the state itself (no transition actions at this time).

Wikipedia defines a Finite State Machine as a behavior model composed of a finite number of states, transitions between those states, and actions, similar to a flow graph in which one can inspect the way logic runs when certain conditions are met.  In the case of my implementation, the conditions are basically calling the Transition method on the StateMachine object.

Example Usage

For this example, I'm going to build a soda machine state machine, which is a classic example for designing state machines.  The below diagram shows the visual representation of what we will be building.  As you can see, we have a start state (Idle), a number of other states, some unconditional transitions and some conditional transitions.  The only one we can't really implement is the Coin Inserted transition, so we'll just assume that coin inserted is triggered in some way and that the value of the coin set somehow on the state "Increment Paid Amount".  Other than that, the rest of the state machine is easy to implement.

First, we'll need to define our transitions.  We have a couple of different transitions that are necessary, so let's create an enum:

public enum SodaTransitions {
        Start = int.MinValue,
        Idle = 0,
        Always,
        CoinInserted,
        TotalAmountNotEnough, // Total Amount < 1
        TotalAmountEnough, // Total Amount >= 1
        ChangeLeft // TotalAmount &amp;gt; 0
}

That's all we need for the transitions.  The next thing we need to do is create the Transition Handler, which implements the IActionHandler<T> interface where T = SodaTransitions.  It would be something like the following:

public class SodaHandler : IActionHandler<SodaTransitions> {
        public bool IsTerminal(SodaTransitions transition) { return false; } // We don't have any terminal transitions
        public SodaTransitions Start() { get { return SodaTransitions.Start; } }
        public IStateMachine<SodaTransitions> StateMachine { get; set; }
}

The next step is to create all of the states involved in the state machine.  This is the most time consuming part, and we won't be creating all of them in this example.  To make things easier on yourself, I would suggest defining an abstract base class for all of your states that implements the IState<T> interface as abstract functions.  This will save you from a lot of typing if you keep this name short.  Something like the following would suffice:

[csharp]

abstract public class BaseState: IState<SodaTransitions>

{

        abstract public void Run(IStateMachine<SodaTransitions> machine);

}

[/csharp]

This will save you from having to type IState<SodaTransitions> a lot.

So all you need to do for the states is to define a run state.  Super easy!  I'll implement WaitForSelection state below, so you can see how to do conditional states.  Keep in mind that a state's run method does not need to call Transition.  If you don't, the state machine will be suspended until transition is called.  This is great for user interaction.

public class WaitForSelectionState : BaseState {
        override public void Run(IStateMachine<SodaTransitions> machine) {
                if (CoinCounter.TotalAmount >=1)
                        machine.Transition(SodaTransitions.TotalAmountEnough);
                else
                        machine.Transition(SodaTransitions.TotalAmountNotEnough);
 
        }
}

It's that simple.  Most states will be about this simple if your state machine is well defined.

Now, let's talk about the construction of the StateMachine because that is the most complicated part.  We need to first create the StateMachine and then add all of the transitions, so let's do that:

// Create the state machine
IStateMachine<SodaTransitions> MyMachine = StateMachine.Create<SodaTransitions, SodaHandler>
 
// Populate the state machine
MyMachine.AddTransition(null, State<IdleState, SodaTransitions>.Instance, SodaTransitions.Start);
MyMachine.AddTransition(State<IdleState, SodaTransitions>.Instance, State<IncrementPaidAmountState, SodaTransitions.CoinInserted>);
// ...
MyMachine.AddTransition(State<WaitForSelectionTransition, SodaTransitions>.Instance, State<SubstractCostState,SodaTransitions>.Instance, SodaTransitions.TotalAmountEnough);
MyMachine.AddTransition(State<WaitForSelectionTransition, SodaTransitions>.Instance, State<RequestMostState, SodaTransitions>.Instance, SodaTransitions.TotalAmountNotEnough);
// ...

I'll go over the State<T,TT> part below, so just ignore that for now and just read that as the state involved.  First we add the start state.  This is the state where the currentState is null, and the nextState is something.  The transition for this MUST be what is returned by SodaHandler.Start.  After that, you wire it up just like the diagram says.  If you have multiple transitions coming from a single state, just set the currentState to the same state multiple  times.  The key is that every currentState/Transition pairing must be unique.  It does not handle parallel states.  If we did support parallel states, we could have done the dispense soda and dispense change states in parallel, but that isn't supported right now (maybe future improvements if anyone requests it)

And now for the State<T,TT> part.  This is just a generic singleton wrapper that is included with the StateMachine code for your use.  You can use it or not, but the main key is that the states need to remain the SAME object if you have multiple transitions coming out of it, so I choose to always use the state unless it's a generic step like displaying a message or something.

So that's it.  You can pull the code from GitHub in my code library repository for StateMachine

Here is a problem that I've been discussing with people at work, and my solution for it.  We want to be able to store a bunch of dlls (and other binary files) inside of a resource and then, at run time, expand the files, and reference them from within the application.  I solved the problem a little differently than the original question since my focus was on the auto-update scenario where we want to be able to completely replace the current running program with a new version.

The first thing we need to do is get a mapping of the data inside of the resource and the names associated with them.  To make this discovery easier, we had to set a naming convention for the resources.  So, since we are concentrating only on dlls and the symbol files, we assume the following naming convention: <assemblyname>_dll and <assemblyname>_pdb.  These map directly to the assembly and the symbol file for the dll, respectively.

Here is the important bits for enumerating the resources and storing their byte streams:

        private void ReadResources()
        {
            resources = new Dictionary<string, byte[]>();
            string resourceName = PackageAssembly.GetManifestResourceNames()[0];
            using(Stream stream = PackageAssembly.GetManifestResourceStream(resourceName))
            {
                if (stream == null) return;
 
                using (ResourceReader reader = new ResourceReader(stream))
                {
                    GetResourcesFromReader(reader);
                }
            }
        }
 
        private void GetResourcesFromReader(ResourceReader reader)
        {
            IDictionaryEnumerator en = reader.GetEnumerator();
            while(en.MoveNext())
            {
                string name = (string)en.Key;
                byte[] value = (byte[])en.Value;
 
                resources[name] = value;
            }
        }

Going through this, ReadResources will open the resource stream from the manifest of an assembly that has been loaded.  Elsewhere in the class, we open PackageAssembly using a standard LoadAssembly call.  Once we have the stream open and a ResourceReader attached to it, we create a Dictionary enumerator.  This is the old style dictionary where everything (key and value) are just object.  This isn't a dictionary using generics so that you have strongly-typed values and keys in the dictionary.  So the next thing we do is basically convert it from the object,object dictionary into a string,byte[] dictionary which is what GetResourcesFromReader does.  After doing that, we now have a dictionary of the resource name mapping to the contents of the resource.  Easy enough to step through and save each of these to a file.

However, we want to be more clever than that.  What's the point of writing them to a file when all we want to do is load them back into memory.  Now, I'm sure an astute observer will notice that means that there are three copies of each resource assembly in memory: 1 from the package assembly, 1 from the resource "cache/mapping", and a third from the actual assembly loaded.  This is an issue that I'm still solving.  The way I think it can be solved is by spinning up a couple of AppDomains, and then loading things into the AppDomain, and then shutting down the AppDomain once done with it.

Another important piece to answer is "what do you do with this information now that it's cached in a dictionary mapping the resource name to the actual data.  There are a couple of approaches, but the easiest is to utilize .NET's ability to do custom assembly resolving via the AssemblyResolve event on the AppDomain.  First let's show how we find and load the Assembly by name if it exists in the package assembly:

        public Assembly FindAssembly(string name)
        {
            string correctedName = name.Replace(".", "_");
            return ExistsInResources(correctedName) ? GetAssemblyFromResource(correctedName) : null;
        }
 
        private Assembly GetAssemblyFromResource(string name)
        {
            byte[] coffData = resources[string.Format("{0}_dll", name)];
#if DEBUG
            byte[] symbolData = resources[string.Format("{0}_pdb", name)];
            return Assembly.Load(coffData, symbolData);
#else
            return Assembly.Load(coffData);
#endif
        }
 
        private bool ExistsInResources(string name)
        {
            return resources.ContainsKey(string.Format("{0}_dll", name));
        }

Basically, we load the coff and the pdb from the resource dictionary that we created.  We have a conditional compile so we only load the symbols if it's a debug build.  It shows how to do both, and you can pick one or the other, or just do the conditional compile as I do above.

Now, how do we use this?  Simple, after creating the initial ResourceAssemblyLoader instance and loading up the package assembly, you just need to hook up your AppDomain to use this class to resolve assemblies.  Like so:

        private static void SetupPackageLoader()
        {
 
            loader = new ResourceAssemblyLoader("ModelTracker.package.dll");
            loader.LoadAssembly();
        }
 
        private static Assembly CurrentDomain_AssemblyResolve(object sender, ResolveEventArgs args)
        {
            return loader.FindAssembly(args.Name);
        }
 
        static void Main()
        {
            SetupPackageLoader();
            AppDomain.CurrentDomain.AssemblyResolve += CurrentDomain_AssemblyResolve;
            ...
         }

After hooking up the AssemblyResolve, you'll be able to just load assemblies like normal, and they will be found in the package file.

Here is the full class for the ResourceAssemblyLoader that you can cut and paste into your application.  If you have any questions, feel free to comment on this post and I'll try to answer them.

    class ResourceAssemblyLoader
    {
        private readonly string assemblyName;
        private Assembly assembly;
        private Type metadataType;
        private Dictionary<string, byte[]> resources;
 
        public ResourceAssemblyLoader(string assemblyName)
        {
 
            if (!Path.IsPathRooted(assemblyName))
            {
                assemblyName = Path.GetFullPath(assemblyName);
            }
 
            this.assemblyName = assemblyName;
            this.assembly = null;
        }
 
        public Assembly PackageAssembly
        {
            get
            {
                if (assembly == null)
                    LoadAssembly();
 
                return assembly;
            }
        }
 
        public string Namespace
        {
            get
            {
                return assembly.GetName().Name;
            }
        }
 
        public string Version { get; private set; }
        public string UpdateUrl { get; private set; }
 
        internal void LoadAssembly()
        {
            assembly = Assembly.LoadFrom(assemblyName);
            ReadProperties();
            ReadResources();
        }
 
        private void ReadResources()
        {
            resources = new Dictionary<string, byte[]>();
            string resourceName = PackageAssembly.GetManifestResourceNames()[0];
            using(Stream stream = PackageAssembly.GetManifestResourceStream(resourceName))
            {
                if (stream == null) return;
 
                using (ResourceReader reader = new ResourceReader(stream))
                {
                    GetResourcesFromReader(reader);
                }
            }
        }
 
        private void GetResourcesFromReader(ResourceReader reader)
        {
            IDictionaryEnumerator en = reader.GetEnumerator();
            while(en.MoveNext())
            {
                string name = (string)en.Key;
                byte[] value = (byte[])en.Value;
 
                resources[name] = value;
            }
        }
 
        private void ReadProperties()
        {
            Debug.WriteLine(assembly.GetName().Name);
            metadataType = assembly.GetType(GetFullyQualifiedName("Metadata"));
            Version = CallProperty(metadataType, "Version");
            UpdateUrl = CallProperty(metadataType, "UpdateUrl");
        }
 
        private string GetFullyQualifiedName(string className)
        {
            return string.Format("{0}.{1}", Namespace, className);
        }
 
        private static string CallProperty(Type classType, string propertyName)
        {
            PropertyInfo classInfo = classType.GetProperty(propertyName, BindingFlags.Public|BindingFlags.Static|BindingFlags.GetProperty);
            return (string)classInfo.GetValue(null, new object[] { });
        }
 
        public Assembly FindAssembly(string name)
        {
            string correctedName = name.Replace(".", "_");
            return ExistsInResources(correctedName) ? GetAssemblyFromResource(correctedName) : null;
        }
 
        private Assembly GetAssemblyFromResource(string name)
        {
            byte[] coffData = resources[string.Format("{0}_dll", name)];
#if DEBUG
            byte[] symbolData = resources[string.Format("{0}_pdb", name)];
            return Assembly.Load(coffData, symbolData);
#else
            return Assembly.Load(coffData);
#endif
        }
 
        private bool ExistsInResources(string name)
        {
            return resources.ContainsKey(string.Format("{0}_dll", name));
        }
    }
 
    // You'll need to define this inside of your package assembly to use the code above.  This includes metadata about the package assembly.
    static public class Metadata
    {
        static public string Version
        {
            get
            {
                return Assembly.GetAssembly(typeof(Metadata)).ImageRuntimeVersion;
            }
        }
 
        static public string UpdateUrl
        {
            get
            {
                return UPDATE_URL;
            }
        }
 
        private const string UPDATE_URL = @"someurl";
    }