Dave's Box

Herman Ventor, the brains behind FxCop's assembly metadata reader, has just released the next version as an open-source project over on CodePlex.

The Common Compiler Infrastructure (CCI) is a set of components for reading and writing CLR assemblies and was developed as an alternative to Reflection. This is an amazing project and if you've ever been frustrated by the limitations of Reflection and Reflection Emit - I would highly recommend downloading and checking it out.

Find out more at: http://ccimetadata.codeplex.com/

I’ve been writing C# code since one of the early betas back in 2001, yet I’ve never run across the following:

error CS1638: 'ToString_WithCache__ShouldReturnICompositionElementDisplayName' is a reserved identifier and cannot be used when ISO language version mode is used.

I was weirded out by this and thought ‘that’s a hell of a coincidence that the C# compiler has an identifier called ‘ToString_WithCache__ShouldReturnICompositionElementDisplayName’, but as it turns out identifiers with two consecutive underscore’s are actually reserved:

Identifiers containing two consecutive underscore characters (U+005F) are reserved for use by the implementation; however, no diagnostic is required if such an identifier is defined. [Note: For example, an implementation might provide extended keywords that begin with two underscores. end note]

You learn something new (however useless) everyday.

(To see this warning you’ll need to turn on ISO-1 in the Language Version drop down in the Advanced build settings)

I was reading what was new for Visual Basic 10 over on the VB Futures site, when I came across the following statement:

Overall Visual Basic 10.0 and C# 4.0 will have the closest feature parity at any point in the history of the languages, which will greatly benefit .NET developers who do work in both.

This statement refers to the language changes coming in Visual Studio 2010, such as Visual Basic getting some C#-like features, such as multiline lambdas, auto-implemented properties and implicit line continuation, and C# getting some Visual Basic-like features, such as late binding, implicit pass-by-ref and optional parameters.

I’m disappointment that the language teams feel the need to have what they call ‘co-evolution’, where any new features in one language also get added to the other. Is there any point Microsoft investing in two separate languages, if they do exactly the same thing?

In my opinion (which does not represent Microsoft’s), we would better off adding C-style macros to one of the languages, allowing us to #define the syntax differences between the two languages, and then kill off the other.

What do others think? Should Microsoft continue to support two (what are) basically the same languages, or should they attempt to differentiate them like they have done with F#?

I ran across something interesting on an internal alias about a new marketing campaign on the Visual Studio Team System marketing site that allows you to customize an amusing video that you can send to friends and team members. What caught my eye wasn’t the video itself, but the error that some users were encountering when attempting to view one of the Silverlight-based videos:

Message: Unhandled Error in Silverlight 2 Application [Format_InvalidString]
Arguments:

Debugging resource strings are unavailable. Often the key and arguments provide sufficient information to diagnose the problem. See http://go.microsoft.com/fwlink/?linkid=106663&Version=2.0.31005.0&File=mscorlib.dll&Key=Format_InvalidString   at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal)
   at System.Number.ParseDouble(String value, NumberStyles options, NumberFormatInfo numfmt)
   at System.Double.Parse(String s, NumberStyles style, NumberFormatInfo info)
   at System.Double.Parse(String s, IFormatProvider provider)
   at System.Convert.ToDouble(String value)
   at PepTalk_Full.DataObjects.doTextOverlay..ctor(XElement element)
   at PepTalk_Full.DataObjects.doPepTalk.Init()
   at PepTalk_Full.DataObjects.doPepTalk..ctor()
   at PepTalk_Full.App.wc_DownloadStringCompleted(Object sender, DownloadStringCompletedEventArgs e)
   at System.Net.WebClient.OnDownloadStringCompleted(DownloadStringCompletedEventArgs e)
   at System.Net.WebClient.DownloadStringOperationCompleted(Object arg)

To keep download sizes smaller, the client Silverlight plug-in does not contain Exception error messages, but by clicking the URL above, or by looking at the stack can you guess what the problem is? I’ll give you a hint – Convert.ToDouble(String) passes CultureInfo.CurrentCulture as the IFormatProvider argument to Double.Parse.

Yep, you guessed it, basically, the application is attempting to parse a string that it has downloaded from the web and the call to Convert.ToDouble is failing because it is expecting the format of the string to match the current culture, while the string itself is formatted using another culture. For example, the string likely contains a value formatted using the invariant culture, say ‘1.00’, but when a visitor from Germany or Norway comes to visit, Convert.ToDouble(String) is looking for a string in the format ‘1,00’.

The irony in all this is that if the Microsoft team or contractor that wrote the Silverlight application were actually using the Code Analysis feature of Team System, they would have received the following warning from Specify IFormatProvider:

Because the behavior of 'Convert.ToDouble(string)' could vary based on the current user's locale settings, replace this call in 'doTextOverlay.doTextOverlay(XElement)' with a call to 'Convert.ToDouble(string, IFormatProvider)'. If the result of 'Convert.ToDouble(string, IFormatProvider)' will be displayed to the user, specify 'CultureInfo.CurrentCulture' as the 'IFormatProvider' parameter. Otherwise, if the result will be stored and accessed by software, such as when it is persisted to disk or to a database, specify 'CultureInfo.InvariantCulture'.

All the developer has to do is to call the overload of Convert.ToDouble that takes an IFormatProvider passing it CultureInfo.InvariantCulture and the application would work on all systems regardless of the user’s culture.

The moral of the story is, use the tools available at your disposal – if you have Team System, use Code Analysis, otherwise, we still offer FxCop as a free download – they both catch bugs like this one.

I’m speaking at Code Camp Oz 2009 on April 4th in Wagga Wagga, Australia. I’ll be giving a talk called ‘A lap around the Managed Extensibility Framework’, in which I’ll give a run through of the major features of MEF.

Mitch Denny has also announced the rest of the speakers & sessions, which looks like a great line up this year. I had the pleasure of attending the first one in 2005 when I was last in Australia and it’s not only a great (and free) way to learn more about current and new technologies, but is also a great place to network with peers.

It’s free and run over the weekend of April 4th/5th. Sign up today.

One of the things that you might run across when starting to work with MEF, is that by default, all parts are created as singleton (‘shared’) instances.

For example, given the following part:

[Export]
public class MyView
{
}

Running the following code:

TypeCatalog catalog = new TypeCatalog(typeof(MyView));
CompositionContainer container = new CompositionContainer(catalog);

var view1 = container.GetExportedObject<MyView>();
var view2 = container.GetExportedObject<MyView>();

Console.WriteLine("Are same object? " + (view1 == view2));

Outputs:

Are same object? True

In certain scenarios, such as web applications, you typically want the reverse, that is, a new instance (‘non-shared’) of a part created on every request.

We allow you to indicate that, using the CompositionOptionsAttribute:

[Export]
[CompositionOptions(CreationPolicy=CreationPolicy.NonShared)]
public class MyView
{
}

Unfortunately, if you have a lot of them, this can be a bit of a hassle to do this for every part. As luck would have it, the latest drop of MEF allows you modify the default with a little bit of code:

public class TransientCompositionContainer : CompositionContainer
{
    public TransientCompositionContainer(ComposablePartCatalog catalog)
        : base(catalog)
    {
    }

    protected override IEnumerable<Export> GetExportsCore(ImportDefinition definition)
    {
        definition = AdaptDefinition(definition);
        
        return base.GetExportsCore(definition);
    }

    private ImportDefinition AdaptDefinition(ImportDefinition definition)
    {
        ContractBasedImportDefinition namedDefinition = definition as ContractBasedImportDefinition;
        if (namedDefinition != null && namedDefinition.RequiredCreationPolicy == CreationPolicy.Any)
        {   // Only change the creation policy if the importer/requester did not specify one
 
            definition = new ContractBasedImportDefinition(namedDefinition.ContractName,
                                                           namedDefinition.RequiredMetadata,
                                                           namedDefinition.Cardinality,
                                                           namedDefinition.IsRecomposable,
                                                           namedDefinition.IsPrerequisite,
                                                           CreationPolicy.NonShared);
        }

        return definition;
    }
}

Basically what is happening above is that all requests to the container are routed through GetExportsCore. When we see a request (represented as an ImportDefinition) that states that it does not have a preference for a particular creation policy (which is the default when querying from the container, or when using the [Import] attribute), we create a new request that explicitly specifies that it wants a non-shared instance and route that back to the base.

Running the above sample, substituting the standard container with the transient container, should now output:

Are same object? False

There we have it – a container that defaults to non-shared parts. You can download the container below.

We’ve dropped another version of the Managed Extensibility Framework over on our CodePlex site.

Changes since the last drop:

• Diagnostics improvement. We now provide a lot more information in error messages when something goes wrong with composition. We’ve also made the main types look a lot better under the debugger.
• Lifetime/Creation Policy. Importers can now request a specific creation policy (ie shared or non-shared) for an injected object if they so want. Non-disposable non-shared objects can now be GC’d before the container is disposed (before it used to hold onto them indefinitely).
• Ripped AllowNonPublicCompositionAttribute. We now always inject private and internal members.
• Namespace cleanup. We’ve added a couple of sub namespaces under System.ComponentModel.Composition, called ‘Hosting’ and ‘Primitives’. Hosting contains types commonly used only by host applications, such as ExportProvider, CompositionContainer and the catalogs. Primitives contains types for implementing a custom programming model, such as ComposablePart, ImportDefinition and ExportDefinition.
• Type name cleanup. Gone are the hideously long type names; AttributedAssemblyPartCatalog, AttributedTypesPartCatalog, AggregratingComposablePartCatalog, and in their place; AssemblyCatalog, TypeCatalog and AggregateCatalog.

Check it out, and tell us what you think.

ReflectionTypeLoadException takes the cake with this message:

Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.

This is one of the most frustrating messages to encounter outside of the debugger. What do you think the first thing you are going to do when you hit this in production?….Exactly, look at the LoaderExceptions property, which is impossible without forethought or a debugger.

How could have the loader team improved the usability of this exception ten-fold? By simply outputting the first couple of entries from the property:

Unable to load one or more of the requested types. The first few exceptions are provided below. Retrieve the LoaderException property for more information.
---> Could not load file or assembly ‘Library, Version=1.0.0.0, Culture=neutral, PublicKeyToken=8e9c13fee402ae29' or one of its dependencies. Strong name validation failed.

Much better. Straight away the user can see the root cause of the problem and can react without needing to attach a debugger.

(As a side note, CompositionException in MEF used to suffer from the same problem when I first arrived on the team; luckily, our latest release fixes this)

MMayerl asks over on the MSDN wiki, why we don’t document the undocumented C# keywords __makeref, __reftype, __refvalue and __arglist?

Because that defeats of the purpose of them being undocumented.

They are undocumented for a reason; so that the C# team is free to change and remove these keywords without worrying whether customers have taken dependencies on them. Documenting them conveys an implicit support agreement between Microsoft and the user, thereby preventing the team from ever changing the keywords.

We want less people to use them, not more.

I’ve seen recent reports of Visual Studio Code Analysis failing (such as this one) with the following error:

Invalid settings passed to CodeAnalysis task.  See output window for details.

Looking in the output window will show something similar to:

Switch '/targetframeworkversion' is an unknown switch.

As I’ve mentioned earlier, the /targetframeworkversion switch was added in Visual Studio 2008 SP1 to support the new multi-targeting rule. The fact that the FxCop does not understand the switch, is a big indicator that something went astray with the installation of the service pack.

Luckily the fix is rather easy, simple re-install it.

This question came up the other day:

How do I return an empty IEnumerable<T> from a method using the C# yield keyword?

There are actually two ways of doing this. Both, admittedly, are not the most intuitive things in C#:

1. yield break
2. Do not return anything

yield break terminates the iterator. For example, the following iterator returns an IEnumerable<string> containing a single element ‘Value’.

public static IEnumerable<string> GetEnumerator(bool condition)
{
    if (condition)
    {
        yield return "Value";
    }

    yield break;
}

That same method above can actually be rewritten without even specifying yield break and simply not returning anything outside of the condition:

public static IEnumerable<string> GetEnumerator(bool condition)
{
    if (condition)
    {
        yield return "Value";
    }
}

This is perfectly legal; as long there is at least one yield return in a method, the C# compiler will not require you to return a value on all code paths.

One of the problems that we’re facing today in the Framework, mainly stemming from the red bits/green bits design of .NET 3.5, is that certain ‘core’ types are located in the wrong assemblies.

For example, the extremely useful ObservableCollection<T> and ReadOnlyObservableCollection<T> types both live in WindowsBase.dll, which is a part of the Windows Presentation Foundation. For low-level assemblies (such as my team’s very own System.ComponentModel.Composition.dll), this means that to use types such as these, they need to take a reference to assemblies either at a higher level to themselves, or to technologies outside of their own. This is in turn leads to scary dependencies, such as the server assembly System.Web.dll’s current direct dependency on the client assembly System.Windows.Forms.dll in .NET 2.0.

Going forward for .NET 4.0, the Framework Architecture team will be leading an effort (being pushed by my colleague Mitch Trofin) to move these and other core types to lower-level assemblies such as mscorlib.dll and System.dll.

Now, as I’ve mentioned previously, Microsoft takes compatibility very seriously, so you might be thinking ‘how can they move these types without runtime breaking changes’?

Luckily, there’s already a solution for this; type forwarding. Introduced in .NET 2.0, type forwarding allows assembly writers to move types into other assemblies without needing existing consumers to recompile their applications. This done via the TypeFowardedToAttribute.

To move a type, do the following:

1. Move the type from the original assembly to the assembly that will become the type’s new home.
2. In the original assembly, add a reference to the new assembly if it does not already have one.
3. Apply the following assembly-level attribute to the original assembly, replacing MyType with the type you moved in step 1:

   [assembly: TypeForwardedToAttribute(typeof(MyType))]

4. Compile both assemblies and that’s it.

A couple of things to note:

• The type must have the same name and namespace in the target assembly.
• This is considered a source breaking change; that is, users recompiling their application against the new versions of the assemblies, will be required to add a reference to the assembly that now contains the type.
• While fully supported by C# and C++/CLI, Visual Basic does not support moving types using the TypeForwardedToAttribute. It does, however, support consuming assemblies written by other languages that have had their types moved.
• Starting in .NET 4.0, a new attribute called TypeForwardedFromAttribute should be applied to types that have been forwarded indicating which assembly they came from. This allows runtime serialization to serialize these types correctly when interoping with previous versions of these assemblies.

Please read this first: About Dave’s ‘unofficial’ Framework Design Guidelines.

þ DO treat a null reference (Nothing in Visual Basic and nullptr in C++/CLI) and an empty collection or IEnumerable<T> as equivalent.

For example, the following constructor treats both a null an empty IEnumerable<string> instance similar; they both result in an empty StringCollection.

public class StringCollection : Collection<string>
{
    public StringCollection(IEnumerable<string> items)
    {
        if (items != null) // Correct
        {
            foreach (string item in items)
            {
                Add(item);
            }
        }
    }
}

ý DO NOT return a null reference (Nothing in Visual Basic and nullptr in C++/CLI) from an array, collection or IEnumerable<T> property or method; return an empty instance instead.

Similar to string properties, consumers of your class expect to be to enumerate over the returned enumerable instance without also needing to check for null.

For example, in the following code, performing a foreach over the returned Collection<T> from Directory.GetFiles should never cause a NullReferenceException to be thrown.

void PrintDirectory(string path)
{
    Collection<string> fileNames = Directory.GetFiles(path);

    foreach (string fileName in fileNames)
    {
        Console.WriteLine(fileName);
    }
}

þ DO throw ArgumentException if an element in collection or IEnumerable<T> argument contains a null reference (Nothing in Visual Basic and nullptr in C++/CLI) and null is not a valid value for an element. Do not filter or skip over these values.

For example, the following method correctly throws ArgumentException when the passed types IEnumerable<Type> instance contains a null element.

public void RegisterTypes(params Type[] types)
{
    if (types == null)
        throw new ArgumentNullException("types");

    if (types.Length == 0)
        throw new ArgumentException("'types' cannot be empty.");

    foreach (string item in items)
    {
        if (item == null)
            throw new ArgumentException("'items' cannot contain a null element", "items"); // Correct

        [...]
    }
}

Note that ArgumentException is thrown above for the null element instead of ArgumentNullException. Throwing the later would be misleading and confusing to the user; the problem isn't that items is null, but rather one of its elements.

þ CONSIDER providing an params array based overload in addition to an IEnumerable<T> based overload so that user can pass elements without having to explicitly create a temporary array.

For example, the following class has two constructors, one taking an array, and one taking an IEnumerable<T>.

public class TypeCollection : Collection<string>
{
    public TypeCollection(params Type[] types)      // Correct 
        : this((IEnumerable<Type>)types)
    {
    }

    public TypeCollection(IEnumerable<Type> types)
    {
        [...]
    }
}

þ CONSIDER providing an IEnumerable<T> overload in addition to an array based overload. This allows users to pass arguments typed as other enumerable sources other than arrays, such as List<T>.

Instances typed as IEnumerable<T> are common when calling LINQ-based extension methods such as Enumerable.Select and Enumerable.Where. Adding an overload that takes IEnumerable<T> saves the user from having to manually convert an instance returned from one of these methods to an array just to pass it to the method or constructor.

ý AVOID looping over IEnumerable<T> arguments multiple times.  IEnumerable<T> instances returned from LINQ-based and yield generated methods are not cached underneath, which could cause work done in the enumerator to be repeated multiple times if an argument is iterated more than once. This could lead to unexpected behavior and performance problems.

For example, the following shows the pitfalls of looping over IEnumerable<T> instance multiple times.

public void DeleteFiles(IEnumerable<string> fileNames)
{
    if (fileNames == null)
        throw new ArgumentNullException("fileNames");

    if (fileNames.Count() == 0)             // 1st Loop
        throw new ArgumentException("'fileNames' cannot be empty.");

    foreach (string fileName in fileName)   // 2nd Loop
    {
        [...]
    }
}

This could cause a performance problem given the method as follows:

void DeleteDocumentsWithBackups(string path)
{
    string[] fileNames = Directory.GetFiles(path, ".doc");

    // Get every document that already has a backup
    IEnumerable<string> filesToDelete = fileNames.Select(fileName => File.Exists(fileName + ".bak")); 

    DeleteFiles(filesToDelete);
}

The lambda expression within the Enumerable.Select extension method will be executed twice for every file name found in the directory; once for Count and once for the foreach statement.

We hit a similar problem while developing the Managed Extensibility Framework. There was a particular method that returned an IEnumerable<T> (generated using the C# yield keyword), that underneath walked an assembly looking for types marked with a specific attribute. Trying to diagnose a performance problem, one of the developers on our team found that most of our time was spent in Reflection and it was quickly narrowed down to a particular method that was walking over the result from this method multiple times. Suffice to say, both methods were changed; the first to cut the loops down to one and the second to cache the results of the search underneath.

þ DO make a copy of an array or IEnumerable<T> argument if outside modification is not expected. As arrays and IEnumerable<T> instances can be modified after they have been passed to a method, make a clone of the argument if storing it and you want to prevent outside manipulation.

The following example uses the LINQ method, Enumerable.ToArray, to make a copy of the types argument.

public class TypeCollection : ICollection<Type>
{
    private readonly Type[] _types;

    public TypeCollection(params Type[] types)
        : this((IEnumerable<Type>)types)
    {
    }

    public TypeCollection(IEnumerable<Type> types)
    {
        if (types != null)
        {
            // Make a copy
            _types = types.ToArray();       // Correct
        }
    }
}

Note, even though the types argument might be an array (such as when the first constructor overload is called), Enumerable.ToArray will still make a copy of it and will not return the original array.

Related Framework Design Guidelines Section: 8.3 Collections

Nick Blumhardt (of autofac fame) recently joined our team, and just fired up a new blog on blogs.msdn.com. Similar to Mitch (with whom he shares an office), he's another big fan of languages with unreadable syntax.

He's started off with part one in a series on integrating Ruby with the Managed Extensibility Framework. Check it out:

Hosting Ruby Parts in MEF

Jeremy Miller takes a dig at the BCL team with his post, I love Ayende (and OSS):

From one of Ayende's many OSS projects.

/// This class actually already exists in the System.Core assembly...as an internal class.
/// I can only speculate as to why it is internal, but it is obviously much too dangerous
/// for anyone outside of Microsoft to be using... 

And dear MS BCL team, since so many people are already using Reflector to go get the ExpressionVisitor code, would you just make this public?  'K, thx.

This seems like an oversight that this wasn't public in first place. It would just be a couple minutes of a single developer's time to change the accessibility from internal to public, right?

Well, actually, it's a lot more complicated than that. There's very good reasons we don't make things public and virtual by default; Eric Lippert summarizes it pretty well with his post, How many Microsoft employees does it take to change a lightbulb?

• One dev to spend five minutes implementing ChangeLightBulbWindowHandleEx.
• One program manager to write the specification.
• One localization expert to review the specification for localizability issues.
• One usability expert to review the specification for accessibility and usability issues.
• At least one dev, tester and PM to brainstorm security vulnerabilities.
• One PM to add the security model to the specification.
• One tester to write the test plan.
• One test lead to update the test schedule.
• One tester to write the test cases and add them to the nightly automation.
• Three or four testers to participate in an ad hoc bug bash.
• One technical writer to write the documentation.
• One technical reviewer to proofread the documentation.
• One copy editor to proofread the documentation.
• One documentation manager to integrate the new documentation into the existing body of text, update tables of contents, indexes, etc.
• Twenty-five translators to translate the documentation and error messages into all the languages supported by Windows.The managers for the translators live in Ireland (European languages) and Japan (Asian languages), which are both severely time-shifted from Redmond, so dealing with them can be a fairly complex logistical problem.
• A team of senior managers to coordinate all these people, write the cheques, and justify the costs to their Vice President.

In addition to the excellent reasons that Eric lists (and the one that often gets forgotten) is compatibility. Despite what appears to be evidence to the contrary, the compatibility bar between versions of the Framework is extremely high. Every single public member and type and its behavior needs be maintained indefinitely. The more APIs that are public and/or virtual, the more baggage that needs to be brought forward into future Framework versions so as to not break compatibility.

Developing good APIs is extremely difficult, especially the first time around. When we get things wrong (and we do get things wrong), trying to maintain backwards compatibility often stifles innovation as we try to correct these scenarios. Unfortunately, unlike others, we don't have the luxury to make only 95% of our new versions backwards compatible with our previous versions.

It is for these very reasons, we try to make our extension points deliberate and designed.