Dave's Box : Compatibility

Documenting ‘undocumented’ features

David Kean — Mon, 29 Dec 2008 15:00:00 GMT

MMayerl asks over on the MSDN wiki, why we don’t document the undocumented C# keywords __makeref, __reftype, __refvalue and __arglist?

Because that defeats of the purpose of them being undocumented.

They are undocumented for a reason; so that the C# team is free to change and remove these keywords without worrying whether customers have taken dependencies on them. Documenting them conveys an implicit support agreement between Microsoft and the user, thereby preventing the team from ever changing the keywords.

We want less people to use them, not more.

Moving a type from one assembly to another using TypeForwardedToAttribute

David Kean — Fri, 19 Dec 2008 15:00:00 GMT

One of the problems that we’re facing today in the Framework, mainly stemming from the red bits/green bits design of .NET 3.5, is that certain ‘core’ types are located in the wrong assemblies.

For example, the extremely useful ObservableCollection<T> and ReadOnlyObservableCollection<T> types both live in WindowsBase.dll, which is a part of the Windows Presentation Foundation. For low-level assemblies (such as my team’s very own System.ComponentModel.Composition.dll), this means that to use types such as these, they need to take a reference to assemblies either at a higher level to themselves, or to technologies outside of their own. This is in turn leads to scary dependencies, such as the server assembly System.Web.dll’s current direct dependency on the client assembly System.Windows.Forms.dll in .NET 2.0.

Going forward for .NET 4.0, the Framework Architecture team will be leading an effort (being pushed by my colleague Mitch Trofin) to move these and other core types to lower-level assemblies such as mscorlib.dll and System.dll.

Now, as I’ve mentioned previously, Microsoft takes compatibility very seriously, so you might be thinking ‘how can they move these types without runtime breaking changes’?

Luckily, there’s already a solution for this; type forwarding. Introduced in .NET 2.0, type forwarding allows assembly writers to move types into other assemblies without needing existing consumers to recompile their applications. This done via the TypeFowardedToAttribute.

To move a type, do the following:

Move the type from the original assembly to the assembly that will become the type’s new home.
In the original assembly, add a reference to the new assembly if it does not already have one.
Apply the following assembly-level attribute to the original assembly, replacing MyType with the type you moved in step 1:
```
[assembly: TypeForwardedToAttribute(typeof(MyType))]
```
Compile both assemblies and that’s it.

A couple of things to note:

The type must have the same name and namespace in the target assembly.
This is considered a source breaking change; that is, users recompiling their application against the new versions of the assemblies, will be required to add a reference to the assembly that now contains the type.
While fully supported by C# and C++/CLI, Visual Basic does not support moving types using the TypeForwardedToAttribute. It does, however, support consuming assemblies written by other languages that have had their types moved.
Starting in .NET 4.0, a new attribute called TypeForwardedFromAttribute should be applied to types that have been forwarded indicating which assembly they came from. This allows runtime serialization to serialize these types correctly when interoping with previous versions of these assemblies.

"I can't believe Microsoft didn't make [Enter API Name Here] public"

David Kean — Tue, 09 Dec 2008 15:00:00 GMT

Jeremy Miller takes a dig at the BCL team with his post, I love Ayende (and OSS):

From one of Ayende's many OSS projects.
/// This class actually already exists in the System.Core assembly...as an internal class.
/// I can only speculate as to why it is internal, but it is obviously much too dangerous
/// for anyone outside of Microsoft to be using... 
And dear MS BCL team, since so many people are already using Reflector to go get the ExpressionVisitor code, would you just make this public? 'K, thx.

This seems like an oversight that this wasn't public in first place. It would just be a couple minutes of a single developer's time to change the accessibility from internal to public, right?

Well, actually, it's a lot more complicated than that. There's very good reasons we don't make things public and virtual by default; Eric Lippert summarizes it pretty well with his post, How many Microsoft employees does it take to change a lightbulb?

One dev to spend five minutes implementing ChangeLightBulbWindowHandleEx.
One program manager to write the specification.
One localization expert to review the specification for localizability issues.
One usability expert to review the specification for accessibility and usability issues.
At least one dev, tester and PM to brainstorm security vulnerabilities.
One PM to add the security model to the specification.
One tester to write the test plan.
One test lead to update the test schedule.
One tester to write the test cases and add them to the nightly automation.
Three or four testers to participate in an ad hoc bug bash.
One technical writer to write the documentation.
One technical reviewer to proofread the documentation.
One copy editor to proofread the documentation.
One documentation manager to integrate the new documentation into the existing body of text, update tables of contents, indexes, etc.
Twenty-five translators to translate the documentation and error messages into all the languages supported by Windows.The managers for the translators live in Ireland (European languages) and Japan (Asian languages), which are both severely time-shifted from Redmond, so dealing with them can be a fairly complex logistical problem.
A team of senior managers to coordinate all these people, write the cheques, and justify the costs to their Vice President.

In addition to the excellent reasons that Eric lists (and the one that often gets forgotten) is compatibility. Despite what appears to be evidence to the contrary, the compatibility bar between versions of the Framework is extremely high. Every single public member and type and its behavior needs be maintained indefinitely. The more APIs that are public and/or virtual, the more baggage that needs to be brought forward into future Framework versions so as to not break compatibility.

Developing good APIs is extremely difficult, especially the first time around. When we get things wrong (and we do get things wrong), trying to maintain backwards compatibility often stifles innovation as we try to correct these scenarios. Unfortunately, unlike others, we don't have the luxury to make only 95% of our new versions backwards compatible with our previous versions.

It is for these very reasons, we try to make our extension points deliberate and designed.

BUG: Multi-targeting rule not firing on usages of 2.0 SP2 and 3.0 SP2

David Kean — Sat, 29 Nov 2008 15:00:09 GMT

Previously, I've spoken about the new multi-targeting rule that shipped in Visual Studio 2008 SP1 and FxCop 1.36. This rule helps to prevent users from inadvertently taking a dependency on a .NET service pack. Unfortunately, it shipped with a bug that when targeting .NET 3.5, prevented it from firing on usages of types and members only available in .NET 2.0 SP2 and .NET 3.0 SP2 (both installed by 3.5 SP1).

Luckily, the fix is rather simple.

In Visual Studio 2008 SP1:

From an elevated (right-click and Run as administrator) Visual Studio command-prompt, type the following and press <ENTER>:

devenv "%PROGRAMFILES%\Microsoft Visual Studio 9.0\Team Tools\Static Analysis Tools\FxCop\Repository\Compatibility\Desktop2.0SP2.xml"

In the <CompatibilityEntry> element in the file that opens, change the Priority attribute from 3015 to 3515:

<CompatibilityEntry Name=".NET Framework 2.0 Service Pack 2"
                    Platform="Desktop" 
                    Version="2.0.2" 
                    Priority="3515">

Save the file.
From the previously opened command-prompt, type the following and press <ENTER>:
devenv "%PROGRAMFILES%\Microsoft Visual Studio 9.0\Team Tools\Static Analysis Tools\FxCop\Repository\Compatibility\Desktop3.0SP2.xml"

Similar to above, in the <CompatibilityEntry> element, change the Priority attribute from 3020 to 3520:

<CompatibilityEntry Name=".NET Framework 3.0 Service Pack 2"
                    Platform="Desktop" 
                    Version="3.0.2" 
                    Priority="3520">

Save the file.

In FxCop 1.36:

Follow the above steps, changing the paths above from %PROGRAMFILES%\Microsoft Visual Studio 9.0\Team Tools\Static Analysis Tools\FxCop\Repository\Compatibility to %PROGRAMFILES%\Microsoft FxCop 1.36\Repository\Compatibility.

To verify the fix:

In Visual Studio, start a new C# WPF Application, and paste the following code in the code-behind of App.xaml:

public partial class App : Application
{
    System.ComponentModel.DateTimeOffsetConverter field1;
    System.Windows.Data.BindingGroup field2;
}

In Solution Explorer, right-click on the project and choose Run Code Analysis.
If the above steps were followed correctly, you should see warnings similar to the following:

This will be fixed in a future release of Visual Studio.

Thanks to Andrew Smith for bringing this my attention.

Breaking changes to the String class

David Kean — Wed, 12 Nov 2008 15:00:43 GMT

You may have already read Justin Van Patten's post about the upcoming breaking changes to the String class for .NET 4.0. This change will affect the behavior of the String.StartsWith, String.EndsWith, String.IndexOf and String.LastIndexOf methods by changing them to perform an ordinal (byte-for-byte) comparison by default instead of a culture-sensitive comparison using CultureInfo.CurrentCulture. In addition, the default overloads of String.ToUpper, String.ToLower, Char.ToUpper and Char.ToLower will be changed to use CultureInfo.InvariantCulture instead of CultureInfo.CurrentCulture.

How does this change affect me?

If you are using an overload of one of the above methods that does not take a StringComparison or CultureInfo as a parameter, then your application or library will be affected when you move it to .NET 4.0. String.Compare and String.CompareTo are not being changed.

An example of the kind of behavior change you could expect, is the following:

static void Main()
{
    int index = "encyclopædia".IndexOf("encyclopaedia");

    Console.WriteLine(index);
}

On versions previous to .NET 4.0, this will output 0 when the user's current locale is set to some cultures, such en-AU, English (Australia), and -1 when the user's current locale is other cultures, such nn-NO, Norwegian, Nynorsk (Norway).

However in .NET 4.0, the above comparison will always output -1, regardless of the user's current locale.

Why was this change made?

The comparison methods on the String class have always been a little schizophrenic; some methods, ==, String.Equals and String.Contains for example, perform an ordinal comparison by default, whereas the above methods, as well String.Compare and String.CompareTo, perform a culture-sensitive comparison. Because they use the current culture, the default culture-sensitive comparisons can also vary between systems, users and even during the same application session. Which, as pointed out in Justin's post, can lead to security vulnerabilities in applications that make security decisions using the default overloads. The planned breaking changes bring the majority of these methods inline with each other to have the same default behavior.

How do I find and fix these comparisons?

To start, you should run FxCop or Visual Studio Code Analysis over your code base. The rules Specify StringComparison and Specify CultureInfo will fire on call sites that do not explicitly specify a StringComparison or CultureInfo. Specify ordinal StringComparison will fire when you use invariant culture instead of ordinal to compare, which is almost always wrong.

To help determine the correct StringComparison or CultureInfo to use, you cannot go past the excellent article New Recommendations for Using Strings in .NET 2.0. In particular, the section under Choosing a StringComparison Member for Your Method Call provides a table detailing common operations and comparisons to use.

New for Visual Studio 2008 SP1 and FxCop 1.36 – Multi-targeting rule

David Kean — Mon, 25 Aug 2008 14:00:00 GMT

Update: Note there is currently a bug in this rule that prevents it from firing on .NET 2.0 SP2 and .NET 3.0 SP2 added APIs when targeting .NET 3.5. See BUG: Multi-targeting rule not firing on usages of 2.0 SP2 and 3.0 SP2 for more information.

Back in October of last year, Krzysztof Cwalina talked about the multi-targeting changes in Visual Studio 2008 and .NET 3.5. He mentioned the limitations in its design that could cause an application that explicitly targets .NET Framework 2.0 and .NET Framework 3.0 to accidently take a dependency on a new member or type that did not exist in the RTM version of that framework. As a workaround, he posted the source code of a Code Analysis/FxCop rule that could detect and fire on a member that took one of these dependencies.

A little while before he wrote that post, my team at the time, the Code Analysis team, was approached to see if we could ship a similar rule out of the box. Unfortunately, we ran out of time to get it included in Visual Studio 2008 RTM and FxCop Beta 2, however, I’m pleased to announce that we’ve added the rule, Use only API from targeted framework, to Visual Studio 2008 SP1 and FxCop 1.36 RTM release.

Note: The Visual Studio integration for this rule is only available in editions that support Code Analysis; Visual Studio Team Development Edition and Visual Studio Team Suite.

How this rule works

In Visual Studio, this rule reads the current targeted framework stored in the C# or Visual Basic project and fires on any usage of API that was not included in the RTM version of that framework. Whereas, in FxCop, the rule reads the targeted framework stored within the FxCop project or passed via the command-line.

The following table attempts to explain this:

	When Target Framework (in Project properties or in FxCop project options/command-line) is set to:
Code Analysis/FxCop fires on usage of:	.NET Framework 2.0	.NET Framework 3.0	.NET Framework 3.5^‡
.NET Framework 2.0
.NET Framework 2.0 SP1	•	•
.NET Framework 2.0 SP2	•	•
.NET Framework 3.0	•^†
.NET Framework 3.0 SP1	•^†	•
.NET Framework 3.0 SP2	•^†	•
.NET Framework 3.5	•^†	•^†
.NET Framework 3.5 SP1	•^†	•^†	•

^† MSBuild prevents a user from mixing these combinations in Visual Studio, so only FxCop can fire on this.
^‡Client-only Framework subset is not supported.

For example, when a user selects .NET Framework 3.0 as their project’s target framework, Code Analysis and FxCop will fire on any usage of members and types that were introduced in .NET Framework 2.0 SP1 and SP2, .NET Framework 3.0 SP1 and SP2, .NET Framework 3.5 and .NET Framework 3.5 SP1. This is because on a fresh install of .NET Framework 3.0 (such as on Vista RTM), none of these service packs or frameworks are installed.

Enabling in Visual Studio

By default, Visual Studio projects with the default Code Analysis settings will already have the rule enabled. To check this, do the following:

In Solution Explorer, right-click on your project and choose Properties
Choose the Code Analysis tab
Expand the Portability node and ensure that Use only API from targeted framework is checked

Once this rule has been enabled, you need to make sure that you targeting the framework that you plan to release your application on. To verify this, do the following:

For C# projects:

In Solution Explorer, right-click on your project and choose Properties
Choose the Application tab
From the Targeted Framework drop down, choose the framework that your application is targeting

For Visual Basic projects:

In Solution Explorer, right-click on your project and choose Properties
Choose the Build tab and click Advanced Compile Options…
From the Target framework (all configurations) drop down, choose the framework that your application is targeting

Once you’ve done the above, running Code Analysis (Analyze -> Run Code Analysis) should now fire on any service pack dependencies your project has taken on. For example, using DateTimeOffset in a project that is set to target .NET Framework 2.0, will cause the following warning to output.

Enabling in FxCop

In contrast to Visual Studio, Use only API from targeted framework does not fire by default in FxCop. This is because FxCop does not have access to the project (.csproj, .vbproj) of the assembly that it is analyzing, and hence cannot automatically determine the Framework your application is targeting. Because of this, you need explicitly specify it; either via the FxCop project or via the command-line.

Enabling using an FxCop project

By default, FxCop projects will already have the rule enabled. To check this, do the following:

Open your FxCop project in FxCop
Choose the Rules tab
Expand the Portability node and ensure that Use only API from targeted framework is checked

Once this rule has been enabled, it will not fire until you tell FxCop the framework you are targeting. To do this, do the following:

Open your FxCop project in FxCop
Choose Project -> Options and then choose the Spelling & Analysis tab
From the Target Framework drop down, choose the framework you are targeting. Typically this would match the value specified in your C#, Visual Basic or C++ project’s properties.
Click OK

Note: You can make this the default for all new FxCop projects and for the command-line by clicking Defaults… after you open the Project Options dialog and then following the rest of the steps above.

Once you’ve done the above, running analysis (Project -> Analyze) should now fire on any service pack dependencies your application has taken on. For example, using DateTimeOffset in a assembly that is set to target .NET Framework 2.0, will cause the following error to output:

Enabling via the command-line

While the preferred method to specify the target framework is by using an FxCop project as described above, you can also specify it explicitly via the command-line. The new /targetframeworkversion switch was added for this purpose. Its valid values are described below:

Argument	Description
/targetframeworkversion:2.0	Specifies .NET Framework 2.0
/targetframeworkversion:3.0	Specifies .NET Framework 3.0
/targetframeworkversion:3.5	Specifies .NET Framework 3.5
/targetframeworkversion:none	Disables the rule. This is the default. However, it can useful to specify this argument if you want to override the associated setting stored in an FxCop project that is passed on the same command-line.

Specifying any of these arguments on a command-line will override the target framework setting stored in the FxCop project.

An example command-line would be the following:

FxCopCmd.exe /file:MyAssembly.dll /console /targetframeworkversion:2.0

Disabling the rule or targeting specific service packs

Use only API from targeted framework does not have support for targeting specific service packs, such as .NET Framework 2.0 SP1. If you want to deliberately take a dependency on a service pack, simply disable the rule by unchecking instead of checking the rule in the Enabling in Visual Studio and Enabling in FxCop sections.