Sunday, 11 January 2009

Evidence for the AppDomain: The resolution

To recap what has been a continuing saga: I discovered that attempts to get Streams on PackageParts in an OpenXML Package were failing because of an issue in the .Net framework. Under the hood, a MemoryStream was overflowing to disk. The framework examines the evidence of the assembly and then examines the evidence of the AppDomain. Our managed COM component was getting created in the DefaultDomain and didn't have the evidence (a SecurityZone of MyComputer) that was needed to satisfy the framework.

As I discussed here, I resolved this by having the COM interface act as a shim. It created a second AppDomain that had the evidence necessary to satisfy the framework.

Further testing did not reveal any more show-stoppers but it was obvious that performance was not great. Large blobs of XML were being marshaled across AppDomain boundaries (from the child AppDomain to the DefaultDomain) and then marshalled again across the COM boundary to the Win32 application.

I briefly considered hosting the CLR so that I could explicitly create the AppDomain of my dreams but decided I'd really rather not.

Fortunately help came in the form of this post. (Can I just say how great it is that everyone with a keyboard in Microsoft seems to be blogging these days. Shawnfa's posts have been a great help.)

To summarize: you don't have to do all the work of hosting the CLR. You can just write (in managed code) an AppDomainManager and tell the CLR (by setting two environment variables) to use it.

This was almost the complete answer to my problem: when an AppDomain gets created I can add the Evidence I need, namely a SecurityZone of MyComputer. Sure enough, when I set the environment variables and my managed COM component gets called, I see a new instance of my AppDomainManager get called. But the CreateDomain() method never gets called. The CLR is just loading my COM component into the DefaultDomain; it's not making a new AppDomain.

It's almost the solution though. It's apparent that my AppDomainManager is getting called, and that the HostSecurityManager property is being used by the framework to make decisions. All I need is for that property to return my own subclass of a HostSecurityManager with the evidence set. In my subclass, I override the method ProvideAppDomainEvidence to provide the Evidence that I need.

Now my COM component works like a charm! Performance has improved too. We are marshaling large XML blobs across COM boundaries but at least we aren't also marshaling across the boundary to a second AppDomain.

Link multiple .netmodule files

This past week I was attempting to add evidence to an assembly directly, i.e. to bake it into the DLL, to satisfy some security requirements. The .Net linker (al.exe) has an option (/evidence:) that lets you add a serialized Evidence object as a resource with the magic name Security.Evidence. There is some discussion of producing this resource file here.

A little poking around on the web showed me how to compile to .netmodule files, which seemed analogous to .obj files in the C++ Win32 world. I had to rework the .csproj file produced by Visual Studio so that I could invoke msbuild and produce one .netmodule file for each .cs file, build the resources etc and then link.

Everything appeared to work just fine. The link stage didn't give any errors or warnings. Curiously, the resulting DLL was only about 8Kb in size. The assembly when produced in Visual Studio from the .csproj file is a little more than 1Mb.

This turns out to be "by design". You can indeed compile to a .netmodule and link but if you have more than one .netmodule file it really is not analogous to the linking of various C++ obj files to produce a single DLL. Instead, the resulting assembly is a meta-assembly. It contains references to the various .netmodule files, which you would have to deploy with the assembly.

Luckily all was saved when I realized that I could compile all the files and produce a single assembly directly using csc.exe (the command line C# compiler). I just needed to specify a resource file containing the serialized Evidence and name the resource Security.Evidence. Indeed, it would seem there is little that you can do with the linker (al.exe) that you couldn't just do with csc.exe (the command line compiler), a point made here.