WiX: Really, Really Open Source

I haven't blogged about WiX in a while but there has been an pretty significant change recently that I thought I call out and comment on.

As some of you may have already seen, the founder of the WiX toolset, Rob Mensching, has left Microsoft and with it, so did the WiX toolset. Rob plans on focusing full time on WiX and I think that could be a very good thing for the toolset. WiX is a powerful toolset that enables massively scalable setup solutions designed to work for anyone from a one man product all the way up to massive enterprise level products like Office, Visual Studio, and many, many more.

Rob's dedication to the WiX project has been unceasing for 13 years. He put his blood, sweat, and tears into making sure WiX was able to deliver everything its users needed. I have personally worked with Rob for 7 of those years and have many fond memories of late nights solving tough problems. I'll probably follow this post up with some commentary about my experiences with Rob and WiX. Although this current change makes it a little harder for me to contribute to the project, I wish Rob all the best in his new adventure and look forward to seeing what WiX can become with him focused on it full time.

The .NET BCL Team released a preview of Immutable Collections

Check out the BCL team blog here. Anyone working on highly multithreaded software in .NET might find this quite useful.

.NET: .NET Framework 4 setup reports success but my .NET 4 applications wont run and tell me I need to install it.

I'm curious to hear from people that are hitting or have hit this issue. Leave a comment or contact me directly through my blog.

 

The issue here is often that %windir%\system32\mscoree.dll is not updated as part of the install. Behind the scenes of the installer, .NET 4 is trying to install an update to the OS that it depends on but Windows Update is telling the .NET 4 installer that the update is not applicable to the machine. The .NET 4 installer treats this response to mean that is not needed and moves on. In general, if Windows update is returning "not applicable" for and update one of wo things is true. Either its not applicable, or something is really wrong with Windows or Windows Update and no updates including critical security updates are able to install.

 

So far I have found two root causes of this issue:

 

1. %windir%\servicing\trustedinstaller.exe is not present on the machine.

2. The operating system is not an officially released version of the OS.

 

I'm curious to hear from anyone experiencing this issue. I'd like to know which of this issues it wascor if there is a root cause of this that I have not found.

 

Trouble Shooting/Self Help Instructions:

 

1. %windir%\servicing\trustedinstaller.exe is not present on the machine.

 

If this is the root of your issue. I would recommend following the steps in this article to try to get trustedinstaller.exe put back http://support.microsoft.com/kb/929833 .
If anyone has any idea what would have removed trustedinstaller.exe from their machine, I'd love to hear about it.

 

2. The Operating System is not an officially released version of the OS.

 

You can check this by looking at the version of %windir%\servicing\trustedinstaller.exe or looking in %windir%\logs\CBS\CBS.log. In the log you should see logging lines that look something like this: "Loaded Servicing Stack v6.1.7601.17592".

 

For Windows 7, I would expect to see a version of at least v6.1.7600.16385. If you see a number lower than this on Windows 7, the only way I know of to fix this issue is to do an upgrade to a supported released version of Windows 7. The more important thing in this case is that pre-release OS's do not get updated with critical security patches from Microsoft. Thats not a really good situation to be left in, especially if you dont know it.

 

I've seen many reports of people having a build number of v6.1.7600.16384 which is one number below the released build. I'm really curious how people would have gotten this build of the OS. The fact that its so close to the final build makes me wonder if it could have been accidentally released somehow. Please let me know if you got it through an official supported channel and its the wrong version.

I'd like to thank Mike Lewis for contacting me and helping me find the 2nd root cause. I'd love to hear from anyone else hitting this issue to make sure I can write down all the causes of this for people.

.NET: Do You Deploy a Managed App - Part 2

Almost 2 years ago I asked for feedback on your experience deploying .NET applications and using the .NET Framework deployment packages (Do you deploy a managed App).

We took a lot of the feedback from my blog and various other places and implemented features in .NET 4.0. Based on the feedback, we focused on size, performance, and robustness which I also recently posted about (The .NET Framework 4 Installer Improvements).

I’m really interested in getting feedback from everyone on how we did and get ideas for what we can do to make the experience better in the next release. We definitely take every bit of feedback seriously and value it highly and I want to say I always appreciate the time people spend to respond.

Feel free to answer as many of these questions as you are compelled to.

Questions:
1. Have you decided to update your application to .NET 4? Was deployment a factor in that decision?
2. Did .NET Framework 4 solve all of your deployment problems? Did you like it?
3. Where did we miss with .NET Framework 4 deployment? What didn’t you like?
4. What do you think about the .NET Framework 4 Client Profile option?
5. Did you find all the documentation and examples you needed to be able to use .NET 4?
6. What do you think about the new size of the .NET package?
7. Do you know about the redist and web installers? If so, which do you use?
8. Do you use a bootrapper/chainer that preinstalls .NET? If not, do you block? If so,
- Which one (VSI, InstallShield, Wise, ClickOnce, custom)?
- Do you run .NET in silent mode or UI mode?
9. Do you have any specific problems you can tell me about that you have had in deploying the .NET Framework?
Some more question that will help me understand your scenario:
10. How large is your application? (Size, install base)
11. How is your product deployed (Web download, CD, DVD, USB)?

Optional:
12. What company are you with?
13. What is your application?
14. Can I follow up with you? If so, email me your contact info along with your answers

As always. I very much appreciate getting everyone’s feedback. Hope to hear from many of you.

The .NET Framework 4 Installer Improvements

Thanks to everyone that gave feedback both on my blog and through other forums about .NET Framework installs. .NET 4 just released so I thought I’d take this opportunity to talk a little about the stuff we have done to improve the installation. My team and I have been focused over the past year on incorporating feedback and striving to make the installer better for the .NET Framework 4. There has been a particular focus on making it better for client applications to install it with their apps.

The key focus areas for the .NET 4 installer have been Size, Robustness, and Performance. I’ll speak to some of the major things we did and give a brief description below.

Size

You can see from the chart below that we have reduced the size of the .Net redist significantly. It went from client applications needing to carry 237MB to 41MB for the client profile. This is a game changer for a lot of client applications.

Size Comparison of the redist size since 3.5 SP1

	3.5 SP1	4 Beta1	4 Beta2	4 RTM
32bit Client Profile	X	34.5 MB	31.5 MB	28.8 MB
32bit Full	X	77.5 MB	38.5 MB	35.3 MB
32+64bit Client Profile	X	72.5 MB	48.2 MB	41 MB
32+64bit Full	237 MB	162.6 MB	55.9 MB	48.1 MB

 

Better Compression Across Packages

We implemented the use of a better compression technology into our packages which reduced the size of our packages by around 15%.

Separate packages for AMD64 and IA64

We found that there was little to no need to ever install the same package on both amd64 and ia64. Because of this, we decided to produce amd64 packages that excluded ia64 binaries as well as ia64 packages that didn’t contain amd64 binaries.

Client Profile

We determined the subset of framework functionality that was used by 95+% of client applications and produced a first class package for this scenario. The result of this is that, unless you are taking advantage of features such as ASP.NET, you can now take a dependency on a smaller framework. More details of what is in the client profile can be found here.

Remove Duplicate MSIL

We identified many assemblies that were functionally identical but differed by the architecture they were built under. These assemblies were all managed CPU neutral assemblies which meant that it didn’t matter whether they were built for x86 or amd64. Their strong names and functionality are the same. We solved this by only carrying one of them.

Robustness

The numbers are already starting to come in from the RTM release and we are very happy with the success rates of the install. At this point we are seeing greater than a 99% success rate which is much better than previous .NET redist releases right out of the gate. Here are some of the larger things we did to increase robustness.

Remove Prerequisites

In a chain of installs, the chain is only as strong as its weakest link. In addition, small weaknesses in each part of the chain compound to lead to higher failure rates for the whole chain. By removing numerous prerequisites and combining the whole client install into a single MSI, we were able to get rid of the compounding effect of failures as well as focus our efforts on making the single MSI as solid as possible.

Simplify the MSI

Custom actions are very common places for installs to fail. The more you have, the more complex the installer gets and the number of points of failure goes up. Removing the need for customactions in many cases and in the cases where we needed them, simplifying them has increased our success rates.

Remove slipstreamed feature MSP’s

In Beta1, we slipstreamed features into the installer’s msi using patches. This proved to be a point of complexity and the root cause of many unsolvable bugs. Due to that, we simplified our install to be completely contained in a single msi per platform.

Fix and Retry

Through thorough investigation of our past installers, looking through KB articles, feedback from customers, and through our past Beta’s, we found numerous install failure conditions that were fixable after the error. We implemented the KB articles and other workarounds so, in failure cases, we can fix the users machine and try again. We’ve seen quite an increase in our success rates due to this work. My hope is that this will also make the windows installer ecosystem cleaner and that msi’s installed after .NET 4 will have a better chance to succeed because our installer put the machine in a better state.

Triple fallback on Download failures

Through analysis of our download failures in the past, we determined that using a single implementation for downloads left you only as successful as that technology allows. We found that between Winhttp, URLMon, and BITS, their failures were in different scenarios and where one would fail, the others would succeed. In order to take advantage of this, our chainer falls back and retries on different download stacks to do everything we can to get a successful download.

Separated out Server configuration from Client Profile

The Client Profile installer should be more robust for applications now because some of the most common failing custom actions in .NET 3.5 were in configuring things like ASP.NET and WCF which are mainly server scenarios and not used by client applications. By moving these to the full install, we are seeing higher success rates for the client install.

Performance

The key metric here is overall install time for the redist. In 3.5 SP1 the average install time was 12-15 minutes. With v4, the average times are closer to 3-5 minutes for the client profile and 5-7 minutes for Full.

Smart Cabbing

Smart cabbing is a technique used to allow you to install the same file to multiple locations but only carry the file once in the msi’s cabs. This technique has been used for years but during our perf investigations, we found that, depending on how many duplicate files there were and where they were in the cab, performance degraded significantly. We made some bug fixes in the tools we use to smart cab (WiX) to reduce the impact of duplicated items while still gaining the benefits of smart cabbing.

Remove Prerequisites

This one is fairly self explanatory. We need to install less packages so we are faster. This is mostly the result of changing the .NET Framework itself to not have certain dependencies or carry subsets of the dependencies within the framework. In a few cases, this was possible because the base functionality was either built into all the supported OS’s or had enough ubiquity in the ecosystem to not warrant us carrying it.

Remove Slipstreamed Msp’s

We found that when applying large slipstreams to a product, there was a significant perf hit towards the end of the install when Windows Installer is caching the packages for source resiliency. By adding all the features into the MSI, we got rid of this performance hit.

Parallel Ngen and removal of synchronous 64bit assemblies

The CLR implemented the ability to ngen on multiple cores in parallel. We made changes in our installer to take advantage of this so now on a multicore machine, ngen times should be significantly reduced. Also, on 64bit machines, most .NET applications run as 32bit. This means that paying the price of creating 64bit native images is not something most apps need to do.

Client Profile

By producing a subset of the .NET install that contains the features most client applications need, most client applications can take advantage of shorter install times by installing less.

Parallel Download and Install
If you are using the web bootstrapper which we made available for the first time in Beta2, you can use the web bootstrapper to install .NET Framework 4. This has the advantage of downloading and installing the payload in parallel. For example, as it is installing the Client Profile, it will be downloading the rest of the framework. In cases where you have enough bandwidth to download the rest before the Client Profile install finishes, you essentially save the time it took to download the rest.

This was originally posted here and may have additional comments.

Which Version of .NET is Built into Windows?

Over the past few years I have been asked more and more which version of .NET is included in the operating system. The following chart shows from .NET 2.0 through 3.5, which version of .NET is included in both the Client and Server Windows operating systems.



Note that this diagram also shows which pieces are optional and whether or not they are on by default in that particular OS.

WiX: Removing Files with Patches

I've had numerous people ask me how to remove a file using a patch. As I say below again, this should be done carefully.

 

Background

If a file is present in a Baseline package then removed in the Upgrade package, it is not trivial to get the installer to remove the file in a way that uninstalling the patch will put it back.

 

Important!!! MUST READ

 

Before removing a file you need to make sure you have no other options. The only time you should remove a file using a patch is when the mere presence of the file causes problems. Consider removing references to the file from files that use it or patching it to be an "empty" file. Removing a file should be a last resort.

 

For shared files, removing a file with a patch could mean breaking any other product using that file. The removal of the file does not take reference counting into account when determining to remove a file. It will be removed if you tell it to be removed. Make sure you consider everything that uses or could use the file before removing it.

 

If a file must be removed by a patch there are a few options:

If the file is not a the keypath of its component:
- Remove the file from the component- Add a RemoveFile entry to the component.

If the file is the keypath if its component:

If the file is the keypath, it means this file is the identity of the component. In this case the entire component must be removed. You cannot simply remove a component using a patch. If you do, you have to remove the entire feature which is not usually the desired scenario. If it is, then just remove the feature from your patch.

In order to remove a component without removing its feature you need to convince Windows Installer that the component isnt supposed to be installed but the msi database still needs to know about it. You can achieve this by making the component's condition false. In order to cause a recalcalculation of the component during repair/patch you also need to mark the component as transitive. The side-affect of this is that when the patch is uninstalled, the file is not put back on the machine. This is because when the patch is removed, the component is no longer transitive and its condition is not reevaluated. You can solve this by shipping two patches. One that marks the component as transitive and a second that makes the component's condition false.

Example:
<Component Id="Foo" Transitive="yes" ... >
    <Condition>FALSE</Condition>      ...
</Component>

Notes and considerations:

In any case, uninstalling the patch may prompt for source if the file that needs to be put back is not available to the installer.

If building 2 patches for the transitive solution, you cannot base the two patches off of the same build because both changes are made to a single component which is the smallest level at which patches can be filtered.