WiX: Why I Contributed for 7 years

Rob Mensching recently posted a call out to people who may be interested in joining him as a member of the 5th generation of WiX Toolset developers. As a big part of both the 3rd and 4th generations, I thought I'd share a few things about why I contributed to the project for so long. Maybe some of these reasons will resonate with you and WiX is the right fit for you. In order to do that, I'll start from the beginning...

I remember the first time I met Rob and the other WiX developers. I was a new developer at Microsoft basically fresh out of college. I was trying to solve a deployment problem for Visual Studio and my boss and I at the time had some ideas about how to solve it but we needed some changes to WiX. I decided that I should go meet with the WiX guys and bounce the ideas off of them and they held "WiX night" every Thursday night in a conference room at Microsoft.  I went to the meeting where everyone was sitting around writing code and having discussions ranging from software design to what they were up to in their personal lives. I could tell, they all enjoyed working together. They said hi to me and welcomed me to the meeting and I sat down.

After about 30 minutes, Rob turned to me and said, "So what brings you here?". I told him that I was working on Visual Studio Setup and I had a request for a feature in WiX. I wont go into the details of what the feature was even though I vividly remember it. I described the feature to them and they all came back at me saying "Absolutely NOT!". "That is the wrong way to solve that problem.", they said in a good sort of way :). They quickly started diving into the details of why I wanted the feature, what was the problem I was trying to solve and they were all very interested in helping me solve the problem. As we worked through it, we ended up landing on a good, simple, and far more generic solution to the problem. After we came up with a design, they asked me if I wanted to try and code up this new feature. After seeing the investment they made in my problem and having so much fun designing the feature, I couldn't resist and volunteered to do it.

I started going to WiX night every week to work on the feature. I loved the experience of working closely with other developers where I could bounce ideas off of them, get advice, get real time code reviews, and crack some jokes while we were at it. Over the next few weeks I felt like friends with the people there and started to get a sense for how impactful the WiX Toolset was. After my first feature, Rob came up to me and asked me what kind of experience I was looking to get out of being a part of WiX. He then helped figure out which part of the toolset I could start to learn in more depth to gain that experience. I started to feel really invested in the project as a whole and I felt like the WiX team was invested in me as well.

I learned a ton of core coding and design skills while working on WiX. The principle of "Always do the right thing" shone through in every change to the project. There were very clear coding guidelines and standards that helped keep the codebase clean and maintainable. Designs were always well thought through with the bigger picture in mind and not just solving the problem at hand as quickly as possible. There is a community built around WiX and I loved having the opportunity to help others use the tools I wrote and learned a lot about how to work in an open source project.

Beyond all those things that I got out of it from the project side. Rob has been a great coworker, mentor, and friend. I know that's not going to change. If I never would have gone to WiX that first night, I know there are many things I would not have learned and there are many people I may have never met. What happened to me by being a part of the 3rd and 4th generations of WiX Toolset developers was far more than just contributing code to an open source project. I grew in ways I would have never expected, met so many great people, and I built skills that I wouldn't have gotten elsewhere as easily that helped better my career.

I'd like to thank Rob and all the other developers that I've worked with on the project over the years. It was a ton of fun and I learned a lot.

I would encourage anyone reading this that feels that they'd like to join a great community, work with and learn from some great devs, and wants to experience a well run open source project from the inside, to contact Rob on his blog or join wix-devs as rob mentions in his latest post.

WiX: Really, Really Open Source

I haven't blogged about WiX in a while but there has been an pretty significant change recently that I thought I call out and comment on.

As some of you may have already seen, the founder of the WiX toolset, Rob Mensching, has left Microsoft and with it, so did the WiX toolset. Rob plans on focusing full time on WiX and I think that could be a very good thing for the toolset. WiX is a powerful toolset that enables massively scalable setup solutions designed to work for anyone from a one man product all the way up to massive enterprise level products like Office, Visual Studio, and many, many more.

Rob's dedication to the WiX project has been unceasing for 13 years. He put his blood, sweat, and tears into making sure WiX was able to deliver everything its users needed. I have personally worked with Rob for 7 of those years and have many fond memories of late nights solving tough problems. I'll probably follow this post up with some commentary about my experiences with Rob and WiX. Although this current change makes it a little harder for me to contribute to the project, I wish Rob all the best in his new adventure and look forward to seeing what WiX can become with him focused on it full time.

The .NET BCL Team released a preview of Immutable Collections

Check out the BCL team blog here. Anyone working on highly multithreaded software in .NET might find this quite useful.

.NET: .NET Framework 4 setup reports success but my .NET 4 applications wont run and tell me I need to install it.

I'm curious to hear from people that are hitting or have hit this issue. Leave a comment or contact me directly through my blog.

 

The issue here is often that %windir%\system32\mscoree.dll is not updated as part of the install. Behind the scenes of the installer, .NET 4 is trying to install an update to the OS that it depends on but Windows Update is telling the .NET 4 installer that the update is not applicable to the machine. The .NET 4 installer treats this response to mean that is not needed and moves on. In general, if Windows update is returning "not applicable" for and update one of wo things is true. Either its not applicable, or something is really wrong with Windows or Windows Update and no updates including critical security updates are able to install.

 

So far I have found two root causes of this issue:

 

1. %windir%\servicing\trustedinstaller.exe is not present on the machine.

2. The operating system is not an officially released version of the OS.

 

I'm curious to hear from anyone experiencing this issue. I'd like to know which of this issues it wascor if there is a root cause of this that I have not found.

 

Trouble Shooting/Self Help Instructions:

 

1. %windir%\servicing\trustedinstaller.exe is not present on the machine.

 

If this is the root of your issue. I would recommend following the steps in this article to try to get trustedinstaller.exe put back http://support.microsoft.com/kb/929833 .
If anyone has any idea what would have removed trustedinstaller.exe from their machine, I'd love to hear about it.

 

2. The Operating System is not an officially released version of the OS.

 

You can check this by looking at the version of %windir%\servicing\trustedinstaller.exe or looking in %windir%\logs\CBS\CBS.log. In the log you should see logging lines that look something like this: "Loaded Servicing Stack v6.1.7601.17592".

 

For Windows 7, I would expect to see a version of at least v6.1.7600.16385. If you see a number lower than this on Windows 7, the only way I know of to fix this issue is to do an upgrade to a supported released version of Windows 7. The more important thing in this case is that pre-release OS's do not get updated with critical security patches from Microsoft. Thats not a really good situation to be left in, especially if you dont know it.

 

I've seen many reports of people having a build number of v6.1.7600.16384 which is one number below the released build. I'm really curious how people would have gotten this build of the OS. The fact that its so close to the final build makes me wonder if it could have been accidentally released somehow. Please let me know if you got it through an official supported channel and its the wrong version.

I'd like to thank Mike Lewis for contacting me and helping me find the 2nd root cause. I'd love to hear from anyone else hitting this issue to make sure I can write down all the causes of this for people.

.NET: Do You Deploy a Managed App - Part 2

Almost 2 years ago I asked for feedback on your experience deploying .NET applications and using the .NET Framework deployment packages (Do you deploy a managed App).

We took a lot of the feedback from my blog and various other places and implemented features in .NET 4.0. Based on the feedback, we focused on size, performance, and robustness which I also recently posted about (The .NET Framework 4 Installer Improvements).

I’m really interested in getting feedback from everyone on how we did and get ideas for what we can do to make the experience better in the next release. We definitely take every bit of feedback seriously and value it highly and I want to say I always appreciate the time people spend to respond.

Feel free to answer as many of these questions as you are compelled to.

Questions:
1. Have you decided to update your application to .NET 4? Was deployment a factor in that decision?
2. Did .NET Framework 4 solve all of your deployment problems? Did you like it?
3. Where did we miss with .NET Framework 4 deployment? What didn’t you like?
4. What do you think about the .NET Framework 4 Client Profile option?
5. Did you find all the documentation and examples you needed to be able to use .NET 4?
6. What do you think about the new size of the .NET package?
7. Do you know about the redist and web installers? If so, which do you use?
8. Do you use a bootrapper/chainer that preinstalls .NET? If not, do you block? If so,
- Which one (VSI, InstallShield, Wise, ClickOnce, custom)?
- Do you run .NET in silent mode or UI mode?
9. Do you have any specific problems you can tell me about that you have had in deploying the .NET Framework?
Some more question that will help me understand your scenario:
10. How large is your application? (Size, install base)
11. How is your product deployed (Web download, CD, DVD, USB)?

Optional:
12. What company are you with?
13. What is your application?
14. Can I follow up with you? If so, email me your contact info along with your answers

As always. I very much appreciate getting everyone’s feedback. Hope to hear from many of you.

The .NET Framework 4 Installer Improvements

Thanks to everyone that gave feedback both on my blog and through other forums about .NET Framework installs. .NET 4 just released so I thought I’d take this opportunity to talk a little about the stuff we have done to improve the installation. My team and I have been focused over the past year on incorporating feedback and striving to make the installer better for the .NET Framework 4. There has been a particular focus on making it better for client applications to install it with their apps.

The key focus areas for the .NET 4 installer have been Size, Robustness, and Performance. I’ll speak to some of the major things we did and give a brief description below.

Size

You can see from the chart below that we have reduced the size of the .Net redist significantly. It went from client applications needing to carry 237MB to 41MB for the client profile. This is a game changer for a lot of client applications.

Size Comparison of the redist size since 3.5 SP1

	3.5 SP1	4 Beta1	4 Beta2	4 RTM
32bit Client Profile	X	34.5 MB	31.5 MB	28.8 MB
32bit Full	X	77.5 MB	38.5 MB	35.3 MB
32+64bit Client Profile	X	72.5 MB	48.2 MB	41 MB
32+64bit Full	237 MB	162.6 MB	55.9 MB	48.1 MB

 

Better Compression Across Packages

We implemented the use of a better compression technology into our packages which reduced the size of our packages by around 15%.

Separate packages for AMD64 and IA64

We found that there was little to no need to ever install the same package on both amd64 and ia64. Because of this, we decided to produce amd64 packages that excluded ia64 binaries as well as ia64 packages that didn’t contain amd64 binaries.

Client Profile

We determined the subset of framework functionality that was used by 95+% of client applications and produced a first class package for this scenario. The result of this is that, unless you are taking advantage of features such as ASP.NET, you can now take a dependency on a smaller framework. More details of what is in the client profile can be found here.

Remove Duplicate MSIL

We identified many assemblies that were functionally identical but differed by the architecture they were built under. These assemblies were all managed CPU neutral assemblies which meant that it didn’t matter whether they were built for x86 or amd64. Their strong names and functionality are the same. We solved this by only carrying one of them.

Robustness

The numbers are already starting to come in from the RTM release and we are very happy with the success rates of the install. At this point we are seeing greater than a 99% success rate which is much better than previous .NET redist releases right out of the gate. Here are some of the larger things we did to increase robustness.

Remove Prerequisites

In a chain of installs, the chain is only as strong as its weakest link. In addition, small weaknesses in each part of the chain compound to lead to higher failure rates for the whole chain. By removing numerous prerequisites and combining the whole client install into a single MSI, we were able to get rid of the compounding effect of failures as well as focus our efforts on making the single MSI as solid as possible.

Simplify the MSI

Custom actions are very common places for installs to fail. The more you have, the more complex the installer gets and the number of points of failure goes up. Removing the need for customactions in many cases and in the cases where we needed them, simplifying them has increased our success rates.

Remove slipstreamed feature MSP’s

In Beta1, we slipstreamed features into the installer’s msi using patches. This proved to be a point of complexity and the root cause of many unsolvable bugs. Due to that, we simplified our install to be completely contained in a single msi per platform.

Fix and Retry

Through thorough investigation of our past installers, looking through KB articles, feedback from customers, and through our past Beta’s, we found numerous install failure conditions that were fixable after the error. We implemented the KB articles and other workarounds so, in failure cases, we can fix the users machine and try again. We’ve seen quite an increase in our success rates due to this work. My hope is that this will also make the windows installer ecosystem cleaner and that msi’s installed after .NET 4 will have a better chance to succeed because our installer put the machine in a better state.

Triple fallback on Download failures

Through analysis of our download failures in the past, we determined that using a single implementation for downloads left you only as successful as that technology allows. We found that between Winhttp, URLMon, and BITS, their failures were in different scenarios and where one would fail, the others would succeed. In order to take advantage of this, our chainer falls back and retries on different download stacks to do everything we can to get a successful download.

Separated out Server configuration from Client Profile

The Client Profile installer should be more robust for applications now because some of the most common failing custom actions in .NET 3.5 were in configuring things like ASP.NET and WCF which are mainly server scenarios and not used by client applications. By moving these to the full install, we are seeing higher success rates for the client install.

Performance

The key metric here is overall install time for the redist. In 3.5 SP1 the average install time was 12-15 minutes. With v4, the average times are closer to 3-5 minutes for the client profile and 5-7 minutes for Full.

Smart Cabbing

Smart cabbing is a technique used to allow you to install the same file to multiple locations but only carry the file once in the msi’s cabs. This technique has been used for years but during our perf investigations, we found that, depending on how many duplicate files there were and where they were in the cab, performance degraded significantly. We made some bug fixes in the tools we use to smart cab (WiX) to reduce the impact of duplicated items while still gaining the benefits of smart cabbing.

Remove Prerequisites

This one is fairly self explanatory. We need to install less packages so we are faster. This is mostly the result of changing the .NET Framework itself to not have certain dependencies or carry subsets of the dependencies within the framework. In a few cases, this was possible because the base functionality was either built into all the supported OS’s or had enough ubiquity in the ecosystem to not warrant us carrying it.

Remove Slipstreamed Msp’s

We found that when applying large slipstreams to a product, there was a significant perf hit towards the end of the install when Windows Installer is caching the packages for source resiliency. By adding all the features into the MSI, we got rid of this performance hit.

Parallel Ngen and removal of synchronous 64bit assemblies

The CLR implemented the ability to ngen on multiple cores in parallel. We made changes in our installer to take advantage of this so now on a multicore machine, ngen times should be significantly reduced. Also, on 64bit machines, most .NET applications run as 32bit. This means that paying the price of creating 64bit native images is not something most apps need to do.

Client Profile

By producing a subset of the .NET install that contains the features most client applications need, most client applications can take advantage of shorter install times by installing less.

Parallel Download and Install
If you are using the web bootstrapper which we made available for the first time in Beta2, you can use the web bootstrapper to install .NET Framework 4. This has the advantage of downloading and installing the payload in parallel. For example, as it is installing the Client Profile, it will be downloading the rest of the framework. In cases where you have enough bandwidth to download the rest before the Client Profile install finishes, you essentially save the time it took to download the rest.

This was originally posted here and may have additional comments.

Which Version of .NET is Built into Windows?

Over the past few years I have been asked more and more which version of .NET is included in the operating system. The following chart shows from .NET 2.0 through 3.5, which version of .NET is included in both the Client and Server Windows operating systems.



Note that this diagram also shows which pieces are optional and whether or not they are on by default in that particular OS.