WiX: Why I Contributed for 7 years

Rob Mensching recently posted a call out to people who may be interested in joining him as a member of the 5th generation of WiX Toolset developers. As a big part of both the 3rd and 4th generations, I thought I'd share a few things about why I contributed to the project for so long. Maybe some of these reasons will resonate with you and WiX is the right fit for you. In order to do that, I'll start from the beginning...

I remember the first time I met Rob and the other WiX developers. I was a new developer at Microsoft basically fresh out of college. I was trying to solve a deployment problem for Visual Studio and my boss and I at the time had some ideas about how to solve it but we needed some changes to WiX. I decided that I should go meet with the WiX guys and bounce the ideas off of them and they held "WiX night" every Thursday night in a conference room at Microsoft.  I went to the meeting where everyone was sitting around writing code and having discussions ranging from software design to what they were up to in their personal lives. I could tell, they all enjoyed working together. They said hi to me and welcomed me to the meeting and I sat down.

After about 30 minutes, Rob turned to me and said, "So what brings you here?". I told him that I was working on Visual Studio Setup and I had a request for a feature in WiX. I wont go into the details of what the feature was even though I vividly remember it. I described the feature to them and they all came back at me saying "Absolutely NOT!". "That is the wrong way to solve that problem.", they said in a good sort of way :). They quickly started diving into the details of why I wanted the feature, what was the problem I was trying to solve and they were all very interested in helping me solve the problem. As we worked through it, we ended up landing on a good, simple, and far more generic solution to the problem. After we came up with a design, they asked me if I wanted to try and code up this new feature. After seeing the investment they made in my problem and having so much fun designing the feature, I couldn't resist and volunteered to do it.

I started going to WiX night every week to work on the feature. I loved the experience of working closely with other developers where I could bounce ideas off of them, get advice, get real time code reviews, and crack some jokes while we were at it. Over the next few weeks I felt like friends with the people there and started to get a sense for how impactful the WiX Toolset was. After my first feature, Rob came up to me and asked me what kind of experience I was looking to get out of being a part of WiX. He then helped figure out which part of the toolset I could start to learn in more depth to gain that experience. I started to feel really invested in the project as a whole and I felt like the WiX team was invested in me as well.

I learned a ton of core coding and design skills while working on WiX. The principle of "Always do the right thing" shone through in every change to the project. There were very clear coding guidelines and standards that helped keep the codebase clean and maintainable. Designs were always well thought through with the bigger picture in mind and not just solving the problem at hand as quickly as possible. There is a community built around WiX and I loved having the opportunity to help others use the tools I wrote and learned a lot about how to work in an open source project.

Beyond all those things that I got out of it from the project side. Rob has been a great coworker, mentor, and friend. I know that's not going to change. If I never would have gone to WiX that first night, I know there are many things I would not have learned and there are many people I may have never met. What happened to me by being a part of the 3rd and 4th generations of WiX Toolset developers was far more than just contributing code to an open source project. I grew in ways I would have never expected, met so many great people, and I built skills that I wouldn't have gotten elsewhere as easily that helped better my career.

I'd like to thank Rob and all the other developers that I've worked with on the project over the years. It was a ton of fun and I learned a lot.

I would encourage anyone reading this that feels that they'd like to join a great community, work with and learn from some great devs, and wants to experience a well run open source project from the inside, to contact Rob on his blog or join wix-devs as rob mentions in his latest post.

WiX: Really, Really Open Source

I haven't blogged about WiX in a while but there has been an pretty significant change recently that I thought I call out and comment on.

As some of you may have already seen, the founder of the WiX toolset, Rob Mensching, has left Microsoft and with it, so did the WiX toolset. Rob plans on focusing full time on WiX and I think that could be a very good thing for the toolset. WiX is a powerful toolset that enables massively scalable setup solutions designed to work for anyone from a one man product all the way up to massive enterprise level products like Office, Visual Studio, and many, many more.

Rob's dedication to the WiX project has been unceasing for 13 years. He put his blood, sweat, and tears into making sure WiX was able to deliver everything its users needed. I have personally worked with Rob for 7 of those years and have many fond memories of late nights solving tough problems. I'll probably follow this post up with some commentary about my experiences with Rob and WiX. Although this current change makes it a little harder for me to contribute to the project, I wish Rob all the best in his new adventure and look forward to seeing what WiX can become with him focused on it full time.

.NET: .NET Framework 4 setup reports success but my .NET 4 applications wont run and tell me I need to install it.

I'm curious to hear from people that are hitting or have hit this issue. Leave a comment or contact me directly through my blog.

 

The issue here is often that %windir%\system32\mscoree.dll is not updated as part of the install. Behind the scenes of the installer, .NET 4 is trying to install an update to the OS that it depends on but Windows Update is telling the .NET 4 installer that the update is not applicable to the machine. The .NET 4 installer treats this response to mean that is not needed and moves on. In general, if Windows update is returning "not applicable" for and update one of wo things is true. Either its not applicable, or something is really wrong with Windows or Windows Update and no updates including critical security updates are able to install.

 

So far I have found two root causes of this issue:

 

1. %windir%\servicing\trustedinstaller.exe is not present on the machine.

2. The operating system is not an officially released version of the OS.

 

I'm curious to hear from anyone experiencing this issue. I'd like to know which of this issues it wascor if there is a root cause of this that I have not found.

 

Trouble Shooting/Self Help Instructions:

 

1. %windir%\servicing\trustedinstaller.exe is not present on the machine.

 

If this is the root of your issue. I would recommend following the steps in this article to try to get trustedinstaller.exe put back http://support.microsoft.com/kb/929833 .
If anyone has any idea what would have removed trustedinstaller.exe from their machine, I'd love to hear about it.

 

2. The Operating System is not an officially released version of the OS.

 

You can check this by looking at the version of %windir%\servicing\trustedinstaller.exe or looking in %windir%\logs\CBS\CBS.log. In the log you should see logging lines that look something like this: "Loaded Servicing Stack v6.1.7601.17592".

 

For Windows 7, I would expect to see a version of at least v6.1.7600.16385. If you see a number lower than this on Windows 7, the only way I know of to fix this issue is to do an upgrade to a supported released version of Windows 7. The more important thing in this case is that pre-release OS's do not get updated with critical security patches from Microsoft. Thats not a really good situation to be left in, especially if you dont know it.

 

I've seen many reports of people having a build number of v6.1.7600.16384 which is one number below the released build. I'm really curious how people would have gotten this build of the OS. The fact that its so close to the final build makes me wonder if it could have been accidentally released somehow. Please let me know if you got it through an official supported channel and its the wrong version.

I'd like to thank Mike Lewis for contacting me and helping me find the 2nd root cause. I'd love to hear from anyone else hitting this issue to make sure I can write down all the causes of this for people.

.NET: Do You Deploy a Managed App - Part 2

Almost 2 years ago I asked for feedback on your experience deploying .NET applications and using the .NET Framework deployment packages (Do you deploy a managed App).

We took a lot of the feedback from my blog and various other places and implemented features in .NET 4.0. Based on the feedback, we focused on size, performance, and robustness which I also recently posted about (The .NET Framework 4 Installer Improvements).

I’m really interested in getting feedback from everyone on how we did and get ideas for what we can do to make the experience better in the next release. We definitely take every bit of feedback seriously and value it highly and I want to say I always appreciate the time people spend to respond.

Feel free to answer as many of these questions as you are compelled to.

Questions:
1. Have you decided to update your application to .NET 4? Was deployment a factor in that decision?
2. Did .NET Framework 4 solve all of your deployment problems? Did you like it?
3. Where did we miss with .NET Framework 4 deployment? What didn’t you like?
4. What do you think about the .NET Framework 4 Client Profile option?
5. Did you find all the documentation and examples you needed to be able to use .NET 4?
6. What do you think about the new size of the .NET package?
7. Do you know about the redist and web installers? If so, which do you use?
8. Do you use a bootrapper/chainer that preinstalls .NET? If not, do you block? If so,
- Which one (VSI, InstallShield, Wise, ClickOnce, custom)?
- Do you run .NET in silent mode or UI mode?
9. Do you have any specific problems you can tell me about that you have had in deploying the .NET Framework?
Some more question that will help me understand your scenario:
10. How large is your application? (Size, install base)
11. How is your product deployed (Web download, CD, DVD, USB)?

Optional:
12. What company are you with?
13. What is your application?
14. Can I follow up with you? If so, email me your contact info along with your answers

As always. I very much appreciate getting everyone’s feedback. Hope to hear from many of you.

The .NET Framework 4 Installer Improvements

Thanks to everyone that gave feedback both on my blog and through other forums about .NET Framework installs. .NET 4 just released so I thought I’d take this opportunity to talk a little about the stuff we have done to improve the installation. My team and I have been focused over the past year on incorporating feedback and striving to make the installer better for the .NET Framework 4. There has been a particular focus on making it better for client applications to install it with their apps.

The key focus areas for the .NET 4 installer have been Size, Robustness, and Performance. I’ll speak to some of the major things we did and give a brief description below.

Size

You can see from the chart below that we have reduced the size of the .Net redist significantly. It went from client applications needing to carry 237MB to 41MB for the client profile. This is a game changer for a lot of client applications.

Size Comparison of the redist size since 3.5 SP1

	3.5 SP1	4 Beta1	4 Beta2	4 RTM
32bit Client Profile	X	34.5 MB	31.5 MB	28.8 MB
32bit Full	X	77.5 MB	38.5 MB	35.3 MB
32+64bit Client Profile	X	72.5 MB	48.2 MB	41 MB
32+64bit Full	237 MB	162.6 MB	55.9 MB	48.1 MB

 

Better Compression Across Packages

We implemented the use of a better compression technology into our packages which reduced the size of our packages by around 15%.

Separate packages for AMD64 and IA64

We found that there was little to no need to ever install the same package on both amd64 and ia64. Because of this, we decided to produce amd64 packages that excluded ia64 binaries as well as ia64 packages that didn’t contain amd64 binaries.

Client Profile

We determined the subset of framework functionality that was used by 95+% of client applications and produced a first class package for this scenario. The result of this is that, unless you are taking advantage of features such as ASP.NET, you can now take a dependency on a smaller framework. More details of what is in the client profile can be found here.

Remove Duplicate MSIL

We identified many assemblies that were functionally identical but differed by the architecture they were built under. These assemblies were all managed CPU neutral assemblies which meant that it didn’t matter whether they were built for x86 or amd64. Their strong names and functionality are the same. We solved this by only carrying one of them.

Robustness

The numbers are already starting to come in from the RTM release and we are very happy with the success rates of the install. At this point we are seeing greater than a 99% success rate which is much better than previous .NET redist releases right out of the gate. Here are some of the larger things we did to increase robustness.

Remove Prerequisites

In a chain of installs, the chain is only as strong as its weakest link. In addition, small weaknesses in each part of the chain compound to lead to higher failure rates for the whole chain. By removing numerous prerequisites and combining the whole client install into a single MSI, we were able to get rid of the compounding effect of failures as well as focus our efforts on making the single MSI as solid as possible.

Simplify the MSI

Custom actions are very common places for installs to fail. The more you have, the more complex the installer gets and the number of points of failure goes up. Removing the need for customactions in many cases and in the cases where we needed them, simplifying them has increased our success rates.

Remove slipstreamed feature MSP’s

In Beta1, we slipstreamed features into the installer’s msi using patches. This proved to be a point of complexity and the root cause of many unsolvable bugs. Due to that, we simplified our install to be completely contained in a single msi per platform.

Fix and Retry

Through thorough investigation of our past installers, looking through KB articles, feedback from customers, and through our past Beta’s, we found numerous install failure conditions that were fixable after the error. We implemented the KB articles and other workarounds so, in failure cases, we can fix the users machine and try again. We’ve seen quite an increase in our success rates due to this work. My hope is that this will also make the windows installer ecosystem cleaner and that msi’s installed after .NET 4 will have a better chance to succeed because our installer put the machine in a better state.

Triple fallback on Download failures

Through analysis of our download failures in the past, we determined that using a single implementation for downloads left you only as successful as that technology allows. We found that between Winhttp, URLMon, and BITS, their failures were in different scenarios and where one would fail, the others would succeed. In order to take advantage of this, our chainer falls back and retries on different download stacks to do everything we can to get a successful download.

Separated out Server configuration from Client Profile

The Client Profile installer should be more robust for applications now because some of the most common failing custom actions in .NET 3.5 were in configuring things like ASP.NET and WCF which are mainly server scenarios and not used by client applications. By moving these to the full install, we are seeing higher success rates for the client install.

Performance

The key metric here is overall install time for the redist. In 3.5 SP1 the average install time was 12-15 minutes. With v4, the average times are closer to 3-5 minutes for the client profile and 5-7 minutes for Full.

Smart Cabbing

Smart cabbing is a technique used to allow you to install the same file to multiple locations but only carry the file once in the msi’s cabs. This technique has been used for years but during our perf investigations, we found that, depending on how many duplicate files there were and where they were in the cab, performance degraded significantly. We made some bug fixes in the tools we use to smart cab (WiX) to reduce the impact of duplicated items while still gaining the benefits of smart cabbing.

Remove Prerequisites

This one is fairly self explanatory. We need to install less packages so we are faster. This is mostly the result of changing the .NET Framework itself to not have certain dependencies or carry subsets of the dependencies within the framework. In a few cases, this was possible because the base functionality was either built into all the supported OS’s or had enough ubiquity in the ecosystem to not warrant us carrying it.

Remove Slipstreamed Msp’s

We found that when applying large slipstreams to a product, there was a significant perf hit towards the end of the install when Windows Installer is caching the packages for source resiliency. By adding all the features into the MSI, we got rid of this performance hit.

Parallel Ngen and removal of synchronous 64bit assemblies

The CLR implemented the ability to ngen on multiple cores in parallel. We made changes in our installer to take advantage of this so now on a multicore machine, ngen times should be significantly reduced. Also, on 64bit machines, most .NET applications run as 32bit. This means that paying the price of creating 64bit native images is not something most apps need to do.

Client Profile

By producing a subset of the .NET install that contains the features most client applications need, most client applications can take advantage of shorter install times by installing less.

Parallel Download and Install
If you are using the web bootstrapper which we made available for the first time in Beta2, you can use the web bootstrapper to install .NET Framework 4. This has the advantage of downloading and installing the payload in parallel. For example, as it is installing the Client Profile, it will be downloading the rest of the framework. In cases where you have enough bandwidth to download the rest before the Client Profile install finishes, you essentially save the time it took to download the rest.

This was originally posted here and may have additional comments.

WiX: Removing Files with Patches

I've had numerous people ask me how to remove a file using a patch. As I say below again, this should be done carefully.

 

Background

If a file is present in a Baseline package then removed in the Upgrade package, it is not trivial to get the installer to remove the file in a way that uninstalling the patch will put it back.

 

Important!!! MUST READ

 

Before removing a file you need to make sure you have no other options. The only time you should remove a file using a patch is when the mere presence of the file causes problems. Consider removing references to the file from files that use it or patching it to be an "empty" file. Removing a file should be a last resort.

 

For shared files, removing a file with a patch could mean breaking any other product using that file. The removal of the file does not take reference counting into account when determining to remove a file. It will be removed if you tell it to be removed. Make sure you consider everything that uses or could use the file before removing it.

 

If a file must be removed by a patch there are a few options:

If the file is not a the keypath of its component:
- Remove the file from the component- Add a RemoveFile entry to the component.

If the file is the keypath if its component:

If the file is the keypath, it means this file is the identity of the component. In this case the entire component must be removed. You cannot simply remove a component using a patch. If you do, you have to remove the entire feature which is not usually the desired scenario. If it is, then just remove the feature from your patch.

In order to remove a component without removing its feature you need to convince Windows Installer that the component isnt supposed to be installed but the msi database still needs to know about it. You can achieve this by making the component's condition false. In order to cause a recalcalculation of the component during repair/patch you also need to mark the component as transitive. The side-affect of this is that when the patch is uninstalled, the file is not put back on the machine. This is because when the patch is removed, the component is no longer transitive and its condition is not reevaluated. You can solve this by shipping two patches. One that marks the component as transitive and a second that makes the component's condition false.

Example:
<Component Id="Foo" Transitive="yes" ... >
    <Condition>FALSE</Condition>      ...
</Component>

Notes and considerations:

In any case, uninstalling the patch may prompt for source if the file that needs to be put back is not available to the installer.

If building 2 patches for the transitive solution, you cannot base the two patches off of the same build because both changes are made to a single component which is the smallest level at which patches can be filtered. 

.NET: Do you Deploy a Managed App?

(Thanks for the replies so far! It is very insightful to hear all the different ways people deploy .NET apps. I am taking all your answers to the questions and using them to help inform decisions about the future of .NET deployment. I appreciate the responses.)

Do you own the deployment of your product and require the .NET Framework to be installed before you can install your application? If so, I want to hear from you.

I have a few general questions but any comments are welcome as long as they are constructive and actionable. My goal is to use this data to do what I can to improve the .NET Framework Redist experience in future versions.

Do you have a bootstrapper/chainer that preinstalls .NET? If so, which one (VSI, InstallShield, Wise, ClickOnce, custom)?

Do you block and point your user to the location to download and install .NET?

How large is your application?

How is your product deployed (Web download, CD, DVD, USB)?

What do you think about the size of the .NET package?

Do you know about the small package for .NET 3.5 that will download only what your user need? If so, do you use it?

What do you like about the .NET Framework Redist?

What do you dislike about the .NET Framework Redist?

Do you have any specific problems you can tell me about that you have had in deploying the .NET Framework?

Like I said, the info above will be helpful to me but I dont want to limit responses to these questions. We have a lot of data about the various issues people encounter while installing the .NET Framework but we dont have as much data about the ways people are deploying it and using it as part of their deployment. This is the type of info I'm looking to gather.

You can post as a comment or email me the info if you dont want it posted and seen by all.

This was originally posted here and may have additional comments.

WiX: Patching something you didnt build with WiX using WiX

Many times people want to take advantage of the Wix v3 patch building features but didnt build their original product using Wix. for example, they may have used Install Shield, WiX v2 or one of the other tools out there. I ran into this when I needed to create a patch for a product we shipped from Visual Studio before the Visual Studio build moved to WiX v3.

 

 WiX v3 has a feature I added a while back to support this scenario. It allows you to point at an admin image of your msi based install package. This will be very familiar to some of you who have used patchwiz and pcp's in the past. The difference here is that your patch authoring can be done using the WiX v3 Patch element and it follows the same workflow as patching a WiX v3 based msi after the first stage of creating the diffs.

Something I found really easy, and I could see this being used by smaller products who dont have a lot of infrastructure, is the ability to hand edit your admin image to product the patch. For example, I wanted to update a single file in my product:

1. I created an admin image
2. I made a copy of the admin image
3. I updated the loose file in my copy of the admin image

After running it through the series of command lines, I got a patch that updated the file. Pretty simple...

The command lines you need to run are all the same as the ones in my sample: Building a Patch using the new Patch Building System - Part 3. The only difference is that you need to pass the -ax command to torch with a path to extract embedded binaries to instead of the -xi switch and add the -xo to specify to create a wixmst instead of an mst. This tells torch that the input is admin image instead of xml. Also, specify the path to the msi in the admin images instead of the wixpdb as the target and upgrade. Everything after that should follow the same process.

Let me know how it works! :)

Bonus feature: One of the most useful things in the WiX v3 patching system is the ability to filter whats in your patch using patch families to only ship your customers the updates they need. This allows you to grab changes per Fragment from your wix source. When patching from admin images, there is no source so you would think that the filtering isn't suppoted. Not so! This was one of the more fun parts of this implementation. We "auto-fragment" the msi into what we consider to be the optimal chunks to give you the ability to filter if you choose to do so.

This was originally posted here and may have additional comments.

WiX: Introducing the WixPdb

I should have made this post about 6 months ago as I was implementing the WixPdb feature in WiX but better late than never :).  The wixpdb was conceived as a solution for a limitation in the original patch building system. Originally, WiX v3 patching only supported creating transforms between two wixout files. This was flawed in that many things modify the output after the wixout is generated by light which weren't being reflected in patches. I knew I needed the patch build system to be able to create transforms from something closer to the final output. Thats what the WixPdb is. It is all the data in the output right before creating the final msi. With that data, we can now create more accurate transforms to what people expect when diffing two products. I updated my blog with samples on how to build patches to show that I now recommend using the wixpdb instead of the wixout.

 

Down the road, I look forward to seeing what other features we can get out of the wixpdb. It provides a ton of information an allows you to potentially connect install time errors all the way back to the wix source and line number. Static analysis is another benefit. I updated smoke to be able to take the wixpdb so it will give you source and line number information for ICE errors. Statis analysis on the wixpdb itself may also be interesting because it contains more data than in the msi. It is the one place where the wix source data and the msi data connect. Let me know if you have any ideas of how to use the wixpdb in some cool ways...

Design your Application to Fit Your Shipping Container

Let me start by saying I'd love to get people's feedback on the content of this post...
When Toyota designs a car they would never ignore the fact that no matter how great a car they build, they have to be able to get it to their customers. Additionally, they have to be able to get it to them at a reasonable cost. Imagine that Toyota decided that it would be cool to create a car that is 4 seats wide. Assume they ignored that the world's infrastructure is built to handle cars two seats wide. Can you imagine what would happen if they did that? Cars wouldn't fit on boats, on trucks, in garages, etc. In short, the cars would sit at the factory. Assuming the world wants the car people would scramble to create the infrastructure to get the cars to the users. In the end all of this would drive up the cost of shipping the cars and cause customers to pay more money to get it.

Software is no different
Unfortunately, software is often designed without considering its ship vehicle. What usually matters to developers is that it runs on their machine. It is designed and built until it works and is tested. Only then is deployment thought about. At that point, the application usually does tons of things that make software deployment a nightmare and costs way more that it would have if it were considered up front.

There's a way out
The standard for deployment on Windows is the Windows Installer (MSI). Luckily, for the applications that think about deployment last, Windows Installer is very extensible. If you want to run a bunch of custom code during install, you can. Having to do this is usually a sign that your application was not designed to make your deployment simple. You have to build a bunch of infrastructure as a band-aid for something that could have likely been avoided. Your custom code inevitably wont behave properly in some scenarios causing your entire deployment to become fragile. Whenever you see this, consider an upstream design change. This is more likely feasible if you aren't just about to ship your product and you aren't scrambling to build an installer package.

Make it simple
Setup can be really simple. Everything on a machine boils down to very simple things: Files, registry keys, and shortcuts. Windows Installer happens to handle Files, registry keys, and shortcuts very well. It's actually really easy to use if all you need to do to deploy your application is copy files, write registry keys, and add shortcuts to the system.

Its often not your application's fault
IIS did not make it easy to copy files to a location to install a web site. .NET didn't make it easy to GAC an assembly or NGEN it by copying it to a location. Both of these are things I like to call custom stores. Applications that expect other applications to extend them or use them as a platform need to be even more conscious of the deployment problems they cause. When platforms don't consider what it will take to deploy on top of them they make every one of their customers building solutions has to modify their shipping container to handle their custom store. Although we cant get rid of the custom stores that have already shipped, we can hopefully prevent more and more custom stores from being produced. I think people building on top of platforms should push back more when their platforms create more and more custom stores.

Start development with deployment
When building your applications, think about deployment as you design your app. Think, "Can I put this on my customers machine by only copying files"? If it starts to get hard, don't just write custom code to solve it. Think, "Could my application make itself easier to deploy?" Ideally, you could build your application to a folder and just run it without any configuration steps. You should create an empty installer package at the beginning of your project and build it up as you ad functionality to your product.

Application virtualization is on the horizon
If you make your application deployment completely declarative where no custom code has to run on your customers machine, your application will fit into the virtualized application world much better. It will also allow scenarios like drag and drop deployment, running from USB key.

Disclaimer
Many of the things I've said as far as guidance are optimistic. The world isn't as clean as I wish it were. But in the end, I dont think we'll ever be in a better situation unless people start thinking about the problems and stop making things worse for the Windows eco system by creating custom stores in platforms that are not installer friendly.

This was originally posted here and may have additional comments.

WiX: BinderExtensions and the BinderFileManager

As of a few weeks ago, the BinderExtension class in the wix extensibility model has taken on a new meaning. If you currently have a BinderExtension, you will need to rename it to BinderFileManager. Over the past year or so, numerous requests came in to give WixExtensions access to data and bind time to pull disperate data together or get access to data only available at bind time.

For example, if I have a custom table where I need to know the exports of all my native dll's and I want this table to be populated dynamically by wix, I would need to have access to populate my table after files are resolved. With the new BinderExtension class, this is now possible.

Key Changes:

What used to be called BinderExtension was renamed to BinderFileManager. The BinderFileManager handles things specific to a files, mainly file resolution and comparison. Only one instance of a BinderFileManager can be present in the WixExtensions passed to the wix tools and is commonly used in an extension specific to a build environment.

It is now possible to have a BinderExtension defined in each WixExtension passed to the command line tools. This allows you to:

1. Link disperate data associated with compile time language extensions. It essentially enables extensions to make additional decisions that the WiX linker cant do for you because it doesnt natively support your language extension. Once the linker has pulled all the data together, it may be neccessary for you to read that data and populate additional tables.

2. Populate tables with bind time data such as information from files which are resolved during bind.
 In my series abut writing WixExtensions, I plan to show an example of this functionality. I plan to pick that series back up soon.

WiX: More powerful foreach

I've been meaning to post about this for a while but just never got around to it. A few months ago I extended the implementation in WiX to allow foreach loops using any variable over any given set of values. Previously it was limited to only those variables that were defined in your preprocessor extension.

 

While this example wont do much for you, it demonstrates what a foreach loop will do. <?foreach?> loops can simplify authoring in many ways, especially for complex products.

Example:

<?foreach number in 1;2;3 ?>
    <Property Id="Property$(var.number)" Value="value$(var.number) />
<?endforeach ?>

Will result in the following being passed to the compiler:

    <Property Id="Property1" Value="value1 />
    <Property Id="Property2" Value="value2 />
    <Property Id="Property3" Value="value3 />

I want to note as a caveat, It can be very easy to break component rules by changing the contents of a component by simply changing the values you iterate over. I recommend not using foreach loops anywhere near components unless you have really thought it through.  Also, while using the preprocessor can be very handy for minimizing the amount of wix code you need to maintain, it can really confuse some of the tools that are out there that help you generate or maintain your wix source. In general, think about other options before you use the preprocessor as a solution but if you think its your best option, enjoy :)

This was originally posted here and may have additional comments.

WiX: Writing your own WiX Extension Part 2: Preprocessor

The preprocessor in WiX allows extensibilty at a few levels. In this article I will describe how to add a PreprocessorExtension to your WixExtension and have it handle variables and functions you define in your own namespace.

 

[This sample assumes you have already gone through Part 1]

 

1. Add a new class to your project called SamplePreprocessorExtension.

2. If you added a new file for this class, make sure you add: using Microsoft.Tools.WindowsInstallerXml to you file.

3. Make your SamplePreprocessorExtension class implement PreprocessorExtension.

public class SamplePreprocessorExtension : PreprocessorExtension

 

4. Add your SamplePreprocessorExtension to your SampleWixExtension class and override the PreprocessorExtension property from the base class. This will make it so when WiX asks your extension for your preprocessor extension, you extension will know what to do.
private SamplePreprocessorExtension preprocessorExtension;
public override PreprocessorExtension PreprocessorExtension
{
    get      {         if (this.preprocessorExtension == null)
        {
            this.preprocessorExtension = new SamplePreprocessorExtension(); 
        }
        return this.preprocessorExtension;
    }
}

5. Now, back in your SamplePreprocessorExtension class, you need to specify what prefixes (or namespaces) your extension will handle. For example, if you want to be able to define a variable $(sample.ReplaceMe) then you need to specify that your extension will handle the "sample" prefix.

private static string[] prefixes = { "sample" };
public override string[] Prefixes { get { return prefixes; } }

6. Now that you have specified your prefixes, you now need to handle variables and functions that are passed to you from WiX. You do this by overriding the GetVariable and EvaluateFunction methods from the PreprocessorExtension base class.

public override string GetVariableValue(string prefix, string name)
{
     string result = null;
    // Based on the namespace and name, define the resulting string.
    switch (prefix)
    {
        case "sample":
            switch (name)
            {
                case "ReplaceMe":
                   // This could be looked up from any where you can access from your code.
                   result = "replaced";
                   break;
            }
            break;
    } 
    return result;
} 
  
public override string EvaluateFunction(string prefix, string function, string[] args)
{
    string result = null;
    switch (prefix)
    {
        case "sample":
            switch (function) 
            {
                case "ToUpper":
                    if (0 < args.Length) 
                    {
                        result = args[0].ToUpper();
                    }
                    else
                    {
                        result = String.Empty; 
                    }
                    break; 
            } 
            break; 
    } 
    return result;
}

7. Build

8. With this you can now pass your extension on the command line to candle and expect variables and functions in your namespace to be passed to your extension and be evaluated. To prove this, try adding the following properties your WiX source.
    <Property Id="VARIABLETEST" Value="$(sample.ReplaceMe)" />
    <Property Id="FUNCTIONTEST" Value="$(sample.ToUpper(lowercase))" />
You resulting msi should have entries in the Property table with the values "replaced" and "LOWERCASE" in the property table.

WiX: Writing Your Own WiX Extension Part 1

Wix extensions are used to extend and customize what WiX builds and how it builds it. I plan on talking about the 3 most common types of extensions in this article series; ProprocessorExtension, CompilerExtension, and BinderExtension.

 

The first step in creating any set of extensions is to create a class that implements WixExtension. This class will be the container for all the extensions you plan on implementing.

Creating a bare-bones WixExtension
1. Create a new C# library (.dll) project named SampleWixExtension
2. Add a reference to wix.dll to your project
3. Add a using statement: using Microsoft.Tools.WindowsInstallerXml;
4. Make your SampleWixExtension class inherit from WixExtension.

 public class SampleWixExtension : WixExtension {}

5. Add the AssemblyDefaultWixExtensionAttribute to your AssemblyInfo.cs.

 [assembly: AssemblyDefaultWixExtension(typeof(SampleWixExtension.SampleWixExtension))]

6. Build
7. Although it wont do anything yet, you can now pass the your SampleWixExtension.dll on the command line to the candle and light with the -ext flag like this: candle Product.wxs -ext SampleWixExtension.dll

The next article in this series will explain how to add a Preprocessor extension to your WixExtension.

WiX: Building a Patch using the new Patch Building System - Part 3

This example is aimed at demonstrating the workflow and command lines required to build a patch using the Patch element and Pyro.exe. I would appreciate any feedback anyone has.

Setting up the Sample:

1. Create a Directory that you plan on running the sample from. I will call this directory the sample root

2. Under the sample root directory create two subdirectories called "1.0" and "1.1"

3. Create a text file in the 1.0 directory called Sample.txt and put some text in it telling you that it is the 1.0 version of the file.

4. Create a text file in the 1.1 directory called Sample.txt and put some text in it telling you that it is the 1.1 version of the file.

You should now have 2 sub-directories under your sample root called 1.0 and 1.1 each containing a Sample.txt file whose contents state which version it is.

5. Create your product authoring in the sample root folder called Product.wxs with the following contents:

<?xml version="1.0" encoding="UTF-8"?>
<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi">
    <Product Id="48C49ACE-90CF-4161-9C6E-9162115A54DD" Name="WiX Patch Example Product" Language="1033" Version="1.0.0" Manufacturer="Dynamo Corporation" UpgradeCode="48C49ACE-90CF-4161-9C6E-9162115A54DD">
        <Package Description="Installs a file that will be patched." Comments="This Product does not install any executables" InstallerVersion="200" Compressed="yes" />
        <Media Id="1" Cabinet="product.cab" EmbedCab="yes" />
        <FeatureRef Id="SampleProductFeature"/>
    </Product>
    <Fragment>
        <Feature Id="SampleProductFeature" Title="Sample Product Feature" Level="1">
            <ComponentRef Id="SampleComponent" />
        </Feature>
    </Fragment>
    <Fragment>
        <DirectoryRef Id="SampleProductFolder">
            <Component Id="SampleComponent" Guid="{C28843DA-EF08-41CC-BA75-D2B99D8A1983}" DiskId="1">
                <File Id="SampleFile" Name="Sample.txt" Source=".\$(var.Version)\Sample.txt" />
            </Component>
        </DirectoryRef>
    </Fragment>
    <Fragment>
        <Directory Id="TARGETDIR" Name="SourceDir">
            <Directory Id="ProgramFilesFolder" Name="PFiles">
                <Directory Id="SampleProductFolder" Name="Patch Sample Directory">
                </Directory>
            </Directory>
        </Directory>
    </Fragment>
</Wix>

6. Create your patch authoring in the sample root called Patch.wxs with the following contents:
<?xml version="1.0" encoding="UTF-8"?>
<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi">
    <Patch
        AllowRemoval="yes"
        Manufacturer="Dynamo Corp"
        MoreInfoURL="http://www.dynamocorp.com/"
        DisplayName="Sample Patch"
        Description="Small Update Patch"
        Classification="Update"
        >
       
        <Media Id="5000" Cabinet="RTM.cab">
            <PatchBaseline Id="RTM"/>
        </Media>
        <PatchFamilyRef Id="SamplePatchFamily"/>
    </Patch>
    <Fragment>   
        <PatchFamily Id='SamplePatchFamily' Version='1.0.0' Supersede='yes'>
            <ComponentRef Id="SampleComponent"/>
        </PatchFamily>
    </Fragment>
</Wix>

You should now have a Product.wxs and Patch.wxs file in the sample root.

Instructions for building a Patch using the sample:
Open a command prompt where you normally run the WiX tools from.
Required WiX executables:
Candle.exe
Light.exe
Torch.exe
Pyro.exe
Your WiX toolset version should be at least 3.0.3001.0

1. Build Target Layout:
> candle.exe -dVersion=1.0 product.wxs
> light.exe -sval Product.wixobj -out 1.0\Product.msi

2. Build the Upgrade Layout:
> candle.exe -dVersion=1.1 product.wxs
> light.exe -sval Product.wixobj -out 1.1\Product.msi

3. Create the transform between your products:
> torch.exe -p -xi 1.0\Product.wixpdb 1.1\Product.wixpdb -out Patch\Diff.Wixmst

4. Build the Patch:
> candle.exe Patch.wxs
> light.exe Patch.wixobj -out Patch\Patch.WixMsp
> pyro.exe Patch\Patch.WixMsp -out Patch\Patch.msp -t RTM Patch\Diff.wixmst

I am currently working on reducing the number of command line switches required for building a simple patch. Sometime soon I hope to have the commands somewhat simplified by making the defaults match up with the patch building system.

Verifying what you have done here works: 
1. Install the 1.0 Product by running 1.0\Product.msi
2. Go to "Program Files\Patch Sample Directory" and open up Sample.txt. You should see that it is the 1.0 version.
3. Close Sample.txt.
3. Install the Patch from the sample root Patch\Patch.msp.
4. Repeat step 2 and notice that Sample.txt now contains the new 1.1 content.
5. Go to Add/Remove Programs and make sure "Show Updates" is checked.
6. Uninstall the Patch and repeat step 2 to see that the file was rolled back to its original version.
7. Uninstall the Product itself to clean off your system.

This was originally posted here and may have additional comments.

WiX: Building a Patch using the new Patch Building System - Part 2

I dont have much time today to write but I wanted to take a chance to answer a few questions that I've been asked a lot recently.

 

What is Pyro?
Pyro is the new tool in the WiX toolset that will pull together your patch authoring and your product transforms in order to build you an msp.

What is the new system and what is the old?
It is important to distinguish the new patch building system from the previous one provided by WiX. There has been some confusion and I want to try and clear it up. 

The Previous Way:
The PatchCreation element has been supported in WiX for a while to help people build patches. It essentially helps users build a pcp file which is then used as input to the PatchWiz tools. The PatchCreation element is not used in the new patch building system.

The New Way:The Patch Element is a newly added element that lets you describe the contents of your patch. It contains information about where the files your patch contains should be stored as well as summary information that describes your patch to your end user. It also supplies a mechanism to filter out specific changes to be included in your patch. When this element is passed to candle to be compiled and the resulting wixobj's are passed to light to link, the result is a WixMsp. This is an xml representation of the patch database (msp).

What Next?
The next thing I'd like to do is to provide a simple example of how to build a patch using Pyro.exe. I plan on getting this started in the next few days. Keep bugging me if I dont to remind me to do it and feel free to ask me more questions as this new functionality isn't all that well documented yet.

WiX: What is a WiX Extension?

A WiX extension is a dll that allows users to add their own custom elements, preprocessor functionality, and build customizations. These extensions take advantage of the various extension points in WiX. The most commonly used ones are Preprocessor, Compiler, and Binder extentions.

Mosts WiX extensions shipped with the WiX toolset are used to encapsulate common funtionality and make it easy for users to author things into their setups. They are targeted at solving setup problems in a standard way so everyone can benefit from and build on the WiX toolset.

Example: WixNetFxExtension.dll

Many people who write applications for the .NET Framework have various needs. I will discuss 2 here in an effort to describe 2 uses of extensions.

AppSearch (ComponentSearch, RegistrySearch, etc..):

Sometimes people want to find information about the system as part of their install. One way to get current system info is using an AppSearch.

Detecting if the .NET Framework is installed:

In reality, in order to detect if the .NET framework is installed, you need to search for a specific registry key and set a property if that key exists. Using the WixNetFxExtension you can just add the following to detect if the 2.0 Framework is installed.

<PropertyRef Id="NETFRAMEWORK20"/>

 The logic defining this property and all the logic behind it is stored in a WixLib which embedded inside the WixNetFxExtension.dll

Custom Actions:

Sometimes people want to do things that are not supported by Windows Installer. In this case they write custom actions. WiX provides a good mechanism for sharing custom actions.

Ngen’ing an Assembly:

To add a custom action to a product there is a lot of authoring to do. You need to define it and its binary, schedule it, and condition it. This doesn’t even consider that most custom actions come in sets, all sequenced at different times. This can all be done using a CustomActionRef in a similar way as the previous example.

After that, you need to populate any custom tables your custom action may rely on for its data. This is a lot complexity for users to author using standard WiX constructs. One use of a compiler extension can be to add your own element to the authoring and have that translate into a custom table in your msi that your customaction can read. The ngen custom actions have such an element.

<netfx:NativeImage Platform=”all” … />

The compiler will take this line and its context and populate a NativeImage table in the resulting MSI that will contain all the data needed by the ngen custom actions. The custom action also requires a dll binary in the binary table. In WiX v2 you had to ship your binary along with your extension. WiX v3 has support for binary WixLib’s. The key thing here is that the binary will get added into the WixLib for you and you don’t have to worry about shipping 2 separate files.

The resulting extension file:

The result of this is a .NET dll with a WixLib embedded in it as a resource. That WixLib then in turn has a cabinet at the beginning of it that contains the custom action binary. All of this nesting is handled for you by the linker and the binder so all you have to do to use stuff you put in your WixLib is add the extension to your candle and light command lines and reference the things defined in the WixLib from your product authoring.

WiX: Preprocessor error and warning

This is simple but could be handy. You can now add preprocessor instructions to you WiX authoring that will allow you to display a warning or error message when the preprocessor encounters that line.

Often errors and warnings would be used inside other prepropcessor constructs.

Example:

<?warning Must define the variable variableThatMustBeDefined ?>
<?ifndef variableThatMustBeDefined ?>
    <?error You did not define the variable variableThatMustBeDefined ?>
<?endif ?>

WiX: PatchFamily patch filtering

 

Using the Patch system in WiX 3.0 allows you to select which differences between 2 builds you want to ship. By default, all the changes will go into a patch, but sometimes people want to ship a targeted update to fix a security issue or major customer problem. This can be done using the PatchFamily concept built into WiX.

A patch family has 2 meanings in this context, it specifies which changes you would like in any given patch, and also provides a mechanism for supercedence and dependency tracking. I'm not going to get into supercedence here other than to say that for each patch family in a patch will get an entry in the MsiPatchSequence table.

The PatchFamily element is a child of the Patch element. You can have as many PatchFamily elements in a Patch as you want. Under a PatchFamily element, there is a set of Reference elements that can be used to pull various items into your patch. Just as in your product authoring, referencing anything in a Fragment will result in that entire fragment being pulled in. WiX will handle file and media sequencing as well as the Sequence tables for you.

Example:

This PatchFamily would select the Fragment that contains the Component "MyComponent" and add it to the patch.

<PatchFamily Id="MyPatchFamily" supercede="yes">
    <ComponentRef "MyComponent" />
</PatchFamily>

Sidenote:
One thing to note is that there are rules about patch families. Once you ship a patch family, you must keep its references the same or add to it (grow it) but never remove items from it. All items in your product must be a part of only one patch family. When you change the build that you are targeting (re-baseline), patch families start over so you can change the contents and asociations at this time.

WiX: Functional Preprocessing

The preprocessor in WiX has a new feature called Functional Preprocessing. It is a way for WiX users to define functions that they can use in their authoring to dynamically fill in content.

 Most people who use WiX are familiar with preprocessor variables which look like:

 "$(var.VariableName)"

 The preprocessor now has support for functions in addition to variables which look like:

"$(func.FunctionName(arg1,arg2,arg3))"

While there have been no additions as of yet to the standard WiX preprocessor function library many functions have been implemented and are being used in people's preprocessor extensions. In an extension you tell the preprocessor that you would like to have all functions with a specific qualifier be handled by your extension.

Example:

Say you want to keep track of all of your component GUID's in a single file. The file has 4 pieces of data for each GUID. Id, Language, Architecture, and of course the GUID.

MyComponentId, English, Intel, 365453-34235-32525-23325345223

If you implement a preprocessor extension to handle "guid" functions and define a function called "GetGuid" that takes 3 arguments you could represent the guid in your authoring as:

"$(guid.GetGuid(MyComponentId,English,Intel))"

This function would be replaced in your authoring with "365453-34235-32525-23325345223".
 There are a lot of possible uses of this type of functionality. I'd love to hear how other people use it.

This was originally posted here and may have additional comments.