Diagnostics, Deployments, and History in IIS...

Monday, June 25, 2007

In a recent trip back to Nashville, TN., I had the pleasure of meeting with some other IT professionals at a local Microsoft "shop." We discussed several different things such as my feelings of Windows Server 2008, IIS7, and other various questions that most Microsoft employees get when they are chattin with customers.

During this 1 hour conversation with these 3 Operations gurus, I learned some valuable things that I just couldn't hepl but come back and share. The purpose of this post is to just share some of the answers I gave to them to their questions.

<legal>NOTE: These are the sole answers from me to others that hopefully will shed light for others out there who have the same or similar questions. They are not warranted, per se, nor guaranteed in any form. </ legal>

First: What about diagnostics?

The first question that was really brought up was around diagnostics. I quickly started talking about IIS7's Failed Request Tracing and how it is the heart and soul of troubleshooting and diagnostics in IIS, now and in the future. I explained how it worked and further how it can combat a lot of the questions that most folks have on the why something is or is not happening.

Second: Can you drop cutting-edge and talk more about today?

I wasn't shocked at all by this question because it often is the reality of the corporate IT world. Bleeding, or cutting-edge, is a risky business for most and because of that they weren't asking What about diagnostics to hear about something they will deploy in 2+ years. They quickly pointed out that they still have Windows 2000 & IIS 5.0 servers that they can't move to IIS 6.0 for various political and technical reasons. Ouch!

After checking what ego I had at the door, I found myself back in the comfort zone that I so adore - talking about today rather than tomorrow. The next 20 minutes was spent talking about their first question - Diagnostics for IIS 5.0 or 6.0. I am proud of the work that Microsoft, in particular the IIS Team and IIS Product Support, did in this area to try and really aid customers in solving problems quicker, and more efficiently on a already shipped platform.

It shouldn't be a shock to many that I am proud of this as it was something that I drove either from a PM perspective (design, spec, etc.) or as the release manager (get it out the door).

IIS Diagnostics Toolkit

I explained to them where the idea came from to have a toolkit...so let me tell you. I have some friends on the Exchange team that I have had some conversations with in the past and I loved to hear how focus this team is on reducing the labor associated with troubleshooting Exchange. They went as far as to create an Exchange Best Practices (EBP) analyzer to try and reduce the number of people who configure themselves into broken situation. Secondly, they also released a consolidated toolkit that had a ton of cool tools that would help solve many of the problems folks have with Exchange.

It wasn't a light-bulb that went off in my head though it was "something" as I started asking myself - why? Why the heck do we not have an IIS set of tools that cover the top major pain points that our customers hit. Hmmm...

If we only had the tools to begin with ...

Wait, we have...

...SSL Diagnostics to help solve problems with Secure Socket Layer (SSL) issues

...Authentication & Access Control Diagnostics (AuthDiag) to help find problems related to authentication & authorization

...We have Debug Diagnostics to simplify the capture of debugging and analysis

...We have WFetch to help isolate when problems are created by the browser, not IIS

...Exchange released SMTP Diagnostics (SMTPDIAG) to help find issues with the SMTP server shipping with IIS or Exchange

...We have the infamous, and invaluable, Log Parser that lets you zoom in with laser accuracy on the problems using easy SQL-commands to access IIS logs, event viewer, or even NetMon traces.

...oh, and Trace Diagnostics that simplifies the creation of tracing in IIS 6.0 with IISREQMON.exe & IISTRACE.exe. And to boot gives you a nice UI for seeing currently executing requests with IIS Request Viewer.

What do you know...

It wasn't rocket science or anything but just work (e.g. labor.) This means simple labor, some politics, and a little magic voodoo and IIS could have a toolkit. The difference is that the idea came about while DebugDiag, WFetch, TraceDiag were in development. The first toolkit had only 4 tools and was released a couple years back. In January of 2006, we updated this toolkit to support the additional tools mentioned above.

Shocked: "Very cool..."

My IT friends were shocked to hear of the extensive set of tools built to aid them in troubleshooting their Web environment. They hadn't heard of several of the tools and I went on to explain in detail what each can do for them based on the symptoms. They loved Debug Diagnostics & Log Parser (previous knowledge existed) but were stunned by the existence of AuthDiag & TraceDiag.

A few months back, I added to IIS.NET the Diagnostics page where www.iisdiagnostics.com points to for folks to learn about about each of the tools. It wasn't publicize very well so I will use this blog as the portal without all the marketing jibberish ...

Your IIS Diagnostics Tool Cheat Sheet...

Tool Name	What it can do for you...
SSL Diagnostics 1.1	Quickly check your SSL Configuration in the Metabase Find registry keys that are affecting your SSL Configuration Quickly make a SSL requests to a specified website Generate a test certificate to validate \ invalidate your SSL Certificate Create self-signed certificates (command-line only - ssldiag /selfssl) Monitor incoming SSL requests in real-time using SSL Monitor
Authentication & Access Control Diagnostics 1.0	Test authentication (anonymous, basic, integrated, digest) to a specified URL Check website authentication configuration versus desired configuration Check permissions to website content path and key IIS components like asp.dll Check OWA permissions when running Exchange Server 2003 Check system permissions based on original permissions defined in Knowledge base articles from Microsoft View Kerberos Service Principle Names (SPNs) stored in Active Directory Enable AuthMon, a real-time monitoring system, to watch and analyze authentication and authorization requests
Debug Diagnostics 1.0	Capture and pull together with ease all pertinent troubleshooting data like event viewer, metabase, and IIS log files Setup a Crash rule to capture failures causing the worker process to fail Use a hang rule to capture a memory dump at the point of failure Analyze memory failures with ease using DebugDiag's LeakDiag feature for monitoring allocations of memory without de-alloc ...and a whole lot more!
WFetch 1.4	Internet Explorer or Firefox never cause failures...but in case they do use this simple tool to validate server-side or client-side issues Test all types of authentication supported in IIS (minus Passport) Add custom headers to test server behavior See raw HTTP request and response headers and body
SMTP Diagnostics 1.0	Test SMTP Check DNS settings for SMTP success ...a whole lot more I don't know about since we didn't build the tool <g>
Trace Diagnostics 1.0	View currently executing requests in a IIS 6.0 SP1 worker process See what state the request is in and what handler has it ...and how long it has taken Use IISREQMON to enable and capture currently executing requests for a URI from a command-line Use IISTrace to setup a trace of a particular type of request using a certain provider for particular URI (simplified a 3-step process to 1 command)
Log Parser 2.2	Parse log files in seconds to find specific data using simple or complex commands Convert log files from one log type (W3C) to another (IIS) and back again Parse log files for specific data and create, and insert, into a SQL database for archiving Use built-in functions to convert data such as IP address to hostname, etc. along with counting ...a whole lot more!

Third: Deployments...how do you deploy new code to IIS.NET?

I am the person in charge of IIS.NET - for better or for worse <g>. I am the first to say that a Webmaster position is one that takes a lot of patience, guts, and even more work than I originally ever believed possible. The notion of a self-sustaining website is impossible (or at least when you have a marketing department.) after a couple of years of running this site. My gratitude goes out to all you Webmasters out there -

They asked how, as the Webmaster, we deployed new features and functionality to IIS.NET. Carefully, I said...<g>

In truth, though, I said that the nice thing about our situation is that we only say when to deploy and don't actually do any work related to the physical deployment. Thus, I can't say in detail how we deploy to IIS.NET. The development team for IIS.NET is not done by Microsoft but instead by our Microsoft Gold Partner Telligent. They develop the code, manage the infrastructure, and deploy the bits. A nice position to be in as a Webmaster for certain!

Until IIS7, you have choices for Diagnostics

I used to, in college, live from test-to-test (or beer-to-beer). I often found myself very frustrated by the fact that I always was planning from the future back to the present. I decided after graduating to not make that mistake - don't live for tomorrow but instead play today. With that said, it isn't hard for me to get compassion for our customers' whom is undeniably interested in the future of IIS but only within a certain context. The future (e.g. IIS7) is for many a fantasy, an unreachable milestone, that often just makes you more sad than happy. The average IT person isn't even sure if his or her role will be the same by the time they deploy the "future."

Instead, reality is "What can you do for me today" and I can't blame it when they get tired of hearing about the props of the future when IT today is about the past. The one point that I love is that with both IIS7 and the diagnostics toolkit, I am positioned to have a great conversation either way - a Win-Win situation. I love IIS, always will...