WCF, TransportWithMessageCredential and MustUnderstands

Problem:
I had to consume a SOAP 1.1 web-service that was being hosted by Oracle’s Enterprise Service Bus (ESB) using WCF. The web-service was setup with transport security (Http over SSL) and message level security using a UsernameToken.

The above configuration should map to a basicHttpBinding (Soap11) and a security mode of “TransportWithMessageCredential” in WCF. This is what that configuration would look like:

<basicHttpBinding>
                <binding name="basicBinding">
                    <security mode="TransportWithMessageCredential">
                        <transport clientCredentialType="None" proxyCredentialType="None"
                            realm="" />
                        <message clientCredentialType="UserName" algorithmSuite="Default" />
                    </security>
                </binding>
</basicHttpBinding>

But when I attempted to use the web-service using this configuration – I got an error with the following message:

“SOAP must understand error:{http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd}Security, {http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd}Security.”

When I looked at the request message in Fiddler, I found that the mustUnderstand header was being set to true (or 1).

Obviously, the first thing for me to try was to set the mustUnderstand value to 0 or false. Trying this using Fiddler’s response builder, allowed me to confirm that this was indeed true. Unfortunately WCF will not directly allow you to change the mustUnderstand value of the security header. WCF adheres a lot more strictly to the W3C recommendations and hence will always set the mustUnderstand to true on the Security header (as it wants the server to throw a fault if it does not understand the Security header’s contents).

The only way that I could find around the “mustUnderstand” issue is to intercept the message just before the client sends it off to the server and modify the security header manually. I hate this solution as it looks very hacky and would love to hear from anybody else who figured out a much more graceful solution around this.

Solution: WCF messages can be intercepted by any object that implements the IClientMessageInspector interface. In turn this inspector is registered with an end-point using a class that implements the IEndpointBehavior interface. (The inspector is registered in the ApplyClientBehavior method of the interface).

Here is the code for IEndpointBehavior and IClientMessageInspector classes that allows you to modify the headers so that you can add a Security header without the mustUnderstand value set to true.

using System;
using System.ServiceModel.Channels;
using System.ServiceModel.Description;
using System.ServiceModel.Dispatcher;
using System.Xml;

public class MessageBehavior : IEndpointBehavior
{
    string _username;
    string _password;

    public MessageBehavior(string username, string password)
    {
        _username = username;
        _password = password;
    }

    void IEndpointBehavior.AddBindingParameters(ServiceEndpoint endpoint, System.ServiceModel.Channels.BindingParameterCollection bindingParameters)
    { }

    void IEndpointBehavior.ApplyClientBehavior(System.ServiceModel.Description.ServiceEndpoint endpoint, System.ServiceModel.Dispatcher.ClientRuntime clientRuntime)
    {
        clientRuntime.MessageInspectors.Add(new MessageInspector(_username, _password));
    }
    void IEndpointBehavior.ApplyDispatchBehavior(System.ServiceModel.Description.ServiceEndpoint endpoint, System.ServiceModel.Dispatcher.EndpointDispatcher endpointDispatcher)
    { }
    void IEndpointBehavior.Validate(System.ServiceModel.Description.ServiceEndpoint endpoint)
    { }

}

public class MessageInspector : IClientMessageInspector
{
    string _username;
    string _password;

    public MessageInspector(string username, string password)
    {
        _username = username;
        _password = password;
    }


    void IClientMessageInspector.AfterReceiveReply(ref System.ServiceModel.Channels.Message reply,
        Object correlationState)
    {
    }

    object IClientMessageInspector.BeforeSendRequest(ref System.ServiceModel.Channels.Message request, System.ServiceModel.IClientChannel channel)
    {
        request.Headers.Clear();
        string headerText = "<wsse:UsernameToken xmlns:wsse=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd\">" +
                                "<wsse:Username>{0}</wsse:Username>" +
                                "<wsse:Password Type=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText\">" +
                                "{1}</wsse:Password>" +
                            "</wsse:UsernameToken>";

        headerText = string.Format(headerText, _username, _password);

        XmlDocument MyDoc = new XmlDocument();
        MyDoc.LoadXml(headerText);
        XmlElement myElement = MyDoc.DocumentElement;

        System.ServiceModel.Channels.MessageHeader myHeader = MessageHeader.CreateHeader("Security", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd", myElement, false);
        request.Headers.Add(myHeader);

        return Convert.DBNull;
    }
}

The things to notice are:

• In the MessageBehavior class, I register the MessageInspector in the “ApplyClientBehavior” method.
• The message headers are transformed in the “BeforeSendRequest” method of the MessageInspector class. The headerText contains the header just as I would like it to be formatted (without a mustUnderstands value in it). The rest of the code is for passing along the userName and password so that it can be placed into the Security header.

The MessageBehavior needs to be assigned to the WCF client. This is done using the following code:

MessageBehavior messageBehavior = new MessageBehavior("username", "password");
client.Endpoint.Behaviors.Add(messageBehavior);

Where client is an instance of the WCF client.

Finally, and this ended up being a key factor that I had to learn after a lot of trial and error, is that I needed to modify the bindings used by WCF to use basicHttpBinding with just transport level security (https). The reason for this is that if use the TransportWithMessageCredential, then even though you might be adding the security headers in the BeforeSendRequest method, WCF will continue adding its version of the Security headers (and these contain the mustUnderstand set to true and will cause the server to fault). With the correct bindings and the MessageInspector in place…. everything began working.

Here is what the updated bindings looked like:

<basicHttpBinding>
      <binding name="basicBinding">
            <security mode="Transport">
                 <transport clientCredentialType="None" proxyCredentialType="None"
                            realm="" />
                 <message clientCredentialType="UserName" algorithmSuite="Default" />
            </security>
      </binding>
</basicHttpBinding>

Note 1:

Instead of using the pre-defined basicHttpBinding you can use the following custom binding.

<customBinding>
     <binding name="basicHttpTransportSecurityUserNameMessage">
          <textMessageEncoding messageVersion="Soap11"/>
          <httpsTransport />
     </binding>
</customBinding>

Note 2:

Like I mentioned before, even though this is a solution around the mustUnderstand issue when using UserNameToken over https with Soap11 (basicHttpBinding), I dont like the fact that I am inserting my own header XML into the message. The main reason is that this solution might need a lot of testing and learning to get it working for your problems. If you find that this solution does not work for you, the first thing I would suggest is to use SoapUI and determine what the headers being sent look like, when you perform a successful request to the web-service. Compare this header with the one in my BeforeSendRequest method. Try and replace the header text to see if this makes your client work correctly. Another extremely useful tool was Fiddler, which allowed me to monitor the data being sent by the WCF client.

Note 3:

Custom Message Encoders might be another solution for suppressing the mustUnderstand attribute. I haven’t tried this yet. But it looks more flexible, although it is also the more complex solution too.

CustomMessageEncoder sample: http://msdn.microsoft.com/en-us/library/ms751486.aspx

Note 4:

Another issue that you might face is when the certificate being used by the server for the HTTPS connection is not valid for some reason (most common of which is the name of the server does not match the CN name in the certificate). In such cases, you need to call the following code before any call to WCF.

ServicePointManager.ServerCertificateValidationCallback = new RemoteCertificateValidationCallback(delegate{return true;});

More Information:

Message Inspectors: http://msdn.microsoft.com/en-us/library/aa717047.aspx

WCF Security and Silverlight 2: http://www.netfxharmonics.com/2008/11/Understanding-WCF-Services-in-Silverlight-2#WCFSilverlightIntroduction

ESRI – MapObjects – End of Life

MapObjects will be end-of-lifed this summer. My initiation to the GIS world came through MapObjects, while I worked at VLS, many moons ago (Summer, 2003).

So it is kinda sad to see that it is being retired.

My first project with MapObjects was an overview window that I had to create as a proof of concept, that showed a thumbnail of where the user was digitizing when he was zoomed real close to a feature. Ah! the good times! (actually not – MapObjects gave me many heart-burns with its sparse documentation).

MapObjects end-of-life announcement

Debugging .Net source code in VS2010

If you did not know… VS allows you to step into .Net framework code so as to allow you to track down those intractable bugs. (I think this feature has been around since VS 2008)

To enable this you need to first go to Tools –> Options –> and under Debugger Options, Disable “Enable Just My Code” and Enable “Enable .Net Framework source stepping”.

Next make sure that you have a Symbol server defined (in VS 2010, this is already provided).

That's it – you should only have to hit F5 and if everthing is working, you should see an EULA, after which you should be able to step into .Net framework code.

Note: Remember to turn off this option when you dont need to step into the .Net framework code, as loading symbols from the server can be a slow process.

More Info: http://referencesource.microsoft.com/Default.aspx

Old but useful FAQ: http://blogs.msdn.com/b/sburke/archive/2008/01/16/configuring-visual-studio-to-debug-net-framework-source-code.aspx#faq

WCF Tool: Convert binding to a custom binding

An extremely useful tool that converts any WCF binding to a custom binding: WCF Binding Box. Why? Sometimes it is easiest to start from a simple pre-defined binding and then modify it to do your thing. (this is also a good way of learning what goes into the predefined bindings – for example: its clear below that the BasicHttpBinding implments Soap version 1.1)

Here is an example of converting a BasicHttpBinding to a custom binding.

<basicHttpBinding>
        <binding name="AddressBinding">
          <security mode="Transport">
          </security>
        </binding>
</basicHttpBinding>

 

<!-- generated via Yaron Naveh's http://webservices20.blogspot.com/ --> <customBinding>
  <binding name="NewBinding0">
    <textMessageEncoding MessageVersion="Soap11" />
    <httpsTransport />
  </binding>
</customBinding>

C# – Asynchronous Programming using BeginInvoke

In C#, delegates automatically get the BeginInvoke and EndInvoke methods defined for them. BeginInvoke is used for running the processes defined by the method and assigned to the delegate on a different thread (asynchronously).

The BeginInvoke method takes a AsyncCallback delegate parameter as a callback method. A common misconception about the BeginInvoke method is that if one does not care about when the method invoked by BeginInvoke completes or its return value, then one can pass in null for this method. THIS IS WRONG! and is partly based on old documentation. If you look at this MSDN document, you can clearly see that the EndInvoke method is not optional but required.

CAUTION   Always call EndInvoke after your asynchronous call completes

Here is more information from IanG: EndInvoke is Required.

So what do you do in Fire-and-Forget scenarios where you dont care about when the method completes or the return value of the method. Simple – use the CallBack parameter to define a call-back method that then calls EndInvoke on the delegate instance. Important: note that I pass in the delegate itself as a parameter to the BeginInvoke method, this allows me to access the delegate via the AsyncState property of the IAsyncResult object.

Sample Code:

Begin Async processing:

//using Action as a shortcut, you could use any delegate that you define instead
Action<int> t = new Action<int>(LongRunningMethod);
IAsyncResult r = t.BeginInvoke(0, new AsyncCallback(CallBack), t);

CallBack method is defined as follows:

private void CallBack(IAsyncResult ar)
{
      Action<int> t = ar.AsyncState as Action<int>; //AsyncState is set by passing in the delegate to the BeginInvoke’s method.
      t.EndInvoke(ar);//calling EndInvoke - REQUIRED
}

VS 2010 and TFS 2008 and Office 2003

If you are stuck on TFS 2008 and Office 2003, then after you install VS 2010, you will find the office integration to TFS is broken. This is by design as the VS team decided to only support Office 2007 and up with VS 2010.

If you are truly stuck in Office 2003 then here is what you can do, to get back the integration:

1. If you already dont have VS2008 and Team Explorer 2008, then install just Team Explorer 2008.
http://www.microsoft.com/downloads/details.aspx?familyid=0ed12659-3d41-4420-bbb0-a46e51bfca86&displaylang=en

2. Install VS 2008 SP1.
http://www.microsoft.com/downloads/details.aspx?familyid=FBEE1648-7106-44A7-9649-6D9F6D58056E&displaylang=en

3. Install the VS 2008 Forward Compatibility update.
http://www.microsoft.com/downloads/details.aspx?FamilyID=CF13EA45-D17B-4EDC-8E6C-6C5B208EC54D&displaylang=en

The office integration functionality is provided by a COM dll called “TFSOfficeAdd-in.dll”. To get back integration with Office 2003, you need to unregister the VS2010 version of this dll and then register the VS2008 version of this dll. (The dll is found in the “Common7\IDE\PrivateAssemblies” folder of the version specific Visual Studio installation folder).

Here are the commands:

Unregister VS2010 version of TFSOfficeAdd-in.dll
regsvr32 /u "c:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies\TFSOfficeAdd-in.dll"

Register VS2008 version of TFSOfficeAdd-in.dll
regsvr32 "c:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblies\TFSOfficeAdd-in.dll"

When you are ready to upgrade to Office 2007, just reverse the above 2 commands (i.e., run regsvr32 /u on the VS2008 dll and regsvr32 on the VS2010 dll).

Free ebook: Introducing Microsoft SQL Server 2008 R2

http://blogs.msdn.com/b/microsoft_press/archive/2010/04/14/free-ebook-introducing-microsoft-sql-server-2008-r2.aspx

Sample Log4Net Configuration

This sample provides 2 types of loggers – a rolling logfile and an SMTP based logger.

<log4net>
    <appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender">
      <file value="logs//appLog.log"/>
      <immediateFlush>true</immediateFlush>
      <lockingModel type="log4net.Appender.FileAppender+MinimalLock"/>
      <appendToFile value="true"/>
      <datePattern value="yyyyMMdd"/>
      <rollingStyle value="Date"/>
      <MaxSizeRollBackups value="180" />
      <filter type="log4net.Filter.LevelRangeFilter">
        <acceptOnMatch value="true"/>
        <levelMin value="DEBUG"/>
        <levelMax value="FATAL"/>
      </filter>
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%-5p %-25d thr:%-5t ${COMPUTERNAME}-[%property{SessionID}] %9rms %c{1},%M: '%m'%n"/>
      </layout>
    </appender>
    <appender name="SmtpAppender" type="log4net.Appender.SmtpAppender">
      <subject value="App Name - Error Log" />
      <smtpHost value="Relay.gov.dnvr" />
      <from value="fromEmail" />
      <to value="toEmail" />
      <bufferSize value="512" />
      <lossy value="true" />
      <datePattern value="yyyyMMdd"/>
      <threshold value="ERROR" />
      <evaluator type="log4net.Core.LevelEvaluator,log4net">
        <threshold value="ERROR" />
      </evaluator>
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="Application Error%newline${COMPUTERNAME} %date %-5level %logger - %message%newline%newline%newline" />
      </layout>
    </appender>
    <root>
      <appender-ref ref="SmtpAppender" />
      <appender-ref ref="RollingLogFileAppender" />
    </root>
  </log4net>

Notes:

The rolling log file appender is setup to have a minimal lock. This is required if you are using a cluster, otherwise it might be an overkill.

To use the loggers in an app you need to initialize log4Net. I do this in Asp.Net apps in the application startup method using code similar to this:

 

void Application_Start(object sender, EventArgs e) 
{
    //the log4net.ConfigurationFile needs to be set as an appsetting in your config file
    //the following code configures log4Net using the log4net.config file settings
    string logFileName = ConfigurationManager.AppSettings["log4Net.ConfigurationFile"];
    if (string.IsNullOrEmpty(logFileName) == false)
    {
        System.IO.FileInfo fi = new System.IO.FileInfo(System.IO.Path.Combine(System.AppDomain.CurrentDomain.BaseDirectory, logFileName));
        if (fi != null && fi.Exists)
            log4net.Config.XmlConfigurator.Configure(fi);
    } 
}

where log4net.configurationFile is an appSetting in your web.config file pointing to the location of the log4net file.

To use the logger, you first need to get a logger object using the following code:

private static readonly ILog Log = LogManager.GetLogger(MethodBase.GetCurrentMethod().DeclaringType);

Logging is performed using the following code:

Log.Debug(“message”) or Log.Error(“message”), etc.

SQL Server – Clean up log file

Command to reduce the size of the log file used by a database.

backup log [databaseName] with truncate_only
go
DBCC SHRINKDATABASE ([databaseName], 10, TRUNCATEONLY)
go

Another useful setting is to change the recovery mode to simple from full, if you dont require the full recovery mode (simple recovery – allows to recover to the last backup, whereas full recovery allows you to recover to the last point of failure).

Loading IIS logs into a SQL Server Database

Tools required:
Microsoft Log Parser : http://www.microsoft.com/downloads/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en
SQL Server Instance

Command:

LogParser "SELECT * INTO webLog FROM “S:\ex100515.log" -i:W3C -o:SQL -server:sqlServerName -database:IISLogs -driver:"SQL Server" -createTable:ON -username:yourSQLUsername -password:yourSQLPassword

note: if you wish to use integrated security (your windows account) to access the database, then, simply dont use the username and password fields.

If you need to import multiple log files, then you can specify a wild card instead of a specific file name (eg: s:\*.log).

List of fields that are imported:

note 2: Depending on the speed of your machine, size of the log files, etc., this can be a long operation and you wont get any indication on whether it is running or not, just be patient. (it took me about 4 minutes to import a file with 300k lines).