BizTalk Mapping – fixing huge maps

I have been working with several industry standard Xml schema definitions, specifically those defined by the UBL standard by OASIS (www.oasis-open.org).  This organisation has a worthwhile, yet lofty, goal of defining a set of document standards that will cover the majority of communication needs for B2B.

The result of trying to be all things to all men is that the schemas defined are big.  Like really, REALLY big, with a set of included schema file that run about 8-10 deep.

The problem with this, in the BizTalk world, is when you either generate an instance document from the schema, or attempt to map a document to a UBL schema document using the BizTalk Mapper.  Due to the way the BizTalk handles default nodes, you end up with all default values being output into the destination document.

A quick, undocumented fix for this is to change the GenerateDefaultFixedNodes setting in the BizTalk Map.  Where is this setting, you ask?

Open the .btm file using the “XML Editor”, rather than the default “BizTalk Mapper” (i.e. right-click, choose “Open With…”, then XML Editor).  The root node of the map document is called “mapsource”, one of the attributes is called “GenerateDefaultFixedNodes”.  Change this from “Yes” to “No”, save and close, and you’re done.

This undocumented trick brought one of our generated XSLT transforms down from a slightly ridiculous 50 Mb to an easily handled 11 Kb, and the transform execution time from 20 seconds down to about the 50 millisecond mark (on a virtual machine)

Received a “System.NotImplementedException” in custom pipeline component

While developing a custom pipeline component I began receiving a System.NotImplementedException when testing the pipeline from an Isolated Host.

After a bit of research, we found the following:

http://www.biztalkgurus.com/forums/p/11372/22514.aspx

Turns out the problem originates from the Http Adapter, which implements the Setter on the MessagePart.BodyPart.Data property, but not the Getter.

The solution was to change the code that retries the message stream from using MessagePart.BodyPart.Data to using MessagePart.GetOriginalDataStream().

Adding a Sql user that already exists

Occasionally I’ll need to copy a database over to my dev machine in order to be able to work offline.

USE <Target_Database>
EXEC sp_change_users_login 'Update_One', '<Server_User>', '<Database_User>'

This will re-associate the Server Login with the specified Database Login.

Unlocking a file from TFS

Inevitably somebody on the team goes away on holidays and forgets to check everything into TFS (guilty, as charged!)

There’s no facility to unlock somebody else’s files through TFS Explorer, so we have to resort to the command line.  Firstly, get the list of the user’s workspaces:

tf workspaces /server:tfsserver /owner:DOMAIN\offender /computer:*

This will list the workspaces “owned” by the offending little tyke absent user, which we will feed into the next command:

tf undo /server:tfsserver /workspace:workspacename;DOMAIN\offender $/path/to/file/to/unlock

Now you’re free to change the file at will, and reprimand your teammate when he gets back from his holiday, all tanned and relaxed.

Another FYI: just type in tf on the command line, and it will open up the appropriate help file. Sweet!

Setting up the BizTalk SharePoint adapter

While deploying a solution that uses the BizTalk adapter for SharePoint, we ran into the following error in the EventLog on the BizTalk box:

Client found response content type of 'text/html; charset=utf-8', but expected 'text/xml

Not exactly helpful when you’re trying to remotely diagnose the root problem!

After a little fact checking, it turns out that we had configured the SharePoint adapter correctly on the SharePoint box, however the BizTalk side of the configuration was trying to contact the SharePoint adapter on the default port 80.  We had installed the adapter into the non-default web site on the SharePoint machine, and it was listening in on a different port.

So, the actual message being returned to BizTalk was the actual HTML page spat back from IIS, telling us that the requested endpoint did not exist (i.e. we got the port wrong), but the client proxy end of the BizTalk adapter was expecting an Xml document, not an HTML document

Calling custom stored procedures using CodeSmith

Calling a custom stored procedure using the .NetTiers template in CodeSmith couldn’t be easier. You just make sure your procedure uses the standard naming convention of _{TableName}_{CustomMethodName} and Bam!, your accessor is available on the appropriate Table related class.

However (there’s always a “however”) there is one thing to make sure everything returns nicely.  Make sure the fields returned in your custom stored procedure match up with those in your table, otherwise your custom method will return a generic DataSet.  By returning all the fields from your base table, .NetTiers is smart enough to provide you with a nice, enumerable, TList<myTableEntity> return value.  Handy!

“Filtering” a document

Just a quick lesson learnt from a recent project, as I know I’m going to need to do this again.

Filtering down an Xml document in a map based on the existance or value of an element or attribute.  For example, say we want to return a copy of a existing document, but only where the optional Status field has a value:

• Use a map with a Scripting Functoid, using Inline XSLT
• Use the Xslt copy-of function to make a copy of the source based on a critera selection:

  <xsl:copy-of select="/*[local-name()='RootNodeName']/*[local-name()='Summary'][*[local-name()='Header']/@Status='Active']" />

• Wire the output of the functoid to the relevant element on the destination schema

Problems with PropertyBag in custom pipelines

Some advice written by one of my colleagues, Zeb, who spent some time wrestling with a custom pipeline

I was recently tasked with building a custom pipeline component. I hit the net to look for some samples on doing this, and the first thing that I noticed was that there appeared to be a million blogs on how to do this, and they were all using the exact same code – right down to even using the same GUID for their class!

There are plenty of things I could write (and maybe I will), but the biggest issue I had was that none of them were really elaborating on the saving and extraction of custom properties. The basics were simple (implement IPersistPropertyBag), but as is so often the case what can seem simple, often turns out to be far from it! (Although it does tend to become simple and obvious in hindsight)

It was a long journey, and hard to debug, as the symptoms weren’t immediately pointing to the cause. So, below are the two biggest issues I had with the PropertyBag, and how to work around them:

Issue 1: Properties are visible in BizTalk, but not in Visual Studio

Now, this one really didn’t make sense at first, and honestly, it still doesn’t! I know the “how”, but not the “why”. Basically, Visual Studio uses reflection on the class to look for public properties and assumes that they are the properties that can be modified from within Visual Studio. BizTalk on the other hand looks at the actual serialized [sic] properties saved into the pipeline (i.e. if you look in the .btp file “Properties” node inside the component).

So, if you do what I did and move your properties into a seperate class (which I did because I wanted to make two versions of the component – one inheriting from FFDasmComp and one from XMLDasmComp), you need to be aware that you also need to expose them in the calling class as seperate public properties or they will not be visible in Visual Studio.

Issue 2: PropertyBag.Read method returning NULL

This was a strange one, as I was getting inconsistent behaviour. Sometimes my component would work, other times it wouldn’t, and my code changes did not appear to be impacting on the success rate.

The lucky break came when I noticed that the Load method was being called twice, and each instance was returning different values. It appears that BizTalk was calling Load once with the information that was saved when the pipeline was created in Visual Studio, and then a second time to get the customized [sic] values that were modified in the BizTalk Administration Console. The problem here is that if you don’t modify a value in BizTalk, it doesn’t get saved to that property bag, so the read will return NULL.

The way around this is to do the first read and use it to load default values for your properties, and then on the second read only overwrite those values if the Read returns a non-NULL value. I did this by creating a “GetProperty” function that accepted the PropertyBag, the key to read and a default value (which was loaded with the previously read value if available). If Read returned NULL (or error), I loaded the default value.

Thanks for the good work, Zebba.

Yay, we’re still relevant

Just incase you haven’t read this amongst the other 7394 blog posts today, Microsoft have confirmed the next version of BizTalk Server 2006 R3 2009.

Here’s the official Roadmap page.  Looks like there’s still a lot of life in the old girl yet!

Unable to refresh the BizTalk Group Hub

I have occasionally gotten either of the following errors in my BizTalk Administration console, and am then unable to refresh the BizTalk Group Hub:

"An instance of the FreeThreadedDOMDocument class cannot be created"
"Failed to create a CLSID_BizTalkPropertyBagFactory"

I still don’t know why it happens, but the solution is simple: Restart the Windows WMI Service.

With thanks to this blog article.