BizTalk Best Practice – Use Well-Known Identifiers for EDI Global Properties

When working on a BizTalk EDI project recently, I received some errors that seemed to be doing things that I had not configured BizTalk to do. The error messages were mentioning that a problem was occuring when sending an EDI message from Party A to Party B with a message namespace which was for an 850 response message but I had only configured Party B to send to Party A with an 855. It took me a long time to get my mind around what was happening until I looked at the EDI global properties. This new portion of BizTalk Server 2006 R2 provides a default catch-all party for messages that cannot be resolved for a party based on the ISA/GS headers. Essentially what happens when the BizTalk runtime is unable to resolve the ISA/GS headers on a message to a trading partner party, it will change the context properties and ISA/GS headers and then if these do not resolve to party or other errors are encountered, these will be logged. So if you are looking in the logs for an error message, what you see may not always be true to what you would expect. Here is a link to some more information on the EDI global properties – http://msdn.microsoft.com/en-us/library/bb245981.aspx, and the role of the party in EDI processing – http://msdn.microsoft.com/en-us/library/bb226560.aspx.

 

For this reason, I have started using well-known identifiers in my EDI global properties so that if the BizTalk runtime cannot resolve the ISA/GS headers, at least I will know this after reading the Event Viewer log messages. So for the ISA 06/GS02 value I specify GLOBALSENDER and for ISA08/GS03 I specify GLOBALRECEIVER. Although I appreciate that BizTalk provides a fallback party resolver, this seems misleading or worse. Think for example if you specified a value for one trading partner as the fallback and then when you added another partner, it would be easy to wind up sending unsolicited messages to a different partner. Also, I think that if the party cannot be successfully resolved based on the ISA/GS headers, you need to know this rather than have the runtime substitute a value for you. The default values such as BTSSender are ok but do not convey the sense that they are coming from the global properties. BTW, the BTSGuestParty is another one of the built-in parties that operates on behalf of the BizTalk runtime, so watch out for this party as well. As far as I know, though, the ISA/GS headers for this party cannot be configured.

Selectively purge suspended instances in BizTalk

A request that I hear about sometimes is a way to purge messages or service instances in BizTalk. The usual response is to refer people to the built-in SQL jobs for purging the DTA archive or the MessageBox databases. These are excellent parts of a well-designed BizTalk database maintenance plan. But sometimes you see service instances or suspended messages that are still stuck in the BizTalk infrastructure. You can execute all of these maintenance jobs but none of them seem to get the stuck messages. I was in this situation today and I went through all of the maintenance plan jobs but none of them seemed to resolve the stuck messages and service instances. The statistics screen called the BizTalk Group hub page continued to show old suspended messages and service instances. I was investigating the suspended messages and found a way of actually removing these items without using the database jobs.

 

If you click on a link on the group hub page, it will show the results of a query run against the suspended messages, service instances, or other artifacts of the BizTalk system. I clicked on a link for the suspended service instances and then looked down in the search results. When I right-clicked on the search results, I found an item for “Terminate Instance”. This actually removed the suspended item from the BizTalk infrastructure. The screenshot below shows how this would look in the BizTalk admin console:

 

 

 

This technique actually works for multiple item removals as well. It is possible to multi-select the search result items and then right click on them as well. This can be a really helpful way of removing suspended service instances and messages. This is by far the easiest way to selectively purge service instances and messages and does not require connecting to SQL Server either.

 

Thanks,

BizTalk Flat-File Overflows – Look for 80004005

I was recently working on debugging a BizTalk flat-file schema for one of my clients. The flat-file happened to be for a PeopleSoft puchase order format. I was encountering issues where the flat-file disassembler (aka ffdasm) was not recognizing a line feed and was throwing a very ambiguous error number. I was testing with ffdasm and was using a document that included an extra line feed that the schema had not been told about and it was complaining about it. What was interesting about this situation was that the tool was throwing back an HResult with an error number that usually does not specifically represent a BizTalk problem but is actually a more general or "unspecified" problem. This number, 80004005 represents an unspecified error.

 

Doing a little research on the problem I determined this error number is sometimes used to represent an out of range error, like in a stack overflow (see http://classicasp.aspfaq.com/general/why-do-i-get-non-database-related-80004005-errors.html for some examples). Of course it would be nice to have a clearer error message. When doing flat-file debugging, its all about the size of the fields or the number of lines of a file. So to resolve this problem I simply had to change the group max number to unbounded for the schema element I was working with. This effectively enabled the schema to be aware that the number of elements in the group could be more than 1 and prevented the overflow possibility.

 

I am not going to speculate on how memory management works within BizTalk to allow schemas to have the possibility of an infinite number of elements. But it is possible to say that as a BizTalk schema developer, it is important to plan for the possibility that a schema be capable of the unexpected. Specifying many details about a schema will enable the flat-file disassembler to make the least number of assumptions about the range of data. Although min and max occurs are not required field values in a BizTalk schema (in most cases), think about providing these values so that the overflow constraints are clear. If you expect that multiple items may be provided in a message, code this in the schema, otherwise you will see overflows. I think with the complexity of BizTalk it can be very hard to think about building in security with all of the other layers of the product. Some of the conventional overflows, even unintentional ones, can be prevented through full schema design.  But also being aware of the non conventional ones, like with Xml-based overflows is important too. Search on "xml overflow" at a search engine and prepare to be amazed at the overflows documented like in the following article – http://www.securityfocus.com/bid/25301. Overflows are also a problem in maps and in adapters. See this Microsoft Security Bulletin about the HTTP adapter in previous BizTalk releases – http://www.microsoft.com/technet/security/bulletin/MS03-016.mspx. So thinking about overflows in BizTalk should be just another application of defense-in-depth - ensuring security holistically throughout the product.

 

You may be thinking that this error number could appear for many different error categories, which is true. I just wanted to mention that one of the categories for this error is the overflow or out of range category, which is particularly relevant for a BizTalk schema creator.

 

Thanks,

BizTalk 2006 R2 on Visual Studio 2008 Workaround

Lately I have been reading about new .NET 3.5 features and have been trying to keep up with LINQ and the ADO.NET Entity framework. As a BizTalk developer, it is quite difficult keeping up to date with all of the advances in WCF or BizTalk services (aka the Internet Service Bus – ISB). A significant challenge lately in keeping up to date is that the BizTalk Visual Studio extensions work with VS 2005 but not with VS 2008. If you try to install BizTalk 2006 R2 onto a system with only VS 2008 you get an error message because the extensions cannot be installed.

 

So I came up with a workaround. I took a VPC that had BizTalk 2006 R2 with VS 2005 on it and then installed VS 2008 to a separate folder than VS 2005. I was able to install and run both side-by-side. I tested the VS 2005 extensions and they are still intact.

 

So it is a little inconvenient to run both VS 2005 and 2008 on the same box and not be able to deploy directly from VS 2008 to BizTalk, but not horrible. So here is an ordered list of software I installed to get it to work:

 

• Windows Server 2003 R2 SP2
• SQL Server 2005
• Visual Studio 2005
• Visual Studio 2005 SP1
• BizTalk Server 2006 R2
• Visual Studio 2008

If anyone encounters issues with this configuration, please let me know.

 

Thanks,

 

Tips for “The Signing Certificate has not been configured for the AS2 party”

Recently I have been working on an AS2/EDI project at a client and have been struggling for a few weeks with a certificate issue. The situation I was having trouble with is that I needed to send an AS2/EDI message through BizTalk and have it digitially signed as well as encrypted. I had already distributed a public key to my trading partner, it was just that when trying to send the message out via BizTalk I would receive the error:

A BTS MIME error was encountered when attempting to encode a message. Error: The Signing Certificate has not been configured for AS2 party. AS2-From: <From Value> AS2-To: <To Value>
Event ID: 8132

Overall, the MSDN documentation on this issue (http://msdn.microsoft.com/en-us/library/bb967848.aspx) is pretty limited, and in the documentation at the link it mentions that this error occurs with the “BizTalk Server 2006 EDI”, when in fact it also occurs with BizTalk Server 2006 R2 EDI. This is just one of the many misleading details that make this a difficult issue to overcome.

 

So the documentation mentions that you need to provide a certificate for the BizTalk group through the group properties under the Certificate tab. A big gotcha to know about this is that the certificate must have the private key included. The following image shows what you should expect to see when a certificate includes the private key – a grey key symbol and note at the bottom of the view certificate window:

 

 

Depending on what adapters you are using with the certificate, you will also need to import this certificate to other certificate stores for other user accounts. The BizTalk environment I have been working with is a multi-machine install on a machine in a domain controlled by Active Directory. All of the BizTalk service accounts are domain accounts. What I had been doing for importing the certificates was that I created a custom MMC and then added all of the snap-ins for the different service accounts – one for the current user, one for the machine account, and then one each for the BizTalk AppHost and IsolatedHost accounts. One very important detail I did not understand about until today was that a service account’s certificate store was different when accessed from one user than another. So for example, I would login into the BizTalk Administrator account and add the Certificates snap-in for the BizTalk AppHost user (service account for the BizTalk Windows service and drop the certificates into the Personal store but I would still get the error.

 

Today I was looking at the following article – http://msdn.microsoft.com/en-us/library/aa559902.aspx#step5. This is the certificates setup step for the B2B solution. About halfway down the page you find a note that mentions you must login to the service account (interactively) in order to manage the certificates for this user. Usually service accounts do not have the right to logon interactively so this was something I kept overlooking. Once I enabled the BizTalk AppHost and IsolatedHost accounts the ability to login, and then I added the certificate to their personal certificates store I was able to get past the above message. The MMC snap-in framework perhaps disguises the fact that a certificate is not placed in the correct location for Personal certificates when logged in under a different user. The Certificates MMC makes it seem like you are adding a certificate to a shared storage location when in fact Personal certificates are more like an isolated, individual storage.

 

Another very important tip to know about is that you need to avoid enabling private key protection in the certificate with the private key or you will get additional error messages. Here are the messages I received when I did not disable private key protection:

A BTS MIME error was encountered when attempting to encode a message. Error: Exception of type ‘Microsoft.BizTalk.Component.MIMEException’ was thrown. HResult – 1061152225

The next error I saw in the logs was a little more helpful:

There was a failure executing the send pipeline: “Microsoft.BizTalk.EdiInt.DefaultPipelines.AS2EdiSend, Microsoft.BizTalk.Edi.EdiIntPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856as364e25″ Source: “AS2 encoder” Send Port: <Send Port> URI: <HTTP Address> Reason: The MIME encoder failed to sign the message because the certificate has private key protection turned on or the private key does not exist.

Please disable private key protection to allow BizTalk to use a certificate for signing.

This message means that the certificate that was loaded in a couple of certificate stores is protecting the private key and is not allowing it to be used by BizTalk. When exporting, be sure that you do not check the box for “Enable strong protection (requires IE 5.0, NT 4.0 SP4 or above) as seen below:

 

 

Also, when importing, do not check the box for “Enable strong private key protection. You will be prompted every time the private key is used by an application if you enable this option” as seen below in the following shot:

 

 

So as you can see, there are many pitfalls along the way to configuring a certificate for use with BizTalk. These tips should help clear up some of the gaps in the documentation.

 

Thanks,

BizTalk for Windows Server 2008

A question I hear frequently is whether BizTalk 2006 R2 runs on Windows Server 2008. It seems that a lot of people do not consider this software combination to be compatible. In this blog post I want to talk about the pros and cons of this discussion. At this point Microsoft has not announced formal compatibility with Windows Server 2008. On the BizTalk website (www.microsoft.com/biztalk), the system requirements (http://www.microsoft.com/biztalk/en/us/system-requirements.aspx) page does not mention specifically that Windows Server 2008 can be used – only variations of Windows Server 2003 with the exception of the web edition. The system requirements for BizTalk RFID does mention Vista as a compatible OS. But the system requirements page DOES mention that 64-bit versions of Windows Server 2003 are compatible. So from a cursory survey of the system requirements, it appears that Windows Server 2008 is not supported. There are quite a few people taking the opinion that these software versions are not compatible including a ZD-Net writer, Mary-Jo Foley (http://blogs.zdnet.com/microsoft/?p=1363).

 

The reason I thought it was important to blog about this is that I actually used BizTalk Server 2006 R2 on Windows Server 2008 on a recent project. The install of Windows Server 2008 that I used was the Windows Server 2008 Enterprise edition. I did not work with an install of Server Core or of Hypervisor, so a system that includes these features may or may not function properly for BizTalk. I installed BizTalk in a domain controlled network but in a single server environment. To provide details for people who are worried about the compatible featureset, I used the FILE and MSMQ adapters, and the BizTalk Server Adapters for Host Systems. I specifically used the MQSC adapter for connecting to WebSphere MQ with the nontransactional client. I created flat-file schemas for the messages I worked with. I was able to run maps and orchestrations and number of other standard BizTalk features. I used the BizTalk admin console and HAT in addition to several other standard BizTalk tools. Additionally, I worked with Visual Studio 2005 to develop the BizTalk solutions and was able to deploy my BizTalk projects directly from Visual Studio. So from my experience, I did not encounter a problem with BizTalk on a Windows Server 2008 system. As a person reasonably knowledgeable on Windows Server technologies, I realize that there are some definite pitfalls that you can encounter if you try to install BizTalk Server 2006 R2 onto a server with Windows Server 2008, so in the next paragraph I will mention some of these.

 

It is important in this blog post to mention that there are a couple of things to watch out for if you go ahead and install BizTalk Server 2006 R2 onto a system with Windows Server 2008. The following tips should help you avoid the Pitfalls of installing BizTalk on Windows Server 2008:

 

• If you install onto Windows Server 2008, make sure that enough of the server roles have been enabled to install BizTalk. I would use the BizTalk application dependency chart found at the system requirements page (http://www.microsoft.com/biztalk/en/us/system-requirements.aspx) when selecting the Windows Server 2008 server roles. The roles have been expanded in Server 2008 from the handful found in Server 2003 so you will need to check quite a few boxes under Application Server in order to get all the dependencies that BizTalk requires. I would be a little generous in selecting features to install in case you are unsure so that you do not miss important ones.
• If you install BizTalk onto Server 2008 on a system that is setup with Server Core, be aware that IIS is not supported in any version of Server Core (see http://www.microsoft.com/windowsserver2008/en/us/compare-core-installation.aspx) so many of the BizTalk adapters will not work or will not work as expected. Without IIS you do will not be able to host WCF services in IIS or use various parts of the HTTP, MSMQ, or MQSC adapters and of course will not be able to expose orchestrations as web services over HTTP.
• If you are installing onto a 64-bit system, be aware that one of your BizTalk hosts will need to be configured as 32-bit only if you want to use certain 32-bit adapters. See the following article for more information on this – http://msdn.microsoft.com/en-us/library/aa560166.aspx.
• Be sure the most updated version of the .NET Framework and service pack has been installed. This may sound like something that should be installed standard, but it is actually not always.
• Be sure to run Windows Update on the Windows Server 2008 system prior to installing BizTalk
• Be sure that if you are installing onto a computer in a domain in which group policies are being employed (check with your System Administrators or IT support), all of the required dependency services are running on the future BizTalk server and separate SQL Server, including the COM+ System Application Service, and Distributed Transaction Coordinator service. 

I think one reason that Microsoft has not yet provided documentation on using BizTalk with Server 2008 is that it will take a considerable knowledge of the product in order to configure Windows Server 2008 for a successful BizTalk install. Just determining all of the necessary server roles and features can take a while (see the server roles page for more details – http://technet2.microsoft.com/windowsserver2008/en/servermanager/default.mspx). Additionally, because Server Core breaks the functionality of so many of the BizTalk adapters it could be misleading or confusing to users to describe BizTalk as 100% compatible because by design, Server 2008 provides options for securing it to the point that it will not function in some configurations with BizTalk.  I think it is possible there are some gaps in the system requirements page for Vista and Windows Server 2008. Its obviously not taking into consideration the Server 2008 prerequisites. Hopefully this blog post can provide some help in this regard. Now, back to the discussion on compatibility, it is possible to find some sparse documentary evidence that compatibility exists.

 

It is quite difficult (although possible) to find a documented answer to this question. I could only find a few links from some creative searching. Interestingly, I can find more documentary evidence for compatibility with Longhorn than Windows Server 2008. For example, one of the BizTalk SAP adapters mentions it is compatible with Longhorn (see http://www.microsoft.com/downloads/details.aspx?FamilyID=FDAD7F15-8CBF-4151-BFB0-AFF831F6B977&displaylang=en). This version of the adapter lists BizTalk Server 2006 [R1] so at least some compatibility must have existed that far back. I installed Longhorn during a CTP but did not actually work with BizTalk on Longhorn, so I cannot speak from my experience for this combination. Scott Woodgate mentioned Longhorn in the context of post BizTalk Server 2006 in one of his blog entries - http://blogs.msdn.com/scottwoo/archive/2004/12/16/319457.aspx - under a BizTalk Server Roadmap document for BizTalk Server 2006 R2.

 

So in the end I am mostly left with personal experience. I was able to work with BizTalk Server 2006 R2 on Windows Server 2008 but I do not have many resources to back up my experience. If you read this and find some good resources on either side of the argument, be sure to post comments for me. Thanks!