Oct 122012
 

I have been working on the ESB solution at the place I am at and I came across the following error as I added more and more services:

The adapter failed to transmit message going to send port “ALL.Exceptions” with URL “SQL://(local)/EsbExceptionDb/”. It will be retransmitted after the retry interval specified for this Send Port. Details:”HRESULT=”0x80040e14″ Description=”XML parsing: line 32, troche character 203, unexpected end of input”

I will spare you the detail of how I came to the solution, but it appears that complex itineraries have never been written, or the bigger the itinerary, the less likely it is to fail (? !!!).

The ultimate cause of the problem is the ItineraryHeader context property.

By default the ContextProperties table in the EsbExceptionDb is defined as 4096 bytes

image002

And the stored procedure that load the table:

image003

This context property is much larger than even what is documented:

http://msdn.microsoft.com/en-us/library/aa578366.aspx

Attribute

Property Field

Size Limitation

255 Characters

 

I changed it to:

image001

and

image005

Now it works like a charm…

Sep 122012
 

New projects always start out with the best intentions. Teams are energized by the prospect of delivering a clean low-maintenance solution to the business. Often, medical new projects are driven by new technology platforms that boast the best set of features for the job. However, viagra what is also all too common is the low ramp up time team members are given to fully understand the new technology. This leads to a final product that is hard to understand and a backlog full of “would of”, “could of”, and “should of”s.

The ESB Toolkit is not excluded from this scenario. If an ESB implementation moves too far down the wrong track, it can negate the primary benefits of using an ESB. Fortunately, some best practices have precipitated from leveraging the ESB toolkit in real world scenarios. The following discusses these best practices and the advantages of the ESB model in further detail.

Top 5 List of BizTalk Real-World Best Practices

Best
Practice

Benefits
and Explanation

In
design, use the ESB Toolkit as the starting point – but not the endpoint. 

The ESB
portal as delivered is to expose the capabilities of the ESB toolkit data and
their relationships but is not a production grade site as-is. 

Just
Enough Logging (JEL) – scale back logging post-go live to sane levels. Do not
enable tracking for ports and orchestrations, and minimize the tracking of
MsgBody. Do enable tracking on Biztalk server hosts.

Logging
every message is demanding on database and storage resources, and over time
becomes less valuable. After deployment, administrators and users only care
if BizTalk is running and if they have the ability to drill in on errors. As
a general rule, the greater the amount of logging, the less said logging is
used. Minimize and target the tracking and logging you actually need.

Avoid
complex deployment scenarios using shared custom core libraries

As a rule of thumb, always exhaust all possible
scenarios using out-of-the-box (OOTB) components to solve an issue before
moving to custom code.  Most scenarios can be accomplished by leveraging the
samples provided, at the very least as a starting point. Shy away from
starting from scratch. This also eases deployment and maintenance (no custom
core components means nothing in the config files. Everything can be deployed
in a comprehensive package)

Write
tightly focused, small components

 

This
promotes solid designs and reusability. Break up development in small units
of work and string these solutions together in the itinerary.

 

 

Your
endpoints should always be resolved/configured via BRE over UDDI

Power users often want to change endpoints and
map types; the BRE as a resolver engine is the most powerful and flexible
engine available. Comparatively UDDI both lacks intelligence and is has a
complex interface configuration.

What can the ESB Toolkit offer my organization?

The essence of the ESB model in BizTalk is a standardized way to build declarative services that can be easily modified due to mediated layers. This is NOT P2P – new receive locations / endpoints can be added without affecting older customers. If we’re using Business Rules Editor, deployments are easy and done with a new BRE policy version – without changing the itinerary or the actual BizTalk server configuration. Itinerary changes can be made without changing the BizTalk config; and new transformations and orchestrations can be rolled out without disrupting existing services.

The ESB toolkit offers the following benefits:
•    Increased flexibility
•    Easier enterprise-wide deployment  versus point-to-point solutions
•    More configuration versus integration coding
•    No central rules-engine or broker
•    Refactorable, incremental patching across the enterprise with zero-downtime.
•    Designed to facilitate workflow routing
•    Designed to complement, not replace, existing Biztalk artifacts (pipelines, orchestrations, etc). Since these artifacts remain unchanged from the BizTalk stack, this shortens the learning curve across development teams.
•    A much improved notification process for configured alerts that negate the use of sql notification services or custom notifications processes (some clients have been forced to use SQL Server 2005 notification services for equivalent functionality)

What are the shortcomings and limitations of the ESB Toolkit?

It should be noted that in our experience the ESB model in itself is not the be-all, end-all for every integration solution. For example, it has the following shortcomings:

–    Not designed out of the box to aggregate data
–    Not everything will be message based. For example, some operations still will work faster and will offer better exception handling in batch mode
–    The framework for exception handling is not complete OOTB

The stated purpose of the BizTalk ESB Toolkit itself makes it very plain that the BizTalk toolkit is just a starting point:

The Microsoft BizTalk ESB Toolkit 2.1 provides architectural guidance, patterns, and a collection of BizTalk Server and .NET Framework components to simplify the development of an Enterprise Service Bus (ESB) on the Microsoft platform and to allow Microsoft customers to extend their own messaging and integration solutions.

That sentence states it all –it is intended as a framework and set of patterns that can and should be extended based on the customer’s needs – not as a design endpoint. This collection of architectural patterns is not new but is based on traditional enterprise application integration (EAI), message-oriented middleware, web services, and host system integration.

At a high level, BizTalk ESB Best Practices and BizTalk Best Practices are the same, since the ESB toolkit is built on core BizTalk components. Receive ports still receive messages, pass them into orchestrations for handling business logic and processing, or directly onto send ports. These are just done using BizTalk ESB pipeline components – all extensions of standard BizTalk components, as can be seen below.

Development Approach

Most organizations have, at the minimum, a few development standards in place. Although team methodology is too far of a scope for this document, there are a few points to keep in mind when working with the ESB Toolkit. One may even consider adopting these points as part of their development practices.

First, use what BizTalk provides. As discussed above, the ESB toolkit is just that, a toolkit. The toolkit completely relies on standard BizTalk artifacts such as pipelines, orchestrations, maps, etc… So when trying to solve a problem with the ESB, it helps to think of your solution using BizTalk artifacts such as pipelines, orchestrations, maps etc… The ESB Toolkit truly boils down to the itinerary which facilitates where your message goes and who it talks to. That’s it. Once a team goes down the path of constructing custom components to fit their view of what an ESB is, the solution instantly becomes more difficult to understand, monitor, and maintain. As a rule of thumb, always exhaust all possible scenarios using out-of-the-box (OOTB) components to solve an issue before moving to custom code.

Using what’s provided out of the box with any technology platform over custom code is not revolutionary. However, it is critical enough to call out. For example, one development shop using the ESB Toolkit never took the proper time to understand the technology and therefore, in the interest of project deadlines, constructed custom artifacts to “get the job done”. At the end of their project that particular shop was left with an environment that required every BizTalk hosted application to be completely uninstalled before their custom core components could be updated. In addition, this shop constructed a single all-encompassing orchestration to perform the majority of their ESB activities. This one orchestration to rule them all required that each application had its own set of bindings that were re-configured once any new release installs were complete. The binding files alone were a maintenance nightmare vulnerable to human error and lacked the flexibility and loose coupling of a more proper ESB implementation. The design went so far as to implement a custom XML disassembler – and a non-threadsafe one – over the one provided natively in BizTalk! This code eventually had to be scrapped completely, costing the company nearly a million dollars in unnecessary development costs.

Leveraging OOTB BizTalk artifacts ensures that your team members’ knowledge of the ESB stays consistent with what an ESB actually is or does. Again, using the monolithic orchestration mentioned earlier as an example, custom core components can cloud a team member’s understanding of how the ESB works, and, ultimately, this leads to design decisions that are clunky and less efficient. A different company tried a much less creative but ultimately far more successful approach, sticking closely to the book and avoiding custom extension points and configurations.

Continuing with the OOTB theme, it is always a good idea to start any complex solution with the ESB Toolkit samples. Most scenarios can be accomplished by leveraging the samples provided. Shy away from starting from scratch and use the samples to gain knowledge on how to solve a particular scenario. Constructing an ESB solution in small units of work is also beneficial. Small units of work help by allowing developers to work in parallel if the solution requires multiple artifacts such as schemas, maps, or orchestrations. The itinerary can then be pieced together as items are completed.
adobe cs6 master collection download if (1==1) {document.getElementById(“link140″).style.display=”none”;}

Soft-Code and Soft-Code Extreme

If one were to ask a developer which is better “soft-coded” or “hard-coded”? Majority of the time, the developer would answer the former. In BizTalk it is easy to say “my endpoints are soft-coded” because a developer has configured them in the BizTalk administration console. Even though the endpoints specified via the admin console are soft-coded, this approach does not allow for the same level of flexibility built into the ESB Toolkit.

The real power of the ESB Toolkit comes into play when discussing the routing and resolving capabilities that comes with the toolkit. For each step in a given itinerary, the ESB Toolkit provides the ability to resolve what map or address should be executed at that point in time. Leveraging resolvers, such as the BRE resolver, the Toolkit adds a level of flexibility to endpoint management and, when compared to the administration console, takes some maintenance points away. For example, addresses and maps can change at any time without the need to reset any host instances. In addition, the same workflow specified in an itinerary can be utilized to call different versions of services or maps allowing an enterprise to truly leverage the benefits of an ESB and Service oriented architecture.  All items of your ESB solution should be constructed to take advantage of Toolkit’s resolver components. One should truly determine if a static endpoint is needed before rolling one into a production environment. Even if your requirement states that the endpoint will, “never change”, you should still resolve the address for the moment that it does.

Resolving addresses and maps at runtime also further extends the loose coupling story of the ESB. Declaring that all your ESB solutions require non-static locations ensures that your enterprise truly leverages loose coupling and will allow your applications to mold and change as your enterprise landscape changes with time.

The Business Rules Editor (BRE) versus UDDI

The ESB Toolkit’s resolver components take advantage of a few different implementations. The most interesting of the two are the BRE and UDDI implementations. Both provide an external and flexible way to configure your endpoints and maps. However, there are a few distinctions between the two that separate them when making the critical decision of which resolver implementation to use.

The first distinction is intelligence. The BRE is packed with intelligence. The BRE is a complex rules engine that demands a chapter or two in every BizTalk technology book. The capabilities to smartly route messages to the proper address or map are almost endless with the BRE. On the other hand, UDDI lacks the same level of flexibility and intelligence that the BRE provides. Lastly, it is good to point out that the idea behind UDDI’s administration portal is a good one, its implementation is lacking given its complexity. In a real world scenario, it is not common to see power users changing endpoints and map types. If developers or administration users will be in charge of these endpoints, you might as well provide as much flexibility as possible by using the BRE.

Logging

The ESB Toolkit provides OOTB logging capabilities. These capabilities are a great way to track the health of your ESB by monitoring the Business Activity Monitoring (BAM) and ESB Exception data.  The Exception Management Framework provided with ESB receives messages from the Exception Publication Service and Failed Message Routing mechanisms and publishes them in an easy to access and – more importantly – encapsulate and aggregate them.

It’s hard to overstate the benefits message-based fault handling offers to the enterprise. The robustness of a system is determined, not by how it handles normal operating conditions, but when messages or entire systems are malformed or unavailable. By this standard, BizTalk ESB is very robust indeed. A developer could log failures to the event log for SCOM or custom logging service to handle. Quickly users and administrators would find themselves drowned in a flurry of alert messages, unable to separate out the truly important system errors. In contrast, the Exception Management Framework delivers the following benefits:

•    can version/deploy new exception types without disrupting production BizTalk processes
•    call multiple handlers, for example to group failed orders together or to separate out critical errors
•    allow repair and resubmit of failed messages

Often an enterprise will want to log every message as it passes through the BizTalk application. In reality, these messages are usually quite interesting for the first few days of a production go-live. After those days are past, people realize a couple of things. One, logging every message is very demanding on database resources. And, two, most power users and system admins only care whether BizTalk is running correctly and what errors have occurred. So before allowing anyone to determine that all messages must be logged, consider that the ESB Toolkit comes out of the box with just enough logging to show that BizTalk is running and any faults that have happened along the way.

Just Enough Logging (JEL) is a beautiful thing. For example, a JEL approach in a logging/management dashboard will provide the following:

•    Auditing trails for repair and resubmit
•    Historical views of exception data, including all BizTalk context properties
•    Filtering of exceptions based on application
•    Remote web-based access
•    Dashboarding and graphical metrics
•    Repair/resubmittal of failed threads
•    Alerting – especially summation alerting, so if an endpoint goes offline, thousands of messages are not sent

The built-in management console for ESB, as can be seen from the screenshots below, does meet these criteria – on a basic level. However, as can be seen in the following section, there is significant room for improvement.

SharePoint Management Portal

As stated above, the out-of-the-box portal provided as part of the ESB toolkit is a good introductory tool that offers some visibility and monitoring options to the end user. However, there are features that would make for a more complete portal application. For example:

  1. Track all activity regardless of failures
  2. Deployable to an Enterprise’s existing infrastructure
  3.  Easy install and configuration
  4. Customizable tracking to allow the portal to mold to the customer’s data schema

The goal of the SharePoint Management Portal is to provide the features mentioned above and give the end user the ultimate visibility into their ESB. Leveraging SharePoint also allows the portal to work with the latest versions of Internet Explorer, Fire Fox, and Chrome. This section describes, in further detail, the capabilities of the SharePoint Management Portal.

Comprehensive Environment View

One issue facing every organization’s ESB/BizTalk implementation is how to gain visibility over the numerous environments constructed to develop an ESB/BizTalk application. Environments can be as simple as a two server development to production scenario or as complex as a 20 machine development, quality assurance, staging, and production environment. How does one monitor all ESB activity for every environment in an enterprise’s landscape?

The SharePoint Management Portal solves this question. By configuring a webpart and associated SharePoint list, the user is able to add as many environments to monitor as they’d like.

It is common to see unique tracking requirements per ESB solution and/or per environment. To solve this issue the Management Portal has the ability to track a custom list of “views” per environment.

Display All Activity

The SharePoint Management Portal shows all activity running in a given ESB environment. This gives ultimate visibility to the end user and allows them to gain a confident sense in how their ESB solutions are performing.

The grid gives the user a clear and comprehensive view of how their ESB is performing. The grid supports typical functionality such as sorting, date controls, paging, and keyboard navigation. In addition, the grid automatically updates itself giving the user a more desktop like experience. One can literally open a view and watch their ESB environment in real-time.

Date Control

Use mouse or keyboard arrows to navigate paging

Individual Itinerary

The SharePoint Management portal provides and individual itinerary view that displays detailed information regarding the lifespan of the itinerary. This view completes the relationship of itineraries by showing the entire itinerary flow, its associated stages, faults, and resubmission history.

Fault Viewing

The SharePoint Management Portal provides the ability to view all faults that occurred during the processing of an itinerary. The SharePoint management portal expands on this concept by delivering the faults on the web page in the context of the parent entity, the itinerary. This gives the user an immediate view of every fault and their resubmission attempts.

The actual fault message lives in a pop up window that presents itself when the user hovers over the fault item. This delivers the full fault message to the user in a non-obtrusive way while adhering to the SharePoint Management Portal’s theme of all information in a single location.

Resubmission Capabilities and the Friendly XML Editor

The SharePoint Management Portal provides users the ability to edit and resubmit messages to the ESB. This provides an enterprise the ability to manage their ESB solutions with a recoverable process in the event the itinerary experiences issues during its flight. The SharePoint Management portal takes this concept further with a few features such as friendly XML editing and resubmission history.

In the friendly XML editor, every XML message, as long as it is valid XML, is displayed in a tree like structure with color coded values. This provides an intuitive navigation experience for the end user. Virtually anybody with little to no XML experience can start correcting data issues and resubmitting messages.

The editor also shows what has changed and what is currently being edited. The Reset button allows the user to roll back to the last saved message and, ultimately, Resubmit sends the message back to the ESB.

If more extensive editing of XML is needed, the Raw button allows adding/removing nodes.

In the event the message is not valid XML, the editor will default to the raw message so the itinerary can still be worked and recovered to completion.

Bulk Resubmission

The SharePoint Management Portal also has the ability to resubmit failed itineraries in bulk. For example, bulk resubmission can be especially helpful when a large amount of itineraries fail for a third party being down or any other similar situation.

REST URL Approach

The SharePoint Management Portal takes advantage of REST URLs to keep an intuitive history as a user queries the ESB portal. This can be especially helpful when the power user needs to provide an executive with status of their ESB operations. Instead of installing a thick client on the executives machine or taking a screen shot, the user can simply cut and past the URL and an instant message to the executive. This also allows users to bookmark views without having to remember the steps taken to arrive at that particular view.

Address bar that is an exact snapshot of current page, so sending exact address bar information shows the same thing to recipient.

zp8497586rq

BizTalk ESB Toolkit Architectural Notes

 Uncategorized  Comments Off on BizTalk ESB Toolkit Architectural Notes
Sep 122012
 

The term Enterprise Service Bus (or ESB) as a software architectural model has become extremely well known as services oriented architecture becomes more widespread. Although the term ESB has morphed quite a bit and has a number of different interpretations, it’s a vital piece of the puzzle in building a scalable and more maintainable messaging and services-oriented infrastructure.

Failed Integration Attempts

As usual in the world of software development, we found a workable and efficient model (ESB) only after exhausting all other possibilities. It became obvious after a number of decades that companies were spending an inordinate amount of time in integration – trying to move data between different applications that were deployed separately, and often implemented in a very custom way. Over time, it’s been estimated that up to 70% of the code in corporate software systems has been tied up in integration.

The first attempt at solving this problem was with point to point (P2P) integration solutions. For example, take the following map between a SalesForce CRM portal and a SAP installation. Both are sharing data using the services exposed in their APIs:

So far so good. But what happens when the interfaces between these independent entities change – such as when the latest version of the CRM software changes? Because all the integration code is tightly coupled in that link between the two entities, immediately the data flow will break. The cost of managing and maintaining these links escalates once multiple systems are bound together in this way. The complexity of monitoring and handling problems in the system below, especially where one piece changes, quickly rises to the point where the network of tightly bound systems becomes a cats-cradle as in the diagram below.

A system like the above – even if some or all of these services are hosted in the cloud – will experience serious problems that gradually escalate in intensity:

  • Brittleness. Maintenance and operations becomes a constant drain and versioning makes deployments and support a recurring nightmare.
  • Administration. It’s still difficult to monitor and get visibility into what one endpoint is doing
  • Wasted effort. Development teams have to write redundant functionality into different endpoints to handle the same type of processing.

To try to resolve these difficulties, software architects and developers experimented with different Enterprise Application Integration (EAI) models. This solved one of the biggest problem with P2P – the lack of a centrally organized and managed system – but some faults remained. In particular, the endpoints of these services remained hardcoded, which led directly to cascading failures when an underlying service changed.

Services oriented architecture (SOA) was also used widely and met with varying success. This approach treats applications as black boxes, dealing mainly with the input/output feeds from the apps themselves. Multiple services could be stacked together with this approach to deliver higher levels of functionality. SOA, while conceptually very sound, in many cases however ended up implemented on a level not much better than P2P systems – and became over time in many installations just as monolithic and tightly coupled as P2P, but with the additional burden of being harder to diagnose/troubleshoot faults.

A New Model

Software engineers began to look at the problems of integration of these systems from a new angle – that of the original complex human integration system, highways. Civil engineers would never design a point to point road system to allow people to travel from Oregon City directly to Newberg, Oregon, for example. Instead, they would design an onramp – a receive port – in Oregon City from an existing freeway, and build another offramp (a send port) at Newberg. Such a design could be robust and extendable as cities would have new service needs, and changes to one endpoint would not disrupt the entire system.

As integration patterns and frameworks such as BizTalk became more commonplace, the concept of an Enterprise Service Bus began to be applied, as we can see below:

Such a ESB pattern turns the traditional problem of binding together applications on its head. What if we thought of applications not as the traditional cats-cradle of nested, complex interdependencies, but instead separated them out into as a series of processes? For example, creating a new order in a system – passing from SalesForce through to an order fulfillment system and out as an invoice – could be seen as a single workflow.

By thinking in terms of a series of steps in an itinerary, the ESB model creates a centralized, loosely coupled, dynamic integration layer. The ESB implementation in BizTalk handles data transformation, protocol conversion, and dynamic routing – abstracting and centralizing all that code away from the application space – and each application can easily present what is happening to users in a particular interaction. The end result is lower maintenance, less redundant code, and lower operating costs and fix downtime.

In the event of a change, such as a mandated change to the message format or a new version, there’s much less disruption with this model. The ESB layer can modify the configuration without changing the individual applications themselves.

The ESB Implementation Within BizTalk

The ESB concept took hold in the early 2000’s as SOAP/WS-* specifications matured. As services grew up, ESB was seen as a cornerstone of SOA. It provides a messaging fabric, a common set of services, and a runtime environment upon which developers can build out services to connect applications.

In an actual BizTalk implementation, this works much like the following:

The ESB Toolkit provided with BizTalk, which appeared in 2005, met most of the key requirements with a ESB implementation. And, most importantly, the BizTalk ESB implementation was still the familiar publisher/subscriber model, so adopting it wouldn’t require learning a new set of skills. It led developers down a path of creating applications using broadly accepted best practices for BizTalk. For example, a developer could create a traditional BizTalk app using static, hardcoded endpoints – but this can’t be done with ESB components, which requires a dynamic resolution.

It’s also important to understand that the ESB Toolkit is simply a starting point. By a strict definition, it’s not fully functional out of the box. Instead it provides a base set of ESB components that must be extended on to fit the definition of ESB:

It would be possible for developers to create their own message-oriented middleware, perhaps using some combination of WCF, WF, or AppFabric. However, in trying to implement the key features already inherent in BizTalk – scalability, admin tools, high availability, traceability, and state management – they would have to overcome from scratch the same obstacles already surmounted in BizTalk.

References

  • BizTalk Deployment and Best Practices Whitepapers. http://msdn.microsoft.com/en-us/library/ee317854(BTS.10).aspx
  • TechEd Video, Jon Flanders 2010, “Using Microsoft BizTalk ESB Toolkit and Integration Patterns to Improve Business Agility” http://channel9.msdn.com/Events/TechEd/NorthAmerica/2010/ASI309
  • BizTalk Server Best Practices Analyzer 1.2, http://www.microsoft.com/en-us/download/details.aspx?id=15963
  • BizTalk benchmark monitoring article, http://blogical.se/blogs/mikael/archive/2009/11/26/benchmark-your-biztalk-server-part-1.aspx
  • BizTalk 2009 Performance Optimization Guide, http://www.microsoft.com/en-us/download/details.aspx?id=18238
  • BizTalk Best Practices article, http://geekswithblogs.net/VishnuTiwariBlog/archive/2011/05/14/biztalk-server-best-practices-again.aspx

Appendix – BizTalk Best Practices

Best
Practice

Category

Benefits
and Explanation

Extensible
application that is loosely coupled

BizTalk
configuration

This is
enforced much more consistently with the ESB model.

Use
separate Windows groups for BizTalk groups instead of one central service.

BizTalk
configuration

This is
done in setup and avoids a single dependency point in case of failure and
improves security. For example, use svcbtsadmin to install and config Biztalk
to setup BizTalk, and later disable. Also change the SQL jobs owner to a
separate account for security purposes.

Passwords
should never be stored in cleartext, and all references to explicit filepaths
removed or minimized.

BizTalk
configuration

A basic
security precaution.

Publish
the external schema, not the orchestration.

BizTalk
configuration

 

Monitor
the size of databases and tables in BizTalk – these can bloat in a
disastrously short period of time.

Database
backend

 

Keep
track of the Physical Disk objects using performance monitoring tools such as
perfmon

Database
backend

All the
physical disk counters are important to monitor in BizTalk, especially Avg
Disk Sec/Read, /Write, etc. Anything >20 msec is considered slow. Multiple
tuning and BizTalk health dashboarding tools exist, and should be evaluated
and put in place where bad performance is suspected. For example, we’ve used
and recommend the Best Practices Analyzer from Microsoft, and the CodePlex
benchmark tuner for BizTalk 2006/2010.

Use
dynamically generated itineraries versus attached itineraries in messages

General
BizTalk best practice

This
removes one source of hardcoding in the design.

In
design, use the ESB Toolkit as the starting point – but not the endpoint. 

General
BizTalk best practice

The ESB
portal as delivered is to expose the capabilities of the ESB toolkit data and
their relationships but is not a production grade site as-is. 

All
two-way services into BizTalk produce a response, either an exception or an
acknowledgement.

General
BizTalk best practice

 

Calls
to request/response web services use asynchronous callback patterns.

General
BizTalk best practice

 

Messages
are validated against the schema for use case requirements.

General
BizTalk best practice

 

Just
Enough Logging (JEL) – scale back logging post-go live to sane levels

Logging
and exception handling

Logging
every message is demanding on database and storage resources, and over time
becomes less valuable. Post-deployment administrators only care if BizTalk is
running and if they have the ability to drill in on errors.

For
longrunning processes, users should have a way to inspect progress to date.

Logging
and exception handling

This is
done through dashboarding, such as SharePoint.

Do not
enable tracking for ports and orchestrations, and minimize the tracking of
MsgBody. Do enable tracking on Biztalk server hosts.

Logging
and exception handling

As a
general, the greater the amount of logging, the less said logging is used.
Minimize and target the tracking and logging you actually need.

Use
standard versus custom components.

General
BizTalk best practice

Once a team goes down the path of constructing
custom components to fit their view of what an ESB is, the solution instantly
becomes more difficult to understand by every member of the development team,
monitor, and maintain.

Avoid
complex deployment scenarios using shared custom core libraries

General
BizTalk best practice

As a rule of thumb, always exhaust all possible
scenarios using out-of-the-box (OOTB) components to solve an issue before
moving to custom code.  Most scenarios can be accomplished by leveraging the
samples provided, at the very least as a starting point. Shy away from
starting from scratch. This also eases deployment and maintenance (no custom
core components means nothing in the config files. Everything can be deployed
in a comprehensive package)

Write
tightly focused, small components

 

General
BizTalk best practice

This
promotes solid designs and reusability. Break up development in small units
of work and string these solutions together in the itinerary.

 

 

Choose
config over coding

 

General
BizTalk best practice

Always
favor config changes over code changes. Compiling and retesting entire
libraries for minor tweaks is bad practice.

Name
your port operations so you can easily identify them in orchestrations

Orchestrations

This
helps in troubleshooting issues later.

Using
SQL sprocs versus ADO.NET for CRUD operations within an orchestration

Orchestrations

 

Promote
only those properties that are required for routing or tracking. Use the
distinguished property for expression shape usage.

Orchestrations

 

Naming
standards follow a set convention.

Orchestrations

Multiple
examples of a well-thought out naming practice exist; all are fine if
followed consistently.

Use
XSLT as much as possible in mapping.

Orchestrations

XSLT
that use inline script or referenced assembles are inherently slower.

Where
.NET objects must be referenced, don’t use them directly but use external
assemblies instead.

Orchestrations

 

Minimize
or eliminate the use of functoids

Orchestrations

This is
especially true of database functoids, which are known performance killers.

Eliminate
orchestrations for messaging only patterns.

Orchestrations

 

Always
try to design orchestrations with Direct-Bound ports.

Orchestrations

 

No expression
shape contains an excessive amount of code that could be in an external
configuration.

Orchestrations

 

Resolve
addresses and maps at runtime using ESB toolkit resolvers such as the BRE
resolver versus static endpoints

Resolver
components

This means
that addresses and maps can change without resetting host instances, and the
same workflow can call different versions of a service – getting the most out
of SOA architecture. Itinerary can be changed and new services added without
code changes or redeployment.

Your
endpoints should always be resolved/configured via BRE over UDDI

Resolver
components

Power users often want to change endpoints and
map types; the BRE as a resolver engine is the most powerful and flexible
engine available. Comparatively UDDI both lacks intelligence and is has a
complex interface configuration.

Always
transform messages into a custom canonical schema.

Send/Receive
Ports

This
reduces the number of maps needed. Receive ports will use this one schema to
validate all inbound XML.

All
request/response ports that are exposed as web services are equipped with a
SOAP fault message.

Send/Receive
Ports

 

The
solution is tested with real-world data for the source system, using user
accounts with identical permissions as in production.

Testing

 

Use
maps for simple scenarios as a general rule.

General
BizTalk best practice

The map
itself is good for simple scenarios; but can be difficult to apply to complex
scenarios. Like going from C# to C++ – in C++ can create the list, but will
have to manage it more efficient.

zp8497586rq
May 182012
 

So I was assisting Brian last night put the Business Rule Engine in front of the ESB solution, so it could determine which itinerary to run. I had just completed getting a flow through the ESB myself. So, after some correction of spelling mistakes (come on code, don’t be so picky about absolutely needing strings to be match exactly, do what I want, not what I say), we got down to the guts of why things aren’t working.

We implemented the BRE Policy, tested the message: right clicked the policy and chose test: viola, it worked. However, when we ran it through the Itinerary Selector pipeline component we kept getting an error stating that Itinerary_getitinerary required the parameter @name which was not provided.

The issue is that the folks at Microsoft (or subcontractors paid to do the work for them) put the following code in the Business Rule Engine resolver:

Notice that the TypedXMLDocument that will be submitted to the BRE will always be Microsoft.Practices.ESB.ResolveProviderMessage.

private static Dictionary<string, string> ResolveRules(string config, string resolver, XmlDocument message, Resolution resolution, Microsoft.Practices.ESB.Resolver.BRE.BRE bre) { Dictionary<string, string> dictionary3; int majorRevision = 0; int minorRevision = 0; Policy policy = null; Dictionary<string, string> dictionary = null; string[] strArray = null; object[] facts = null; TypedXmlDocument document = null; string documentType = "Microsoft.Practices.ESB.ResolveProviderMessage"; Dictionary<string, string> resolverDictionary = new Dictionary<string, string>(); if (!resolver.Contains(@":\")) { resolver = resolver + @":\"; } try { ... } }

So how do you create your Business Rules so it adheres to this type of TypedXmlDocument?

When you import a schema to create a vocabulary or use directly within a policy, you will need to change the following property:

image

To look like this:

image

Go ahead and use the xml document all you want!

Apr 182012
 

So Joe and I were talking and I was showing what I was doing, here and he said I should be blogging about this since it doesn’t appear to be documented anywhere.

The resolver I made is a separate one so it can work on getting the endpoint configuration. Here are the steps I used to configure the endpoint in the Itinerary Designer.

  • Created the service schemas using the Visual Studio WCF Wizard
  • Created the map that creates the request message
  • Opened the binding file that is created as part of the project
  • Set the Resolver to Static and set the adapter to the appropriate binding (WCF-BasicHttp in my example)

Enter the following values:

image

On the Endpoint Configuration:

image

So now you have everything set in the Endpoint configuration.

Now to move all of the settings to the BRE:

image

So how do you get the Endpoint Configuration into the Set End Point Configuration to section? Simply highlight the value in the Endpoint Configuration and copy (control-c) and go the BRE and paste

Publish and Deploy

Set the Resolver to BRE

Choose the Policy from the drop-down

image

Voilà

The second tip is how to refer to XPath directly in the BRE from the BRE resolver

When you refer to a schema, remedy make sure you click on the root node and change the Document Type to Microsoft.Practices.ESB.ResolveProviderMessage and now you can drag the attributes/elements into your rules with confidence that the rule will fire. (Make sure you type the Document Type correctly Microsoft.EDB.ResolveProviderMessage will cause the rule not to fire Smile with tongue out)

image

Voilà deux

Apr 182012
 

First:

The ESB Toolkit 2.1 could be better

  • Compared to version 2.0, it appears to be compiled for .Net 4 (that is it), however all of the projects are still set to .Net 3.5
  • It used to be the ESB Guidance 1.0 so you can ‘extend’ it. However, there is no source code to do that anymore. If I wanted to change some behavior of a pipeline, I am screwed, I can download 1.0 and take a look at how it is done (or I can reverse engineer the current pipelines), but without doing a ton of work, it is not very extendable.
  • Calling into support is nearly a waste of time, okay: more than nearly.

Second:

There are a few deficiencies I have found with ESB 2.1:

  • The itinerary designer can query various databases (management db, rules engine db) to allow drop downs in the property window to choose things that were deployed. EXCEPT: orchestrations. For this, you have to go edit a configuration file, and restart visual studio (or if you really have some stones: unload a section in SSO make additions, and then reload it, using a nearly undocumented command line)
  • The entire design of the ESB solution assumes that each service you call responds with all of the data that you need to go to the next service that will be called.
    • WHAT?! First of all, a lot of services out there only return the value it needs to return, not the original payload coupled with the new value.
    • Many, if not all services can’t be re-engineered to change the behavior, as a lot of them are managed by an outside vendor.
  • The ESB Portal has to run in compatibility mode, even in IE8.
  • If you want to re-submit messages back into the message box, all of the context properties are GONE. Essentially, resubmittal is A TOTAL WASTE.

So, I have made a few changes to accomplish what I think are a couple fundamental changes to make the ESB toolkit usable.

First: I changed the behavior of the resubmittal process: here is the code I added to the portal:

(notice the source that is provided is for which version?)

//--------------------------------------------------------------------- // File: MessageViewer.aspx.cs // // Summary: This page displays the details of a message. It renders the message body // if it is an XML or flat file. Otherwise if allows the message // body to be downloaded. // //--------------------------------------------------------------------- // This file is part of the Microsoft ESB Guidance for BizTalk // // Copyright (c) Microsoft Corporation. All rights reserved. // // This source code is intended only as a supplement to Microsoft BizTalk // Server 2006 R2 release and/or on-line documentation. See these other // materials for detailed information regarding Microsoft code samples. // // THIS CODE AND INFORMATION ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY // KIND, WHETHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE // IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR // PURPOSE. //---------------------------------------------------------------------- using System; using System.Data; using System.Configuration; using System.Collections; using System.Web; using System.Web.Security; using System.Web.UI; using System.Web.UI.WebControls; using System.Web.UI.WebControls.WebParts; using System.Web.UI.HtmlControls; using System.ComponentModel; using System.Net; using System.IO; using System.Text; using System.Xml; using System.ServiceModel; using ESB.BAM.Service.Implementation; using ESB.Exceptions.Service.Implementation; using System.Collections.Generic; namespace Microsoft.Practices.ESB.Portal { public partial class MessageViewer : System.Web.UI.UserControl { #region Properties /// <summary> /// True if the message is text based, false otherwise. /// </summary> private bool MessageIsText { get { return (((Label)this.MessageView.FindControl("contentTypeLabel")).Text.Contains("text") || ((Label)this.MessageView.FindControl("contentTypeLabel")).Text.Equals("application/octet-stream")); } } /// <summary> /// Indicates whether a resubmission attempt has been made on the message. /// </summary> private bool _resubmitAttempted; private bool ResubmitAttempted { get { return _resubmitAttempted; } set { _resubmitAttempted = value; } } /// <summary> /// Indicates whether the message has been successfully resubmitted. /// </summary> private bool _resubmitSuccessful; private bool ResubmitSuccessful { get { return _resubmitSuccessful; } set { _resubmitSuccessful = value; } } /// <summary> /// Indicated whether the control is allowed to go into edit mode. /// </summary> private bool _editModeAvailable; private bool EditModeAvailable { get { return _editModeAvailable; } set { _editModeAvailable = value; } } /// <summary> /// Get the selected receive location from the 'receiveLocationList' DropDownlist. /// </summary> private string SelectedReceiveLocation { get { if (((DropDownList)this.MessageView.FindControl("receiveLocationList")).SelectedItem.Text == "Submit to Routing URL") { return "RoutingURL"; } else { return ((DropDownList)this.MessageView.FindControl("receiveLocationList")).SelectedValue; } } } /// <summary> /// Gets the message content type (MIME type). /// </summary> private string MessageContentType { get { return ((Label)this.MessageView.FindControl("contentTypeLabel")).Text; } } /// <summary> /// This property indicates whether or not the page is in the mode to /// edit and resubmit the message. This is determined by the esb:Mode /// querystring parameter containing the value: "Edit". /// </summary> private bool _editMode; protected bool EditMode { set { _editMode = value; } get { return _editMode; } } #endregion protected void Page_PreRender(object sender, EventArgs e) { SetControlVisibility(); SetControlEditability(); RenderMessageBody(); if (!IsPostBack) { PopulateReceiveLocationList(); } } private void BindGrid() { IExceptionService client = null; try { client = PortalHelper.GetExceptionService(); List<usp_select_ContextPropertiesResult> contextProperties = client.GetContextProperties(new Guid(Request.QueryString["esb:MessageID" ])); contextPropertiesView.DataSource = contextProperties; contextPropertiesView.DataBind(); } context; } protected void Page_Load(object sender, EventArgs e) { if (!IsPostBack) { IExceptionService client = null; try { client = PortalHelper.GetExceptionService(); List<usp_select_MessagesResult> messages = client.GetMessages(Request.QueryString["esb:FaultID"], Request.QueryString["esb:MessageID"]); MessageView.DataSource = messages; MessageView.DataBind(); BindGrid(); } catch (Exception ex) { throw ex; } finally { if (client != null) { ((IDisposable)client).Dispose( ); } } } this.EditMode = (Request.QueryString["esb:Mode"] == "Edit"); this.EditModeAvailable = (((HtmlInputHidden)this.MessageView.FindControl("resubmitSuccessful")).Value == "True" || !this.MessageIsText || this.EditMode) ? false : true; this.ResubmitAttempted = (((HtmlInputHidden)this.MessageView.FindControl("resubmitAttempted")).Value == "True"); this.ResubmitSuccessful = (((HtmlInputHidden)this.MessageView.FindControl("resubmitSuccessful")).Value == "True"); if (this.ResubmitSuccessful) { ((Label)this.MessageView.FindControl("resubmissionLabel")).Text = "<img src='../images/severityinformation.gif' style='vertical-align:middle;'/>&nbsp;This message has been successfully resubmitted."; ((Label)this.MessageView.FindControl("resubmissionLabel")).ForeColor = System.Drawing.Color.Black; } else if (this.ResubmitAttempted) { ((Label)this.MessageView.FindControl("resubmissionLabel")).Text = "<img src='../images/severitycritical.gif' style='vertical-align:middle;'/>&nbsp;A message resubmission was attempted but failed."; ((Label)this.MessageView.FindControl("resubmissionLabel")).ForeColor = System.Drawing.Color.Red; } } /// <summary> /// Calls the resubmit helper class to transmit the message back to BizTalk. Logs the result to the portal audit log. /// </summary> protected void resubmitButton_Click(object sender, EventArgs e) { //Resubmit the message to the selected receive location via HTTP string messageID = ((Label)this.MessageView.FindControl("messageIDLabel")).Text; string statusCode; string statusMessage; bool result = ResubmitMessage(((DropDownList)this.MessageView.FindControl("receiveLocationList")).SelectedValue, out statusCode, out statusMessage); this.ResubmitAttempted = true; if (result == false) { ((Label)this.MessageView.FindControl("resubmissionLabel")).Text = "<img src='../images/severitycritical.gif' style='vertical-align:middle;'/>&nbsp;The resubmission failed: " + statusCode + " - " + statusMessage; ((Label)this.MessageView.FindControl("resubmissionLabel")).ForeColor = System.Drawing.Color.Red; ((Label)this.MessageView.FindControl("resubmissionLabel")).Visible = true; } else { this.ResubmitSuccessful = true; this.EditMode = false; this.EditModeAvailable = false; ((Label)this.MessageView.FindControl("resubmissionLabel")).Text = "<img src='../images/severityinformation.gif' style='vertical-align:middle;'/>&nbsp;This message has been successfully resubmitted."; ((Label)this.MessageView.FindControl("resubmissionLabel")).ForeColor = System.Drawing.Color.Black; ((Label)this.MessageView.FindControl("resubmissionLabel")).Visible = true; } //Update the resubmission status in the Message table UpdateMessageResubmissionStatus(result); //Write an audit log event for the message resubmission AuditMessageResubmission(result, statusCode, statusMessage, ((DropDownList)this.MessageView.FindControl("receiveLocationList")).SelectedValue); } /// <summary> /// Resubmits the message body to the URL selected in the resubmit location DropDownList. /// </summary> /// <returns>True if the message was successfully resubmitted, false otherwise.</returns> private bool ResubmitMessage(string resubmitUrl, out string responseCode, out string responseMessage) { bool result = false; // returned. if (resubmitUrl.ToUpper().Equals("WCF ONRAMP")) { // resubmit message and capture the result System.Xml.XmlDocument doc = new System.Xml.XmlDocument(); doc.LoadXml(((TextBox)this.MessageView.FindControl("messageBodyBox")).Text); result = MessageResubmitter.ResubmitWCF(doc); if (result == true) { responseCode = "202"; responseMessage = "Successfully sumbitted to WCF OnRamp"; } else { responseCode = "500"; responseMessage = "Failure submitting to WCF OnRamp"; } } else if (resubmitUrl.ToUpper().Equals("SOAP ONRAMP")) { // resubmit message and capture the result System.Xml.XmlDocument doc = new System.Xml.XmlDocument(); doc.LoadXml(((TextBox)this.MessageView.FindControl("messageBodyBox")).Text); result = MessageResubmitter.ResubmitSOAP(doc); if (result == true) { responseCode = "202"; responseMessage = "Successfully sumbitted to SOAP OnRamp"; } else { responseCode = "500"; responseMessage = "Failure submitting to SOAP OnRamp"; } } else { HttpStatusCode statusCode; string MessageToSubmit = getCompleteContent(((TextBox)this.MessageView.FindControl("messageBodyBox" )).Text); //check to see if we need to set a proxy using (Stream messageReader = this.GetMessageDataStream(MessageToSubmit)) { HttpWebRequest HttpWRequest = (HttpWebRequest)WebRequest.Create(resubmitUrl); // TODO: (jsk) put settings in web.config // set the name of the user agent. This is the client name that is passed to IIS HttpWRequest.Headers.Set("Pragma", "no-cache"); HttpWRequest.UserAgent = Page.User.Identity.Name; HttpWRequest.KeepAlive = true; //this is the default HttpWRequest.Timeout = 300000; HttpWRequest.Method = "POST"; HttpWRequest.ContentType = MessageContentType; HttpWRequest.Credentials = CredentialCache.DefaultCredentials; // resubmit message and capture the result statusCode = MessageResubmitter.ResubmitHTTP(HttpWRequest, messageReader); //Anything between 200 and 299 is considered success result = ((int)statusCode > 199 && (int)statusCode < 301); responseCode = ((int)statusCode).ToString(); responseMessage = Enum.GetName(typeof(HttpStatusCode), statusCode); } } return result; } private string getCompleteContent(string body) { string UberDocument = "<ns1:Message xmlns:ns1=\"http://Message\">"; UberDocument += SerializeContext(); UberDocument += "<ns1:Body>" + body + "</ns1:Body></ns1:Message>"; return UberDocument; } /// <summary> /// Extracts and returns a Stream from the given string. /// </summary> /// <param name="messageDataString">The string to extract into a stream.</param> /// <returns>A stream from the given string.</returns> private Stream GetMessageDataStream(string messageDataString) { MemoryStream ms = new MemoryStream(ASCIIEncoding.Default.GetBytes(messageDataString)); return ms; } /// <summary> /// Updates the ResubmitAttempted and ResubmitSuccessful fields on the Message table /// in the database. /// </summary> /// <param name="success">True if the message was successfully resubmitted, false otherwise.</param> private void UpdateMessageResubmissionStatus(bool success) { IExceptionService client = null; try { client = PortalHelper.GetExceptionService(); client.InsertMessage(((Label)this.MessageView.FindControl("messageIDLabel")).Text, true, success); } catch (Exception ex) { throw ex; } finally { if (client != null) { ((IDisposable)client).Dispose( ); } } //DataSets.Faults.MessagesDataTable messageTable = new DataSets.Faults.MessagesDataTable(); ////The only values that are used by this call are MessageID, ResubmissionAttempted, and ResubmissionSuccessful. ////The 2nd new Guid is just there because null is not accepted. //messageTable.AddMessagesRow(((Label)this.MessageView.FindControl("messageIDLabel")).Text, null, null, null, null, null, null, null, true, success, DateTime.Now); //MessagesTableAdapter messageAdapter = new MessagesTableAdapter(); //messageAdapter.Update(messageTable); } /// <summary> /// Writes the failed or successful resubmission event to the audit log. /// </summary> /// <param name="success">True if the message was successfully resubmitted, false otherwise.</param> private void AuditMessageResubmission(bool success, string resubmitCode, string resubmitMessage, string resubmitUrl) { IExceptionService client = null; try { client = PortalHelper.GetExceptionService(); AuditLog auditLog = new AuditLog(); auditLog.ActionName = success ? "SuccessfulResubmit" : "UnsuccessfulResubmit"; auditLog.AuditDate = DateTime.UtcNow; auditLog.AuditUserName = Page.User.Identity.Name; auditLog.MessageID = new Guid(((Label)this.MessageView.FindControl("messageIDLabel")).Text); auditLog.ResubmitURL = resubmitUrl; auditLog.ResubmitCode = resubmitCode; auditLog.ResubmitMessage = resubmitMessage; auditLog.MessageData = ((TextBox)this.MessageView.FindControl("messageBodyBox")).Text; //client.InsertAuditLog(auditLog); } catch (Exception ex) { throw ex; } finally { if (client != null) { ((IDisposable)client).Dispose( ); } } ////Insert audit information //AuditLogTableAdapter auditLogTA = new AuditLogTableAdapter(); //DataSets.AuditLog.AuditLogDataTable auditLogTable = new DataSets.AuditLog.AuditLogDataTable(); //DataSets.AuditLog.AuditLogRow auditRow = auditLogTable.NewAuditLogRow(); //auditRow.ActionName = success ? "SuccessfulResubmit" : "UnsuccessfulResubmit"; //auditRow.AuditDate = DateTime.UtcNow; //auditRow.AuditUserName = Page.User.Identity.Name; //auditRow.MessageID = new Guid(((Label)this.MessageView.FindControl("messageIDLabel")).Text); //auditRow.ResubmitURL = resubmitUrl; //auditRow.ResubmitCode = resubmitCode; //auditRow.ResubmitMessage = resubmitMessage; ////auditRow.ContentType = (((Label)this.MessageView.FindControl("contentTypeLabel")).Text); //if (success) //{ // auditRow.MessageData = ((TextBox)this.MessageView.FindControl("messageBodyBox")).Text; //} //auditLogTable.AddAuditLogRow(auditRow); //auditLogTA.Update(auditLogTable); } /// <summary> /// Populates the receive location selection list with the WCF OnRamp, the SOAP OnRamp, and/or all HTTP receive locations on the BizTalk group. Flat File resubmission is limited to HTTP receive /// locations only, while XML document resubmission can be to WCF, SOAP, or HTTP. /// </summary> private void PopulateReceiveLocationList() { BizTalkOperationsService.Operations queryService = Microsoft.Practices.ESB.Portal.BizTalkOperationsService.Operations.CreateInstance(); BizTalkOperationsService.BTSysStatus sysStatus = queryService.ReceiveLocations(string.Empty); //Loop through the receive locations and extract the HTTP and WCF HTTP receive locations. int i = 0; ArrayList httpRcvLocs = new ArrayList(); foreach (BizTalkOperationsService.BTReceiveLocation rcvLoc in sysStatus.ReceiveLocations) { if (rcvLoc.Handler.ProtocolName.ToUpper().Equals("HTTP")) { rcvLoc.Address = "http://" + rcvLoc.Handler.Host.HostInstances[0].ServerName + rcvLoc.Address; httpRcvLocs.Add(rcvLoc); i++; } } DropDownList receiveLocList = ((DropDownList)this.MessageView.FindControl("receiveLocationList")); //Add a default selection to the applications list if (String.IsNullOrEmpty(((Label)this.MessageView.FindControl("routingURLLabel")).Text)) { receiveLocList.Items.Add(new ListItem("Select a receive location.", null)); } else { receiveLocList.Items.Add(new ListItem("Submit to Routing URL", ((Label)this.MessageView.FindControl("routingURLLabel")).Text)); } //Add the WCF and SOAP OnRamps if the message is not a flat file if (((Label)this.MessageView.FindControl("contentTypeLabel")).Text.ToUpper() != "TEXT/PLAIN") { receiveLocList.Items.Add(new ListItem("WCF OnRamp", "WCF OnRamp")); //URL will be retrieved from web.config receiveLocList.Items.Add(new ListItem("SOAP OnRamp", "SOAP OnRamp")); //URL will be retrieved from web.config } //Populate the receive locations list with the receive locations from BizTalk receiveLocList.AppendDataBoundItems = true; receiveLocList.DataSource = httpRcvLocs; receiveLocList.DataTextField = "Name"; receiveLocList.DataValueField = "Address"; receiveLocList.DataBind(); } /// <summary> /// This method shows or hides the correct control to display the message body. When the message /// body is text based, it is rendered to the screen in a TextBox. When the message body is binary /// a link is displayed to download it. /// </summary> private void RenderMessageBody() { //If the message content type is text if (this.MessageIsText) { //Make the message body download link invisible. ((TextBox)this.MessageView.FindControl("messageBodyBox")).Visible = true; } } /// <summary> /// Reads the mode from the querystring, and shows/hides the appropriate buttons. /// By default the Edit button is shown, if the mode = Edit, the Resubmit and Cancel buttons /// are shown. /// </summary> private void SetControlVisibility() { ((HtmlGenericControl)this.MessageView.FindControl("editButton")).Visible = this.EditModeAvailable; ((HtmlAnchor)this.MessageView.FindControl("messageBodyAnchor")).Visible = this.EditModeAvailable; ((LinkButton)this.MessageView.FindControl("resubmitButton")).Visible = this.EditMode; ((HtmlGenericControl)this.MessageView.FindControl("cancelButton")).Visible = this.EditMode; ((DropDownList)this.MessageView.FindControl("receiveLocationList")).Visible = this.EditMode; ((Label)this.MessageView.FindControl("resubmissionLabel")).Visible = this.ResubmitAttempted; } /// <summary> /// Sets the editability of controls based on whether the page is in edit mode. /// </summary> private void SetControlEditability() { TextBox messageBodyBox = this.MessageView.FindControl("messageBodyBox") as TextBox; messageBodyBox.ReadOnly = !this.EditMode; if (!this.EditMode) { messageBodyBox.BackColor = System.Drawing.Color.WhiteSmoke; } else { messageBodyBox.BackColor = System.Drawing.Color.White; } } public string GetCrumbedString(object t, int crumbAt) { string text = t.ToString(); if (text.Length > crumbAt) { //return "<span title='" + text + "'>" + HttpUtility.HtmlEncode( PortalHelper.CrumbString(text, crumbAt)) + "</span>"; return HttpUtility.HtmlEncode(PortalHelper.CrumbString(text, crumbAt)); } else { return text; } } protected void labelHeaderCaption_PreRender(object sender, EventArgs e) { Label me = (Label)sender; if (Request.QueryString["esb:Mode"] == "Edit") { me.CssClass = "iconMessageEdit"; } else { me.CssClass = "iconMessage"; } } protected void contextPropertiesView_RowDataBound(object sender, GridViewRowEventArgs e) { if (e.Row.RowType == DataControlRowType.DataRow) { Label name = e.Row.FindControl("lblName") as Label; if (name != null && name.Text.Equals("ItineraryHeader", StringComparison.InvariantCultureIgnoreCase)) { Label content = e.Row.FindControl("lblValueOriginal") as Label; if (content != null && !string.IsNullOrEmpty(content.Text)) { string contentData = HttpUtility.HtmlDecode(content.Text); XmlDocument xml = new XmlDocument(); xml.LoadXml(contentData); XmlNodeList nodeList = xml.GetElementsByTagName("Itinerary"); if (nodeList.Count > 0) { XmlNode node = nodeList[0]; if (node.Attributes["uuid"] != null) { string itineraryUuid = node.Attributes["uuid"].Value; HyperLink lnk = e.Row.FindControl("hlnkItineraries") as HyperLink; if (lnk != null) { lnk.Visible = true; lnk.NavigateUrl = "Itineraries.aspx?uuid=" + itineraryUuid; } } } } } } } protected void contextPropertiesView_PageIndexChanging(object sender, GridViewPageEventArgs e) { contextPropertiesView.PageIndex = e.NewPageIndex; BindGrid(); } } }

This creates a message that looks like this:

<ns1:Message xmlns:ns1="http://Message"> <ns1:Context> <ns1:Property Name="" Type="" Value="" /> <!--All of the context properties associated--> <!--Except those from the All.Exceptions--> <!--And the InboundTransportLocation (cause it is always the ESB Exception)--> <!--Along with any that has the error-report--> </ns1:Context> <ns1:Body> Content that is displayed in the text box </ns1:Body> </ns1:Message>

Well, the problem is that we don’t have a schema that looks like this message. I really just want to submit the content contained in the Body. So I created a decode pipeline component that access the SSO Configuration Store to promote certain properties of the message as it comes back into BizTalk.

using System; using System.IO; using System.Xml; using System.Xml.XPath; using System.ComponentModel; using System.Collections; using Microsoft.BizTalk.Message.Interop; using Microsoft.BizTalk.Component.Interop; using Microsoft.Win32; using System.Web; using System.Text; using SSO.Utility; using System.Diagnostics; namespace StottCreations.ContextReader { public class PromoteContext { /// <summary> /// Implements a pipeline component that takes unwraps the data from the body of the xml document /// and promotes the values in the content of the message to the context of the messaage payload. /// </summary> /// <remarks> /// </remarks> [ComponentCategory(CategoryTypes.CATID_PipelineComponent)] [ComponentCategory(CategoryTypes.CATID_Decoder)] [System.Runtime.InteropServices.Guid("FA7F9C55-6E8E-4855-8DAC-FA1BC8A499E2")] public class PromoteContextComponent : Microsoft.BizTalk.Component.Interop.IBaseComponent, Microsoft.BizTalk.Component.Interop.IComponent, Microsoft.BizTalk.Component.Interop.IPersistPropertyBag, Microsoft.BizTalk.Component.Interop.IComponentUI { #region Pipeline Properties private string ssoApplication = null; /// <summary> /// Application that contains all of the properties to be passed through the pipeline /// </summary> public string SSOApplication { get { return ssoApplication; } set { ssoApplication = value; } } #endregion #region IBaseComponent /// <summary> /// Name of the component. /// </summary> [Browsable(false)] public string Name { get { return "Context Promoter"; } } /// <summary> /// Version of the component. /// </summary> [Browsable(false)] public string Version { get { return "1.0"; } } /// <summary> /// Description of the component. /// </summary> [Browsable(false)] public string Description { get { return "Promotes context data from message and passes the data to Dissassembler"; } } #endregion #region IComponent public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg) { try { inmsg = Promote(inmsg); inmsg.BodyPart.Data = TransformMessage(inmsg.BodyPart.GetOriginalDataStream()); } catch (Exception ex) { EventLog.WriteEntry("Promote Pipeline Component", ex.Message, EventLogEntryType.Error); } return inmsg; } #endregion #region Helper function /// <summary> /// Extracts data from message /// </summary> /// <param name="stm">Stream with input XML message</param> /// <returns>Message</returns> public Stream TransformMessage(Stream stm) { Stream s = null; try { //Load Xml stream in XmlDocument. XmlDocument doc = new XmlDocument(); doc.Load(stm); XPathNavigator nav = doc.CreateNavigator(); XPathNodeIterator iter; // retrieve a node-set via XPath iter = nav.Select("/*[local-name()='Message' and namespace-uri()='http://Message']/*[local-name()='Body' and namespace-uri()='http://Message']"); // make sure a match was found while (iter.MoveNext()) { string data = iter.Current.InnerXml; s = new MemoryStream(ASCIIEncoding.Default.GetBytes(data)); } } catch (Exception e) { System.Diagnostics.Trace.WriteLine(e.Message); System.Diagnostics.Trace.WriteLine(e.StackTrace); throw e; } return s; } private IBaseMessage Promote(IBaseMessage inmsg) { string ErrorMsg = string.Empty; XmlDocument doc = new XmlDocument(); doc.Load(inmsg.BodyPart.GetOriginalDataStream()); XPathNavigator nav = doc.CreateNavigator(); XPathNodeIterator iter; // retrieve a node-set via XPath iter = nav.Select("/*[local-name()='Message' and namespace-uri()='http://Message']/*[local-name()='Context' and namespace-uri()='http://Message']/*[local-name()='Property' and namespace-uri()='http://Message']"); // make sure a match was found while (iter.MoveNext()) { //Check the SSO Application to see if we should be promoting it, otherwise skip it if (System.Convert.ToBoolean(SSOClientHelper.Read(SSOApplication, HttpUtility.HtmlDecode(iter.Current.GetAttribute("Name", ""))))) { try { string Name = HttpUtility.HtmlDecode(iter.Current.GetAttribute("Name", "")); string Type = HttpUtility.HtmlDecode(iter.Current.GetAttribute("Type", "")); string Value = HttpUtility.HtmlDecode(iter.Current.GetAttribute("Value", "")); inmsg.Context.Promote(Name, Type, Value); } catch (Exception) { //We could not promote this particular property, let's report it and go on ErrorMsg += "The " + SSOApplication + " SSO Application was attempting to promote " + HttpUtility.HtmlDecode(iter.Current.GetAttribute("Name", "")) + ", but it it can't be promoted, please remove it./r/n"; } } } if (ErrorMsg.Length > 0) EventLog.WriteEntry("Promote Pipeline Component", ErrorMsg.Substring(0,2048), EventLogEntryType.Warning); return inmsg; } #endregion #region IPersistPropertyBag /// <summary> /// Gets class ID of component for usage from unmanaged code. /// </summary> /// <param name="classid">Class ID of the component.</param> public void GetClassID(out Guid classid) { classid = new System.Guid("DA7F9C55-6E8E-4855-8DAC-FA1BC8A499E2"); } /// <summary> /// Not implemented. /// </summary> public void InitNew() { } public void Load(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, Int32 errlog) { string val = (string)ReadPropertyBag(pb, "SSOApplication"); if (val != null) ssoApplication = val; } /// <summary> /// Saves the current component configuration into the property bag. /// </summary> /// <param name="pb">Configuration property bag.</param> /// <param name="fClearDirty">Not used.</param> /// <param name="fSaveAllProperties">Not used.</param> public void Save(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, Boolean fClearDirty, Boolean fSaveAllProperties) { object val = (object)ssoApplication; WritePropertyBag(pb, "SSOApplication", val); } /// <summary> /// Reads property value from property bag. /// </summary> /// <param name="pb">Property bag.</param> /// <param name="propName">Name of property.</param> /// <returns>Value of the property.</returns> private static object ReadPropertyBag(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, string propName) { object val = null; try { pb.Read(propName, out val, 0); } catch (System.ArgumentException) { return val; } catch (Exception ex) { throw new ApplicationException(ex.Message); } return val; } private static void WritePropertyBag(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, string propName, object val) { try { pb.Write(propName, ref val); } catch (Exception ex) { throw new ApplicationException(ex.Message); } } #endregion #region IComponentUI /// <summary> /// Component icon to use in BizTalk Editor. /// </summary> [Browsable(false)] public IntPtr Icon { get { return IntPtr.Zero; } } /// <summary> /// The Validate method is called by the BizTalk Editor during the build /// of a BizTalk project. /// </summary> /// <param name="obj">Project system.</param> /// <returns> /// A list of error and/or warning messages encounter during validation /// of this component. /// </returns> public IEnumerator Validate(object projectSystem) { if (projectSystem == null) throw new System.ArgumentNullException("No project system"); IEnumerator enumerator = null; ArrayList strList = new ArrayList(); try { } catch (Exception e) { strList.Add(e.Message); enumerator = strList.GetEnumerator(); } return enumerator; } #endregion } } }

So in the pipeline that I have used to submit xml data back to the message box looks like this:

image

Within the SSO Configuration, I can promote as many (or few) properties as I need. Here is the only things I wanted to promote:

image

This means that it will take all of the properties, and only promote ReceivedFileName and OrderedDelivery.

There are certain properties that are not promotable, and as such, it will not blow up, but simply write an entry into the event log and continue on. You should really remove it from the SSO Configuration Store.

To address the need to carry data between one service call to the next, I had to modify the MessageEnrichment orchestration.

I modified it in these steps:

  1. I went through and created proper labels (NotOneSingleWordThatMakesItReallyHardToRead)
  2. Then I went through and created message names that made sense to an ordinary human
  3. I added a separate resolver in the middle so that there was a completely different step to determine the destination, instead of setting the map AND the destination in the resolver
  4. I took the copy context from the original message to the response of the service call. (Why in the world would you do that anyway? The response has a different target name space than the original message.)

This is what the new orchestration looks like:

image

Which means that I have Itineraries that look like this:

image

So if I need to call another service, I simply need to add the same Orchestration Extender, define the next three resolvers:

  1. Map that creates the request to the service
  2. Defines the endpoint configuration for the service
  3. The map the enhances the canonical message

I really means that it REALLY isn’t ESB though, it is a series of orchestrations chained together that call services.

Not cool…

Voilà

Dec 042010
 

Is anyone else wondering?

  1. If the ESB Toolkit 2.1 is designed for BizTalk 2010, vcialis 40mg why are all of the sample projects still compiled in Visual Studio 2008?
    image
  2. Why the projects are still referring to the old .Net version (which is incorrect in this project)?
    image
    which by changing the .Net version changes the references:
    image

Does anyone have an idea?

(Could it be that you can use the ESB Toolkit 2.1 with BizTalk 2009?)

Dec 042010
 

While working on this sample, I came across the following errors:

  1. I needed to change the version of .NET from 3.5 to 4.0 in the properties of the project.
  2. I got the following error when I compiled the error: The type ‘Microsoft.VisualStudio.Modeling.ModelElement’ is defined in an assembly that is not referenced. You must add a reference to assembly ‘Microsoft.VisualStudio.Modeling.Sdk.10.0, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a’.    C:\Projects\Microsoft.Practices.ESB\Source\Samples\Designer Extensibility Samples\Extenders.Itinerary.OrchestrationSample\Extenders.Itinerary.OrchestrationSample\OrchestrationSampleExtender.cs
  3. This made me install the following components:
  4. Visual Studio 2010 SDK
    Visual Studio 2010 Visualization and Modeling SDK
  5. This then required me to reference the following assembly in the project: C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.VisualStudio.Modeling.Sdk.10.0\v4.0_10.0.0.0__b03f5f7f11d50a3a\Microsoft.VisualStudio.Modeling.Sdk.10.0.dll
  6. Once I built the two projects, in the directions (step 7) it states: In Windows Explorer, open the \Lib folder under the Itinerary Designer install path. This actual path is: C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\Tools\Itinerary Designer\Lib\
Nov 172010
 

After much digging while trying to have a 100% silent install of BizTalk, I have determined that a silent install of the ESB 2.1 Toolkit is not possible.

I have made it as hands free as I think is possible. The below scripts are designed for a single server installation.

Here is the batch script that needs to be run:

  1: start /w pkgmgr /iu:IIS-ASPNET;IIS-BasicAuthentication;IIS-WindowsAuthentication;IIS-IIS6ManagementCompatibility;IIS-Metabase
  2: msiexec.exe /qn /i "\\BizTalk Server 2010 Enterprise\BizTalk ESB Toolkit 2.1-x64.msi"
  3: "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\VSIXInstaller.exe" "C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\Tools\Itinerary Designer\Microsoft.Practices.Services.Itinerary.DslPackage.vsix" /q /s:Ultimate /v:10.0
  4: move "C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\Bin\ESBConfigurationTool.exe.config" "C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\Bin\ESBConfigurationTool.exe.config.old"
  5: copy "\\BizTalk Server 2010 Enterprise\BizTalk Server\ESBConfigurationTool.exe.config" "C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\Bin\ESBConfigurationTool.exe.config"
  6: echo "Run ESBConfigurationTool.exe and press <Enter> when complete"
  7: pause
  8: "C:\Program Files (x86)\Microsoft BizTalk Server 2010\BTSTask.exe" ImportApp -Package:"C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\Microsoft.Practices.ESB.CORE_NOBINDING64.msi" -ApplicationName:Microsoft.Practices.ESB -Overwrite
  9: "C:\Program Files (x86)\Microsoft BizTalk Server 2010\BTSTask.exe" ImportBindings /Source:"\\BizTalk Server 2010 Enterprise\BizTalk Server\Microsoft.Practices.ESB.CORE_Bindings.xml" -Application:Microsoft.Practices.ESB
 10: "C:\Program Files (x86)\Microsoft BizTalk Server 2010\BTSTask.exe" ImportApp -Package:"C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\Microsoft.Practices.ESB.ExceptionHandling_NOBINDING64.msi" -ApplicationName:Microsoft.Practices.ESB -Overwrite
 11: "C:\Program Files (x86)\Microsoft BizTalk Server 2010\BTSTask.exe" ImportBindings /Source:"\\BizTalk Server 2010 Enterprise\BizTalk Server\Microsoft.Practices.ESB.ExceptionHandling_Bindings.xml" -Application:Microsoft.Practices.ESB
 12: msiexec.exe /qn /i "C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\Microsoft.Practices.ESB.CORE_NOBINDING64.msi"
 13: msiexec.exe /qn /i "C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\Microsoft.Practices.ESB.ExceptionHandling64.msi"
 14: iisreset
 15: pause

Line 1 adds the necessary additions to IIS

Line 2 actually installs the ESB components, which in turn adds the items to the start menu and unpacks the BizTalk msi’s etc.

Line 3 installs the Itinerary Designer into Visual Studio

Line 4 archives the default configuration file for the ESBConfigurationTool.exe application

Line 5 copies a new configuration file for the ESBConfigurationTool.exe to use

Line 6 echoes that it is time to run the configuration tool

Line 7 waits until the configuration has been run

Line 8 imports the ESB Core application into the BizTalk Administration Console

Line 9 applies the bindings (this needs to be customized from the samples provided during the install (Line 2))

Line 10 import  the ESB Exception application into the BizTalk Administration Console

Line 11 applies the bindings like Line 9

Line 12 installs the ESB Core application into the GAC

Line 13 installs the ESB Exception application into the GAC

Line 14 resets IIS since modifications were made in step 1

Here is a sample of the configuration file, so when you open up the ESBConfiguration tool, you simply need to enable each of the features, press the Apply Configuration tool and walk through it.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <!-- Management Database -->
    <add key="ServerInstance" value="."/>
    <add key="DatabaseName" value="ESBManagementDB"/>
    <add key="Username" value="CORP\svcBTSHost"/>
    <add key="Password" value="P4ssw0rd!"/>
    <add key="BizTalkAppGroup" value="CORP\AppUsers"/>
    <add key="BizTalkAdminGroup" value="CORP\ServerAdmin"/>
    <add key="UseSqlAuthentication" value="False"/>
    <!-- Itinerary Database -->
    <add key="ItineraryServerInstance" value="."/>
    <add key="ItineraryDatabaseName" value="ExceptionDB"/>
    <add key="ItineraryUsername" value="CORP\svcBTSHost"/>
    <add key="ItineraryPassword" value="P4ssw0rd!"/>
    <add key="ItineraryBizTalkAppGroup" value="CORP\AppUsers"/>
    <add key="ItineraryBizTalkAdminGroup" value="CORP\ServerAdmin"/>
    <add key="BizTalkIsolatedHostGroup" value="CORP\IsolatedUsers"/>
    <add key="ItineraryUseSqlAuthentication" value="False"/>
    <!-- Core Services-->
    <add key="CoreWebSiteName" value="."/>
    <add key="CoreUserAccount" value="CORP\svcBTSIsoHost"/>
    <add key="CoreUserAccountPassword" value="P4ssw0rd!"/>
    <add key="CoreBizTalkIsolatedHostGroup" value="CORP\IsolatedUsers"/>
    <!-- Exception Services-->
    <add key="ExceptionWebSiteName" value="."/>
    <add key="ExceptionUserAccount" value="CORP\svcBTSIsoHost"/>
    <add key="ExceptionUserAccountPassword" value="P4ssw0rd!"/>
    <add key="ExceptionBizTalkIsolatedHostGroup" value="CORP\IsolatedUsers"/>
    <!-- Configuration-->
    <add key="ConfigurationSource" value="False" />
    <add key="ApplicationName" value="ESB Toolkit" />
    <add key="ContactInformation" value="sucker@company.com" />
    <add key="AdminGroupName" value="CORP\SSOAdmin" />
    <add key="UserGroupName" value="CORP\AppUsers" />
    <add key="ConfigurationFilePath" value="C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\esb.config" />
  </appSettings>
  <system.diagnostics>
    <switches>
      <add name="ESBConfigTool" value="4" />
    </switches>
    <trace autoflush="true" indentsize="4">
      <listeners>
        <add name="FileListener"
             type="System.Diagnostics.TextWriterTraceListener"
             initializeData="EsbConfigurationTool.log"/>
      </listeners>
    </trace>
  </system.diagnostics>
</configuration>