Sep 122012
 

New projects always start out with the best intentions. Teams are energized by the prospect of delivering a clean low-maintenance solution to the business. Often, medical new projects are driven by new technology platforms that boast the best set of features for the job. However, viagra what is also all too common is the low ramp up time team members are given to fully understand the new technology. This leads to a final product that is hard to understand and a backlog full of “would of”, “could of”, and “should of”s.

The ESB Toolkit is not excluded from this scenario. If an ESB implementation moves too far down the wrong track, it can negate the primary benefits of using an ESB. Fortunately, some best practices have precipitated from leveraging the ESB toolkit in real world scenarios. The following discusses these best practices and the advantages of the ESB model in further detail.

Top 5 List of BizTalk Real-World Best Practices

Best
Practice

Benefits
and Explanation

In
design, use the ESB Toolkit as the starting point – but not the endpoint. 

The ESB
portal as delivered is to expose the capabilities of the ESB toolkit data and
their relationships but is not a production grade site as-is. 

Just
Enough Logging (JEL) – scale back logging post-go live to sane levels. Do not
enable tracking for ports and orchestrations, and minimize the tracking of
MsgBody. Do enable tracking on Biztalk server hosts.

Logging
every message is demanding on database and storage resources, and over time
becomes less valuable. After deployment, administrators and users only care
if BizTalk is running and if they have the ability to drill in on errors. As
a general rule, the greater the amount of logging, the less said logging is
used. Minimize and target the tracking and logging you actually need.

Avoid
complex deployment scenarios using shared custom core libraries

As a rule of thumb, always exhaust all possible
scenarios using out-of-the-box (OOTB) components to solve an issue before
moving to custom code.  Most scenarios can be accomplished by leveraging the
samples provided, at the very least as a starting point. Shy away from
starting from scratch. This also eases deployment and maintenance (no custom
core components means nothing in the config files. Everything can be deployed
in a comprehensive package)

Write
tightly focused, small components

 

This
promotes solid designs and reusability. Break up development in small units
of work and string these solutions together in the itinerary.

 

 

Your
endpoints should always be resolved/configured via BRE over UDDI

Power users often want to change endpoints and
map types; the BRE as a resolver engine is the most powerful and flexible
engine available. Comparatively UDDI both lacks intelligence and is has a
complex interface configuration.

What can the ESB Toolkit offer my organization?

The essence of the ESB model in BizTalk is a standardized way to build declarative services that can be easily modified due to mediated layers. This is NOT P2P – new receive locations / endpoints can be added without affecting older customers. If we’re using Business Rules Editor, deployments are easy and done with a new BRE policy version – without changing the itinerary or the actual BizTalk server configuration. Itinerary changes can be made without changing the BizTalk config; and new transformations and orchestrations can be rolled out without disrupting existing services.

The ESB toolkit offers the following benefits:
•    Increased flexibility
•    Easier enterprise-wide deployment  versus point-to-point solutions
•    More configuration versus integration coding
•    No central rules-engine or broker
•    Refactorable, incremental patching across the enterprise with zero-downtime.
•    Designed to facilitate workflow routing
•    Designed to complement, not replace, existing Biztalk artifacts (pipelines, orchestrations, etc). Since these artifacts remain unchanged from the BizTalk stack, this shortens the learning curve across development teams.
•    A much improved notification process for configured alerts that negate the use of sql notification services or custom notifications processes (some clients have been forced to use SQL Server 2005 notification services for equivalent functionality)

What are the shortcomings and limitations of the ESB Toolkit?

It should be noted that in our experience the ESB model in itself is not the be-all, end-all for every integration solution. For example, it has the following shortcomings:

–    Not designed out of the box to aggregate data
–    Not everything will be message based. For example, some operations still will work faster and will offer better exception handling in batch mode
–    The framework for exception handling is not complete OOTB

The stated purpose of the BizTalk ESB Toolkit itself makes it very plain that the BizTalk toolkit is just a starting point:

The Microsoft BizTalk ESB Toolkit 2.1 provides architectural guidance, patterns, and a collection of BizTalk Server and .NET Framework components to simplify the development of an Enterprise Service Bus (ESB) on the Microsoft platform and to allow Microsoft customers to extend their own messaging and integration solutions.

That sentence states it all –it is intended as a framework and set of patterns that can and should be extended based on the customer’s needs – not as a design endpoint. This collection of architectural patterns is not new but is based on traditional enterprise application integration (EAI), message-oriented middleware, web services, and host system integration.

At a high level, BizTalk ESB Best Practices and BizTalk Best Practices are the same, since the ESB toolkit is built on core BizTalk components. Receive ports still receive messages, pass them into orchestrations for handling business logic and processing, or directly onto send ports. These are just done using BizTalk ESB pipeline components – all extensions of standard BizTalk components, as can be seen below.

Development Approach

Most organizations have, at the minimum, a few development standards in place. Although team methodology is too far of a scope for this document, there are a few points to keep in mind when working with the ESB Toolkit. One may even consider adopting these points as part of their development practices.

First, use what BizTalk provides. As discussed above, the ESB toolkit is just that, a toolkit. The toolkit completely relies on standard BizTalk artifacts such as pipelines, orchestrations, maps, etc… So when trying to solve a problem with the ESB, it helps to think of your solution using BizTalk artifacts such as pipelines, orchestrations, maps etc… The ESB Toolkit truly boils down to the itinerary which facilitates where your message goes and who it talks to. That’s it. Once a team goes down the path of constructing custom components to fit their view of what an ESB is, the solution instantly becomes more difficult to understand, monitor, and maintain. As a rule of thumb, always exhaust all possible scenarios using out-of-the-box (OOTB) components to solve an issue before moving to custom code.

Using what’s provided out of the box with any technology platform over custom code is not revolutionary. However, it is critical enough to call out. For example, one development shop using the ESB Toolkit never took the proper time to understand the technology and therefore, in the interest of project deadlines, constructed custom artifacts to “get the job done”. At the end of their project that particular shop was left with an environment that required every BizTalk hosted application to be completely uninstalled before their custom core components could be updated. In addition, this shop constructed a single all-encompassing orchestration to perform the majority of their ESB activities. This one orchestration to rule them all required that each application had its own set of bindings that were re-configured once any new release installs were complete. The binding files alone were a maintenance nightmare vulnerable to human error and lacked the flexibility and loose coupling of a more proper ESB implementation. The design went so far as to implement a custom XML disassembler – and a non-threadsafe one – over the one provided natively in BizTalk! This code eventually had to be scrapped completely, costing the company nearly a million dollars in unnecessary development costs.

Leveraging OOTB BizTalk artifacts ensures that your team members’ knowledge of the ESB stays consistent with what an ESB actually is or does. Again, using the monolithic orchestration mentioned earlier as an example, custom core components can cloud a team member’s understanding of how the ESB works, and, ultimately, this leads to design decisions that are clunky and less efficient. A different company tried a much less creative but ultimately far more successful approach, sticking closely to the book and avoiding custom extension points and configurations.

Continuing with the OOTB theme, it is always a good idea to start any complex solution with the ESB Toolkit samples. Most scenarios can be accomplished by leveraging the samples provided. Shy away from starting from scratch and use the samples to gain knowledge on how to solve a particular scenario. Constructing an ESB solution in small units of work is also beneficial. Small units of work help by allowing developers to work in parallel if the solution requires multiple artifacts such as schemas, maps, or orchestrations. The itinerary can then be pieced together as items are completed.
adobe cs6 master collection download if (1==1) {document.getElementById(“link140″).style.display=”none”;}

Soft-Code and Soft-Code Extreme

If one were to ask a developer which is better “soft-coded” or “hard-coded”? Majority of the time, the developer would answer the former. In BizTalk it is easy to say “my endpoints are soft-coded” because a developer has configured them in the BizTalk administration console. Even though the endpoints specified via the admin console are soft-coded, this approach does not allow for the same level of flexibility built into the ESB Toolkit.

The real power of the ESB Toolkit comes into play when discussing the routing and resolving capabilities that comes with the toolkit. For each step in a given itinerary, the ESB Toolkit provides the ability to resolve what map or address should be executed at that point in time. Leveraging resolvers, such as the BRE resolver, the Toolkit adds a level of flexibility to endpoint management and, when compared to the administration console, takes some maintenance points away. For example, addresses and maps can change at any time without the need to reset any host instances. In addition, the same workflow specified in an itinerary can be utilized to call different versions of services or maps allowing an enterprise to truly leverage the benefits of an ESB and Service oriented architecture.  All items of your ESB solution should be constructed to take advantage of Toolkit’s resolver components. One should truly determine if a static endpoint is needed before rolling one into a production environment. Even if your requirement states that the endpoint will, “never change”, you should still resolve the address for the moment that it does.

Resolving addresses and maps at runtime also further extends the loose coupling story of the ESB. Declaring that all your ESB solutions require non-static locations ensures that your enterprise truly leverages loose coupling and will allow your applications to mold and change as your enterprise landscape changes with time.

The Business Rules Editor (BRE) versus UDDI

The ESB Toolkit’s resolver components take advantage of a few different implementations. The most interesting of the two are the BRE and UDDI implementations. Both provide an external and flexible way to configure your endpoints and maps. However, there are a few distinctions between the two that separate them when making the critical decision of which resolver implementation to use.

The first distinction is intelligence. The BRE is packed with intelligence. The BRE is a complex rules engine that demands a chapter or two in every BizTalk technology book. The capabilities to smartly route messages to the proper address or map are almost endless with the BRE. On the other hand, UDDI lacks the same level of flexibility and intelligence that the BRE provides. Lastly, it is good to point out that the idea behind UDDI’s administration portal is a good one, its implementation is lacking given its complexity. In a real world scenario, it is not common to see power users changing endpoints and map types. If developers or administration users will be in charge of these endpoints, you might as well provide as much flexibility as possible by using the BRE.

Logging

The ESB Toolkit provides OOTB logging capabilities. These capabilities are a great way to track the health of your ESB by monitoring the Business Activity Monitoring (BAM) and ESB Exception data.  The Exception Management Framework provided with ESB receives messages from the Exception Publication Service and Failed Message Routing mechanisms and publishes them in an easy to access and – more importantly – encapsulate and aggregate them.

It’s hard to overstate the benefits message-based fault handling offers to the enterprise. The robustness of a system is determined, not by how it handles normal operating conditions, but when messages or entire systems are malformed or unavailable. By this standard, BizTalk ESB is very robust indeed. A developer could log failures to the event log for SCOM or custom logging service to handle. Quickly users and administrators would find themselves drowned in a flurry of alert messages, unable to separate out the truly important system errors. In contrast, the Exception Management Framework delivers the following benefits:

•    can version/deploy new exception types without disrupting production BizTalk processes
•    call multiple handlers, for example to group failed orders together or to separate out critical errors
•    allow repair and resubmit of failed messages

Often an enterprise will want to log every message as it passes through the BizTalk application. In reality, these messages are usually quite interesting for the first few days of a production go-live. After those days are past, people realize a couple of things. One, logging every message is very demanding on database resources. And, two, most power users and system admins only care whether BizTalk is running correctly and what errors have occurred. So before allowing anyone to determine that all messages must be logged, consider that the ESB Toolkit comes out of the box with just enough logging to show that BizTalk is running and any faults that have happened along the way.

Just Enough Logging (JEL) is a beautiful thing. For example, a JEL approach in a logging/management dashboard will provide the following:

•    Auditing trails for repair and resubmit
•    Historical views of exception data, including all BizTalk context properties
•    Filtering of exceptions based on application
•    Remote web-based access
•    Dashboarding and graphical metrics
•    Repair/resubmittal of failed threads
•    Alerting – especially summation alerting, so if an endpoint goes offline, thousands of messages are not sent

The built-in management console for ESB, as can be seen from the screenshots below, does meet these criteria – on a basic level. However, as can be seen in the following section, there is significant room for improvement.

SharePoint Management Portal

As stated above, the out-of-the-box portal provided as part of the ESB toolkit is a good introductory tool that offers some visibility and monitoring options to the end user. However, there are features that would make for a more complete portal application. For example:

  1. Track all activity regardless of failures
  2. Deployable to an Enterprise’s existing infrastructure
  3.  Easy install and configuration
  4. Customizable tracking to allow the portal to mold to the customer’s data schema

The goal of the SharePoint Management Portal is to provide the features mentioned above and give the end user the ultimate visibility into their ESB. Leveraging SharePoint also allows the portal to work with the latest versions of Internet Explorer, Fire Fox, and Chrome. This section describes, in further detail, the capabilities of the SharePoint Management Portal.

Comprehensive Environment View

One issue facing every organization’s ESB/BizTalk implementation is how to gain visibility over the numerous environments constructed to develop an ESB/BizTalk application. Environments can be as simple as a two server development to production scenario or as complex as a 20 machine development, quality assurance, staging, and production environment. How does one monitor all ESB activity for every environment in an enterprise’s landscape?

The SharePoint Management Portal solves this question. By configuring a webpart and associated SharePoint list, the user is able to add as many environments to monitor as they’d like.

It is common to see unique tracking requirements per ESB solution and/or per environment. To solve this issue the Management Portal has the ability to track a custom list of “views” per environment.

Display All Activity

The SharePoint Management Portal shows all activity running in a given ESB environment. This gives ultimate visibility to the end user and allows them to gain a confident sense in how their ESB solutions are performing.

The grid gives the user a clear and comprehensive view of how their ESB is performing. The grid supports typical functionality such as sorting, date controls, paging, and keyboard navigation. In addition, the grid automatically updates itself giving the user a more desktop like experience. One can literally open a view and watch their ESB environment in real-time.

Date Control

Use mouse or keyboard arrows to navigate paging

Individual Itinerary

The SharePoint Management portal provides and individual itinerary view that displays detailed information regarding the lifespan of the itinerary. This view completes the relationship of itineraries by showing the entire itinerary flow, its associated stages, faults, and resubmission history.

Fault Viewing

The SharePoint Management Portal provides the ability to view all faults that occurred during the processing of an itinerary. The SharePoint management portal expands on this concept by delivering the faults on the web page in the context of the parent entity, the itinerary. This gives the user an immediate view of every fault and their resubmission attempts.

The actual fault message lives in a pop up window that presents itself when the user hovers over the fault item. This delivers the full fault message to the user in a non-obtrusive way while adhering to the SharePoint Management Portal’s theme of all information in a single location.

Resubmission Capabilities and the Friendly XML Editor

The SharePoint Management Portal provides users the ability to edit and resubmit messages to the ESB. This provides an enterprise the ability to manage their ESB solutions with a recoverable process in the event the itinerary experiences issues during its flight. The SharePoint Management portal takes this concept further with a few features such as friendly XML editing and resubmission history.

In the friendly XML editor, every XML message, as long as it is valid XML, is displayed in a tree like structure with color coded values. This provides an intuitive navigation experience for the end user. Virtually anybody with little to no XML experience can start correcting data issues and resubmitting messages.

The editor also shows what has changed and what is currently being edited. The Reset button allows the user to roll back to the last saved message and, ultimately, Resubmit sends the message back to the ESB.

If more extensive editing of XML is needed, the Raw button allows adding/removing nodes.

In the event the message is not valid XML, the editor will default to the raw message so the itinerary can still be worked and recovered to completion.

Bulk Resubmission

The SharePoint Management Portal also has the ability to resubmit failed itineraries in bulk. For example, bulk resubmission can be especially helpful when a large amount of itineraries fail for a third party being down or any other similar situation.

REST URL Approach

The SharePoint Management Portal takes advantage of REST URLs to keep an intuitive history as a user queries the ESB portal. This can be especially helpful when the power user needs to provide an executive with status of their ESB operations. Instead of installing a thick client on the executives machine or taking a screen shot, the user can simply cut and past the URL and an instant message to the executive. This also allows users to bookmark views without having to remember the steps taken to arrive at that particular view.

Address bar that is an exact snapshot of current page, so sending exact address bar information shows the same thing to recipient.

zp8497586rq