A Real-World Experience using AIA Productivity Tools

One of the joys of working in product development, especially in my earlier customer success advocate role, is the satisfaction received when helping resolve customers needs with your products!

A few months back, a key telecom customer needed to describe the semantic Enterprise Business Object (EBO) structure in a hierarchical/indented model to a flat spreadsheet so that designers can perform application mapping for the Process Integration Packs (a.k.a. PIP, basically pre-packaged integration accelerators for specific business process flows). I pointed them to a not-well publicised (and highly underestimated tool) called the XSL Mapping Analyzer (a.k.a XMAN).

Below is an email from the customer describing how useful they found the tool, in fact just one of the utilities! Thanks to the OCS Team (Balaji S G and Kurt) and KT (Sungwoo Yu) for this valuable feedback!

Side note: XMAN is actually a set of utilities my team built to parse XSL and EBO XSDs into various formats for semantic analysis of attribute level mapping; including HTML, CSV, flattened CSV and XML. It’s real power is in providing tactical SOA governance if you are using AIA canonicals, as it can help flag functional interoperability issues across projects using the same EBO, thereby encouraging service reuse and providing the ROI on your SOA. To learn more about XMAN usage, see my earlier blog post and OOW 2009 session.

— Email below published verbatim with prior owner’s permission —

Hi Balaji,

I’m glad to share my experience with Rajesh’ blog

Specially, I use to visit and learn a best practices using aia from that popular blog.

I’d like to briefly introduce my situation.

I’m working at telecom Project aims to build next generation BSS/OSS for 35 mil subscribers utilized by Oracle RODOD(Siebel,BRM,OSM, OPH and etc) .

I’m in charge of Integration team.

As we try to fit OOTB PIPs with our business, we need to analysis OOTB PIPs and design extension of PIPs.

These days our team cooperation with OCS for Communication PIP extension design on AIA Technical Document Design.

Although we cannot totally understand what existing input/output message and mapping in complex process integrations, we have to provide explain how prebuilt aia integration build interoperability with Siebel, BRM.

Basically, we can refer PIP implementation documents. However, if we closely looked at message level input/output and mapping details for reviewing service detail communication with edge application. It had been challenge.

At that time, I concerned that below so that I requested the OCS guidance

For we describe edge ABM structure or complex EBMs on spreadsheet, how can we describe different level elements on spreadsheet automatically?

Why we need to describe on spreadsheet, most of edge application function designer is not familiar with complex related xsd and different level of structure specially ebo.

So, when designed for extension of pips, Edge application function designer is unable to consider EBO structure.

If AIA team share used each operation EBM by spreadsheet, they will be easy to approach understanding EBO

Do you have any idea for that?

1.       How to automatically convert message structure xsd to table. (It could be different structure distinguished by indentation)

2.       If we describe different level elements distinguish using XPath, How we automatically extract XPath of all nodes from EBM.

Thanks to OCS(Balaji, Kurt) helps

We can utilize AIA Development Tools such as XMAN, XSD Flatter and generated XMAN reports RV11.1.

Particularly, XSD Flatter is right to solve above two problems.

For instance, I’m going to explain how I can extract element tree on spreadsheet from SalesOrderEBM.xsd

source $AIA_INSTANCE/bin/aiaenv.sh.

cd $AIA_Installed Directory/DeveloperTools/XSDFlattener/bin/

 

xsd2csv.sh -inputSchemaURL $AIA_Metadata_Directory /AIAComponents/EnterpriseObjectLibrary/Industry/Communications/EBO/SalesOrder/V2/SalesOrderEBO.xsd -rootElement ProcessSalesOrderEBM

As converted, I’ve got a csv file with x-path, cardinality, annotation and element tree with display hierarchy using indentation, and then reformatted below.

Even, I can convert ABM of Siebel Integration Object Message as well.

xsd2csv.sh -inputSchemaURL $AIA_Metadata_Directory /AIAComponents/ApplicationObjectLibrary/Siebel/V1/schemas/SWIOrderIO.xsd -rootElement ListOfSWIOrderIO

 

In addition, I aware generated RV11.1 XMAN report already provided whole EBM csv information by support.oracle.com

Also I can generate csv report from all of EBM using eol2csv.

eol2csv.sh -inputDir $AIA_Metadata_Directory /AIAComponents/ApplicationObjectLibrary/Siebel/V1/schemas -outputDir ~/eoloutput/AIAComponents/ApplicationObjectLibrary/Siebel/V1/schemas

Thanks

Sungwoo

Advertisements

Five Frequently Asked Questions About EXTERNAL Web Services in Fusion Applications

The Fusion Applications OER white paper (introduced in my previous post on Fusion Applications Integration) touches upon the EXTERNAL keyword tagged on services.

FA OER Service External KeywordAs described in the paper, these services are used by customers/partners to extend and integrate with Oracle Fusion Applications, whether in on-premise or SaaS mode. This post describes the keyword in detail by addressing five frequently asked questions.

Q1. What does the EXTERNAL keyword imply for integration scenarios and SaaS deployments?

To understand this, let’s look at the Fusion Applications deployment topology below Note: this shows only SOA Composite services, but is also applicable for ADF Services.

Fusion Applications TopologyThe topology splits visibility to web and service resources into two categories – internal and external. This is achieved by fronting all HTTP requests through two different Oracle HTTP Servers (OHS), one for internal traffic routed between Fusion Applications servers, and another for external sources, which has only a small subset of the routing rules to prevent unauthorized access by external systems. This approach facilitates unhindered access within the Fusion Applications domain and acts as a firewall to restrict access to external systems.

In the example shown above, internal clients (i.e. those deployed in the same Fusion Applications domain) would have access to all composite services – A, B, and C and any service endpoints that they provide. In contrast, all other external clients are restricted to accessing only services provided through composites A and B.

Another characteristic of services that are exposed externally is that they also enforce a more stringent Oracle Web Services Manager (OWSM) policy using SSL or WS-Security message protection. By contrast, services NOT marked as EXTERNAL are protected using the OWSM Global Policy Attachments (GPA) feature, which in the case of Fusion Apps do not enforce message protection and rely only on username token or SAML identity propagation.

As you may have guessed by now, services tagged with the EXTERNAL keyword in OER, are the only ones provisioned with routing rules on the External OHS and protected with a stringent message protected OWSM policy. This is what makes them suitable for application integration use cases; even more so in the cloud where deploying custom services to the SaaS WebLogic Server domain may not be possible.

Q2. The service I want is not marked as EXTERNAL. Can I just tag it with the keyword for my integration project?

As explained above, making a service “EXTERNAL” is more than just tagging the keyword in OER. The service also needs to be secured using a message protection enabled local OWSM policy and requires modification of the OHS routing rules to allow service URI access to the external world. For Oracle-shipped services, this is done by the Provisioning framework. For custom services, this can be possible in on-premise and hosted installations, but may not be possible in the SaaS mode.

Most likely, if an Oracle-shipped service has not been marked as EXTERNAL, it is due to specific functional and/or security reasons, which is usually not addressable at the customer site.

Q3. If I can’t change non-EXTERNAL services, why document them in OER?

While it is true that non-EXTERNAL services cannot be consumed by external clients, they can still be consumed by internal clients, such as custom implemented composites that are deployed in the Fusion Applications SOA domain in on-premise or hosted deployments. Moreover, apart from service invocations, many composite services also have capabilities for layered customizations for BPEL processes and Oracle Business Rules. The OER entries for these non-EXTERNAL services therefore serve as API documentation.

Q4. Are EXTERNAL services the same as “Public” services?

The two terms — “EXTERNAL” and “Public” — are frequently used interchangeably, however, they are not synonymous! “EXTERNAL” refers to service visibility in the topology, while “Public” basically amounts to the level of API support provided by Oracle. In fact, there is no attribute called “Public”, instead it is referenced by the Compatibility attribute value of Supported.

While most EXTERNAL services tend to be Public, there are exceptions. For instance, mobile-enabled services accessed by iPad or iPhone apps (external to Fusion Applications) will be tagged as EXTERNAL. However, if the only intended supported client is the pre-certified mobile app, then the service may be marked with a Compatibility value of Not Supported to effectively make it “Private”.

The Fusion Applications OER white paper also lists the various combinations of the Keyword and Compatibility tags and their implications for service use in integration projects.

Q5. How do I find all available EXTERNAL services?

Using the cloud hosted OER instance. Simply search for EXTERNAL and limit the Asset Type to ADF Service or Composite Service to filter the results.

FA OER Service External SearchIMPORTANT NOTE: This is a simple text search and will result in services that have the word EXTERNAL in any metadata, including descriptions. Therefore, you may get false positives. To confirm the service accessibility, always check the Keyword value on the Taxonomy tab of the service detail page as shown in the screenshot at the top of this post.

Hopefully this post has addressed most of the questions about the EXTERNAL keyword usage. If you still have some, feel free to sound off in the comments.

Integrating with Oracle Fusion Applications: Discovering Integration Artifacts

This is a first in a series of posts I hope to write on the topic of integrating with Oracle Fusion Applications, which is the next generation of enterprise applications built on top of Oracle Fusion Middleware. My goal is to share the ease with which integrations are now possible using standards-based technologies with enterprise applications.

Oracle Fusion Applications are designed and built from the ground up using a service-oriented architecture (SOA). As I have discussed in an earlier post, visibility and control are the key tenets to SOA governance, and the first step in integrating with Oracle Fusion Applications is to find out what are the integration options available. Oracle Enterprise Repository, an industry-leading metadata repository, provides this visibility.

Oracle Enterprise Repository for Oracle Fusion Applications can be used to discover functional and technical information about integrating with other applications, including web services, business events, interface tables and data model documentation among other artifacts.

Oracle Enterprise Repository for Oracle Fusion Applications

Oracle Enterprise Repository for Oracle Fusion Applications

To maximize the usage of the repository, I have posted a white paper “Making the Most of the Oracle Enterprise Repository for Oracle Fusion Applications” – also available from the Help link on the top right of the repository. I thank Nigel King, Tim Hall and Kris Doherty for their valuable inputs and reviews on the paper. I hope that our customers and partners will find this a useful resource to mine the repository content. In future posts, I will discuss various best practice integration patterns i.e. how to integrate with Oracle Fusion Applications.

Note: Access to the repository and the white paper requires an Oracle account.

SOA Content and Technical Governance using AIA Foundation Pack

In an enterprise application integration scenario, SOA Governance is about visibility and control during the entire service lifecycle; including design time, where this boils down to maximizing the investments made in building the service portfolio to ensure that it provides for interoperability, reusability and extensibility.

AIA Foundation Pack Tenets

AIA Foundation Pack Tenets

The AIA Foundation Pack allows you to satisfy these criteria using content and technical level governance.

Content level governance – the key to Interoperability and Reusability

The AIA Foundation Pack provides content level governance at all the abstraction layers. The Enterprise Business Objects (EBO) and Enterprise Business Services (EBS) development go through thorough scrutiny with Subject Matter Experts (SME) from each of the Oracle application areas (yes, including Fusion applications) to come up with a rationalized object model. It then undergoes a standardization pass, which ensures adherence to standards such as UN/CEFACT Core Component Technical Specifications (CCTS) and XML Naming and Design Rules (NDR).

AIA EBO Development Methodology

AIA EBO Development Methodology

This level of governance renders the common object and services model instantly relevant as opposed to a theoretical model that is hard to implement, since it has been already vetted for mapping against most of the Oracle applications logical data models as part of the rationalization process.

Data models aside, the reason why the rationalization was even initiated in the first place is due to a compelling need at the business process level. The reference process models supplied with the Foundation Pack also allow you to drill down from the lowest levels of the process models right into the associated canonical objects and services. This allows you to now take your pick of the application capability that can best satisfy the business task at hand. Since the canonical layer has just “virtualized” your application capabilities, so you can start with the business problem instead of the constraints of the technology. This level of content governance not only relieves you from having to tackle with the typical SOA governance issues of having to decide on the right-sized service granularity, but it also creates a common platform for business and IT stakeholders to collaborate on i.e. shared semantics and a business process approach to integrations.

The content level governance just does not stop here, but goes deep into the implementation. Once the canonical objects are mapped to the actual application interfaces (Web Services, Component Interfaces, Interface Tables, APIs etc), these mappings can be then viewed using the XSLT Mapping Analyzer (XMAN) utility which analyzes the XSL maps and presents a tabular view of the EBO – Application mapping.

 

AIA XSL Mapping Analyzer (XMAN) Report

AIA XSL Mapping Analyzer (XMAN) Report

In the case of Oracle AIA PIPs, custom extensions to the Oracle provided maps are highlighted which allows for easier identification during upgrades.  In addition, map reports from existing Oracle PIP assets can be viewed to assist in and accelerate custom Application Business Connector Service (ABCS) development as well as ensure that your custom maps are in accordance with the Oracle best practices.

The tool can also be invaluable when comparing the same report from different transformations (potentially developed by different teams) to ensure that there are no mapping conflicts, which can be otherwise very difficult to detect.  (This is currently manual but in the process of being automated). More on the XMAN tool can be found in this post.

Technical level governance – the key to Extensibility and Reusability

With an ever-growing asset inventory of design patterns and programming guidelines, manual checking for compliance to the AIA Integration Developers Guides standards checklist can get a little tedious.

AIA PIP Auditor Summary Report

AIA PIP Auditor Summary Report

PIP Auditor is a tool designed to enhance productivity and improve the quality of the services by checking your AIA services for compliance to programming best practices and patterns found in the AIA Integration Developers Guide, as well as other industry best practices as appropriate. The report output is categorized per TOGAF standards and a compliance “score” is presented.

AIA PIP Auditor Detail Report

AIA PIP Auditor Detail Report

Developers can drill down into the problem areas and identify the areas to fix in order to achieve compliance. Violations can be drilled down into right down to the file level and where relevant, even down to the line/column number e.g. in the case of naming convention violations. The design of the tool is flexible enough to allow for incremental patching of new rules – more than 70 rules have already been implemented as part of an internal beta release, with more coming soon before we make it available on Metalink. More on the PIP Auditor tool can be found in this post.

There are a lot more productivity enhancing tools and utilities that are available as part of the AIA Foundation Pack, such as an Artifacts Generator and a Build Install Generator tools, which we will be discussing on this forum in future posts.

Do I really need AIA?

One of the frequently asked questions from those new to AIA or other canonical object based methodologies, is whether AIA is really needed in their specific SOA project.

In short, it depends.

Now, the longer version. AIA provides a robust platform and framework for business process driven, SOA based enterprise application integrations. If your business process already has a pre-built Process Integration Pack (PIP) available from Oracle for your specific enterprise applications, then it is a no-brainer to use the PIP.

For example, if you are implementing the Order to Cash process with Oracle Siebel CRM as the front-end system and Oracle E-Business Suite as the ERP system, then the Siebel CRM Integration Pack for Oracle Order Management: Order to Cash PIP should definitely be on your shopping list.  Even if the exact applications (or versions) are not a perfect fit i.e. say, you want the CRM integration to PeopleSoft or a legacy application, then you can start with the PIP and replace the Application Business Connector Services (ABCS) from the Oracle E-Business Suite to the other application. Compared to a 100% custom development solution, this provides you with all the benefits of a flexible integration framework and will be faster to implement since one end of the PIP has already been developed.

So the question “Do I really need AIA” only has relevance for business process driven SOA integrations for which there are no pre-packaged PIPs i.e. comparing use of AIA Foundation Pack (FP) vs a custom point-to-point development using Fusion Middleware. Now, the Foundation Pack is built on top of Fusion Middleware, so it must be providing value-add on top of it. I like analogies, so let’s take the following example.

Which is the faster, better way to build a plane?

 

AIA - Building Raw vs with Components

Building Raw vs with Components

I think everyone would agree that having pre-built components, engineered and tested to the specifications of what a plane needs, is a much faster and better way, than to do-it-yourself with basic raw materials, even if you have access to the same tools in both cases. The AIA Foundation Pack provides these pre-built components, that are engineered to the demands of EAI based SOA integrations. It provides the pre-built components such as the Enterprise Business Objects (EBO), the AIA reference architecture and programming model, governance tools (e.g. service repository, integration scenarios) and framework utilities (e.g. error handling, message level testing).

So in the case of an EAI-based SOA integration, the answer is absolutely Yes, you do need AIA!

Now, I know many would argue, “but I am not really building a plane, I just want a one-time, quick, cheap solution, which may never need any enhancements”.  In this case, going the route of direct point to point integration using SOA may suffice, but not without risks.  Once you end up building a passenger train (the cheap, quick solution instead of a plane), which runs on a direct point to point track between SFO and LAX, it becomes next to impossible to later turn it into a plane flying random segments (e.g. NY) with a higher capacity and faster performance. In almost all cases, you would have to re-implement the project to start building a plane.

So the key is to be clear on the business needs and then plan for the right enterprise architecture to implement it. If the business needs a plane, go for AIA. Evaluate the use of native SOA technologies with point to point interfaces, only when it is absolutely clear – at all levels of management – that you are building a train.