Getting Started with the Real Time Information Director (RTID) Table of Contents Foundation for a Real-Time Solution Data Store Facilities for Loading and Extracting Data Enterprise Application Integration Auditing and Security Features Management Facilities Client Software Data Store Application Server Message and Document Formats Metadata Classes Transport Protocols Implementation Tasks Installing Required Software On the Windows Workstation On the NonStop Server On the EAI Platform Design and Definition
At the simplest level, a real-time solution demands ● A data store ● Facilities for loading and extracting data from the data store ● Options for integrating diverse enterprise applications with the data store ● Security and auditing features to assure privacy and regulatory compliance ● Management facilities Data Store The real-time data store includes the enterprise data that must be available at all times to enterprise clients.
Facilities for Loading and Extracting Data The messages enterprise applications exchange must be decomposed and mapped for insertion into a relational data store. For example, different fields in the same SAP IDoc, or in the same record of a patient encounter, might need to be stored in different tables. In addition, some fields of an incoming record might not need to be kept in the data store.
Figure 2. Loading and Extracting Data For more information about the input, output, and metadata mapping facilities of the Director, see the document entitled Technical Overview of the Real Time Information Director. Enterprise Application Integration To integrate enterprise applications with the data store, you need tools for enterprise application integration (EAI). EAI tools include modules to perform data transformations and to implement routing based on predefined business processes and scenarios.
which they are entitled. Furthermore, it is crucial to be able to verify who accessed or modified what information at what time. To support these demands, the Director supports ● Metadata to describe security relationships and policies ● Data tables for security-related data ● Features that permit auditing of interactions with the data store For more information, see the document entitled RTID Security and Auditing.
Figure 4. Client Access You can develop your own clients, using any of several interfaces: ● A servlet interface ● An Enterprise Java Bean (EJB) interface ● An interface to the Director Dispatcher component (used when running the Director outside of NonStop WLS) ● A low-level interface that accepts and returns objects, rather than XML documents The following figure illustrates these interfaces.
Figure 5. Programmatic Interfaces For more information, see the document entitled RTID Message Format and Programmatic Interfaces. Standards-Based Infrastructure The Director uses a standards-based infrastructure for ease of use and compatibility with available solution development and deployment tools. Data Store The data store is based on NonStop SQL/MX, an implementation of the ANSI SQL-92 standard for the NonStop Server. The Director can use either SQL/MX or SQL/MP tables.
The EHR example handles messages in two formats: ● Health Level 7. As already mentioned, the SeeBeyond e*Gate product suite maps between HL7 and the XML format the Director uses. ● Native XML. XML messages are mapped to and from records and fields in the data model, according to document definitions expressed in metadata. For your real-time solution, you can use EAI tools to map other message formats to native XML. Metadata Classes A solution designer uses a text editor or Java IDE to specify metadata.
WLS on the NonStop Server before you design an EAI interface (because the EAI interface must be configured to point to the WLS configuration). The Director itself includes files to be installed on the workstation and files to be installed on the NonStop server. Workstation components are packaged in a zip file, NonStop server components in a PAX file. For information about installing these components, see the RTID readme file.
● Metadata file for the type of solution (e.g., rtscDefs.jar) ● RTID deployment scripts ● Data store scripts for the type of solution On the EAI Platform To use the webMethods Integration Server: ● webMethods 6.0.1 Integration Server ● webMethods 6.0.1 Developer ● webMethods JMS Adapter for 6.0.1 ● webMethods 4.6 SAP Adapter ● WebLogic JAR file To use the SAP Exchange Infrastructure: ● SAP Web Application Server (WAS) 6.20 ● SAP XI 2.
● SeeBeyond HL7 OTD library for e*Gate 5.x ● SeeBeyond XML Toolkit for e*Gate 5.x ● SeeBeyond e*Gate Monitor for e*Gate 5 Design and Definition To customize the infrastructure for your solution, perform the following design tasks, in this order: 1. Define the requirements for your solution by considering the formats of messages that traverse the enterprise (or solution domain).
starting metadata class definitions in c:\rtid\chin\com after you install the Director on the workstation.
For detailed information about defining subscriptions, see the document entitled RTID Subscriptions. 6. Design EAI Interfaces using the SAP XI Integration Builder, the webMethods Developer, or the SeeBeyond Enterprise Manager. Deployment To deploy the solution on the NonStop Server, perform the following tasks in order: 1. Create the data tables, using the DDL file you created in the design phase. For information about creating the data store for RTSC, see the readme.txt file in c:\rtid\rtsc.
9. Test the configuration, end-to-end, using the JMS or HTTP adapters of your chosen EAI tool. 10. Test the batch input functions, using the webMethods built-in file service, the XI File Adapter, or the SeeBeyond File e*Way, to transfer data from the EAI platform to the NonStop Server, and using the Delsert program to load the data into the data store. For information about running the Delsert program, see the document entitled RTID Extensibility. 11.
● Test the new functionality For information about these tasks, see the document entitled RTID Extensibility. Supply Chain Scenario An early user of RTSC wants to create a message flow in which an SAP order-processing application sends an IDoc representing an order (ORDERS05) to the hub, where the order is enriched with customer data and forwarded to a supply-chain visibility application.
2. Create metadata for an XML document describing the HL7 message, as it will be stored in the EHR data store. If an appropriate document definition already exists, you might still want to modify it to specify security and auditing requirements. Given the new or modified document definition, the Director automatically generates metadata for a corresponding query, i.e., the query you'd use to retrieve the document just as it was submitted. 3.
Alias A metadata option used to qualify the names of names of columns obtained from joined tables so as to prevent ambiguity in SQL statements and in XML output. Aliased document definition A document definition that represents one of several possible mappings of the same IDoc (as described in a base document definition), and that uses a distinctive top tag (specified to the IDocSegment constructor) to identify the mapping to be used. AuditDetails A database table to which audit details are written.
Content metadata Java-based metadata used to create and execute prepared SQL statements that delete old records and insert new ones related to an IDoc or other XML message. The metadata describes relationships among tables and associations among records in multiple tables. Each real-time solution comes with starting content metadata that you can customize.
file. Dispatch runs from a PC and uses JDBC for database access. Dispatcher The component that parses an incoming message and assigns an appropriate document handler. DOCCLASS table An SQL/MX table that indicates what class to load to process a given IDoc type. When you create a document definition, you must add a corresponding entry to the DOCCLASS table. Document definition The metadata file that describes a document type.
adapters. The webMethods Integration Server supports FTP as a built-in service. The NonStop Server supports FTP by means of an FTP server process. Format metadata Java-based metadata that describes the mapping between an IDoc and the RTSC Data Store. Data elements from an IDoc can be stored in different database records and fields, depending on the data model. RTSC includes predefined metadata corresponding to the starting data model; however, both the data model and the metadata are extensible.
XI Integration Server. iHUB Nickname for the HP supply chain implementation, which uses the Director. The RTSC Data Model is based on the iHUB data model. Insert The type of insertion performed by the Director when an inbound document is certain to be new. InstanceId A metadata option used to assign a line number or subitem number to a specified column of a record that is logically a child to another record.
The correspondence between a tag in an XML document and a record or field in the Data Store. Also, a mechanism for correlating external IDs in documents with internal IDs used in the Data Store. In EAI, the correspondence between a data element in an incoming message and a data element in the message format expected by a target system. MBeanClient Name of the Director management client class.
QueueReceive program A test driver that you can use to test the deployment of the Subscription Module under WLS without having to install and configure EAI software. QueueReceive retrieves a message that the Subscription Module has posted to the JMS queue. QueueSend program A test driver that you can use to test the deployment of the Director under WLS without having to install and configure EAI software. Queuesend places a message on the JMS queue for processing by the Director.
Selector An IDoc header element whose value determines which of several possible mappings will be used for an IDoc. For example, the value of the element SNDPOR in the IDoc header specifies which of several possible mappings to use for an ORDERS05 IDoc. Subscription A request, by an application, to be notified when a specified change or type of change occurs in the Data Store. Support for subscriptions is a feature of NonStop SQL/MX.
Variant document definition A document definition that represents one of several possible mappings of the same IDoc (as described in a base document definition), and that uses a distinctive value of some IDoc header element (defined as the "selector") to specify which mapping to use. Warning threshold The number of tags, in an inbound XML document, that will cause a warning to the effect that the document is almost too large. (Compare Abort threshold.
Technical Overview of the Real Time Information Director Technical Overview of the Real Time Information Director (RTID) Table of Contents Director Document Handling .......................................................................................... 3 Loading the Data Store.................................................................................................... 5 Partitioning the Data Store ..............................................................................................
Technical Overview of the Real Time Information Director The Real Time Information Director (RTID or Director) is a major component in several NonStop Enterprise Division Solutions: • • • Real Time Supply Chain Real Time Health Care Real Time Financial Services The name “Real Time Information Director” reflects the purpose of the product. The product supports Real-Time solutions, giving enterprises access to business data in seconds instead of days or weeks.
Technical Overview of the Real Time Information Director Director Document Handling The Director maps the content of incoming documents to an SQL/MX Data Store and retrieves, from that Data Store, the content required for outgoing documents. The Director object classes that perform the SQL statements and transactions are called Document Handlers: • • An update document handler stores segments and fields from an incoming XML message in the corresponding rows and columns in the Data Store.
Technical Overview of the Real Time Information Director • • • • • • • To assign a line number or subitem number to a child record To join one table with another To create aliases for column names To list the columns that will be included in a document To specify the order of records in an outbound (query) document To specify requirements for partitioning data tables on the basis of artificial or natural keys To invoke external services, such as business rules or data cleansing The ability to generate uni
Technical Overview of the Real Time Information Director Loading the Data Store Among the most important functions of the Director is to let enterprise applications submit event and other data to the Data Store. Any enterprise application can submit data, provided that it can post XML documents, using HTTP or Java Messaging Service (JMS), directly or through an EAI tool (often called an Integration Server).
Technical Overview of the Real Time Information Director The Real Time Information Director runs in a NonStop WebLogic Server (WLS) environment on the NonStop Server. It consists primarily of a set of Enterprise Java Beans (EJBs or MDBs), which provide access to the RTSC Data Store and optionally to message de-duplication and cleansing services. When the Director application starts up, beans are activated in each instance of NonStop WLS on the NonStop server.
Technical Overview of the Real Time Information Director Clinical System HL 7 Integration Server (SeeBeyond, TIBCO) XML via HTTP Director Servlet Content Metadata Real Time Information Director EJB NonStop WLS Document Handlers EHR Data Store Figure 2. Loading the EHR Data Store Partitioning the Data Store Partitioning provides the ability to spread a table across multiple disks to help balance workload and improve the performance of a solution.
Technical Overview of the Real Time Information Director The advantage of artificial partitioning is that the Director can use your knowledge of how the data is typically accessed to help ensure that the data is both evenly distributed and efficiently clustered. The logical key of a table can be either a natural key--data present in the input record, for example a customer name--or an artificial key assigned by the Director.
Technical Overview of the Real Time Information Director Queries and Subscriptions The Director provides two ways of retrieving data from the Data Store: queries and subscriptions. Queries A client queries the data store by sending a query document to the Director servlet. Both the query document and its response are in XML and are defined by Director metadata. The following figure represents the query interface in the context of an Electronic Health Record (EHR) solution.
Technical Overview of the Real Time Information Director Subscriptions The Subscription Module serves the needs of applications that must be notified of specific changes in the Data Store. A “subscription” specifies the triggering event and the format of the document to be delivered to the subscribing application. The Subscription Module supports any document format defined in the metadata; it uses a Document Handler to construct the required message.
Technical Overview of the Real Time Information Director Subscription Registration Interface A user (shown in the upper right corner of the figure) uses the Subscription Registration Interface to specify a subscription. A subscription has three attributes: • • • A SQL Select statement, which defines the event in the data store that will trigger the subscription A Java Messaging Service (JMS) topic or queue name. This name indicates where the subscription server must publish the message.
Technical Overview of the Real Time Information Director The Subscription Server posts the outgoing message to the JMS topic or queue specified in the subscription.
Technical Overview of the Real Time Information Director Security Features The Director supports the definition of security policies that specify which agents have access to each consumer’s data. For example, a standard self-service policy gives consumers access to their own data; a personal agent policy gives specific agents access to the data of specific consumers. Custom policies can be implemented as concrete subclasses of abstract policy classes provided with the Director.
Technical Overview of the Real Time Information Director • • • • • • WLS. Others use HTTP or JMS for access to the Director. SQL statement tracing is a valuable feature of some drivers. A utility to generate XML schema corresponding to document metadata. Clients and integration servers can use the schema to validate documents prior to submission.
RTID Deployment Real Time Information Director (RTID) Deployment Table of Contents Overview......................................................................................................................... 3 Deployment Process ....................................................................................................... 4 Prepare for Deployment ................................................................................................. 4 Install Prerequisite Software .................
RTID Deployment Properties Related to Input File Processing ........................................................... 20 ABORT_THRESHOLD ........................................................................................ 20 CheckDocumentSequence ..................................................................................... 20 MaxDocumentHandlers......................................................................................... 21 WARNING_THRESHOLD ........................................
RTID Deployment Overview This document describes the process for deploying the components of the Real Time Information Director (also known as RTID or Director) to the BEA Weblogic Server version 8.1. The Director is a Java application that is deployed as an Enterprise application under WebLogic. The application has the following components: • Dispatch EJB.
RTID Deployment Deployment Process The deployment instructions in this document show how to deploy the Director components under WebLogic, first under WebLogic running on your PC workstation, and then under WebLogic on the NonStop Server. The deployment process is as follows: 1. • • • • • Prepare for deployment Install prerequisite software Install the Director Set up the database Specify properties Create the Director Enterprise application deployment file 2.
RTID Deployment 1. Use the IPSetup utility to install the NonStop and Workstation components of RTID. The Workstation components are packaged as a zip file “rtidws.zip”. 2. Unzip rtidws.zip and specify “c:\” as the target folder. This step will create the c:\rtid folder and install the required files there. The J2EE components required for a deployment of a solution using the Director are released in an ear file called Dispatch.ear, which you’ll find in the folder c:\rtid\lib.
RTID Deployment Edit and use the c:\rtid\rtsc\rtsc.properties for running the RTSC example on a workstation. Edit and use the c:\rtid\chin\chin.properties for running the CHIN example on a workstation. Edit and use the c:\rtid\lab\library.properties for running the LAB example on a workstation. Note: To access the NonStop SQL/MX database from the workstation, download and install the jdbc type 4 driver for SQL/MX (T1249), and make sure the t4sqlmx.
RTID Deployment Create the Dispatch EAR File To deploy a solution, you must create an Enterprise Application deployment file (Dispatch.ear) that describes your Director Enterprise application. Create the EAR file by modifying the default Dispatch.ear file to include data for your application and target system. 1. Open the BEA Weblogic Build tool from the start menu under BEA Weblogic 8.1 Platform -> Other Development Tools -> Weblogic Builder 2. Open the c:\rtid\lib\Dispatch.
RTID Deployment Select ‘Development Mode’ and ‘BEA Supplied SDK – Sun SDK,’ and click ‘Next’ Enter the domain name, for example ‘mydomain,’ and click ‘Create’ Check the ‘Start Admin Server’ box and click ‘Done’ 2. If the server starts without complications, a DOS window will come up, and the server log will display the message ‘Listening on Port 7001.’ 3. Copy the zli.properties file from the c:\rtid\scripts folder to your WebLogic folder.
RTID Deployment /usr/bea/user_projects/domains/RTID/scripts 3. Copy the setenv_server.sh and setenv.sh scripts from the /usr/tandem/rtid/scripts/ folder to the /scripts folder. 4. Edit setenv_server.sh and setenv.sh to set the RTID_APP_HOME value to the domain folder name, for example: RTID_APP_HOME=/usr/bea/user_projects/domains/RTID If you want to deploy the RTSC example, uncomment the line #CLASSPATH=$RTID_HOME/examples/rtsc/lib/rtscDefs.
RTID Deployment Configure Weblogic Server for the Director Before you start the Director server for the first time, use the WebLogic console to add Director-specific configuration information. WLS saves the changes, so they’ll apply to any future restarts of the server. You connect to the WebLogic console by specifying its URL, for example http:// 16.107.199.52:7011/console). Add Director Specific Configuration 1.
RTID Deployment If your solution requires subscriptions, refer to the RTID Subscriptions documentation for details about adding the subscription startup class. Create the JMS resources for LoaderMDB If your Director Enterprise application does not use the Loader MDB interface, you can skip this step. 1. Create a JMS ConnectionFactory This step will create a JMS connection factory called “LoaderJMSConnectionFactory”.
RTID Deployment Create and configure a queue for the LoaderJMSServer and call it “weblogic.jms.LoaderQueue”. • • • • On the left panel of the Weblogic Console, expand the ‘Services’ and ‘JMS’ nodes. Select the ‘Servers’ element and the new LoaderJMSServer. Select the ‘Destinations’ element under the LoaderJMSServer node. Click ‘Configure a new JMS Queue.’ Set the following values for the queue: Name = ‘LoaderQueue’ JNDI Name = ‘weblogic.jms.
RTID Deployment either Dispatch or appsdir_Dispatch_Ear You should see the DispatchEJB, LoaderMDB and DispatchServlet listed. • To verify the DispatchServlet is running you can run it from your web browser by going to it’s url, for example http://16.107.199.52:7011//RTIDServlet/Dispatch On the page returned, you should see the Dispatch Server version and an indication that it is connected to the Dispatch EJB.
RTID Deployment Deploy the Director on the NonStop Server as a Generic Process The Director can be configured as a Generic process to provide failover if the process terminates abnormally. (Abnormal termination could be due to a CPU failure, a runtime error affecting the JVM process, or some other cause.) The HP NonStop Server Toolkit for BEA WebLogic Server 8.1 provides scripts for running your WebLogic Servers as Generic process.
RTID Deployment typeset GPPNAME="\$RTIDG" • If necessary, update the priority of the Generic process: typeset PRIORITY=150 • # As appropriate for your system Update the user id. The Generic process and the Director WebLogic process will both be started using this user id. Make sure this user id has access to both the WebLogic server and the application database used by the Director. typeset USERID="240,10" # The WLS Administrator's Guardian userid 3. Open startgpServer.
RTID Deployment 5. Open the stopServer.sh and change the URL, user, and password for the WebLogic server. Start the Director as a Generic Process 1. Make sure the RTID WebLogic server is in a stopped state. 2. Open a new OSS session. Log in as super.super user. 3. Change directory to /scripts. 4. Grant execute permissions to the script files: chmod +x *.sh 5. Alter the ownership of the “startgp.sh” and “stopgp.sh” to super.super. You need super user access to create the Generic process.
RTID Deployment 10. If you don’t see the success message, check the /rtidstderr.log and see why the WebLogic server failed to start. Stop the Director Generic Process 1. Open a new OSS session, using the SUPER user id to logon to OSS. 2. Change directory to /scripts. 3. Set the environment variables . ./setenv.sh 4. Stop the Director Generic process using the following command: stop.
RTID Deployment Manage Server1 Manage Server 2 D r 7011 8011 8021 Parallel TCP/IP Formatted 8011 7011 7011 Server1 Formatted 8021 Server2 Figure 1. Shared and Management URLs Create the Director WebLogic Instances Use the WebLogic configuration wizard to create your WebLogic servers, specifying the same Listen Address and Port for each. This becomes the shared URL.
RTID Deployment USERID=119,10 For server2, CPUSTRING=FIRSTOF(2,3) GPNAME=RTIDGPServer2 GPPNAME=\$RTG2 USERID=119,10 2. Review the following entries in the startgpServer.sh script, and make any necessary changes. RTID_APP_HOME SERVER_NAME 3. Add the following define statements to the top of the startgpServer.sh script for each Director installation on the NonStop server. These defines specify the Parallel TCP/IP entries that enable the load balancing. 4.
RTID Deployment Start / Stop Load-Balanced Directors Use the instructions for starting the Director as a generic process to start the individual Director instances. As you can see from the STDOUT, each Director instance is listening on two ports (the shared port and the management port). Use the stop.sh script to stop the individual Director instances. Appendix: Director Properties File Configuration properties used by Director components are specified in a Java properties file.
RTID Deployment If document sequence checking is enabled and an out-of-sequence document arrives, the Director returns a DocumentOutOfSequence exception. Behavior is the same for Delsert and Upsert documents. The default value is “true”. MaxDocumentHandlers Maximum number of document handlers, and hence of SQL connections, allowed in the configuration. When the number of document handlers exceeds the threshold, the Director removes the least recently used document handler.
RTID Deployment • CLOSED_DEFER. In this mode, the Director saves each incoming document in a file for later processing. Files are saved in the OSS error folder. The Director returns a response of 200, indicating success. No entry is written to the server log. This option is used to capture documents while troubleshooting mapping errors, making it unnecessary for the source system to resend the documents. • CLOSED_NOERROR.
RTID Deployment This property does not apply when the Director is running in a NonStop WLS environment. It applies only when you run the Director in standalone mode, for instance when you run the Dispatch test driver. In such a case, if you don’t specify a value for this property, errors are logged to the standard output file.
RTID Deployment JdbcDriver Class name of the JDBC driver used to connect to the NonStop SQL/MX database. In the properties file that you use on the NonStop Server, the value must be the com.tandem.sqlmx.SQLMXDriver. In the properties file that you use on the workstation, the value must be either • • sun.jdbc.odbc.JdbcOdbcDriver, for the JDBC-ODBC bridge, or com.tandem.t4jdbc.SQLMXDriver for the NonStop SQL/MX Type 4 driver The default value is JdbcDriver=com.tandem.sqlmx.
RTID Deployment jdbc:t4sqlmx://16.107.198.57:18650:/ User User name used to connect to the SQL/MX database. This property applies only to the PC. You don’t need it on the NonStop Server. Examples: rtscuser zliuser Password Password used to connect to the SQL/MX database. This property applies only to the PC. You don’t need it on the NonStop Server. Examples: rtscuser zliuserpw catalog The SQL/MX catalog name. This value must be in uppercase.
RTID Deployment AuditPartitionCount Number of partitions used by auditing tables, if auditing options are enabled, as described in the document entitled RTID Security and Auditing. A value of 0 means the tables are not partitioned, and is the default. For information about how to specify partitioning for other tables, see the document entitled RTID Metadata Language. AuditPartitionStartingId Starting partition ID generated for auditing tables. The default value is 0.
RTID Deployment JMS connection factory used to connect to the JMS queue. The default value is ConnectionFactory=weblogic.jms.LoaderJMSConnectionFactory. Destination JNDI name of the JMS queue to which the drivers connect. The default value is Destination=weblogic.jms.LoaderQueue. Url URL of WebLogic server where the Director is running. The value should be of the format: IPAddress:port like 19.109.98.102:7001. Do not specify a Parallel TCP/IP shared URL. Do not include the t3:// prefix.
RTID Deployment AuditPartitionCount = 4 AuditPartitionColumnName = PartitionId AuditPartitionStartingId = 0 AuditPartitionRange = 1000000 Hewlett-Packard Company 28 529618 - 002
RTID Metadata Language Real Time Information Director (RTID) Metadata Language Table of Contents Introduction to Director Metadata.................................................................................... 3 Overview of this Document............................................................................................. 4 Role of Metadata............................................................................................................. 4 Reasons to Customize Metadata ...............
RTID Metadata Language Record Segments....................................................................................................... 35 RecordSegment ......................................................................................................... 37 Segment .................................................................................................................... 38 Inheriting Columns..................................................................................................
RTID Metadata Language Introduction to Director Metadata To create a real-time solution using the Real Time Information Director (called Director for short in the rest of this document), you don’t necessarily have to write any custom code. What you do have to do is create metadata describing the business documents your solution requires.
RTID Metadata Language Overview of this Document This document covers the following topics. Words in boldface type are terms that will be defined in context in this document.
RTID Metadata Language • To specify audit options for specific types of documents Reasons to Customize Metadata The starting data model and metadata for a vertical industry provide a head start toward developing a real-time solution for a customer, but every company will want to enhance the data model and metadata. Different companies, and even regional subsidiaries of companies, use the SAP data model differently.
RTID Metadata Language Director Metadata Is Descriptive Metadata is data about data. The first important characteristic of the Director metadata is that it is descriptive: it describes the data without describing how to use it. This characteristic is in contrast with a procedural language such as XSLT, which specifies an operation or series of operations to perform on the data. (XSLT is a high-level scripting language, implemented in XML, whose purpose is to convert XML documents to other forms.
RTID Metadata Language Director Metadata Is Implemented in Java Director metadata is implemented in Java, a language that supports polymorphism. To specify the attributes of a specific document type, you pass those attributes as parameters to Java constructors. To invoke a constructor, you use the Java new statement and provide the parameters applicable to the object.
RTID Metadata Language if a query response corresponds exactly to an insert or update document, the same metadata can serve for the insert/update and the query response.) In addition to the metadata you explicitly define, the Director creates certain metadata automatically: • For each insert or update document definition that you create, the Director creates a corresponding query document definition, suffixed with the characters “_R”.
RTID Metadata Language containing one or more fields, or elements), the metadata language provides for many levels of nesting. The metadata language also supports “flattening” incoming data to store fields from multiple segments in the same database record. Content Metadata and Native Document Definitions Content metadata is much less detailed than format metadata and is required for all documents, so let’s explore it first, in the context of a Native Document definition.
RTID Metadata Language public NativeOrders() throws MetadataInconsistency, SQLException super(documentRecord, "NATIVEORDERS"); } { } This native document definition contains most of the data captured from an SAP sales order document. The class is called NativeOrders and extends the class NativeDocumentDefinition. package com.hp.rtsc.documents; import com.hp.rtsc.metadata.*; import java.sql.
RTID Metadata Language package com.hp.rtsc.documents; import com.hp.rtsc.metadata.*; import java.sql.
RTID Metadata Language The Director automatically constructs the prepared SQL statements required to select, insert, and delete these records. package com.hp.rtsc.documents; import com.hp.rtsc.metadata.*; import java.sql.
RTID Metadata Language DocumentRecord The DocumentRecord object identifies the parent record of the document. This is the object you’ll pass to the superclass constructor. The DocumentRecord has four possible parameters: • • • • The name of the primary table of the document (tableName). You must specify either this parameter or an enriched record, as described shortly. An array of related records. Use RelatedRecords[] to specify multiple records or RelatedRecords to specify just one related record.
RTID Metadata Language The following parameters are specified in various constructors of RequestRecord: • String[] keys. These elements of the request record serve as the selection keys for the query. • An array of related records. Use RelatedRecords[] to specify multiple records or RelatedRecords to specify just one record. An enrichment. Enrichments are ways of enhancing inbound or outbound data. Options are discussed under the heading “Enriching the Data,” later in this document.
RTID Metadata Language SplitRecord The SplitRecord class identifies a single record in another table that is related to the parent record. In the XML Document, fields of this record are at the same level as the parent record, rather than under a nested tag. The matter of whether to split a record across multiple tables is a database design issue. SplitRecord is a subclass of RelatedRecords, so you can put a SplitRecord in the array of RelatedRecords.
RTID Metadata Language DependentLinks You use DependentLinks, rather than Links, when a record is linked to the parent by a different criterion than was used to select the parent records. For example, as long as all the tables in an order document are related by order number, there is no need for DependentLinks. The need arises in a case like the record for a product; such a record will not have an order number, because many orders can refer to the same product.
RTID Metadata Language Title TitleAction PK ISBN I1 I2 I2 Title LastName FirstName CheckedOutCopies CumulativeCheckouts Category PK,FK1 PK ISBN RequestDate FK2,I1 FK3,I2 BranchIDFrom BranchIDTo ActionCode Branch Item PK ItemID FK1 FK3,I1 FK2 ISBN BranchID CheckedOutID BranchID I1 BranchName FictionCheckedOut NonFictionCheckedOut Checkout CheckoutItem PK,FK2 PK,FK1 PK CheckoutID ItemID CheckOutDate DueDate ReturnedDate LibraryCard PK CheckoutID PK CardID FK1,I1 FK2 CardID BranchID
RTID Metadata Language HistoryRecord The HistoryRecord class identifies a single record to be inserted into another table every time that the parent record is inserted or replaced. The purpose of a HistoryRecord is to track changes in the parent record; later in this document, you’ll see an example of HistoryRecord used to track schedule changes for line items in an order. (See “Metadata Excerpts: An IDoc Definition.
RTID Metadata Language relevant error message and a query document that could be used to retrieve the records just created) is included as an audit detail. For more information, see the document called RTID Security and Auditing. UPSERT. Causes the Director to update existing records related to this document. (If no such records exist, the Director inserts the new document, unless the EXISTING_DOCUMENT_ONLY option is also specified.
RTID Metadata Language SQL Keys The Director queries the SQL schema to discover the following information about each table that a document uses: • • • • • • The key fields for each table The name of each field The datatype of each field The scale and precision of each field The default value of each field Whether a field allows nulls The leverage gained from the SQL schema greatly reduces the amount of metadata that you need to specify for individual documents.
RTID Metadata Language Specifying a Unique IDoc To specify an IDoc, you create a class that has a unique name, such as the name of the IDoc, and that extends the BaseDocumentDefinition class. (Remember that a native document extends NativeDocumentDefinition.) You construct a DocumentRecord, just as for a native document, to specify the document content metadata, but you also construct an IDocSegment, which describes how to map specific IDoc elements to the data store.
RTID Metadata Language The following excerpt comes from the bottom of the same document definition: private static DocumentRecord documentRecord = new DocumentRecord("SALES_ORDER", new RelatedRecords[] { new RelatedRecords("ROLE_SPEC_SLS_ORD_EVENT"), new RelatedRecords("SALES_ORDER_DETAIL", new RelatedRecords[] { new SplitRecord("SLS_ORD_SPEC_BUSINESS"), new SplitRecord("SALES_ORDER_FLOW", new Links(new String[] { "SALES_ORDER_NO","PRECEDING_DOCUMENT_NO", "SALES_ORDER_ITEM_NO","PRECEDING_DOCUMENT_ITEM_NO"})
RTID Metadata Language private static DocumentRecord documentRecord = new DocumentRecord("SALES_ORDER", new RelatedRecords[] { new RelatedRecords("ROLE_SPEC_SLS_ORD_EVENT"), new RelatedRecords("SALES_ORDER_DETAIL", new RelatedRecords[] { new SplitRecord("SLS_ORD_SPEC_BUSINESS"), new SplitRecord("SALES_ORDER_FLOW", new Links(new String[] { "SALES_ORDER_NO","PRECEDING_DOCUMENT_NO", "SALES_ORDER_ITEM_NO","PRECEDING_DOCUMENT_ITEM_NO"})), new RelatedRecords("RL_SPEC_SO_DT_EV_OVRRD"), new RelatedRecords("SLS_ORD
RTID Metadata Language XML IDoc Format The next several pages show excerpts from an actual ORDERS05 IDoc. Looking at the structure of the IDoc will make it easier to understand how to specify different kinds of elements in the document definition.
RTID Metadata Language The header contains general information about the document. This example omits a few fields.
RTID Metadata Language J328 0300060645 01 STANDARD W0J008399 00 New orders 006 ZD 007 ZZ Following the E1EDK01 segment are several E1EDK14 segments, each containing only two elements.
RTID Metadata Language private static final IDocSegment idocSegment = new IDocSegment("ORDERS05", "SALES_ORDER", new TagDefinition[] { IDOC.
RTID Metadata Language private static final IDocSegment idocSegment = new IDocSegment("ORDERS05", "SALES_ORDER", new TagDefinition[] { IDOC.
RTID Metadata Language Note: IDocs have a single tag, named IDOC, immediately below the top tag. You don’t specify this tag; the Director provides for it automatically. Here are the possible constructors for the IDocSegment class: public IDocSegment(String tagName, String tableName, Segment[] tagDefinitions) public IDocSegment(String name, String tableName, SegmentFormatter segmentFormatter, Segment[] tagDefinitions) The IDoc Header The first tag under the new IDocSegment should be IDOC.HEADER.
RTID Metadata Language For ORDERS05, the initial segment is E1EDK01. The initial segment contains information for the top level record. Parameters for the Segment class are given under “Segment,” later in this document. FieldElement The FieldElement class maps an element to a column of some table. FieldElement has the following parameters: • • • String tagName. This parameter identifies the XML tag and is required. String columnName.
RTID Metadata Language public FieldsElement(String tagName, String[] columnNames) public FieldsElement(String tagName, String columnNames) Examples: new FieldsElement("BELNR", new String[] { "SALES_ORDER_NO", "PRECEDING_DOCUMENT_NO" }), new FieldsElement( new FieldElement[] { new FieldElement("ZEILE", "REFERENCE_DOCUMENT_ITEM_NO"), new FieldElement("ZEILE", "PRECEDING_DOCUMENT_ITEM_NO") }) Here is an example of how to pass multiple column names as a single string, instead of an array.
RTID Metadata Language The SegmentQualifier [] parameter is an array of SegmentQualifier.
RTID Metadata Language new SegmentQualifier("022", "DATUM","CUSTOMER_PURCHASE_ORDER_DT"), new SegmentQualifier("025", new FieldElement[] { new FieldElement("DATUM", "RECORD_CREATE_DT"), new FieldElement("UZEIT", "ORDER_ENTRY_TM") }), new SegmentQualifier("023","DATUM","PRICE_AND_EXCHANGE_RATE_DT") }), The same segment qualifier can define multiple fields. In the highlighted text below, the second parameter passed to SegmentQualifier is an array of tag definitions.
RTID Metadata Language • SegmentQualifier[] segmentQualifiers. This required parameter specifies how to map data elements in a segment according to the qualifier value, as described next.
RTID Metadata Language public SegmentQualifier(String qualifierValue, FieldElement tag) Examples: In the following example, an array of tag definitions (in this case FieldElement) specifies how to map two data elements if the qualifier has the value “025”: new SegmentQualifier("025", new FieldElement[] { new FieldElement("DATUM", "RECORD_CREATE_DT"), new FieldElement("UZEIT", "ORDER_ENTRY_TM") }) In the following example, if the qualifier has the value “ZZV,” the data elements BELNR, ZEILE, and DATUM are
RTID Metadata Language new new new new new FieldElement("LDDAT", FieldElement("MBDAT", FieldElement("WADAT", FieldElement("EDATU", FieldElement("WMENG", "LOADING_DT"), "MATERIAL_AVAILABILITY_DT"), "CURRENT_SCHEDULED_SHIP_DT"), "CUSTOMER_REQUESTED_RECEIPT_DT"), "REQUESTED_QT") }) }), new RecordSegment("E1EDPA1","RL_SPEC_SO_DT_EV_OVRRD",ORDER_EVENT_TAGS), The highlighted text below shows two record segments under E1EDP01. (Other elements and qualified segments under E1EDP01 are omitted from the excerpt.
RTID Metadata Language new FieldsElement("POSEX", new String[] {"SALES_ORDER_ITEM_NO","PRECEDING_DOCUMENT_ITEM_NO"}, IDOC.ITEM_NO), new FieldElement("ACTION", "SALES_ORDER_ITEM_ACTION_CD"), … new RecordSegment("E1EDP20","SLS_ORD_SCHEDULE_DT",new TagDefinition[]{ new FieldElement("WMENG", "CONFIRMED_QT"), new FieldElement("EDATU", "SCHEDULE_LINE_DT"), new FieldElement("EZEIT", "DELIVERY_TM"), new Segment("ZEEDP20", new FieldElement[] { new FieldElement("ETENR", "SCHEDULE_LINE_NO", IDOC.
RTID Metadata Language Segment Segment defines a segment that does NOT introduce a new record into the data store. Use Segment only when the segment does not occur more than once within any instance of the parent segment. The usual way to handle custom fields in an RTSC solution is to wrap them in a segment with a name beginning with “Z”. Segment has the following parameters: • • • String tagName. This parameter is the XML tag that identifies the segment and is required. TagDefinitions[] tagDefinitions.
RTID Metadata Language Segmented Tables A RecordSegment can actually refer to more than one table: • • • Tables can be horizontally segmented, so that different columns of the same entity are stored in different tables. The same entities may be represented in different tables, having different primary keys for mapping or lookup purposes. Each such segmented record is identified in the content metadata as a SplitRecord.
RTID Metadata Language QualifiedParentElement The QualifiedParentElement class maps an element in a specific instance of a segment to a column in the parent table. This option addresses the ambiguity that can arise in the case of ParentElement. QualifiedParentElement has the following parameters: • • • • String tagName. This parameter is the XML tag of the data element to be mapped. String columnName. This parameter identifies the database column, in the parent segment, in which the data will be stored.
RTID Metadata Language 978200014001 EN FixedElement FixedElement inserts a constant value into an outbound document (in contrast with ImpliedField, which inserts a constant value into the data store). FixedElement has the following parameters: • • String tagName. This parameter is the XML tag of the data element to be included in the outbound document. String value. This parameter is the value to be generated for the data element.
RTID Metadata Language Here are the constructors for the PrependFieldElement and AppendFieldElement classes: public PrependFieldElement(String tagName, String columnName, char separator) public PrependFieldElement(String tagName, String columnName, int numberOfChars) public AppendFieldElement(String tagName, String columnName, char separator) public AppendFieldElement(String tagName, String columnName, int numberOfChars) Example: The following example concatenates the values of the elements CREDAT and CRE
RTID Metadata Language PredicatedFieldElement The PredicatedFieldElement class maps a data element into a column if a specified column has a specified value. PredicatedFieldElement has the following required parameters: • • • • String tagName. This parameter is the XML tag of the data element to be mapped. String columnName. This parameter is the database column to which to map the data element if the specified condition is met. String qualifierName.
RTID Metadata Language Formatter Classes Formatter classes convert data from the IDoc document format (or a custom format) to the format required by the data store, or vice versa. For example, certain IDoc elements are stored in the data store using numeric types but are expressed in the IDoc with a fixed number of positions, using leading zeros. A formatter’s unformat operation converts the data, as it appears in the inbound message, to the format required by the data store.
RTID Metadata Language new new new new new new new new new new new FieldElement("GLTRI", FieldElement("GLTRP", FieldElement("GLTRS", FieldElement("GLUZP", FieldElement("GLUZS", FieldElement("GMEIN", FieldElement("GSTRI", FieldElement("GSTRP", FieldElement("GSTRS", FieldElement("GSUZP", FieldElement("GSUZS", "ACTUAL_FINISH_DT", IDOC.DASHED_DATE), "BASIC_FINISH_DT", IDOC.DASHED_DATE), "SCHEDULED_FINISH_DT", IDOC.DASHED_DATE), "BASIC_FINISH_TM", IDOC.TIME), "SCHEDULED_FINISH_TM", IDOC.
RTID Metadata Language • SAP dates are usually expressed as eight consecutive digits. A formatter inserts hyphens between the year, month, and day for an SQL date field, e.g., 20030630 is converted to 2003-06-30. This type of conversion occurs automatically. You needn’t specify it in your metadata. Segment Formatter Classes Whereas a formatter class applies special processing to a data element, a segment formatter class applies special processing to a whole record.
RTID Metadata Language Variant Documents Typically, each SAP instance handles some data elements differently, or the same instance requires different tag mappings in different circumstances. Variant document definitions provide a way to deal with the differences, by creating a separate document metadata class for each SAP instance or circumstance. You can use a file-comparison tool to see the differences between variant document definitions.
RTID Metadata Language • • The second parameter is a logical name used to select an outbound document for this variant. The third parameter is the reference to the format metadata. package com.hp.rtsc.documents; import com.hp.rtsc.metadata.*; import com.hp.rtsc.metadata.format.*; import java.sql.
RTID Metadata Language To specify an aliased document, you create a class that has a unique name and extends AliasedDocumentDefinition. You implement a constructor for your aliased document. This constructor calls the superclass with the following parameters: • • • IDocSegment idocSegment. This parameter contains the format metadata for the document. String baseDocumentName. This parameter gives the name of the document on which aliased document is based.
RTID Metadata Language Because you cannot reload a Java class once it has been loaded into the JVM, a substitute document definition must have a new name, not the same name as the document definition it replaces. A substitute document cannot replace a variant or aliased document definition. You replace a variant or an aliased document definition by deploying a new variant or aliased document definition.
RTID Metadata Language Common Characteristics of Variant, Aliased and Substitute Documents Although variant, aliased, and substitute document definitions serve different purposes, they have some important characteristics in common. One is that the same SQL statements generated for the base document are used for the related variant, aliased, or substitute documents.
RTID Metadata Language new LeftJoin("CHECKOUTITEM", "CI", "CO.CHECKOUTID=CI.CHECKOUTID"), new LeftJoin("ITEM", "I", "CI.ITEMID=I.ITEMID"), new LeftJoin("TITLE", "T", "I.ISBN=T.ISBN"), new OrderBy("CO.CARDID, CO.CHECKOUTDATE, T.LASTNAME, T.FIRSTNAME, T.TITLE"), new ColumnList("CO.CARDID CO.BRANCHID CO.CHECKOUTDATE CI.ITEMID CI.CHECKOUTID CI.CHECKOUTDATE CI.DUEDATE CI.RETURNEDDATE I.ISBN I.BRANCHID I.CHECKEDOUTID T.TITLE T.LASTNAME T.FIRSTNAME T.CHECKEDOUTCOPIES T.CUMULATIVECHECKOUTS T.
RTID Metadata Language Mapping The Mapping enrichment looks up fields in a table that correlates external identifying information, such as a person’s name, with an internal identifier assigned by the Director. The enrichment is reversible, in that it performs a lookup in one direction for an inbound document and in the opposite direction for an outbound document. For example, a Mapping enrichment is used to obtain the internal ID of a patient from the external ID of the patient in an update document.
RTID Metadata Language 2. Construct a QualifiedMapper, by calling the qualify method of the UnqualifiedMapper object. The qualify method requires you to specify a value for the typecode, so a search of the mapping table is restricted to the appropriate rows. 3. Construct an actual mapping, within a document definition, by calling the map method of the QualifiedMapper object. This call looks exactly the same as a call to the map method of a Mapper object.
RTID Metadata Language • • • master data, for example a data table representing a customer, a product, or a factory. Parent-child affinity. A new record is assigned the same partition ID as the parent record in the document. This strategy helps to cluster the data, at least when the data is first inserted. (Note that the relationship between this child record and the parent record might be temporary and non-exclusive.
RTID Metadata Language Partitioning with Artificial Keys In the case of an artificial key, the partition ID is assigned according to any of the following strategies (defined earlier): • • • Round-robin partitioning Parent-child affinity System key affinity Given the partition ID, the Director assigns an artificial key (or system key). The Director divides the key space by partition, so the partition ID can be deduced from the system key. To provide for this kind of partitioning, you 1.
RTID Metadata Language 3. Create a PartitionParameters object with parameters consistent with those you specified in the KEYS table: • • • • Assign to the keySpaceName parameter the value you put in the NAME column for the namespace. Assign to the startingPartition parameter the smallest value you put in the PARTITIONID column for any row in the namespace. Assign to the numberPartitions parameter the value you put in the MAX_PARTIONID column for the namespace.
RTID Metadata Language public InstanceId(String fieldName) LeftJoin The LeftJoin enrichment joins one table with another. It causes a “LEFT JOIN ON ” clause to be added to the SQL statements generated for this record. Two mandatory String parameters specify the and the . An optional parameter specifies the alias: an “AS” qualifier appended to the join clause.
RTID Metadata Language The ColumnList constructor looks like this: public ColumnList (String[] columnNames) // Array of columnNames public ColumnList (String columnNames) OrderBy The OrderBy enrichment specifies the order in which records must be returned in an outbound (query response) document. By default, rows are returned in the order returned by SQL/MX when no “Order By” clause is specified.
RTID Metadata Language public VirtualColumn (String columnNames) Example: The following metadata comes from the branch library example. In this case, the input document supplies the RETURNBRANCHID and RETURNEDDATE. CARDID is obtained from another record. The inbound enrichment ReturnItemEnrichment uses these columns for processing but does not store them in the Item record. Library.itemRecord.
RTID Metadata Language Generating a Unique ID One common motivation for a real-time solution using the Director is to gain the benefits of a comprehensive, customer-centric data store. For example, the EHR demo illustrates the benefits of capturing and correlating information related to a patient (named Michael in the demo). When a source system or client refers to a person, that system or client uses an external ID, which might be a number, a name, or some other kind of code.
RTID Metadata Language public static EnrichedRecord encounterRecord = new EnrichedRecord("ENCOUNTER", new PartitionedSystemKey("ENCOUNTERIDENTIFIER", "ENCOUNTERPID", encounterIdDispenser, new SystemKeyAffinity("UNIQUEIDENTIFIER", personIdDispenser))); Here is an excerpt from the document definition PatientEvents.java, defining the other records related to an Encounter.
RTID Metadata Language the next record the value 2, and so on. Given the automatic generation of IDs, an input file, such as the ChinnAddEvent2.xml sample file, need not specify values for TESTID in all TESTS records. Partitioning Tables This section contains information and examples related to partitioning data in the Director. If you haven’t read the overview in “Partitioning,” in this document, do so before proceeding. Electronic Health Record Example The class EHR.
RTID Metadata Language The block size is the number of keys that will be allocated at once. Only one key is needed for each document, of course, but a PartitionedKeyDispenser class fetches a block of keys at a time; any other instance of the Director would have to start with the next set of that many keys.
RTID Metadata Language Now we’ll see how the metadata prescribes partitioning for another table, called OTHERSTATUS, which is where the unique identifier for a patient is explicitly requested.
RTID Metadata Language public abstract class Library The next few lines of metadata define values for partitioning parameters. In this case, the data will be divided across four partitions, starting with partition 0. You need not use the same number of partitions for all the tables in your solution, but the Library application does so for simplicity. The partition range actually specifies the number of keys available for assignment to the partition.
RTID Metadata Language new Mapper(branchRecord, "BRANCHID", "BRANCHNAME"); The next statement invokes the new branchMapper to incorporate the external ID into the enriched record: public static final EnrichedRecord externalBranchRecord = branchRecord.enrich(branchMapper.map()); Now we’ll see how the metadata prescribes partitioning for a table called ITEM, which has a natural key called ITEMID. The metadata creates an enriched record called itemRecord.
RTID Metadata Language new EnrichedRecord("CHECKOUT", new PartitionedSystemKey("CHECKOUTID", "CHECKOUTPID", checkoutDispenser, new NaturalKeyAffinity("CARDID", partitionParameters))); The Checkout record is further enriched with an Alias, for use in joins. public static EnrichedRecord aliasedCheckoutRecord = checkoutRecord.enrich(new Alias("CO")); Now another record, CheckoutItem, is enriched with the artificial key that represents the Checkout (CHECKOUTID).
RTID Metadata Language Using Dependent Links to Select Child Records Although the section of a query is of crucial importance for selecting records, related (child) records must be selected by other means. For example, consider the following query document, called ChinnQueryMichaelEventsHistory.xml, in the folder C:\rtid\chin\inputs:
RTID Metadata Language Using a Lookup Table to Select Data for a Person The previous example showed how to select data for a single person based on an EXTERNALIDENTIFIER value provided in the input document. But what if the external identifier is not recognized? Is this a new patient who needs to be registered, or was the same person registered earlier under another identifier. In such a case, we can resolve the UNIQUEIDENTIFIER column by using a lookup table. For example, consider the PatientByDOB.
RTID Metadata Language }) In this example, the Alias enrichment redefines the name of PLOOKUP table as PL, to be used in later statements. The LeftJoin enrichment defines how to select records from other tables by matching the UNIQUEIDENTIFIER column in the lookup table. Finally, the output is restricted to the columns defined in the ColumnList metadata, plus the key columns of the lookup table. Mapping External to Internal IDs The Mapping enrichment defines a reversible table lookup process.
RTID Metadata Language Once you have constructed the Mapper object, you can use it to produce Mapping enrichments in individual Document Definitions. The effect of such an enrichment, remember, is to map from external identifiers to internal ones on input, and to perform the reverse mapping on output, so the application deals only with the external values, and the database stores the internal ones. You create a Mapping enrichment by calling the map method of the Mapper class.
RTID Metadata Language These enrichments come into play when a patient’s chart needs to be updated as the result of a medical encounter. Here is an excerpt from the UpdateEncounter document definition. This document definition refers to several of the enriched records defined in EHR.java: public class UpdateEncounter extends NativeDocumentDefinition { private static DocumentRecord documentRecord = new DocumentRecord(EHR.lookupEncounterRecord, new RelatedRecords(EHR.
RTID Metadata Language The first step is to define the mapping table by constructing, statically, the UnqualifiedMapper object. The constructor of UnqualifiedMapper takes the same parameters that the constructor of Mapper takes, plus one additional parameter: the name of the column that contains the category name. The second step is to identify a mapping category by calling the qualify method of the UnqualifiedMapper. The qualify method takes one parameter, the name of the category.
RTID Metadata Language document has to specify the category ENCOUNTERTYPE; that category was specified when the encounterTypeMapper was constructed. Here are some examples of the use of QualifiedMapping enrichments from the EHR example. Notice that the invocations of the map method are indistinguishable from the use of simple Mapping enrichments. Only the names of the Mappers hint that they are qualified.
RTID Metadata Language Enriching a Delsert Transaction In the case of a Delsert, where all previous records related to a document are deleted and replaced with data from an incoming document, you can use enrichments to validate or supply data missing from the input document.
RTID Metadata Language The Enrichment Class The Enrichment class, Enrichment.java, includes methods for initialization and for implementing inbound, outbound, top-down, and bottom-up enrichments. For information about the syntax for invoking these methods, see the Javadoc for the class Enrichment.java. The first step in creating an Enrichment is to create a constructor that defines the parameters. These parameters can be strings, or immutable or singleton objects.
RTID Metadata Language Compound Enrichments The CompoundEnrichment class provides a way of applying multiple enrichments to the same document or record. You pass to this class a bracketed list of enrichments, to be performed in the order specified. Wherever the metadata syntax allows you to specify an Enrichment, you can specify a CompoundEnrichment instead. In general, enriched records are more convenient to use than compound enrichments.
RTID Metadata Language Index The following index is meant primarily to help you look up syntax terms you see in existing metadata.
RTID Metadata Language Library example objects illustrated, 16 Links, 15 dependent, 69 Looking up fields from a related table, 53 Lookup table, 70 Management interface, 51 Map method of Mapper class, 72 Mapper, 71 Mapping internal and external IDs, 53, 71 qualified by type of lookup value, 73 Mapping (enrichment), 72 Metadata definition and characteristics, 5 Native XML document, 9 format, 20 specifying, 12 NativeDocumentDefinition class, 12 Natural keys, partitioning with, 55 partitioning, 54 Order of reco
RTID Metadata Language passing multiple values in, 31 STRIP_BLANKS formatter, 45 Subscription enriching, 76 triggers predefined query, 8 Substitute document, 49 SQL statements used for, 51 Superclass constructor options, 18 System key used in partitioning, 56 SystemKey, 52, 61 Table specifying in content metadata, 10 Tables segmented, 39 Tags need not be enumerated if not mapped to data store, 51 order of, 51 Testing metadata, 3 TIME formatter, 44 Top-down enrichment, 76 Unformat operation, 44 Unique ID, 61
Real Time Information Director (RTID) Extensibility Table of Contents Overview ______________________________________________________________ 3 Guidelines for Analyzing Mapping Requirements _____________________________ 4 Defining a New Native Document __________________________________________ 5 Delsert or Upsert? ____________________________________________________ 6 Creating Native Document Metadata ____________________________________ 6 Defining a New IDoc _______________________________________________
RTID Extensibility Testing the Metadata____________________________________________________ 21 Zli.
RTID Extensibility Overview The Real Time Information Director (also known as RTID or Director) uses two kinds of metadata: content metadata and format metadata. Content metadata defines, for each document type, the corresponding database tables and records and how the records are related. Format metadata defines, for each IDoc, the correspondence between XML tags in the IDoc and records or columns in the data store. For documents in native format (not IDocs), you use only content metadata.
RTID Extensibility Guidelines for Analyzing Mapping Requirements Think about a mapping as the answer to this question: From what element of the incoming XML document do I insert data into a specific database column? The same mapping is used in the reverse process, creating an XML document from the database. The Director explicitly supports two kinds of documents. A native format document is one in which XML segments and elements correspond exactly to records and columns in the database.
RTID Extensibility A sales order schedule detail record is inserted into the SLS_ORD_SCHEDULE_DT table for each E1EDP20 tag encountered in the input data. For each sales order schedule detail record the CURRENT_SCHEDULED_SHIP_DT column is populated from the WADAT tag. Note that the WADAT tag is nested under the custom segment ZEEDP20.
RTID Extensibility Delsert or Upsert? For each inbound native document type, you must establish (and specify in the metadata) whether to load the data using a delsert or an upsert operation. If an input document will always contain complete information (i.e., for different versions of the same document, each new document is a superset of the older versions of the document), then it is appropriate to delsert the document.
RTID Extensibility By default, an incoming document is loaded in a Delsert operation; if an Upsert is appropriate to the document you’re defining, you must specify that option in the document definition. For example, the following line from a document definition specifies that a document of type PERSONALDATAUPDATE requires an Upsert: super(documentRecord,“PERSONALDATAUPDATE”,“UPSERT”) No two native documents can have the same top-level tag. 2.
RTID Extensibility • • • • Analyze the information to be captured. Update your data model to hold this additional information. Determine if the inbound documents are delserted or upserted. This step is the same as for a native document. Understand the mappings needed to store the information in the data store. Basically, you need identify which elements in the document will be mapped to which columns in the database.
RTID Extensibility 5. If the output from Dispatch or Reselect shows that inbound data and outbound data sometimes vary in their format, you might need to use formatters to do data conversions. See “Converting Data Representations,” later in this module, for a discussion of this topic. 6. Validate the IDoc mapping against the SAP schema for the IDoc, as described below. Validating an IDoc mapping A client can use an XML Schema file to verify that a given XML document is valid.
RTID Extensibility Adding Alternate Mappings for an IDoc Perhaps you already have mappings for an IDoc type, for example the ORDERS05 IDoc. These mappings include content and format metadata and are based on an existing data model. In some cases you might need to create an additional mapping for the same IDoc type: in other words, the Director needs to map the same IDoc in several different ways.
RTID Extensibility 2. Let’s assume that the top tag is different from the base definition’s top tag. Then edit Orders05EXT.java, and change the class definition to extend AliasedDocumentDefinition instead of BaseDocumentDefinition. 3. Change the top-level tag to ORDERS05EXT as shown below: private static final IDocSegment idocSegment = new IDocSegment("ORDERS05EXT", "SALES_ORDER", new Segment[] { IDOC.HEADER, . . . } 4. Remove the definition of the content metadata (DocumentRecord). 5.
RTID Extensibility Converting Data Representations Sometimes a data element is represented in one way in an IDoc and in another way in the data store. Examples are numeric values with leading zeros, and date or date-time fields that are formatted differently from the corresponding SQL date or SQL date-time fields. In order for the Director to generate outbound documents from the database with the same format as the inbound documents, you must specify Formatter metadata objects.
RTID Extensibility public SubStringFormatter(int size) { this.separator = null; this.size = size; } public String format(String value) { return value; } public String unformat(String value) { if(value == null) return value; String result = ""; if(separator == null) { if(size > 0 && size < value.length()) return value.substring(0,size); else return value; } int index = value.indexOf(separator); if(index == -1) return value; return value.substring(0,index); } } 3.
RTID Extensibility Updating Metadata Mappings for an IDoc The metadata you initially define for an IDoc might not map all the IDoc elements to database columns. The decision to map some but not others would be based on business requirements. But business requirements change, and so you might some day need to capture additional elements; this might mean adding columns or even new tables to the data model.
RTID Extensibility 2. Compile the Java source file, as described in “Compiling Metadata.” 3. Test the metadata, and deploy the updated Java class files to the RTID runtime environment. See “Deploying Metadata” for deployment steps. To test inbound IDocs, use the Delsert or Disassemble standalone program. To test outbound IDocs, use the Select standalone program. To test both inbound and outbound IDocs, use the Reselect program. See “Testing the Metadata” for information about syntax.
RTID Extensibility To add internal to external mappings, first declare the Mapper object that defines the mapping table and the mapping columns. Then add an enrichment to the EnrichedRecord object used in RelatedRecords (or a subclass of RelatedRecords) that calls the Mapper.map() method. 3. Compile the Java source file, as described in “Compiling Metadata.” Modify any test files, if needed.
RTID Extensibility 3. Use the –schema option of the Management Client to reload the schema for any tables whose schemas had to change. Use spaces to separate the table names. Changes like adding or dropping a partition or an index don’t change the table definition and therefore don’t require a schema reload. The following example reloads the schema for two tables, SALES_ORDER and SERIAL_NUMBER.
RTID Extensibility Deploying Metadata After compiling the metadata, you must deploy it to the NonStop server. Follow these steps to ensure that the metadata will be available whenever RTID is started. 1. Use FTP to copy the class file in binary mode to the /examples/rtsc/com/hp/rtsc/documents folder in your RTID OSS install directory on the NonStop server. 2. If this is a new metadata definition, create an entry in the DOCCLASS table.
RTID Extensibility Replacing Existing Format Metadata The Director also supports dynamic replacement of existing metadata for Base, Aliased and Variant documents: • Dynamically deploying an Aliased document definition automatically replaces the previous Aliased document definition for the same IDoc. • Dynamically deploying a Variant document definition automatically replaces the previous Variant document definition of the same IDoc.
RTID Extensibility name to match with the file name. 3. Change the superclass from BaseDocumentDefinition to SubstituteDocumentDefinition. For example: public class Orders05v1 extends SubstituteDocumentDefinition {…} Then remove the declaration of the DocumentRecord object, and remove the object documentRecord from the call to the superclass. (Remember that the documentRecord represents content metadata, and that a substitute document, by definition, includes only format metadata.
RTID Extensibility Note: A class that you’ve deployed dynamically, following this procedure, is not automatically added to the DOCCLASS table and will therefore not be available after a server restart. To make the class available on a permanent basis, you should deploy the fixes to the corresponding class in DOCCLASS, to be used when the server is next restarted. Testing the Metadata The Director comes with a set of test drivers that you can use in the process of developing the metadata for your solution.
RTID Extensibility • • Dispatch calls the Dispatcher object directly and is useful for testing the RTID from a workstation using JDBC for database access. In this scenario, the complete RTID runs in the local workstation and only the database calls are remote, allowing the use of workstation tools for debugging of RTID code. The Delsert, Disassemble, and Reselect drivers test various internal RTID functions. These tests can also be run remotely in the same way as the Dispatch test.
RTID Extensibility Logging Related Properties • LogFileName • LogLevel • ErrorHandlerClass • SaveFilePrefix • SaveDirectory Driver Arguments All drivers use the same argument definitions for their operation, but not all argument definitions are supported for all drivers. For instance, QueueSend, which reads in a file and sends it to a JMS Queue doesn’t use the ‘–o outputDirectory’ option, because it produces no output.
RTID Extensibility java –Dzli.propertyfile=zli.properties Dispatch –o verify –c com.hp.rtsc.documents.NativeDelivery DELIVERY –e In contrast, the following command, which specifies the arguments in a different order, exports to the current directory (not to the directory specified by –o) the schema for each document defined in the DOCCLASS table (not the NativeDelivery document type). Then it processes a document called DELIVERY and stores the response in the ‘verify’ directory.
RTID Extensibility -ro -rs -sgt -sgo -sgs -sgn -sft -sfo -sfs -sfn -sdt docname -sdo docname -sds docname -sdn docname -st -su -toption Includes operations history in XML responses to update documents. Includes only summary statistics in XML responses to update documents. This is the default setting. Accumulates global statistics by table. Accumulates global operation histories. Accumulates global summary statistics only. This is the default setting. Disables accumulation of global statistics.
RTID Extensibility 'i' turns on tracing of INSERT statements -t delay -u urlString -y 'm' turns on tracing of SELECT statements that map between internal and external IDs. Sets a time delay between files for –l & -g processing Sets the URL for Http drivers The protocol part of the URL should NOT be included in the urlString. For example 104.199.107.14:7001 is a valid URL but t3://localhost:7001 is not. Also, for the HttpPost driver, the URL must include the servlet URI.
RTID Extensibility Disassemble Driver The Disassemble program parses (disassembles) the input XML document and creates the internal XML representation. Given an input file ‘InFile.xml’, Disassemble generates an output file with the filename InFileRecords.xml The usual command syntax for the Disassemble program is: java -Dzli.
RTID Extensibility Example 1. 2. 3. 4. Open a DOS window Change directory to c:\rtid Run setenv.bat to set the environment. Run the following command to Disassemble an ORDERS05 IDoc: cd rtsc java -Dzli.propertyFile=rtsc.properties Disassemble -i c:\rtid\rtsc\inputs\ -o c:\rtid\rtsc\outputs\ ORDERS05_FusionAM_1.xml This command will create the file orders05_fusionAM_1Records.xml in the c:\rtid\rtsc\outputs directory. You can view this file to verify that your metadata is defined correctly. 5.
RTID Extensibility [-i inputFileDirectory] Specifies the name of a directory containing input XML files. [-c class name] Loads the specified class, which is not in the DOCCLASS table. [filename(s)] Specifies one or more input XML files.
RTID Extensibility This command should insert the XML IDoc in the database. You can verify by querying the database. Reselect Driver The Reselect program inserts a document into database and selects the same document from the database. It is easier to use than Select, because it doesn’t require an input file to list the primary key fields; Reselect gets the primary key fields when it inserts the document in the database and uses these fields to select it back from the database.
RTID Extensibility Before running this program, you must have a directory for the input files and a separate directory for the output files. Let’s assume you have the following directories and files: c:\rtid\rtsc\inputs\orders05_fusionam_1.xml c:\rtid\rtsc\outputs 1. 2. 3. 4. Open a DOS window Change directory to c:\rtid Run setenv.bat to set the environment. Run the following command to Reselect an ORDER05 IDoc from the database: cd rtsc java -Dzli.propertyFile=rtsc.
RTID Extensibility Processes all files in the test/testIn directory putting all the responses into the test/testOut directory java -Dzli.propertyFile=my.properties Dispatch -o test/testOut -i test/In testFile.xml Process the testFile.xml file from the test/In directory and put the response in the test/testOut directory java -Dzli.propertyFile=my.properties Dispatch -o test/testOut –e Produces xml schema definition (.
RTID Extensibility Example: java -Dzli.propertyFile=my.properties QueueReceive -q rtsc.TestQueue -o test/testOut Listens to the JMS Queue named rtsc.TestQueue and puts all messages received on the queue to the test/testOut directory QueueSend Driver This driver sends the specified input file(s) to the specified JMS Queue. This driver generates no output files. For syntax information, see the Javadoc for QueueSend. Example: java -Dzli.propertyFile=my.properties QueueSend –u 11.22.33.44:7001 -q rtsc.
RTID Extensibility To use the Harness driver, specify it on the command line (or in the IDE environment) as follows: java –Dzli.propertyFile=my.properties Harness arguments The first argument is the name of a driver class, such as Dispatch. or HttpPost. The remaining arguments are the parameters for the driver. After the driver starts, it validates metadata and then displays a prompt (the '>' symbol) to let you type a command.
Real Time Information Director (RTID) Management Table of Contents Overview......................................................................................................................... 2 Director Management Interface..................................................................................... 2 Director Management Client.......................................................................................... 4 Setting Statistics Gathering Options.........................................
RTID Management Overview This document describes the management features of the Real Time Information Director (also known as RTID or Director). The Director management features supports the dynamic reconfiguration of the Director at runtime without having to bring down the server.
RTID Management long getTotalDocumentProcessFailures(); long getResponseTime(); long getTimeOuts(); int getDocumentHandlerCount(); int getDocumentHandlerInUseCount(); int getHandlersAllocated(); int getAllocateHandlerMilliseconds(); int getHandlersPrepared(); int getInsertedRows(); int getInsertMilliseconds(); int getInsertStatements(); int getDeletedRows(); int getParseMilliseconds(); int getSelectMilliseconds(); int getDeleteMilliseconds(); int getUpdateMilliseconds(); int getSerializeMilliseconds(); int
RTID Management } Director Management Client The Director provides a built-in command line management client that can be used to exercise the management functions described in this document. The name of the management client class is “MBeanClient”. Because the management interface is based on JMX, it should be possible to use any JMX management client to invoke the Director management functions. Use the following syntax, all on one line, to invoke the management client.
RTID Management -password [password] Password for the user who access to the WebLogic Server. If you omit this parameter, the password is 'weblogic' by default. Example: java MBeanClient -url 127.0.0.1:7001 -server myServer -user rtidAdmin -password mypassword -at Use this option to specify the maximum number of XML start tags permitted in an input document. If the number of start tags in a document exceeds the threshold, the Director rejects the document with an InvalidInputException.
RTID Management Use this option to reload the schema for the specified tables.
is a list of table names, separated by spaces. Example: java MBeanClient -url 127.0.0.1:7001 -server myServer -user rtidAdmin -password mypassword -schema SALES_ORDER_DETAIL SHIPMENT_DETAIL -mode … Use this option to set the Director processing mode, as described in “Set Director Processing Mode,” later in this document.RTID Management Note: Global statistics are combined statistics for all document types for which you have not requested separate statistics. If you request separate accumulation of statistics for a document type, global statistics will exclude statistics for that document type.
RTID Management -wt Use this option to set the WARNING_THRESOLD. is a long value that specifies the number of XML start tags beyond which parser reports a warning message that the warning threshold was exceeded. Example: java MBeanClient -url 127.0.0.1:7001 -server myServer -user rtidAdmin -password mypassword -wt 300000 In this example, if the Director received a document that had 400000 XML tags, a warning message would be logged to indicate that the threshold was exceeded.
RTID Management java MBeanClient -url 127.0.0.1:7001 -server myServer -user rtidAdmin password mypassword -mode SAPR00=CLOSED_ERROR SAPPAP=CLOSED_DEFER SAPPAP processing mode set to CLOSED_DEFER SAPR00 processing mode set to CLOSED_ERROR To query the current modes, use the -mode option without any values as shown below. It shows the source level modes and the system level mode (SYSTEM): java MBeanClient -url 127.0.0.
RTID Management accumulation. Thus, if you request statistics gathering for ORDERS05 documents, global statistics will exclude statistical data for ORDERS05. Global statistics for documents that were processed successfully are accumulated separately from those for documents that failed. For a report of global, document class, and failure statistics combined, use the –sx option to harvest statistics to a file. Some statistics gathering options (those ending with s) specify collection of summary statistics.
RTID Management SQL Operations The following XML tags can be present in summary or detailed statistics and represent SQL operations: • • • • SELECT UPDATE INSERT DELETE Any of these tags can have one or more attributes, depending on the statistics gathering options you’ve specified. An attribute name is followed by an equals sign (=) and a value enclosed in quotation marks (“”).
RTID Management
RTID Management • • • HANDLERS_BY_CLASS. The total number of document handlers for this document type. INUSE_ALL. The total number of document handlers currently in use. INUSE_BY_CLASS. The total number of document handlers that process this type of document. Tags representing SQL operations are described in “SQL Operations,” above. The remaining tags represent other steps in the processing of a document. If you’ve requested timing of operations, these tags include the MILLISECONDS attribute.
RTID Management The DOCUMENT_CLASS and DOCUMENT_HANDLERS XML segments are nested under the top-level tag RTID_OPERATIONS_REPORT.
RTID Management Counts of SQL operations apply only to successful transactions. The Director also maintains such counts for transactions that failed, but those are not available on the management interface. The only failure-related counters available on the management interface are TotalAllocateHandlerFailures, TotalDocumentProcessFailures, and DocumentsOutofSequence. Most counters are unaffected by the statistics gathering options you set in the Management Client.
RTID Management DocumentsOutOfSequence HandlersAllocated configuration Number of documents found to be out of sequence (having timestamps earlier than those on documents already processed) Cumulative count of handlers allocated since the last restart of the server. Since the document handler holds the SQL/MX connection and the prepared statements, counters pertaining to document handlers can be helpful for understanding the resource usage of the Director server.
RTID Management This counter is incremented if even a document fails to load. SelectedRows Number of rows selected from the data store during assembly of an output document. SelectStatements Number of Select statements executed SelectMilliseconds Time spent selecting records from the data store SerializeMilliseconds Time spent serializing outbound documents TimeOuts Number of failures due to SQL timeouts.
RTID Management To load a class dynamically, use the –load option of the Director management client (MBeanClient) or invoke the following Director management method from your own client: void loadDocumentClass(String name) throws Exception; The parameter is the name of the document class. The name must be the fully qualified path with the package name. Example: com.hp.rtsc.documents.
RTID Management Setting Log Level The Director logs events to the Director WebLogic server log. The events are categorized by severity level: 1. 2. 3. 4. ERROR WARNING INFO DEBUG The default log level is INFO. That means INFO and all categories above it (ERROR and WARNING) are logged. This level can be dynamically modified at runtime. For example you can turn on the log level to DEBUG to trouble shoot and reset it back to INFO when you are done. The updates to the log level are immediate.
RTID Management The source is either the name of the source system to which the mode applies, or the value SYSTEM to signify that the mode applies to all source systems. The mode can have any of the values described in the next several paragraphs. The default processing mode is OPEN, unless a different default mode is specified in the properties file. In this mode, incoming documents are processed by the Director and inserted appropriately into the data store.
RTID Management limit on the number of document handlers, and hence on the number of database connections, active at a given time. You specify the maximum number of document handlers for your configuration as the value of the –quota option specified to the management client or by invoking the SetDocumentHandlerQuota method of the management bean. Alternatively, you can specify the quota as the value of the MaxDocumentHandlers property in the Director properties file.
RTID Management 3. Select the “Deployments->Startup & Shutdown” option under your domain name, as shown in the picture below. 4. Select the “Configure a new Startup Class..” option. 5. In the “Create a new StartupClass..” window, enter the following values for the fields in the window. Name=DirectorManagerMBeanRegistrationClass ClassName=com.hp.rtid.management.beans.DirectorManager 6. Check the “Run before application deployments” option. 7. Click on “Create”. 8. Click on the “Targets” tab. 9.
RTID Management Troubleshooting This section provides an overview of the tools and resources you can use to troubleshoot problems that might arise after you deploy your solution. It also recommends approaches for addressing some typical types of problems.
RTID Management • • • A SystemException requires an operator to correct the system, often because of a database problem. Director error messages include nested SQL error messages wherever applicable; the document entitled “RTID Message Formats and Programmatic Interfaces” includes an example. A MetadataInconsistency requires the application customizer to correct the metadata. For detailed information about the metadata language, see the document entitled “RTID Metadata Language Reference.
RTID Management The Director can accumulate and report various statistics, including counts of SQL operations, documents processed or serialized, and time spent in specific types of operations. You can use the management client to set statistics gathering options. For example, you can isolate statistics pertaining to a document type or have statistics reported on a tableby-table basis. For more information about the statistics gathering options, see “Setting Statistics Gathering Options,” in this document.
RTID Management Downstream User Reports Missing Data If a user of the data store reports that expected data is missing, proceed as follows: 1. Try to discover what the source system should have been, and find out whether a whole document or only some data is missing. 2.
RTID Management operating correctly. Also, use the management client to check the processing mode for the source system, to see whether you’ve deferred or shut off processing from the source system in question. (The server log will also reflect the processing mode for a given feed.) For information about processing modes, see “Setting the Director Processing Mode.” 5. If the server log is empty, it makes sense to verify that the client or source system actually sent the data to the correct URL. 6.
RTID Management • Change the metadata to map the incoming data differently Transient Errors A great many SQL errors are transient. For example, a table is locked, or a TMF error occurs, or an online database change has occurred but the new schema hasn’t been loaded yet. In such cases, you can just resubmit the document after a short delay. Documents that failed on their initial submittal are in the errors folder configured in the Director properties file.
RTID Message Format and Programmatic Interfaces Real Time Information Director (RTID) Message Format and Programmatic Interfaces Table of Contents Director XML Message Format....................................................................................... 2 Native Document Format ............................................................................................ 2 Excerpts from a Native Document ...........................................................................
RTID Message Format and Programmatic Interfaces This document describes the message format and programmatic interfaces for access to the Real Time Information Director (also known as RTID or Director). Director XML Message Format You don’t have to write a client for access to the Director. You can use a JMS client, such as the JMS adapter of an Integration Server, to submit and retrieve XML documents.
RTID Message Format and Programmatic Interfaces Excerpts from a Native Document The following excerpt comes from a native document complying with a document definition called NativeOrders.
RTID Message Format and Programmatic Interfaces YOR 0200114533 2002-1018 DELIVERED DUTY UNPAID ORDERS05SAPAPL … ZHPF400 E …
RTID Message Format and Programmatic Interfaces characters like kanji—you must specify the encoding in the tag. The Director converts double-byte characters to single byte before storing them in the database. Outbound encoding is ISO 8859 by default, but you can specify a different encoding in the properties file, using the "OutboundEncoding" property, for example: OutboundEncoding=UTF-8 The RTID outbound encoding must match the encoding of the SPQ/MXMX database.
RTID Message Format and Programmatic Interfaces 00001 PAT00002 CRN … BRTH … Keys in Query Documents An important part of any query document is the section. It lists the primary key columns and the input values corresponding to each column.
RTID Message Format and Programmatic Interfaces The guiding principle in all these cases is that where the input specifies a value (even if it consists of spaces), the Director inserts that value. Where the input does not specify a value (where the element is absent or empty), the Director uses a default value in the case of an insert and retains the value in the data store in the case of an update. The next several paragraphs show how this principle applies, case by case.
RTID Message Format and Programmatic Interfaces 22001 -29256 Most Director exceptions fall into one of the following categories (each corresponding to an exception class): • • • • InvalidInputException requires the sending application to correct the data. SystemException requires an operator to correct the system (typically because of a database problem). MetadataInconsistency requires the application customizer to correct the metadata.
RTID Message Format and Programmatic Interfaces
RTID Message Format and Programmatic Interfaces For more detailed information about Director statistics XML formats, see the document entitled RTID Management. Director Programmatic Interfaces This section describes the interfaces for programmatic access to the Director.
RTID Message Format and Programmatic Interfaces • • A query, if the client sent a new or updated document. (The client can use this query later, to retrieve the document just loaded.) An error response, if the request could not be completed successfully In addition to these interfaces for submitting and retrieving documents, the Director supports a management interface for setting configuration and operational parameters, loading metadata dynamically, and monitoring Director operations.
RTID Message Format and Programmatic Interfaces 5. Use the setRequestMethod method of the HttpURLConnection object to specify a POST operation: urlConn.setRequestMethod("POST"); Use the getOutputStream method of the HttpURLConnection object to identify the destination, and the input and output objects to write the stream.
RTID Message Format and Programmatic Interfaces //If server returned an error, check the error stream InputStream errorStream = urlConn.getErrorStream(); if(errorStream == null) { System.println("No response error message"); }else{ responseReader = new BufferedReader(new InputStreamReader(errorStream)); String inputLine; while((inputLine = input.readLine())!= null) { output.write(inputLine); } } 8. Use the disconnect method of the HttpURLConnection object to close the connection to the servlet: UrlConn.
RTID Message Format and Programmatic Interfaces dispatch method using streams returns an integer result, which is 0 if the request was successful, 1 if the request failed. The dispatch method using strings returns a response document if the request was successful, an error document if the request failed. For information about how XML document formats correlate with the document definition metadata, see the document entitled RTID Metadata Language.
RTID Message Format and Programmatic Interfaces import com.hp.rtid.beans.DispatchBean; import com.hp.rtid.beans.DispatchBeanHome; 2. Get the initial context by retrieving the ContextFactory, the URL, and the JNDI name for the class from the properties file, which is called zli.properties by default.
RTID Message Format and Programmatic Interfaces 5. Invoke the dispatch method with an input stream and an output stream. For example: try{ int result = dispatchEjb.dispatch(message, response); } 6. Catch any exceptions. For example: catch(Exception e) { e.printStackTrace(); } Invoking the Dispatcher Directly from a Client on the NonStop Server You can invoke the RTID Dispatcher directly from a Java program and use streams to send and receive XML messages.
RTID Message Format and Programmatic Interfaces dispatcher.dispatch(inputStream); If you prefer to use strings for input and output, use the following syntax: Dispatcher.dispatch(inputString, outputString); 5. Catch any exceptions, and perform any suitable processing: Catch (Exception e) { Systen.out.println(“Error calling dispatch” + e); … } 6. Use the freeResources method to release the Dispatcher. For example: dispatcher.
RTID Message Format and Programmatic Interfaces • • • • DocumentHelper initializes the Director, sets various modes (for instance, to enable statistics gathering), and eventually frees document processing resources. UpdateHelper constructs and submits a new update document. QueryHelper constructs and submits a new query document. Group represents the document internally and provides methods for adding and reading records and their constituent elements.
RTID Message Format and Programmatic Interfaces 5. Use the query method to execute the query. Document responseDoc = queryHelper.query(queryDoc); 6. Use methods of the Group object to read elements or for access to child groups within the document. Group responseGroup = responseDocument.getfirst(); … Constructing and Processing an Insert or Update 1. Construct an UpdateHelper object, specifying the document name. (You must already have created and deployed the corresponding document definition.
RTID Security and Auditing Real Time Information Director (RTID) Security and Auditing Table of Contents Director Security and Auditing Features.......................................................................... 2 Consumers and Agents ................................................................................................ 2 Event Date................................................................................................................... 4 Role .....................................
RTID Security and Auditing This document describes the security, auditing, and transaction logging features of the Director. Director Security and Auditing Features The Director has security and auditing features to guarantee the rights of the consumer. Security policies restrict access to data pertaining to the consumer. Auditing provides a record of who read or modified what data. The metadata defining a document indicates whether the document requires security and/or auditing.
RTID Security and Auditing - PAT00002 CRN … BRTH - … Consumers and agents are identified in a document header by external IDs. An external ID is a character string that consists of an ID assigned by an issuing agency, followed by the name or code of the agency.
RTID Security and Auditing Event Date If you look back at the example above, you’ll notice that the header includes a date. This date indicates when the reported event occurred and is automatically recorded in the audit record as the event date. An audit record also includes the system time (on the NonStop Server) when the document was processed. Role The role signifies the relation of the agent to the consumer. In the example above, the agent claims to be the consumer’s personal physician.
RTID Security and Auditing The Director relies on its infrastructure--the NonStop Server or WLS--or on the client for several of these functions. For example, the Director does not perform authentication: it assumes that the user is who he says he is and that the connection between a client, such as a portal, and the Director is secured as appropriate to the solution. See the WLS documentation for information about security features of WLS.
RTID Security and Auditing Standard and Custom Policies The Director supports the definition of security policies that specify which agents have access to each consumer’s data. Thus far, two policies are predefined, or standard: • • The SelfService policy gives consumers access to their own data The PersonalAgent policy gives specific agents access to the data of specific consumers. A policy is implemented as a Java class. Policy classes reside in the com.hp.rtid.security package.
RTID Security and Auditing CompoundPolicy allows multiple policies to be applied to the same transaction; if the criteria of any policy named in the compound policy class are satisfied, the agent is allowed access to the requested data. For example, the compound policy SelfOrPersonalPhysician grants access if the agent is either the patient or the patient’s physician. SelfOrPersonalPhysician is a subclass of CompoundPolicy and calls the super class with the string "SelfService PersonalPhysician.
RTID Security and Auditing new policy classes. For example, a business might want to customize a provided data model or base policy decisions on other information in its own data model. For more information about the Director policy classes, see the Javadocs for the RTID Security package.
RTID Security and Auditing Auditing: Who Did What and When Like security, auditing is implemented primarily for the consumer’s benefit. Its primary use is to protect the privacy rights of the consumer and to protect against malpractice. Director Role in Auditing Implementation A real-time solution might have auditing requirements of several kinds: • • • To quantify resource use, as a basis for billing and/or capacity planning To monitor employee or customer activity To ensure regulatory compliance, e.g.
RTID Security and Auditing • • The date and time when the request was processed by the Director The name of the document type received or retrieved The Director also records audit details: the inbound document received by the Director, the outbound document sent by the Director, or both.
RTID Security and Auditing You must also modify the DDL to include the partition ID column. You need not write any custom code or modify any metadata. The partitioning strategy for audit tables is round-robin: each audit record is assigned the next partition ID in a repeating sequence. Specifying Audit Requirements for a Document A document definition specifies whether a document is audited or not and whether to record the inbound data, the outbound data, or both as audit details.
RTID Security and Auditing Here is the response to the query. It repeats the header from the query and returns the headers of four audited events pertaining to the child:
RTID Security and Auditing OBSERVATIONDETAIL_R_RECORDS 3 MICHAEL JONES ALEC GARDINER - 1100 2004-05-07T09:57:12 1000 CRN OBSERVATIONSUMMARY_R_RECOR DS 3 MICHAEL JONES ALEC GARDINER
RTID Security and Auditing Here is an example of the corresponding query. The person making the query has the external ID ABC and is a doctor (if we assume that PCP stands for Primary Care Physician). The role identifies ABC as a detective. He or she requests information pertaining to CDE, who is also a doctor (PCP). The AUDITDOCTYPE of 1 signifies audit was of inbound data, such as a record being inserted or updated in the data store. (An audit of outbound data has an AUDITDOCTYPE of 2.)
RTID Security and Auditing Retrieving Audit Detail Records To obtain the full audit detail record, ABC can use an audit document, as described by the following metadata. (You can’t tell by looking at the metadata itself, but a document of this type causes the Director to read from the AuditDetail table.) package com.hp.ehr.documents; import com.hp.rtid.metadata.AuditDocumentDefinition; import com.hp.rtid.metadata.DocumentRecord; import com.hp.rtid.metadata.MetadataInconsistency; import com.hp.rtid.
RTID Security and Auditing If you look at the header, you’ll notice that it doesn’t include the consumer ID. The reason, in this case, is that the audited event was the registration of a new patient; such a patient would not yet have had a consumer ID assigned. The consumer, in this case, happens also to be a doctor, the doctor we know as CDE from earlier transactions.
RTID Security and Auditing Transaction logging is an option that you choose as part of the persistent configuration of the Director. If you want transactions to be logged, specify the name of the log table as the value of the property RTID_TRANSACTION_LOG_TABLE in your properties file. If the value of that property is blank, or if the property isn’t specified at all, transaction logging does not occur.
RTID Security and Auditing SOURCE_REFERENCE_DOC_NO DOCUMENT_STATUS_CD RECORDS_AFFECTED_CT MAP_VERSION_NO MESSAGE_TX The format metadata for an IDoc maps the appropriate data field to SOURCE_REFERENCE_DOC_NO in the top level record. For example for ORDERS, the E1EDK01\BELNR element (sales_order_no) is mapped to SOURCE_REFERENCE_DOC_NO. “Error”, “Warning” or “OK”. The current status code that is returned in the XML response. The number of records affected by the transaction.
RTID Subscriptions Real Time Information Director (RTID) Subscriptions Table of Contents Introduction..................................................................................................................... 2 Subscription Module Overview ................................................................................... 2 Required Software and Related Documentation........................................................... 4 User Tasks ..........................................................
RTID Subscriptions Introduction Subscription Module Overview The Real Time Information Director (RTID, or Director) provides the basis for a realtime solution that integrates systems and applications across the enterprise. For example, the Real Time Supply Chain (RTSC) solution integrates multiple Supply Chain instances.
RTID Subscriptions JMS Queue/Topic Canned Subscriptions (XML File) OutboundDocumentHandler RegSubs SubscriptionRequestHandler DBSubscriber SUBINFO DATA STORE Figure 1.
RTID Subscriptions Required Software and Related Documentation The following products must be installed on the NonStop Server before you configure the Subscription Module: • • • BEA WebLogic Server 8.1 and NonStop Server Toolkit for WebLogic Server NonStop SQL/MX 2.0 Real Time Information Director executable JAR file (rtid.jar) User Tasks The following steps are required to include the Subscription module in your solution: 1.
RTID Subscriptions The SubServ Runtime Directory Structure The expression $RTID_HOME in this document refers to the directory where the Real Time Information Director and other Foundation Layer components are installed on the NonStop Server. See the document entitled RTID Deployment for information about deploying the Director. The Subscription Module is configured in your RTID environment with the directory structure shown in Figure 2. $RTID_HOME bin conf scripts subservstore Figure 2.
RTID Subscriptions The Subscription Store The Subscription module maintains all registered subscriptions in a Subscription Store, which is created during the Install process. The Subscription Store supports the following operations: • • • Inserting new subscriptions Browsing existing subscriptions Deleting existing subscriptions Subscriptions are described using XML that conforms to the schema $RTID_HOME/bin/SubServ.xsd. You use the RegSubs.
RTID Subscriptions Configuration Requirements for SubServ WebLogic Configuration Requirements Perform the following configuration tasks, using the WebLogic console, before running the Subscription module: • • • • • • Configure SubServ as a Startup Class within WebLogic. Configure a JDBC connection pool for the NonStop SQL/MX JDBC driver. Configure a JDBC Tx DataSource using the connection pool created. Ensure that the “Emulate Two-Phase Commit for non-XA Driver” is not selected.
RTID Subscriptions Installing and Configuring the Subscription Module Dependencies The Subscription module requires the Director to be installed and configured in the WLS environment. Installing and Configuring SubServ on the NonStop Server Make sure that your WebLogic Server is running with the Director configured. Carry out the following steps to install and configure the Subscription Module as part of your solution: 1. Make the RegSubs.
RTID Subscriptions • Grant execute permissions to sub.sh script chmod +x sub.sh • Execute the sub.sh from the OSS promt. sub.sh 5. Using the WLS console, configure the data source by navigating as follows: ÆServicesÆJDBCÆConnection PoolsÆConfigure New JDBC Connection Pool Database Type = Other Database Properties – Name = mxPool Driver Classname = com.tandem.sqlmx.
RTID Subscriptions 7. Ensure that the NonStop Server XA Resource is configured with WebLogic. Use the WebLogic Server console or the config.xml file to configure the startup class for the WLSNonStopTxHelper: ÆDeploymentsÆStartup & ShutdownÆConfigure New Startup Class Name=”NSMXARResource” ClassName=”com.hp.nsk.xares.
RTID Subscriptions Click ‘Create’ and then, on the Deploy screen, select Server and click ‘Apply.’ 9. Using the WebLogic console, configure the JMS Server’s file store by navigating as follows: ÆServicesÆJMSÆStoresÆConfigure New JMSFile Store Set the following values to configure the file store: Name= SubServStore Directory= /subservstore 10.
RTID Subscriptions The SubServ Properties File The SubServ properties file is used to specify the configuration parameters for the Subscription Module. Each property in the properties file is specified in = format. Table 3 describes the supported properties. Property name DATASOURCE URL USERNAME PASSWORD JMSSERVER INPUTFILE CATALOG SCHEMA Description The JNDI name for the JDBC TxDataSource configured within WLS. This name is used to get a handle for database connections. e.g.
RTID Subscriptions # XML Input File to register Subscriptions (optional) INPUTFILE=SubServ.xml # ANSI NAME of catalog CATALOG=DATA1 # ANSI NAME of schema SCHEMA=RTSCDB Registering Subscriptions Once the Subscription module is running, you can configure subscriptions to deliver documents to your applications. You use the RegSubs.sh program to register subscriptions. You describe subscriptions in an XML file that serves as input to RegSubs.sh.
RTID Subscriptions Table 4 lists and defines the XML tags used in this schema. Tag Name Subscriptions SubInfo DocName DestQueueName DestTopicName SQLstmt Description Root element of the XML document. Used to describe a Subscription. Specifies the name of the document that is to be generated.
RTID Subscriptions It is a requirement that the master table on which the subscription is registered contain a DATETIME field that contains a unique value for each record. It is a requirement that this DATETIME field be the last column in the SELECT clause list It is a requirement that this DATETIME field be the only column in the WHERE clause of the SELECT statement to which a value is set at run time.
RTID Subscriptions Using RegSubs.sh You use the RegSubs.sh program to • • • Add subscriptions to the Subscription Store, Browse the Subscription Store and list the currently active subscriptions Delete registered subscriptions Before running RegSubs.sh, execute the setenv.sh script from the /scripts folder. Then invoke RegSubs.sh with any of the following command lines: RegSubs.sh -i [-p ] [-f ] RegSubs.
RTID Subscriptions The following are examples of command lines that can be used in the Insert, Browse, and Delete modes: . /scripts/setenv.sh RegSubs.sh -i RegSubs.sh -i -p RTSCapp.prop -f RTSCsubs.xml RegSubs.sh -b RegSubs.sh -d -n 13 -p RTSCapp.prop Note: Make sure to create an index on the top-level table for each subscription you create. The tables to which you subscribe must have an index on the DATETIME column and the primary key columns.
RTID Subscriptions Testing and Troubleshooting The Subscription module logs errors to the Director’s WebLogic log file in the server’s log directory (serverName.log in WLS_DOMAIN_HOME/serverName/), enabling you to troubleshoot problems in development and production environments. The following tips are useful in ensuring that the Subscription module is configured and functioning correctly: 1. After configuring SubServ and starting NonStop WLS, make sure NonStop WLS has started without error.
RTID Subscriptions -o outputFileDirectory - sets the directory for output files. If the output directory is NOT specified, the drivers use the following logic to save any output file generated: First a ‘Response_’ string is added to the input filename. Then this file is saved where the input file is found (in the input directory if –i (or –d) is specified, or in the root directory where the program is running. Output filenames have the format MDBtimestamp.xml.
