Sun GlassFish Enterprise Server 2.1 Performance Tuning Guide Sun Microsystems, Inc. 4150 Network Circle Santa Clara, CA 95054 U.S.A.
Copyright 2009 Sun Microsystems, Inc. 4150 Network Circle, Santa Clara, CA 95054 U.S.A. All rights reserved. Sun Microsystems, Inc. has intellectual property rights relating to technology embodied in the product that is described in this document. In particular, and without limitation, these intellectual property rights may include one or more U.S. patents or pending patent applications in the U.S. and in other countries. U.S. Government Rights – Commercial software.
Contents Preface ...................................................................................................................................................13 1 Overview of Enterprise Server Performance Tuning ..................................................................... 17 Process Overview ................................................................................................................................ 17 ▼ Performance Tuning Sequence .......................................
Contents Tuning Tips for Specific Types of EJB Components ............................................................... 42 JDBC and Database Access ......................................................................................................... 46 Tuning Message-Driven Beans .................................................................................................. 47 3 Tuning the Enterprise Server .........................................................................................
Contents Improving ORB Performance with Java Serialization ............................................................. 75 Thread Pool Settings ........................................................................................................................... 76 Tuning Thread Pools (Unix /Linux only) ................................................................................. 76 Resources .........................................................................................................
Contents Disk I/O Settings ........................................................................................................................ 102 TCP/IP Settings .......................................................................................................................... 102 Tuning UltraSPARC T1–Based Systems ........................................................................................ 103 Tuning Operating System and TCP Settings ...............................................
Figures FIGURE 1–1 Java EE Application Model .......................................................................................
Tables TABLE 1–1 TABLE 1–2 TABLE 3–1 TABLE 3–2 TABLE 3–3 TABLE 3–4 TABLE 4–1 TABLE 5–1 TABLE 5–2 Performance Tuning Roadmap ............................................................................... 17 Factors That Affect Performance ............................................................................. 24 Bean Type Pooling or Caching ................................................................................ 53 EJB Cache and Pool Settings .................................................
Examples EXAMPLE 4–1 EXAMPLE 4–2 Heap Configuration on Solaris ................................................................................ 89 Heap Configuration on Windows ...........................................................................
Preface The Performance Tuning Guide describes how to get the best performance with Enterprise Server. This preface contains information about and conventions for the entire Sun GlassFishTM Enterprise Server documentation set. Sun GlassFish Enterprise Server Documentation Set TABLE P–1 Books in the Enterprise Server Documentation Set Book Title Description Documentation Center Enterprise Server documentation topics organized by task and subject.
Preface TABLE P–1 Books in the Enterprise Server Documentation Set (Continued) Book Title Description High Availability Administration Guide Setting up clusters, working with node agents, and using load balancers. Administration Reference Editing the Enterprise Server configuration file, domain.xml. Performance Tuning Guide Tuning the Enterprise Server to improve performance. Reference Manual Utility commands available with the Enterprise Server; written in man page style.
Preface Typographic Conventions The following table describes the typographic changes that are used in this book. TABLE P–3 Typographic Conventions Typeface Meaning Example AaBbCc123 The names of commands, files, and directories, and onscreen computer output Edit your .login file. Use ls -a to list all files. machine_name% you have mail.
Preface TABLE P–4 Symbol Conventions (Continued) Symbol Description Example Meaning → Indicates menu item selection in a graphical user interface. File → New → Templates From the File menu, choose New. From the New submenu, choose Templates. Documentation, Support, and Training The Sun web site provides information about the following additional resources: ■ ■ ■ Documentation (http://www.sun.com/documentation/) Support (http://www.sun.com/support/) Training (http://www.sun.
1 C H A P T E R 1 Overview of Enterprise Server Performance Tuning You can significantly improve performance of the Sun GlassFish Enterprise Server and of applications deployed to it by adjusting a few deployment and server configuration settings. However, it is important to understand the environment and performance goals. An optimal configuration for a production environment might not be optimal for a development environment.
Process Overview TABLE 1–1 Performance Tuning Roadmap (Continued) Step Description of Task Location of Instructions 3 Installation: If you are using HADB for session persistence, ensure that the HADB software is installed. Installation Guide 4 Deployment: Install and run your applications. Familiarize yourself with how to configure and administer the Enterprise Server.
Understanding Operational Requirements Understanding Operational Requirements Before you begin to deploy and tune your application on the Application Server, it is important to clearly define the operational environment.
Understanding Operational Requirements Client-Side Presentation Server-Side Presentation Server-Side Business Logic Browser Web Server EJB Container Pure HTML JSP EJB Java Applet JSP EJB Java Servlet EJB J2EE Platform J2EE Platform Enterprise Information System Desktop Java Application Other Device J2EE Client FIGURE 1–1 Java EE Application Model Moderately complex enterprise applications can be developed entirely using servlets and JSP technology.
Understanding Operational Requirements Security Requirements Most business applications require security. This section discusses security considerations and decisions. User Authentication and Authorization Application users must be authenticated. The Application Server provides three different choices for user authentication: file-based, LDAP, and Solaris. The default file based security realm is suitable for developer environments, where new applications are developed and tested.
Understanding Operational Requirements ■ What is the nature of the applications with respect to security? Do they encrypt all or only a part of the application inputs and output? What percentage of the information needs to be securely transmitted? ■ Are the applications going to be deployed on an application server that is directly connected to the Internet? Will a web server exist in a demilitarized zone (DMZ) separate from the application server tier and backend enterprise systems? A DMZ-style deploym
General Tuning Concepts Administration A single Application Server installation on a server can encompass multiple instances. A group of one or more instances that are administered by a single Administration Server is called a domain. Grouping server instances into domains permits different people to independently administer the groups. You can use a single-instance domain to create a “sandbox” for a particular developer and environment.
General Tuning Concepts TABLE 1–2 Factors That Affect Performance Concept In practice Measurement Value sources User Load Concurrent sessions at peak load Transactions Per Minute (TPM) (Max. number of concurrent users) * (expected response time) / (time between clicks) Web Interactions Per Second (WIPS) Example: (100 users * 2 sec) / 10 sec = 20 Application Scalability Transaction rate measured on one CPU TPM or WIPS Measured from workload benchmark. Perform at each tier.
General Tuning Concepts ▼ To Determine Capacity 1 Determine performance on a single CPU. First determine the largest load that a single processor can sustain. You can obtain this figure by measuring the performance of the application on a single-processor machine. Either leverage the performance numbers of an existing application with similar processing characteristics or, ideally, use the actual application and workload in a testing environment.
Further Information fraction of them are logged in and performing business transactions. A common mistake during capacity planning is to use the total size of customer population as the basis and not the average and peak numbers for concurrent users. The number of concurrent users also may exhibit patterns over time. ■ What is the average and peak amount of data transferred per request? This value is also application-specific.
2 C H A P T E R 2 Tuning Your Application This chapter provides information on tuning applications for maximum performance. A complete guide to writing high performance Java and Java EE applications is beyond the scope of this document.
Java Programming Guidelines String str = "testing"; str = str + "abc"; The compiler translates this code as: String str = "testing"; StringBuffer tmp = new StringBuffer(str); tmp.append("abc"); str = tmp.toString(); Therefore, copying is inherently expensive and overusing it can reduce performance significantly.
Java Server Page and Servlet Tuning Declare Method Arguments final Declare method arguments final if they are not modified in the method. In general, declare all variables final if they are not modified after being initialized or set to some value. Synchronize Only When Necessary Do not synchronize code blocks or methods unless synchronization is required. Keep synchronized blocks or methods as short as possible to avoid scalability bottlenecks.
Java Server Page and Servlet Tuning Suggested Coding Practices This section provides some tips on coding practices that improve servlet and JSP application performance. General Guidelines Follow these general guidelines to increase performance of the presentation tier: ■ ■ ■ ■ Minimize Java synchronization in servlets. Don’t use the single thread model for servlets. Use the servlet’s init() method to perform expensive one-time initialization. Avoid using System.out.println() calls.
Java Server Page and Servlet Tuning Configuration and Deployment Tips Follow these configuration tips to improve performance. These tips are intended for production environments, not development environments. ■ To improve class loading time, avoid having excessive directories in the server CLASSPATH. Put application-related classes into JAR files. ■ HTTP response times are dependent on how the keep-alive subsystem and the HTTP server is tuned in general.
EJB Performance Tuning EJB Performance Tuning The Enterprise Server’s high-performance EJB container has numerous parameters that affect performance. Individual EJB components also have parameters that affect performance. The value of individual EJB component’s parameter overrides the value of the same parameter for the EJB container. The default values are designed for a single-processor computer system—change them to optimize for other system configurations.
EJB Performance Tuning asadmin get --user admin --host e4800-241-a --port 4848 -m specjcmp.application.SPECjAppServer.ejb-module. supplier_jar.stateful-session-bean.BuyerSes.bean-cache.
EJB Performance Tuning collection is happening too frequently, and the pool size is growing, but the cache hit rate is small, then the pool-idle-timeout-in-seconds can be reduced to destroy the instances. Note – Specifying a max-pool-size of zero (0) means that the pool is unbounded. The pooled beans remain in memory unless they are removed by specifying a small interval for pool-idle-timeout-in-seconds. For production systems, specifying the pool as unbounded is NOT recommended.
EJB Performance Tuning To find the possible objects (applications, modules, beans, and methods) and object attributes that can be monitored, use the Admin Console. For more information, see Chapter 18, “Monitoring Components and Services,” in Sun GlassFish Enterprise Server 2.1 Administration Guide. Alternatively, use the asadmin list command. For more information, see list(1). For statistics on stateful session bean passivations, use this command: asadmin get -m serverInstance.application.appName.
EJB Performance Tuning ■ Cache EJB resources: Use setSessionContext() or ejbCreate() to cache bean resources. This is again an example of using bean lifecycle methods to perform application actions only once where possible. Remember to release acquired resources in the ejbRemove() method. Use the Appropriate Stubs The stub classes needed by EJB applications are generated dynamically at runtime when an EJB client needs them.
EJB Performance Tuning Prefer Local Interfaces An EJB component can have remote and local interfaces. Clients not located in the same application server instance as the bean (remote clients) use the remote interface to access the bean. Calls to the remote interface require marshalling arguments, transportation of the marshalled data over the network, un-marshaling the arguments, and dispatch at the receiving end. Thus, using the remote interface entails significant overhead.
EJB Performance Tuning semantics. See “Value Added Features” in Sun GlassFish Enterprise Server 2.1 Developer’s Guide for more details about the pass-by-reference flag. To specify that an EJB component will use pass by reference semantics, use the following tag in the sun-ejb-jar.xml deployment descriptor: true. This avoids copying arguments when the EJB component’s methods are invoked and avoids copying results when methods return.
EJB Performance Tuning source are going to be involved in a transaction. If a database participates in some distributed transactions, but mostly in local or single database transactions, it is advisable to register two separate JDBC resources and use the appropriate resource in the application.
EJB Performance Tuning Use version consistency to improve performance while protecting the integrity of data in the database. Since the application server can use multiple copies of an EJB component simultaneously, an EJB component’s state can potentially become corrupted through simultaneous access. The standard way of preventing corruption is to lock the database row associated with a particular bean. This prevents the bean from being accessed by two simultaneous transactions and thus protects data.
EJB Performance Tuning ... OrderTable.VC_VERSION_NUMBER In addition, you must establish a trigger on the database to automatically update the version column when data in the specified table is modified. The Application Server requires such a trigger to use version consistency.
EJB Performance Tuning For example, enable threadpools named priority-1 and priority-2 to the element as follows: 3 Include the threadpool ID in the use-thread-pool-id element of the EJB component’s sun-ejb-jar.xml deployment descriptor. For example, the following sun-ejb-jar.
EJB Performance Tuning to the steady load of users), beans would be frequently passivated and activated, causing a negative impact on the response times, due to CPU intensive serialization and deserialization as well as disk I/O. Another important variable for tuning is cache-idle-timeout-in-seconds where at periodic intervals of cache-idle-timeout-in-seconds, all the beans in the cache that have not been accessed for more than cache-idle-timeout-in-seconds time, are passivated.
EJB Performance Tuning ■ ■ Database rows represented by the bean do not change. The application can tolerate using out-of-date values for the bean. For example, an application might use a read-only bean to represent a list of best-seller books. Although the list might change occasionally in the database (say, from another bean entirely), the change need not be reflected immediately in an application. The ejbLoad() method of a read-only bean is handled differently for CMP and BMP beans.
EJB Performance Tuning For example, you have this relationship defined in the ejb-jar.
EJB Performance Tuning Pre-fetching generally improves performance because it reduces the number of database accesses. However, if the business logic often uses Orders without referencing their OrderLines, then this can have a performance penalty, that is, the system has spent the effort to pre-fetch the OrderLines that are not actually needed. Avoid pre-fetching for specific finder methods; this can often avoid that penalty.
EJB Performance Tuning Reduce the database transaction isolation level when appropriate. Reduced isolation levels reduce work in the database tier, and could lead to better application performance. However, this must be done after carefully analyzing the database table usage patterns. Set the database transaction isolation level with the Admin Console on the Resources > JDBC > Connection Pools > PoolName page.
EJB Performance Tuning Cache Bean-Specific Resources Use the setMessageDrivenContext() or ejbCreate() method to cache bean specific resources, and release those resources from the ejbRemove() method. Limit Use of JMS Connections When designing an application that uses JMS connections make sure you use a methodology that sparingly uses connections, by either pooling them or using the same connection for multiple sessions. The JMS connection uses two threads and the sessions use one thread each.
3 C H A P T E R 3 Tuning the Enterprise Server This chapter describes some ways to tune the Enterprise Server for optimum performance, including the following topics: ■ “Deployment Settings” on page 49 ■ “Logger Settings” on page 50 ■ “Web Container Settings” on page 51 ■ “EJB Container Settings” on page 53 ■ “Java Message Service Settings” on page 58 ■ “Transaction Service Settings” on page 58 ■ “HTTP Service Settings” on page 60 ■ “ORB Settings” on page 70 ■ “Thread Pool Settings” on
Logger Settings Disable Auto-deployment Enabling auto-deployment will adversely affect deployment, though it is a convenience in a development environment. For a production system, disable auto-deploy to optimize performance. If auto-deployment is enabled, then the Reload Poll Interval setting can have a significant performance impact. Disable auto-deployment with the Admin Console under Stand-Alone Instances > server (Admin Server) on the Advanced/Applications Configuration tab.
Web Container Settings General Settings In general, writing to the system log slows down performance slightly; and increased disk access (increasing the log level, decreasing the file rotation limit or time limit) also slows down the application. Also, make sure that any custom log handler doesn’t log to a slow device like a network file system since this can adversely affect performance.
Web Container Settings Manager Properties: Reap Interval Modifying the reap interval can improve performance, but setting it without considering the nature of your sessions and business logic can cause data inconsistency, especially for time-based persistence-frequency. For example, if you set the reap interval to 60 seconds, the value of session data will be recorded every 60 seconds. But if a client accesses a servlet to update a value at 20 second increments, then inconsistencies will result.
EJB Container Settings EJB Container Settings The EJB Container has many settings that affect performance. As with other areas, use monitor the EJB Container to track its execution and performance. Monitoring the EJB Container Monitoring the EJB container is disabled by default. Enable monitoring with the Admin Console under Configurations > config-name > Monitoring. Set the monitoring level to LOW for to monitor all deployed EJB components, EJB pools, and EJB caches.
EJB Container Settings Note – If you develop and deploy your EJB components using Sun Java Studio, then you need to edit the individual bean descriptor settings for bean pool and bean cache. These settings might not be suitable for production-level deployment. Tuning the EJB Pool A bean in the pool represents the pooled state in the EJB lifecycle. This means that the bean does not have an identity. The advantage of having beans in the pool is that the time to create a bean can be saved for a request.
EJB Container Settings ■ Pool Idle Timeout: the maximum time that a stateless session bean, entity bean, or message-driven bean is allowed to be idle in the pool. After this time, the bean is destroyed if the bean in case is a stateless session bean or a message driver bean. This is a hint to server. The default value is 600 seconds. The corresponding EJB deployment descriptor attribute is pool-idle-timeout-in-seconds.
Max Cache Size Max Cache Size Maximum number of beans in the cache. Make this setting greater than one. The default value is 512. A value of zero indicates the cache is unbounded, which means the size of the cache is governed by Cache Idle Timeout and Cache Resize Quantity. The corresponding EJB deployment descriptor attribute is max-cache-size. Cache Resize Quantity Number of beans to be created or deleted when the cache is serviced by the server.
Refresh period TABLE 3–2 EJB Cache and Pool Settings (Continued) Cache Settings Pool Settings Type of Bean cacheresizequantity max- cachesize cacheidletimeoutinseconds Entity X X X X X Entity Readonly X X X X X removaltimeout- inseconds victimselectionpolicy refreshperiodinseconds X Message Driven Bean steadypool-size poolresizequantity maxpoolsize pool-idletimeout-inseconds X X X X X X X X X X X Commit Option The commit option controls the action taken by the EJB c
Java Message Service Settings than cache misses, then option B is an appropriate choice. You might still have to change the max-cache-size and cache-resize-quantity to get the best result. If the cache hits are too low and cache misses are very high, then the application is not reusing the bean instances and hence increasing the cache size (using max-cache-size) will not help (assuming that the access pattern remains the same). In this case you might use commit option C.
Transaction Service Settings asadmin get -m serverInstance.transaction-service.* The following statistics are gathered on the transaction service: ■ ■ ■ ■ ■ total-tx-completed Completed transactions. total-tx-rolled-back Total rolled back transactions. total-tx-inflight Total inflight (active) transactions. isFrozen Whether transaction system is frozen (true or false) inflight-tx List of inflight (active) transactions.
HTTP Service Settings Recover On Restart (Automatic Recovery) To set the Recover on Restart attribute with the Admin Console, go to Configurations > config-name > Transaction Service. Click the Recover check box to set it to true (checked, the default) or false (un-checked). You can also set automatic recovery with asadmin, for example: asadmin set server1.transaction-service.
HTTP Service Settings With asadmin, use the following command to list the monitoring parameters available: list --user admin --port 4848 -m server-instance-name.http-service.* where server-instance-name is the name of the server instance. Use the following command to get the values: get --user admin --port 4848 -m server.http-service.parameter-name.* where parameter-name is the name of the parameter to monitor. Statistics collection is enabled by default.
HTTP Service Settings HitRatio The hit ratio is the number of cache hits divided by the number of cache lookups. This setting is not tunable. Note – If you turn off DNS lookups on your server, host name restrictions will not work and IP addresses will appear instead of host names in log files. Caching DNS Entries It is possible to also specify whether to cache the DNS entries. If you enable the DNS cache, the server can store hostname information after receiving it.
HTTP Service Settings File Cache Information (file-cache) The file cache caches static content so that the server handles requests for static content quickly. The file-cache section provides statistics on how your file cache is being used. For information on tuning the file cache, see “HTTP File Cache” on page 67.
HTTP Service Settings Connection Queue ■ Total Connections Queued: Total connections queued is the total number of times a connection has been queued. This includes newly accepted connections and connections from the keep-alive system. ■ Average Queuing Delay: Average queueing delay is the average amount of time a connection spends in the connection queue.
HTTP Service Settings ■ ■ Request Timeout Buffer Length Thread Count The Thread Count parameter specifies the maximum number of simultaneous requests the server can handle. The default value is 5. When the server has reached the limit or request threads, it defers processing new requests until the number of active requests drops below the maximum amount. Increasing this value will reduce HTTP response latency times.
HTTP Service Settings Buffer Length The size (in bytes) of the buffer used by each of the request processing threads for reading the request data from the client. Adjust the value based on the actual request size and observe the impact on performance. In most cases the default should suffice. If the request size is large, increase this parameter. Keep Alive Both HTTP 1.0 and HTTP 1.1 support the ability to send multiple requests across a single HTTP session.
HTTP Service Settings Time Out Time Out determines the maximum time (in seconds) that the server holds open an HTTP keep alive connection. A client can keep a connection to the server open so that multiple requests to one server can be serviced by a single network connection. Since the number of open connections that the server can handle is limited, a high number of open connections will prevent new clients from connecting. The default time out value is 30 seconds.
HTTP Service Settings Max Files Count Max Files Count determines how many files are in the cache. If the value is too big, the server caches little-needed files, which wastes memory. If the value is too small, the benefit of caching is lost. Try different values of this attribute to find the optimal solution for specific applications—generally, the effects will not be great. Hash Init Size Hash Init Size affects memory use and search time, but rarely will have a measurable effect on performance.
HTTP Service Settings Tuning HTTP Listener Settings Change HTTP listener settings in the Admin Console under Configurations > config-name > HTTP Service > HTTP Listeners > listener-name. Network Address For machines with only one network interface card (NIC), set the network address to the IP address of the machine (for example, 192.18.80.23 instead of default 0.0.0.0). If you specify an IP address other than 0.0.0.0, the server will make one less system call per connection.
ORB Settings ORB Settings The Enterprise Server includes a high performance and scalable CORBA Object Request Broker (ORB). The ORB is the foundation of the EJB Container on the server. Overview The ORB is primarily used by EJB components via: ■ RMI/IIOP path from an application client (or rich client) using the application client container. ■ RMI/IIOP path from another Enterprise Server instance ORB. ■ RMI/IIOP path from another vendor’s ORB.
ORB Settings set serverInstance.iiop-service.orb.system.monitoringEnabled=true reconfig serverInstance Connection Statistics The following statistics are gathered on ORB connections: ■ ■ total-inbound-connections Total inbound connections to ORB. total-outbound-connections Total outbound connections from ORB. Use this command to get ORB connection statistics: asadmin get --monitor serverInstance.iiop-service.orb.system.orb-connection.
ORB Settings TABLE 3–3 Tunable ORB Settings (Continued) RMI/ IIOP from ORB to Enterprise Server communication infrastructure, thread pool steady-thread-pool-size, max-thread-pool-size, idle-thread-timeout-in-seconds RMI/ IIOP from a vendor ORB parts of communication infrastructure, thread pool steady-thread-pool-size, max-thread-pool-size, idle-thread-timeout-in-seconds In-process thread pool steady-thread-pool-size, max-thread-pool-size, idle-thread-timeout-in-seconds Tunable ORB Parameters Tu
ORB Settings ■ Minimum Pool Size: The minimum number of threads in the ORB thread pool. Set to the average number of threads needed at a steady (RMI/ IIOP) load. ■ Maximum Pool Size: The maximum number of threads in the ORB thread pool. ■ Idle Timeout: Number of seconds to wait before removing an idle thread from pool. Allows shrinking of the thread pool. ■ Number of Work Queues In particular, the maximum pool size is important to performance.
ORB Settings Load Balancing For information on how to configure RMI/IIOP for multiple application server instances in a cluster, Chapter 9, “RMI-IIOP Load Balancing and Failover,” in Sun GlassFish Enterprise Server 2.1 High Availability Administration Guide. When tuning the client ORB for load-balancing and connections, consider the number of connections opened on the server ORB. Start from a low number of connections and then increase it to observe any performance benefits.
ORB Settings ++++++++++++++++++++++++++++++ Message(Thread[ORB Client-side Reader, conn to 192.18.80.118:1050,5,main]): createFromStream: type is 4 < MessageBase(Thread[ORB Client-side Reader, conn to 192.18.80.118:1050,5,main]): Message GIOP version: 1.2 MessageBase(Thread[ORB Client-side Reader, conn to 192.18.80.118:1050,5,main]): ORB Max GIOP Version: 1.2 Message(Thread[ORB Client-side Reader, conn to 192.18.80.118:1050,5,main]): createFromStream: message construction complete. com.sun.corba.ee.
Thread Pool Settings ▼ To Enable Java Serialization You must set this property on all servers that you want to use JSG. 1 In the tree component, expand the Configurations node. 2 Expand the desired node. 3 Select the JVM Settings node 4 In the JVM Settings page, choose the JVM Options tab. 5 Click Add JVM Option, and enter the following value: -Dcom.sun.CORBA.encoding.ORBEnableJavaSerialization=true 6 Click Save 7 Restart the Application Server.
Resources is not offered in a Unix/Linux user interface. However, it is possible to edit the OS-scheduled thread pools and add new thread pools, if needed, using the Admin Console. Resources ■ ■ “JDBC Connection Pool Settings” on page 77 “Connector Connection Pool Settings” on page 80 JDBC Connection Pool Settings For optimum performance of database-intensive applications, tune the JDBC Connection Pools managed by the Enterprise Server.
Initial and Mimimum Pool Size ■ ■ ■ “Timeout Settings” on page 78 “Isolation Level Settings” on page 79 “Connection Validation Settings” on page 79 Pool Size Settings The following settings control the size of the connection pool: Initial and Mimimum Pool Size Size of the pool when created, and its minimum allowable size. Maximum Pool Size Upper limit of size of the pool. Pool Resize Quantity Number of connections to be removed when the idle timeout expires.
Pool Resize Quantity ■ Idle Timeout: Maximum time in seconds that a connection can remain idle in the pool. After this time, the pool can close this connection. This property does not control connection timeouts on the database server. Keep this timeout shorter than the database server timeout (if such timeouts are configured on the database), to prevent accumulation of unusable connection in Enterprise Server.
Connection Validation Required Connection Validation Required If true, the pool validates connections (checks to find out if they are usable) before providing them to an application. If possible, keep the default value, false. Requiring connection validation forces the server to apply the validation algorithm every time the pool returns a connection, which adds overhead to the latency of getConnection(). If the database connectivity is reliable, you can omit validation.
Close All Connections On Any Failure asadmin> create-connector-connection-pool --raname jdbcra --connectiondefinition javax.sql.
4 C H A P T E R 4 Tuning the Java Runtime System This chapter discusses the following topics: ■ ■ ■ “Java Virtual Machine Settings” on page 83 “Managing Memory and Garbage Collection” on page 84 “Further Information” on page 91 Java Virtual Machine Settings J2SE 5.0 provides two implementations of the HotSpot Java virtual machine (JVM): ■ The client VM is tuned for reducing start-up time and memory footprint. Invoke it by using the -client JVM command-line option.
Managing Memory and Garbage Collection Managing Memory and Garbage Collection The efficiency of any application depends on how well memory and garbage collection are managed.
Managing Memory and Garbage Collection When the new generation fills up, it triggers a minor collection in which the surviving objects are moved to the old generation. When the old generation fills up, it triggers a major collection which involves the entire object heap. Both HotSpot and Solaris JDK use thread local object allocation pools for lock-free, fast, and scalable object allocation. So, custom object pooling is not often required.
Managing Memory and Garbage Collection Tracing Garbage Collection The two primary measures of garbage collection performance are throughput and pauses. Throughput is the percentage of the total time spent on other activities apart from GC. Pauses are times when an application appears unresponsive due to GC. Two other considerations are footprint and promptness. Footprint is the working size of the JVM process, measured in pages and cache lines.
Managing Memory and Garbage Collection Although applications can explicitly invoke GC with the System.gc() method, doing so is a bad idea since this forces major collections, and inhibits scalability on large systems. It is best to disable explicit GC by using the flag -XX:+DisableExplicitGC. The Enterprise Server uses RMI in the Administration module for monitoring.
Managing Memory and Garbage Collection memory structures. The difference between the maximum address space and the total of those values is the amount of memory that can be allocated to the heap. You can improve performance by increasing your heap size or using a different garbage collector. In general, for long-running server applications, use the J2SE throughput collector on machines with multiple processors (-XX:+AggressiveHeap) and as large a heap as you can fit in the free memory of your machine.
Managing Memory and Garbage Collection ■ Decide the total amount of memory you can afford for the JVM. Accordingly, graph your own performance metric against young generation sizes to find the best setting. ■ Make plenty of memory available to the young generation. The default is calculated from NewRatio and the -Xmx setting. ■ Larger eden or younger generation spaces increase the spacing between full GCs. But young space collections could take a proportionally longer time.
Managing Memory and Garbage Collection To prevent load address collisions, set preferred base addresses with the rebase utilty that comes with Visual Studio and the Platform SDK. Use the rebase utility to reassign the base addresses of the Application Server DLLs to prevent relocations at load time and increase the available process memory for the Java heap. There are a few Application Server DLLs that have non-default base addresses that can cause collisions.
Further Information Further Information For more information on tuning the JVM, see: ■ ■ ■ ■ ■ ■ Java HotSpot VM Options Frequently Asked Questions About the Java HotSpot Virtual Machine Performance Documentation for the Java HotSpot VM Java performance web page Monitoring and Management for the Java Platform (J2SE 5.
5 C H A P T E R 5 Tuning the Operating System and Platform This chapter discusses tuning the operating system (OS) for optimum performance.
Server Scaling the relative amount of time it spends in I/O versus CPU activity. Studies have shown that doubling the number of CPUs increases servlet performance by 50 to 80 percent. Memory See the section Hardware and Software Requirements in the Sun Java System Application Server Release Notes for specific memory recommendations for each supported operating system. Disk Space It is best to have enough disk space for the OS, document tree, and log files. In most cases 2GB total is sufficient.
Tuning for the Solaris OS Solaris 10 Platform-Specific Tuning Information SolarisTM Dynamic Tracing (DTrace) is a comprehensive dynamic tracing framework for the Solaris Operating System (OS). You can use the DTrace Toolkit to monitor the system. The DTrace Toolkit is available through the OpenSolarisTM project from the DTraceToolkit page (http://www.opensolaris.org/os/community/dtrace/dtracetoolkit/).
Tuning for the Solaris OS TABLE 5–1 Tuning Parameters for Solaris (Continued) Parameter Scope Default Tuned Value Comments tcp_conn_req_max_q ndd /dev/tcp 128 1024 tcp_conn_req_max_q0 ndd /dev/tcp 1024 4096 tcp_ip_abort_interval ndd /dev/tcp 480000 60000 tcp_keepalive_interval ndd /dev/tcp 7200000 900000 For high traffic web sites, lower this value. tcp_rexmit_interval_initial ndd /dev/tcp 3000 3000 If retransmission is greater than 30-40%, you should increase this value.
Linux Configuration File Descriptor Setting On the Solaris OS, setting the maximum number of open files property using ulimit has the biggest impact on efforts to support the maximum number of RMI/IIOP clients.
Tuning for Solaris on x86 echo 262143 > /proc/sys/net/core/rmem_default #above configuration for 2.4.
Tuning for Solaris on x86 Some of the values depend on the system resources available. After making any changes to /etc/system, reboot the machines. File Descriptors Add (or edit) the following lines in the /etc/system file: set set set set set set rlim_fd_max=65536 rlim_fd_cur=65536 sq_max_size=0 tcp:tcp_conn_hash_size=8192 autoup=60 pcisch:pci_stream_buf_enable=0 These settings affect the file descriptors.
Tuning for Linux platforms Tuning for Linux platforms To tune for maximum performance on Linux, you need to make adjustments to the following: ■ ■ ■ ■ ■ “File Descriptors” on page 100 “Virtual Memory” on page 101 “Network Interface” on page 102 “Disk I/O Settings” on page 102 “TCP/IP Settings” on page 102 File Descriptors You may need to increase the number of file descriptors from the default.
Tuning for Linux platforms cputime filesize datasize stacksize coredumpsize memoryuse descriptors memorylocked maxproc openfiles unlimited unlimited unlimited 8192 kbytes 0 kbytes unlimited 1024 unlimited 8146 1024 The openfiles and descriptors show a limit of 1024. To increase the limit to 65535 for all users, edit /etc/security/limits.
Tuning for Linux platforms Network Interface To ensure that the network interface is operating in full duplex mode, add the following entry into /etc/rc.local: mii-tool -F 100baseTx-FD eth0 where eth0 is the name of the network interface card (NIC). Disk I/O Settings ▼ To tune disk I/O performance for non SCSI disks 1 Test the disk speed. Use this command: /sbin/hdparm -t /dev/hdX 2 Enable direct memory access (DMA).
Tuning UltraSPARC® T1–Based Systems 2 Add the following to /etc/sysctl.conf # Disables packet forwarding net.ipv4.ip_forward = 0 # Enables source route verification net.ipv4.conf.default.rp_filter = 1 # Disables the magic-sysrq key kernel.sysrq = 0 net.ipv4.ip_local_port_range = 1204 65000 net.core.rmem_max = 262140 net.core.rmem_default = 262140 net.ipv4.tcp_rmem = 4096 131072 262140 net.ipv4.tcp_wmem = 4096 131072 262140 net.ipv4.tcp_sack = 0 net.ipv4.tcp_timestamps = 0 net.ipv4.
Tuning UltraSPARC® T1–Based Systems TABLE 5–2 Tuning 64–bit Systems for Performance Benchmarking Parameter Scope Default Value Tuned Value Comments rlim_fd_max /etc/system 65536 260000 Process open file descriptors limit; should account for the expected load (for the associated sockets, files, pipes if any).
Tuning UltraSPARC® T1–Based Systems Disk Configuration If HTTP access is logged, follow these guidelines for the disk: ■ Write access logs on faster disks or attached storage. ■ If running multiple instances, move the logs for each instance onto separate disks as much as possible. ■ Enable the disk read/write cache. Note that if you enable write cache on the disk, some writes might be lost if the disk fails.
Tuning UltraSPARC® T1–Based Systems ■ To start the 32–bit Enterprise Server with 4–Mbyte pages, use the following options: LD_PRELOAD_32=/usr/lib/mpss.so.1 ; export LD_PRELOAD_32; export MPSSHEAP=4M; ./bin/startserv; unset LD_PRELOAD_32; unset MPSSHEAP ■ To start the 64–bit Enterprise Server with 4–Mbyte pages, use the following options: LD_PRELOAD_64=/usr/lib/64/mpss.so.1; export LD_PRELOAD_64; export MPSSHEAP=4M; .
6 C H A P T E R 6 Tuning for High-Availability This chapter discusses the following topics: ■ ■ ■ “Tuning HADB” on page 107 “Tuning the Enterprise Server for High-Availability” on page 116 “Configuring the Load Balancer” on page 120 Tuning HADB The Application Server uses the high-availability database (HADB) to store persistent session state data. To optimize performance, tune the HADB according to the load of the Enterprise Server.
Tuning HADB If the database runs out of device space, the HADB returns error codes 4593 or 4592 to the Enterprise Server. Note – See Sun Java System Application Server Error Message Reference for more information on these error messages. HADB also writes these error messages to history files. In this case, HADB blocks any client requests to insert, or update data. However, it will accept delete operations. HADB stores session states as binary data.
Tuning HADB Note – hadbm does not add data devices to a running database instance. Placing HADB files on Physical Disks For best performance, data devices should be allocated on separate physical disks. This applies if there are nodes with more than one data device, or if there are multiple nodes on the same host. Place devices belonging to different nodes on different devices. Doing this is especially important for Red Hat AS 2.
Tuning HADB Performance For best performance, all HADB processes (clu_xxx_srv) must fit in physical memory. They should not be paged or swapped. The same applies for shared memory segments in use. You can configure the size of some of the shared memory segments. If these segments are too small, performance suffers, and user transactions are delayed or even aborted. If the segments are too large, then the physical memory is wasted.
Tuning HADB ■ FreeSize: free size in MB. ■ Usage: percent used. Use the hadbm resourceinfo command to monitor resource usage, for example the following command displays data buffer pool information: %hadbm resourceinfo --databuf NodeNo Avail Free Access 0 32 0 205910260 1 32 0 218908192 Misses 8342738 8642222 Copy-on-write 400330 403466 The columns in the output are: ■ Avail: Size of buffer, in Mbytes. ■ Free: Free size, when the data volume is larger than the buffer.
Tuning HADB The log records remain in the buffer until they are processed locally and shipped to the mirror node. The log records are kept until the outcome (commit or abort) of the transaction is certain. If the HADB node runs low on tuple log, the user transactions are delayed, and possibly timed out. Tuning LogBufferSize Begin with the default value. Look for HIGH LOAD informational messages in the history files.
Tuning HADB Large BLOBs necessarily allocate many disk blocks, and thus create a high load on the node internal log. This is normally not a problem, since each entry in the nilog is small. Tuning InternalLogbufferSize Begin with the default value. Look out for HIGH LOAD informational messages in the history files. The relevant messages contain nilog, and a description of the internal resource contention that occurred.
Tuning HADB Calculating the number of locks To calculate the number of locks needed, estimate the following parameters: ■ Number of concurrent users that request session data to be stored in HADB (one session record per user) ■ Maximum size of the BLOB session ■ Persistence scope (max session data size in case of session/modified session and maximum number of attributes in case of modified session). This requires setAttribute() to be called every time the session data is modified.
Tuning HADB For example, the output displayed by this command might look something like this: Node No. 0 1 Avail 50000 50000 Free 50000 50000 Waits na na ■ Avail: Number of locks available. ■ Free: Number of locks in use. ■ Waits: Number of transactions that have waited for a lock.“na” (not applicable) if all locks are available. To change the number of locks, use the following command: hadbm set NumberOfLocks The hadbm restarts all the nodes, one by one, for the change to take effect.
Tuning the Enterprise Server for High-Availability For more information on configuring the load balancer plug-in, see “Configuring the HTTP Load Balancer” in Sun GlassFish Enterprise Server 2.1 High Availability Administration Guide. HADB timeouts The sql_client time out value may affect performance.
Tuning the Enterprise Server for High-Availability Tuning Session Persistence Frequency The Enterprise Server provides HTTP session persistence and failover by writing session data to HADB. You can control the frequency at which the server writes to HADB by specifying the persistence frequency. Specify the persistence frequency in the Admin Console under Configurations > config-name > Availability Service (Web Container Availability).
Tuning the Enterprise Server for High-Availability Session Persistence Scope You can specify the scope of the persistence in addition to persistence frequency on the same page in the Admin Console where you specify persistence frequency, Configurations > config-name > Availability Service (Web Container Availability). For detailed description of different persistence scopes, see Chapter 7, “Configuring High Availability Session Persistence and Failover,” in Sun GlassFish Enterprise Server 2.
Tuning the Enterprise Server for High-Availability It is important to pay attention while determining the HTTP session size. If you are creating large HTTP session objects, calculate the HADB nodes as discussed in “Tuning HADB” on page 107. Checkpointing Stateful Session Beans Checkpointing saves a stateful session bean (SFSB) state to the HADB so that if the server instance fails, the SFSB is failed over to another instance in the cluster and the bean state recovered.
Configuring the Load Balancer For optimal performance, use a pool with eight to 16 connections per node. For example, if you have four nodes configured, then the steady-pool size must be set to 32 and the maximum pool size must be 64. Adjust the Idle Timeout and Pool Resize Quantity values based on monitoring statistics.
Configuring the Load Balancer ■ interval-in-seconds: Specifies the interval at which health checks of instances occur. The default is 30 seconds. ■ timeout-in-seconds: Specifies the timeout interval within which a response must be obtained for a listener to be considered healthy. The default is 10 seconds. If the typical response from the server takes n seconds and under peak load takes m seconds, then set the timeout-in-seconds property to m + n, as follows: PAGE 122
Index A Acceptor Threads, 69 access log, 64 AddrLookups, 62 application architecture, 19 scalability, 24 tuning, 27 arrays, 27 authentication, 21 authorization, 21 automatic recovery, 60 Average Queuing Delay, 64 B B commit option, 57 bandwidth, 94 benchmarking, tuning Solaris for, 104 best practices, 27 Buffer Length, HTTP Service, 66 caching (Continued) servlet results, 31 capacity planning, 24 checkpointing, 43, 119 class variables, shared, 30 Client ORB Properties, 73-74 Close All Connections On Any
Index deployment settings, 49 tips, 31 deserialization, 27-29 disabling network interrupts, 105 disk configuration, 105 disk I/O performance, 102 disk space, 94 distributed transaction logging, disabling, 59 DNS cache, 61-62 DNS lookups, 62, 67 dynamic reloading, disabling, 50 E EJB components cache tuning, 35-36, 36, 55-56 commit options, 57-58 monitoring individual, 34-35 performance of types, 35 pool tuning, 36, 54-55 stubs, using, 36 transactions, 38-39 EJB container, 53-58 cache settings, 55-56 cachi
Index HTTP Service (Continued) tuning, 64 HTTP sessions, 30 JSP files, 29 pre-compiling, 50 reloading, 52 tuning, 29-31 jvmstat utility, 85 I idle timeout EJB cache, 56 EJB pool, 55 IIOP Client Authentication Required, 72 IIOP messages, 74-75 Initial Thread Count, HTTP Service, 65 InternalLogbufferSize, 112-113 ip:ip_squeue_bind, 104 ip:ip_squeue_fanout, 104 IP stack, 99 ipge:ipge_bcopy_thresh, 104 ipge:ipge_srv_fifo_depth, 104 ipge:ipge_taskq_disable, 104 ipge:ipge_tx_ring_size, 104 ipge:ipge_tx_syncq,
Index monitoring (Continued) transaction service, 58 processors, 93 programming guidelines, 27-29 promptness, 86 N NameLookups, 62 Network Address, 69 network configuration, 105 network interface, 102 network interrupts, disabling, 105 NewRatio, 88 NewSize, 88 Node Supervisor Process (NSUP), 109 null, assigning, 28 NumberOfLocks, 113-115 O open files, 97, 101 operating system, tuning, 93-106 operational requirements, 19-23 ORB, 70-76 Client properties, 73-74 IIOP Client Authentication Required, 72 Max M
Index session (Continued) timeout, 51 Small/Medium File Size, HTTP file cache, 68 SOAP attachments, 29 Solaris JDK, 85 TCP/IP settings, 95 tuning for performance benchmarking, 104 version 9, 31 sq_max_size, 95, 104 SSL, 21 start options, 105-106 stateful session beans, 42-43, 119 stateless session beans, 43 storing persistent session state, 107 StringBuffer, 27-28 Strings, 27-28 -sun.rmi.dgc.client.gcInterval, 87 Survivor Ratio Sizing, 89 synchronizing code, 29 System.
Index X x86, 98 XA-capable data sources, 38-39 -Xms, 88 -Xmx, 88 -XX +DisableExplicitGC, 87 MaxHeapFreeRatio, 88 MaxPermSize, 86 MinHeapFreeRatio, 88 128 Sun GlassFish Enterprise Server 2.
