Configuring Dynamic Instrumentation
To configure dynamic instrumentation, select Configuration -> Configure Dynamic Instrumentation on the top menu. This section will allow you to deploy or undeploy profiling agent, configure Dynamic Instrumentations and optionally set up method profiling.
Enable Agents on Server
To Ensure profiling and monitoring function correctly, the Applicare profiling agent must be enabled on the app server. This can be accomplished on the IntelliTrace Deployment tab by selecting the appropriate server from the list on the right.
Disable Dynamic Profiling
Selecting this option will disable all instrumentation on all the attached agents.
Standard Profiling
This option activates HTTP, SQL and JDBC connection profiling on the application server or JVMs where the agent is enabled.
Advanced Mode
In advanced mode, you can select and deploy individual aspects. For example you maybe only interested in detecting JDBC Connection leaks and might want to improve performance by taking out the other aspects you don't want.
Following are the available aspects.
1. HTTP Profiling
2. HTTP Error Code Profiling
3. SQL Profiling
4. JDBC Connection Profiling
5. Business Transactions
6. IntelliTrace profiling - Deploying or undeploying IntelliTrace profiling requires a restart of the agent server for change to take effect.
7. Method Profiling
8. External WebServices
9. End User Experience Profiling
10. End Client Process Monitoring (Slow Calls only)
11. End Client Process Monitoring (Every Call)
12. JNDI Lookup
13. External JMS Messages
14. LDAP Calls Profiling
15. Mail Monitoring
16. Collections Monitoring
17. Network Connection Profiling
18. Heap Tracker (BETA)
Please refer to the section on InelliTrace Configuration (for Applicare 6.9.0 and later) & Enabling IntelliTrace on Production Servers for guidance using IntelliTrace in production environments.
Deploy Aspects In Advanced Mode
1. Choose the desired aspects from the Available Aspects list and click the Add button.
2. The selected Aspects will be transferred to the Deployed Aspects list.
3. Click Save button and apply the configuration.
Deploying or undeploying IntelliTrace profiling requires the agent servers to be restarted for the changes to take effect.
Undeploy Aspects In Advanced Mode
1. Select the aspects you want to undeploy from the Deployed Aspects list and click the Remove button.
2. Selected Aspects will be removed to the Deployed Aspects list.
3. Click Save button to save the configuration.
Per Server Advanced Mode
In this mode, each server can have its own customized set of aspects.
Activating IntelliTrace for Specific Applications or a Group of Applications
Starting with Applicare 6.9.0 dynamic IntelliTrace can be enabled for an app or a particular set of apps. This helps only focus intrumentation on important applications and keep the overhead extremly low.
This is enabled by adding ",perappinstrumentation=true,perappinstapps=/JPetStore:/qos" to the Applicare -javaagent argument on app server start. In this example JPetStore and qos are the applications that will be instrumented by Applicare Profiling Agent. Since app servers allow you to deploy two different apps as JPetStore and jpetstore, application name for this should be same case as application name for deployment in app server. Application name here is case sensitive.
Depending upon how you pass Applicare startup arguments either you will need to modify setAgentEnv.sh /setAgentenv.bat file in scripts directory or changes will need to be made to however you pass Applicare arguments to your app server. If you are using WebLogic and passing the startup arguments via 'Server Start' tab under configuration then you will make this change there.
-javaagent:$ARCTURUS_HOME/applicare-aspects.jar=verbosity=0,logfile=false,perappinstrumentation=true,perappinstapps=/JPetStore:/qos -Darcturus_home=$ARCTURUS_HOME "
Static IntelliTrace instrumention on a particular app has been available on Applicare since beginning but that requires instrumenting the application jars and redeploying them. With this perapp dynamic instrumentation there is no need to instrument the jar and redeploy the instrumented jar.
Intellitrace Configuration
IntelliTrace configuration provides precise control on what packages are instrumented and what packages should be excluded from instrumentation. Instrumenting an excessive no of packages or classes can impact performance and only necessary packages should be instrumented. This feature is added in Applicare 6.9.0 version and is not available in prior versions.
Diagnostic mode enables you to see packages and how many classes would be instrumented by the current settings, without actually performing the instrumentation. This is helpful for app servers where many packages/apps are deployed and server startup time and performance needs to be tuned.
When Diagnostic mode is enabled, no classes are instrumented but all the data shown on IntelliTrace Configuration tab is gathered. You should use this mode to find out what packages/classes are instrumented and fine tune the list of packages that are required for desirable results.
By default the packages in the file "trace-exclusions.txt" in the applicare home directory are considered as the exclusion list. These packages are not transformed. If you update the package details here on IntelliTrace Configuration tab and "apply" changes, then a new exclusion file is created in the applicare home directory with the name "trace-exclusions_$ServerName" and exclusion package details are persisted in this file. These changes take effect when the app server is restarted.
Threshold limit: This limits the # of classes intrumented by Applicare. This is meant to prevent app server start taking too long in case many apps are deployed on it. Classes are transformed until the threshold limit is reached and after that no classes are transformed. A message is shown in IntelliTrace and BTM Summary tab for each server where this limit is reached.
Diagnostic mode: When diagnostic mode is enabled, no classes will be transformed. If diagnostic mode is not enabled, classes from packages not listed in the exclusion list will be transformed up to the threshold limit.
Force apply: This options allows you to re-instrument the classes without needing to restart the application server.
Instrumentation count details: This displays the number of transformed/untransformed classes/packages.
Packages included in instrumentation: This displays the list of packages which "included in instrumentation".
Packages excluded from instrumentation: This shows the list of packages which "excluded from instrumentation".
Right arrow: This moves the selected packages from "included packages from instrumentation" to "excluded packages from instrumentation".
Left arrow: This moves the selected packages from "excluded packages from instrumentation" to "included packages in instrumentation".
If the exclude packages list is empty by default, the following packages will be excluded:
"javax.", "org.apache.commons.", "weblogic.", "com.bea.", "bea.", "jrockit.", "com.certicom.", "org.quartz.", "com.arcturus.", "kodo.", "com.rsa.", "org.omg.", "com.octetstring.", "com.google.", "org.snmp4j.", "org.nsclient4j.", "com.tangosol."
i) All changes applied by clicking the "Apply" button in the bottom right of the page take effect after app server restart. But if "Foce Apply" checkbox is enabled, then it will take effect without app server restart.
ii) In case if the changes are applied (without Force Apply) when app server is down, then app server will run with older changes when it starts. Otherwise if the changes are applied with Force Apply when app server is down, then the app server will run with the latest changes when it starts.
iii) If you remove the file "trace-exclusions_$ServerName.txt" in the applicare home directory, then the packages in the file "trace-exclusions.txt" will be considered as exclusion list.
iv) Below packages are not allowed to be added to inclusion list/included in instrumentation:
"javax.", "org.apache.commons.", "org.apache.struts", "com.bea.", "bea.", "jrockit.", "com.certicom.", "org.quartz.", "com.arcturus.", "kodo.", "com.rsa.", "org.omg.", "com.octetstring.", "com.google.", "org.snmp4j.", "org.nsclient4j.", "com.tangosol.", "weblogic."
Method Profiling Configuration
Go to the Method Profiling Configuration tab to create or modify the filter criteria for Method Profiling Aspect.
Method Type: Select the type of methods to filter (All, public, private or protected)
Return Type: Return type of the methods (eg. java.lang.String ). Use * to match all types including void.
Package/Class/Interface: Expression specifying the package, class or interface name
Eg.
org.apache..* - Match all classes and method names in org.apache package and sub packages
org.apache.*.* - Match all classes and method names in org.apache package
org.apache.commons.StringUtils.escape - Match methods named 'escape' in org.apache.commons.StringUtils class.
Parameter Type(s): Parameter types of the methods. Use .. to match all parameter types
Eg. java.lang.String, java.lang.Long � Match methods accepting String and a Long as a parameter
Inverse Filter: When selected, adds a logical NOT condition to the filter
Add your conditions by clicking the + button. You can create a complex filter by including multiple conditions as shown in the screenshot. Once all the conditions are added to the list, save the configuration by clicking the Save button.
Additionally, remember to deploy the Method Profiling Aspect from the Profiling Deployment tab, if it is not already deployed.
IntelliTrace Analyzer
The IntelliTrace Analyzer features distinct views for displaying data collected by IntelliTrace aspects. To access the IntelliTrace Analyzer, click on Server Analyzer and then select IntelliTrace.
SQL View
SQL Statement Profiling provides real-time performance statistics of database requests. You can view what statements are taking the most time and quickly diagnose performance impacting SQL/s.
Errors
The Errors view shows any SQL, JNDI, HTTP, LDAP and WebServices errors that occured on the selected server for the selected time period. Error detection is enabled when respective profiling is enabled. Selecting a SQL in the table displays the stack trace of the code that invoked the SQL. SQL Error detection allows you to quickly track down SQL issues to the exact line in the code plus the reason for the failure.
JDBC Connection View
The JDBC Connection Profiling feature allows you to detect leaked JDBC connections, connections held open for long periods of time and code responsible for connection leaks. You can view the connections that are currently in open state and clicking on an open connection will display the call stack which requested the connection. Statistics displayed are for the selected server in the UI.
HTTP View
HTTP Profiling feature allows you to easily view the performance statistics of real-time HTTP requests on-the-fly. Statistics displayed are for the selected server in the UI.
IntelliTrace View
IntelliTrace decides what methods to track dynamically and decides the depth of the call stack based on invocation counts and time spent on each individual method to minimize the overhead of tracing. IntelliTrace does what the real experts do with the profiling data - iliminate data that has little impact but it does it in real time. This way no time/resources are spent on gathering data that doesn't have meanigful impact on performance.
The Tree view displays the invocations as a call graph and shows the invocation count of each method and the total time spent within the methods. HTTP requests and SQLs are displayed inline with the URL and SQL prefixes respectively.
The three pie charts below displays following data:
1. Children Methods - Time Spent: Displays the absolute time spent in each child method below the selected point in call graph.
2. Children Methods - Invocation Count: Shows the total number of times each child method below the selected point in call graph.
3. Cumulative Time (Code vs Database): Indicates the total time spent in Java code and Database call below the selected point in call graph.
Initial IntelliTrace view charts show time spent on selected calls e.g. 94.04 % time went into viewCategory and 5.535 % in viewItem.
Selecting the Hotspots tab in IntelliTrace view displays performance data of all children methods below the selected point in call graph. This is useful in identifying the most expensive calls in a call graph without drilling down through the tree view. Hotspot view shows percentile data.
IntelliSense View
IntelliSense is a feature that, works when IntelliTrace is enabled, to automatically detect transaction executions that deviate considerably from the normal execution times and saves a complete call graph of the execution for later analysis.
Method Profiling
The Method Profiling feature provides you to easily view the performance statistics of real-time method executions on-the-fly. This Profiling Aspect needs to be configured and deployed before you can view performance data. See Method Profiling Configuration for more information on configuration.
Parameter Value Masking
The Parameter Value Masking feature allows you to mask the value of HTTP parameters in Applicare. This way you can hide user's sensitive information.
Enabling IntelliTrace on Production Servers
IntelliTrace was designed to run on production servers with very minimal overhead so that the end users don't experience a noticeable performance degradation. However on large applications where lot of third party libraries are used, it is recommended to enable IntelliTrace on a test server and perform the following optimization before enabling IntelliTrace on production servers.
Basically you can tune IntelliTrace to ignore third party libraries altogether by adding the package names of the third party code to trace-exclusions.txt file in Applicare home directory. In most cases you are not interested in the internal working of these third party libraries and excluding them from tracing will reduce the overhead significantly.
Enable IntelliTrace on the test server (requires restart) and run through few different transactions or run a load script. Now in IntelliTrace window if you expand the transactions to view call graphs you may see third party classes/methods in the call graph.
Open trace-exclusions.txt file in a text editor and you can see that we have already added most of the common libraries such as Spring framework and Apache Commons in this file. So if you see some third party classes/methods in you IntelliTrace window that you are not concerned about, you should add the base package name of that library to trace-exclusions.txt in a new line.
e.g. To disable tracing all classes in org.apache and its sub packages, you should add the following line
org.apache.
Remember to copy the modified file to all agent Applicare directories on other machines too and you will have to restart application servers for the changes to take effect.
Now you can use the modified file in production Applicare installation to get best performance while IntelliTrace is enabled. Also you can send the packages you want excluded to Arcturus Tech so they will be included in trace-exclusions.txt file by default in future versions.
Enable or Disable IntelliTrace Profiling and Business Transaction Tracing Without Restarts
By default enabling / disabling IntelliTrace Profiling and Business Trasaction tracing requires affected servers to be restarted for the changes to take effect. If for some reason you need to apply these changes without restarting, you can do so by editing the relevant agent startup script in Applicare/scripts/ folder (e.g. for Tomcat servers on Windows, setTomcatAgentEnv.bat) and change norestart-intellitrace=false to norestart-intellitrace=true. Enabling this flag is not recommended for production servers. On a loaded server with this flag set to true when you enable IntelliTrace the server may stop responding for few minutes. This happens because on enabling IntelliTrace Applicare Profiling agent would start instrumenting classes and that has impact on server performance.
Click here for Custom HTTP Tags help and here for Remote Method Definitions.
Comments
0 comments
Article is closed for comments.