Package org.cougaar.core.qos.profile

Utility components to periodically log performance metrics, such as the cpu load and message traffic.

See:
Description

Interface Summary

BlackboardSizeService	This service registers agents with the node-level BlackboardSize profiler.
BlackboardSizeService.Client
ProfilerService	This service registers profilers with the ProfilerCoordinator
ProfilerService.Client
ServiceFinder.Callback

Class Summary

BlackboardSize	This component profiles the per-agent blackboard size and add/change/remove counts for standard blackboard object types (UniqueObject, Task, etc).
BlackboardSizeInAgent	This component supports the node-level BlackboardSize profiler.
JavaHeapSize	This component profiles the JVM heap size (used, free, total, max).
Load	This component profiles the MetricsService's load metrics for the cpu, messaging, and persistence load of the node aggegate, each agent, and core services (mts & metrics).
LocalTraffic	This component profiles the message traffic (message count and byte count) of each local agents to any target.
NodeTraffic	This component profiles the aggregate message traffic of all local agents to any target.
PersistSize	This component profiles persistence activity for each agent and the aggegate node "sum".
ProcLoadAvg	This component profiles the CPU load from "/proc/loadavg".
ProcMemInfo	This component profiles system memory usage from "/proc/meminfo".
ProcNetDev	This component profiles system network activity from "/proc/net/dev".
ProcStat	This component profiles system activity from "/proc/stat".
ProcStatus	This component profiles the JVM's process size from "/proc/self/status" (VmSize & VmRSS).
ProfilerBase	This component is the common base class for profiler components.
ProfilerCoordinator	This component coordinates the profiler components to log at the same time.
RemoteTraffic	This component profiles the message traffic (message count and byte count) from the local aggregated agents to each specific remote target agent.
Resources	This component profiles the MetricsService's summary resource metrics (load average, tcp use, etc).
ServiceFinder	A utility class to simplify late-binding service lookup.
ServiceProviderBase	A base class for ServiceProviders.
Throughput	This component profiles the aggregate "messages per second" throughput for all agents on the local node.
TrafficMatrix	This component profiles the agent traffic matrix for all local-to-local and local-to-remote agent messaging.
VmStat	This component profiles the output of the "/usr/bin/vmstat" command, which summarizes the system performance.

Package org.cougaar.core.qos.profile Description

Utility components to periodically log performance metrics, such as the cpu load and message traffic.

The simplest configuration option is to profile the aggregate "messages per second" throughput. Add the following components to all nodes:

  <!-- required profiler-management component -->
  <component
      class='org.cougaar.core.qos.profile.ProfilerCoordinator'
      priority='HIGH'
      insertionpoint='Node.AgentManager.Agent.Component'>
    <!-- optional period between logs, defaults to 60000 millis -->
    <argument>period=10000</argument>
  </component>

      <!-- message transport sensor -->
      <component
          class='org.cougaar.mts.std.StatisticsAspect'
          insertionpoint='Node.AgentManager.Agent.MessageTransport.Aspect'/>

  <!-- messaging throughput profiler -->
  <component
      class='org.cougaar.core.qos.profile.Throughput'/>

Example output:

2005-04-29 13:23:39,297 SHOUT [throughput] - NodeA: #agent_send_count_per_second, total_send_count_per_second, agent_send_count, total_send_count,
2005-04-29 13:23:39,362 SHOUT [throughput] - NodeA: 0.00, 0.00, 0, 3
2005-04-29 13:23:49,103 SHOUT [throughput] - NodeA: 0.56, 1.78, 5, 19
2005-04-29 13:23:59,101 SHOUT [throughput] - NodeA: 27.22, 27.22, 250, 264
2005-04-29 13:24:09,102 SHOUT [throughput] - NodeA: 121.50, 121.50, 1465, 1479
2005-04-29 13:24:19,101 SHOUT [throughput] - NodeA: 160.89, 161.11, 2913, 2929

In the above example, the agents on NodeA are sending an aggregate 161 messages/second. This output is from a two-node "ping" test from AgentA on NodeA to a remote AgentB on NodeB.

For more detailed profiling, add the following components to all nodes:

  <!-- required profiler-management component -->
  <component
      class='org.cougaar.core.qos.profile.ProfilerCoordinator'
      priority='HIGH'
      insertionpoint='Node.AgentManager.Agent.Component'/>

  <!-- mts components -->
      <component
          class='org.cougaar.mts.std.StatisticsAspect'
          insertionpoint='Node.AgentManager.Agent.MessageTransport.Aspect'/>
      <component
          class='org.cougaar.core.qos.tmatrix.AgentFlowAspect'
          insertionpoint='Node.AgentManager.Agent.MessageTransport.Aspect'/>

  <!-- optional mts-internal profilers (select none, any, or all) -->
  <component
      class='org.cougaar.core.qos.profile.LocalTraffic'
      insertionpoint='Node.AgentManager.Agent.MessageTransport.Component'/>
  <component
      class='org.cougaar.core.qos.profile.RemoteTraffic'
      insertionpoint='Node.AgentManager.Agent.MessageTransport.Component'/>
  <component
      class='org.cougaar.core.qos.profile.Throughput'
      insertionpoint='Node.AgentManager.Agent.MessageTransport.Component'/>
  <component
      class='org.cougaar.core.qos.profile.TrafficMatrix'
      insertionpoint='Node.AgentManager.Agent.MessageTransport.Component'/>

  <!-- metrics components and plugins -->
      <component
          class='org.cougaar.core.qos.rss.AgentHostUpdaterComponent'
          insertionpoint='Node.AgentManager.Agent.MetricsServices.Component'/>
      <component
          class='org.cougaar.lib.mquo.SyscondFactory'
          insertionpoint='Node.AgentManager.Agent.MetricsServices.Component'/>
      <component
          class='org.cougaar.core.qos.rss.ConfigFinderDataFeedComponent'
          insertionpoint='Node.AgentManager.Agent.MetricsServices.Component'>
        <argument>name=sites</argument>
        <argument>url=cougaarconfig:TIC-Sites.conf</argument>
      </component>
      <component
          class='org.cougaar.core.qos.rss.ConfigFinderDataFeedComponent'
          insertionpoint='Node.AgentManager.Agent.MetricsServices.Component'>
        <argument>name=hosts</argument>
        <argument>url=cougaarconfig:TIC-Hosts.conf</argument>
      </component>

      <component
          class='org.cougaar.core.qos.tmatrix.AgentFlowRatePlugin'/>
      <component
          class='org.cougaar.core.qos.metrics.AgentStatusRatePlugin'/>
      <component
          class='org.cougaar.core.thread.AgentLoadSensorPlugin'/>
      <component
          class='org.cougaar.core.thread.AgentLoadRatePlugin'/>
      <component
          class='org.cougaar.mts.std.StatisticsPlugin'/>
  
  <!-- optional profilers (select none, any, or all) -->
  <component
      class='org.cougaar.core.qos.profile.BlackboardSize'/>
  <component
      class='org.cougaar.core.qos.profile.JvmHeapSize'/>
  <component
      class='org.cougaar.core.qos.profile.Load'/>
  <component
      class='org.cougaar.core.qos.profile.NodeTraffic'/>
  <component
      class='org.cougaar.core.qos.profile.PersistSize'/>
  <component
      class='org.cougaar.core.qos.profile.ProcLoadAvg'/>
  <component
      class='org.cougaar.core.qos.profile.ProcNetDev'/>
  <component
      class='org.cougaar.core.qos.profile.ProcMemInfo'/>
  <component
      class='org.cougaar.core.qos.profile.ProcStat'/>
  <component
      class='org.cougaar.core.qos.profile.ProcStatus'/>
  <component
      class='org.cougaar.core.qos.profile.Resources'/>
  <component
      class='org.cougaar.core.qos.profile.VmStat'/>

  <!-- optional, required if above BlackboardSize is loaded -->
  <component
      class='org.cougaar.core.qos.profile.BlackboardSizeInAgent'/>

Plus add the following components to all agents:

  <!-- optional, required if above PersistSize is loaded -->
  <component 
      class='org.cougaar.core.qos.metrics.PersistenceAdapterPlugin'/>

  <!-- optional, required if above BlackboardSize is loaded -->
  <component
      class='org.cougaar.core.qos.profile.BlackboardSizeInAgent'/>

Some useful system properties:

  <!-- enable persistence, for PersistSize activity -->
  <vm_parameter>
    -Dorg.cougaar.core.persistence.enable=true
  </vm_parameter>

  <!-- disable the "NodeA: " logging prefix -->
  <vm_parameter>
    -Dorg.cougaar.core.logging.addAgentPrefix=false
  </vm_parameter>

  <!-- turn off +/- logging on message activity -->
  <vm_parameter>
    -Dorg.cougaar.core.agent.quiet=true
  </vm_parameter>

  <!-- reduce naming & mts startup time -->
  <vm_parameter>
    -Dorg.cougaar.core.wp.resolver.rmi.minLookup=500
  </vm_parameter>
  <vm_parameter>
    -Dorg.cougaar.core.wp.resolver.rmi.maxLookup=2000
  </vm_parameter>
  <vm_parameter>
    -Dorg.cougaar.core.wp.server.failTTD=1000
  </vm_parameter>
  <vm_parameter>
    -Dorg.cougaar.core.mts.destq.retry.maxTimeout=30000
  </vm_parameter>