PingAM 8.0.0

Maintenance

This guide covers how to perform maintenance tasks in PingAM such as backing up and restoring, monitoring, and others.

This guide is written for anyone that sets up and maintains PingAM services for their organizations. This guide covers tasks and configurations you might repeat throughout the life cycle of a deployment in your organization.

Back up and restore configurations

How to back up and restore AM.
Monitor instances

Monitor AM through any of the included interfaces.
Tune instances

Best practices about tuning your AM environment.
Enable debug logging

Capture troubleshooting information.

Name changes for ForgeRock products

Product names changed when ForgeRock became part of Ping Identity.

The following name changes have been in effect since early 2024:

Old name New name

ForgeRock Identity Cloud

PingOne Advanced Identity Cloud

ForgeRock Access Management

PingAM

ForgeRock Directory Services

PingDS

ForgeRock Identity Management

PingIDM

ForgeRock Identity Gateway

PingGateway

Learn more about the name changes in New names for ForgeRock products in the Knowledge Base.

Back up configurations

During normal production operations, you rely on directory replication to maintain multiple, current copies of AM’s configuration. To recover from the loss of a server or from a serious administrative error, back up directory data and configuration files.

Find information on backing up your configuration directory server in Backup and Restore in the DS documentation.

Back up instance configuration data

This procedure backs up the configuration files stored with the server. You can restore this backup when rebuilding a failed server.

Consider the following when using this procedure:

  • Refer to the documentation for your external directory server or work with your directory server administrator to back up and restore configuration data stored in the directory server.

    For PingDS, find information in Backup and restore in the DS documentation.

  • Do not restore configuration data from a backup of a different major version of AM. The structure of the configuration data can change from release to release.

Follow these steps for each AM server that you want to back up:

  1. Stop AM or the container in which it runs.

  2. Back up AM server files.

    This example uses the default configuration location, and excludes logs. $HOME is the home directory of the user who runs the web container where AM is deployed. AM is deployed in Apache Tomcat under am:

    $ cd $HOME
$ zip -r AM-config-dir-backup-`date -u +%F-%H-%M`.zip am .openamcfg/* \
  -x am/var/debug/* am/var/audit/* am/var/stats*
…​
$ ls AM-config-dir-backup-*.zip
AM-config-dir-backup-2022-10-01-05-07-50.zip

  3. Start AM or the container in which it runs.

Monitor AM instances

AM provides a number of interfaces for you to check the status of AM instances and to gather monitoring data. This section describes the steps to enable monitoring and use the monitoring endpoints.

Check the status of an AM instance

In certain deployments, such as Kubernetes, it can be useful to poll endpoints to check if an instance is running, and ready to handle requests. You can determine the status of an AM instance by using the following endpoints:

/json/health/live

Use the live endpoint to determine if AM instances are up and running.

If the instance is running the endpoint returns an HTTP status code of 200. If not, it returns a 503 response.

For example, use the following curl command to determine when an AM instance is alive:

$ curl --include --retry 10 --retry-connrefused 'https://am.example.com:8443/am/json/health/live'
Warning: Transient problem: connection refused Will retry in 1 seconds. 10
Warning: retries left.
Warning: Transient problem: connection refused Will retry in 2 seconds. 9
Warning: retries left.
Warning: Transient problem: connection refused Will retry in 4 seconds. 8
Warning: retries left.
HTTP/1.1 200
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
Cache-Control: private
Content-API-Version: resource=1.0
Content-Length: 0
Date: Mon, 09 Nov 2020 12:22:38 GMT
/json/health/ready

Use the ready endpoint to determine if AM instances are ready to process requests.

If the instance is ready to process requests, the endpoint returns an HTTP status code of 200. If not, it returns a 503 response.

For example, use the following curl command to determine when an AM instance is ready to process requests:

$ curl --include --retry 10 --retry-connrefused 'https://am.example.com:8443/am/json/health/ready'
Warning: Transient problem: connection refused Will retry in 1 seconds. 10
Warning: retries left.
Warning: Transient problem: connection refused Will retry in 2 seconds. 9
Warning: retries left.
Warning: Transient problem: connection refused Will retry in 4 seconds. 8
Warning: retries left.
Warning: Transient problem: connection refused Will retry in 8 seconds. 7
Warning: retries left.
HTTP/1.1 200
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
Cache-Control: private
Content-API-Version: resource=1.0
Content-Length: 0
Date: Mon, 09 Nov 2020 12:45:03 GMT
Deprecated isAlive.jsp page

AM provides a deprecated isAlive.jsp page, to check whether AM is up. Point your application to the file under the deployment URL, such as https://am.example.com:8443/am/isAlive.jsp.

If you get a success code (with Server is ALIVE: in the body of the page returned), then the instance is in operation.

The isAlive.jsp page is deprecated and will be removed in a future release. Update your environment to use the live and ready endpoints instead.

The endpoints and isAlive.jsp page do not require authentication. You might want to restrict public access, and only allow access from internal infrastructure.

Enable monitoring in the console

To query some of the monitoring endpoints, such as Prometheus or CREST, you must enable the Monitoring service:

  1. In the AM admin UI, go to Configure > Global Services > Monitoring.

  2. Set Monitoring Status to enabled.

  3. Click Save Changes.

Learn about configuring the service in Monitoring.

JMX monitoring

This functionality is deprecated.

You can configure AM to let you listen for Java Management eXtension (JMX) clients, by default on port 9999. Either go to Configure > Global Services > Monitoring in the AM admin UI and make sure both Monitoring Status and Monitoring RMI interface status are enabled, or use the ssoadm command:

$ ssoadm \
set-attr-defs \
--servicename iPlanetAMMonitoringService \
--schematype Global \
--adminid uid=amAdmin,ou=People,dc=am,dc=example,dc=com \
--password-file /tmp/pwd.txt \
--attributevalues iplanet-am-monitoring-enabled=true \
iplanet-am-monitoring-rmi-enabled=true

A number of tools support JMX, including jvisualvm and jconsole. When you use jconsole to browse AM MBeans for example, the default URL for the AM running on the local system is service:jmx:rmi:///jndi/rmi://localhost:9999/server.

$ jconsole service:jmx:rmi:///jndi/rmi://localhost:9999/server &

To browse MBeans, connect to your web application container, and go to the AM MBeans. By default, JMX monitoring for the container is likely to be accessible only locally, using the process ID.

You can monitor an instance over JMX.
Figure 1. JConsole browsing MBeans

Also refer to Monitoring and Management Using JMX for instructions on how to connect remotely, how to use SSL, and so on.

JMX has a limitation in that some Operations and CTS tables cannot be properly serialized from AM to JMX. As a result, only a portion of AM’s monitoring information is available through JMX.

Use Prometheus, Graphite, or common REST monitoring if possible.

For monitoring metrics reference, refer to Monitoring metrics.

Monitor with Prometheus

Prometheus is third-party software used for gathering and processing monitoring data.

Prometheus can monitor and process AM metrics but your deployment might benefit from further analysis and visualization. You can use tools such as Grafana to create customized charts and graphs based on the information Prometheus collects.

Learn more about installing and running Grafana in the Grafana documentation.

Prometheus endpoints

AM exposes endpoints that Prometheus uses to gather metrics from the AM instance.

When you enable Prometheus monitoring, AM makes the Prometheus-formatted metrics available at the following endpoints:

  • /metrics/prometheus

    The path of this endpoint is format-agnostic, but the response payload is identical to that from the /json/metrics/prometheus endpoint.

    Although this endpoint is new, it is also deprecated in this release and support for its use will be removed in a future release. Move to the /metrics/prometheus/0.0.4 endpoint as soon as convenient.

  • /metrics/prometheus/0.0.4

    The path of this endpoint is format-agnostic, but the response payload is slightly different to that from the /metrics/prometheus endpoint.

    Metrics that were previously suffixed with _total are suffixed with _sum, which conforms better to the latest version of Prometheus. For example:

    • /metrics/prometheus/0.0.4 returns the metric am_authorization_policy_set_evaluate_seconds_sum{outcome=outcome,policy_set=policy-set,}

    • /metrics/prometheus returns the metric am_authorization_policy_set_evaluate_seconds_total{outcome=outcome,policy_set=policy-set,}

The /json/metrics/prometheus endpoint is deprecated. If you’re using this endpoint, change your Prometheus configuration to use one of the new endpoints:

  • If your deployment doesn’t rely on the response payload provided from the /json/metrics/prometheus endpoint, move straight to the /metrics/prometheus/0.0.4 endpoint.

  • If your deployment relies on the exact response payload provided from the /json/metrics/prometheus endpoint, move to the /metrics/prometheus endpoint first. Adapt your deployment to expect the different payload then move to the /metrics/prometheus/0.0.4 endpoint.

Configure Prometheus

Configure Prometheus to monitor the AM endpoints in the Prometheus configuration file, prometheus.yml.

Learn more about configuring Prometheus in the Prometheus configuration documentation.

Learn more about PingAM monitoring metrics in Monitoring metrics.

Enable Prometheus monitoring

  1. Ensure you have enabled monitoring.

  2. Go to Configure > Global Services > Monitoring.

  3. On Secondary Configurations tab, click prometheus, then select Enabled.

  4. In the Authentication Type menu, select one of the following options:

    • None. Prometheus doesn’t need to authenticate when accessing the endpoint.

    • HTTP Basic. Prometheus must authenticate using a username and a password when accessing the endpoint.

  5. If you selected HTTP Basic, specify a Username and Password.

    If you set a Secret Label Identifier and AM finds a matching secret in a secret store, the Password is ignored.

  6. For greater security, define a Secret Label Identifier to use a secret in a secret store and rotate the secret periodically.

    AM uses this identifier to create a specific secret label, using the template am.services.monitoring.prometheus.identifier.secret where identifier is the value of Secret Label Identifier.

    The identifier can only contain alphanumeric characters a-z, A-Z, 0-9, and periods (.). It can’t start or end with a period.

    Learn more about using secrets in Map and rotate secrets.

  7. Save your work.

Common REST monitoring

Common REST refers to the REST framework supported by all Ping Identity Platform products. AM exposes an endpoint that lets REST clients gather information about your AM installation in JSON format.

When enabled, AM makes the common REST-formatted metrics available at the /json/metrics/api endpoint.

For example, to query all monitoring metrics:

$ curl \
--request GET \
--header 'Content-Type: application/json' \
--header 'iPlanetDirectoryPro: P8Eri6d…​xAAA.*' \
'https://am.example.com:8443/am/json/metrics/api?_queryFilter=true&_prettyPrint=true'
{
  "result" : [ {
    "_id" : "session.authentication-client-based.get-matching-sessions.failure",
    "count" : 0,
    "max" : 0.0,
    "mean" : 0.0,
    "min" : 0.0,
    "p50" : 0.0,
    "p75" : 0.0,
    "p95" : 0.0,
    "p98" : 0.0,
    "p99" : 0.0,
    "p999" : 0.0,
    "stddev" : 0.0,
    "m15_rate" : 0.0,
    "m1_rate" : 0.0,
    "m5_rate" : 0.0,
    "mean_rate" : 0.0,
    "duration_units" : "milliseconds",
    "rate_units" : "calls/second",
    "total" : 0.0,
    "_type" : "timer"
  },
  …​

To output values for a specific metric, use the format /json/metric/api/metric name.

For example:

$ curl \
--request GET \
--header 'Content-Type: application/json' \
--header 'iPlanetDirectoryPro: P8Eri6d…​xAAA.*' \
'https://am.example.com:8443/am/json/metrics/api/authentication.success'
{
    "_id":"authentication.success",
    "m15_rate":2.3882880682497324E-4,
    "m1_rate":1.0818067450729532E-12,
    "m5_rate":3.057223961594952E-5,
    "mean_rate":1.072595825176473E-4,
    "units":"events/second",
    "total":9.0,
    "count":9,
    "_type":"summary"
}

For monitoring metrics reference, refer to Monitoring metrics.

Enable the common REST monitoring interface

  1. Ensure you have enabled monitoring.

  2. Go to Configure > Global Services > Monitoring.

  3. On the Secondary Configurations tab, click crest.

  4. Set CREST to Enabled.

  5. Click Save Changes.

Graphite monitoring

Graphite is third-party software used for storing monitoring data, and rendering graphs of the data. For more information about installing and running Graphite, see the Graphite documentation.

For monitoring metrics reference, refer to Monitoring metrics.

Enable the Graphite monitoring interface

  1. Ensure you have enabled monitoring.

  2. Go to Configure > Global Services > Monitoring.

  3. On the Secondary Configurations tab, click Add a Secondary Configuration.

  4. Select Graphite Reporter.

  5. Specify the Name and Hostname of the Graphite instance to push the metrics data to.

  6. Click Create.

MBean monitoring (legacy)

This legacy functionality is deprecated.

You can configure AM to let you access a web-based view of AM MBeans on port 8082 where the core server runs, such as https://am.example.com:8443/. Either use the console (Configure > Global Services > Monitoring), or use the ssoadm command:

$ ssoadm \
set-attr-defs \
--servicename iPlanetAMMonitoringService \
--schematype Global \
--adminid uid=amAdmin,ou=People,dc=am,dc=example,dc=com \
--password-file /tmp/pwd.txt \
--attributevalues iplanet-am-monitoring-http-enabled=true

The default authentication file lets you authenticate over HTTP as user demo, password changeit. The user name and password are kept in the file specified, with the password encrypted:

$ cat openam/security/openam_mon_auth
demo AQICMBCKlwx6G3vzK3TYYRbtTpNYAagVIPNP

You can encrypt a new password using the ampassword command. After changing the authentication file, you must restart AM for the changes to take effect.

You can monitor an instance through a web browser.
Figure 2. MBeans in a browser

Trace incoming and outgoing requests

When a user interacts with Ping Identity Platform, the request can travel through multiple services before it completes. Distributed tracing lets you monitor the request flow through Ping Identity Platform.

Tracing provides a single view of a request’s journey and makes it easier to locate bottlenecks and errors. If issues arise, tracing makes it easier to identify the service causing the problem. It’s more efficient and effective than sifting through isolated logs.

By identifying slow services, tracing helps you optimize application performance and reduce debugging time. This improves the end user experience as users are less likely to encounter errors or slow loading times.

AM supports the OpenTelemetry framework (OTEL) for collecting distributed tracing data.

Which requests are traced?

AM supports distributed tracing for the following request types:

  • Incoming HTTP requests

  • Outgoing HTTP requests to PingIDM (Ping Identity Platform deployments only)

  • Outgoing LDAP requests

    These requests are searchable and identifiable for the following LDAP operations:

    • ADD

    • MODIFY

    • SEARCH

    • DELETE

    • BIND

  • Outgoing scripting HTTP requests

Understand a trace object

This section provides a brief overview of an OTEL trace object. Learn more about trace objects in Traces.

A trace represents the path of a request through an application. A trace is made up of one or more spans.

Each span includes the following elements:

  • traceId representing the trace that the span is part of

  • spanId a unique ID for that span

  • parentSpanId the ID of the originating request

The root span indicates the start and end of an entire operation. The parentSpanId of the root span is null because the root span isn’t part of an existing trace.

Subsequent spans in the trace have their own unique spanId. Their traceId is the same as that of the root span. Their parentId matches the spanId of the root span.

Example trace object 
{
  "spans":[
    {
      "traceId":"8c3ebde938a6cf04f5bb34dd03135d45",
      "spanId":"335776e2b57ee9cf",
      "parentSpanId":"4065c5580cf47c7",
      "name":"HTTP POST /am/json/alpha/authenticate",
      "kind":1,
      "startTimeUnixNano":"1718811677526761641",
      "endTimeUnixNano":"1718811677526761641",
      "attributes":[],
      "events":[],
      "Links":[],
      "status":{
        "code":1
      },
      "flags":257
    }
  ]
}

Learn more in Traces in the OpenTelemetry documentation.

Enable and disable distributed tracing

Distributed tracing is disabled by default.

Enable distributed tracing

  1. Stop AM or the container in which it runs.

  2. In the /path/to/am/config directory, create a deployment/trace directory. For example:

    • Linux

    • Windows

    $ mkdir -p /path/to/am/config/deployment/trace
    PS C:\Users\Administrator> New-Item -ItemType Directory 'C:\path\to\am\config\deployment\trace'

  3. In the deployment/trace directory, create a file named config.json with, at least, the following contents:

    {
  "tracing": {
    "enabled": true
  }
}

    Find information on additional configuration properties in Configure distributed tracing.

    If the content of config.json is invalid JSON, distributed tracing remains disabled, even if you set "enabled": true.

  4. Restart AM or the container in which it runs.

Disable distributed tracing

  1. Stop AM or the container in which it runs.

  2. In the /path/to/am/config/deployment/trace/config.json file, set "enabled": false.

  3. Restart AM or the container in which it runs.

Configure distributed tracing

The Ping Identity Platform supports a common set of configuration properties for OpenTelemetry support.

  • The stability of this configuration interface is classified as evolving.

  • Any changes to the configuration require a server restart.

To change the default OpenTelemetry configuration, add the configuration properties to your /path/to/am/config/deployment/trace/config.json file, for example:

{
  "tracing": {
    "enabled": true,
    "resourceAttributes": {
      "service.instance.id": "am-server-1"
    },
    "exporter": {
      "config": {
        "headers": {
          "X-CUSTOM-HEADER": "custom-value"
        }
      }
    },
    "spanLimits": {
      "maxNumberOfAttributesPerEvent": 128
      }
    }
  }
}
Distributed tracing configuration properties
enabled: boolean, optional

Set to true to enable OpenTelemetry tracing.

Default: false

resourceAttributes: object, optional

A map of additional resource attributes for processing traces. Find more information in the OpenTelemetry documentation on Semantic Attributes with SDK-provided Default Value.

For example, if there are multiple AM instances in a deployment, you could set the "service.instance.id" resource attribute differently for each one to distinguish between them:

{
    "resourceAttributes": {
        "service.instance.id": "am-server-1"
    }
}
exporter: object, optional

Configuration for the exporter, which pushes traces to the OpenTelemetry service:

type: string, optional

Set to otlp for OpenTelemetry Protocol (OTLP) support. This is currently the only supported protocol.

Default: otlp

config: object, optional

Endpoint and timeout configuration:

  • compressionMethod: enumeration, optional

    Method used to compress trace data; either gzip or none.

    Default: gzip

  • connectionTimeout: duration, optional

    Time out a connection to the endpoint after this duration.

    Default: 10 seconds.

  • endpoint: string, optional

    The endpoint to publish traces to.

    For HTTPS, AM trusts the default JVM CAs. To override this, set the -Djavax.net.ssl.trustStore and associated JVM settings when starting AM. Learn more about the optional settings in the Java Secure Socket Extension (JSSE) Reference Guide.

    AM doesn’t support TLS configuration for the tracing endpoint at this time.

    Default: http://localhost:4318/v1/traces

  • headers: object, optional

    Map of additional headers to include in the export span request.

    The following example sets the authorization header, Authorization: Bearer ${bearer.token}:

    "headers": { "Authorization": "Bearer ${bearer.token}" }

  • retries: object, optional

    Defines a retry policy for the export span requests.

    Default: Enabled

    • backoffMultiplier: number, optional Multiplier for the backoff wait time before retries.

      Default: 1.5

    • enabled: boolean, optional

      Retry failed requests.

      Default: true

    • initialBackoff: duration, optional

      How long to wait before the first retry.

      Default: 1 second

    • maxAttempts: number, optional

      Maximum number of retries.

      Default: 5

    • maxBackoff: duration, optional

      Maximum wait time between retries.

      Default: 5 seconds

  • "timeout": duration, optional

    Time out a request to publish data to the endpoint after this duration.

    Default: 10 seconds.

batch: object, optional

Enable and configure batch processing for trace data.

  • compressionMethod: enumeration, optional

    Method used to compress trace data; either gzip or none.

    Default: gzip

  • enabled: boolean, optional

    Leave batch processing enabled in deployment.

    Default: true

  • exporterTimeout: duration, optional

    Time out a data exporter after this duration.

    Default: 30 seconds

  • exportUnsampledSpans: boolean, optional

    Whether to report on unsampled spans.

    Default: false

  • maxExportBatchSize: number, optional

    Maximum number of spans in a batch.

    Default: 512

  • maxQueueSize: number, optional

    Maximum number of spans to queue before dropping them.

    Default: 2048

  • scheduleDelay: duration, optional

    Maximum interval between sending batches of trace data.

    Default: 50 seconds

sampler: object, optional

Configuration for sampling spans.

ratio: number, optional

For ratio-based types, a percentage of spans to process.

Default: 50 (percent)

type: string, optional

The sampler strategy to use is one of the following:

  • alwaysOn: Send every span for processing.

  • alwaysOff: Never send any span for processing.

  • traceIdRatio: Sample the specified ratio of spans, deterministically based on the trace IDs of the spans.

  • parentBasedAlwaysOn: Always send the span for processing if the parent span was sampled. (Default)

  • parentBasedAlwaysOff: Never send the span for processing if the parent span was sampled.

  • parentBasedTraceIdRatio: Send the specified ratio of spans for processing if the parent span was sampled.

spanLimits: object, optional

Configuration for limits enforced when recording spans.

maxNumberOfAttributes: number, optional

The maximum number of attributes per span.

Default: 128

maxNumberOfAttributesPerEvent: number, optional

The maximum number of metadata items (attributes) attached to a span per event. An event is an annotation to span at a particular, meaningful point in time during the span’s duration.

Default: 128

maxNumberOfAttributesPerLink: number, optional

The maximum number of attributes per link.

Default: 128

maxNumberOfEvents: number, optional

The maximum number of events per span.

Default: 128

maxNumberOfLinks: number, optional

The maximum number of links per span. Links associate the current span with one or more other spans.

Default: 128

Secure trace exports

To protect trace exports, configure a secure connection between AM and the trace collector.

These steps assume you’re using the OpenTelemetry (OTel) Collector, but you can choose any trace collector. Adjust the steps accordingly.

Configuring a secure connection involves the following steps:

  1. Configure the collector to receive traces over HTTPS.

    Specify the path to your collector’s TLS certificate and private key and, optionally, the path to the CA certificate.

    If you’re using the OTel collector, read OTel’s collector documentation for information on configuring TLS.

  2. In the /path/to/am/config/deployment/trace/config.json file, make sure the exporter endpoint references HTTPS. For example:

    {
    "tracing": {
        "enabled": true,
        "exporter": {
            "config": {
                "endpoint": "https://otelcol.localtest.me:4318/v1/traces"
            }
        }
    }
}

  3. Add the trace collector’s TLS certificate to the AM truststore. For example:

    $ keytool \
-importcert \
-file /path/to/cert_file.pem \
-keystore /path/to/am/security/keystores/truststore

  4. Restart AM or the container in which it runs.

Visualize traces with Jaeger

You can use Jaeger to collect trace data directly from AM (or from your chosen telemetry collector) and to visualize that trace data.

Example visualization with Jaeger

This example assumes a local AM deployment in Apache Tomcat.

  1. Start Jaeger.

    Jaeger runs in a Docker container. Start Jaeger with a command similar to the following:

    docker run --rm --name jaeger \
  -p 5778:5778 \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  -p 14250:14250 \
  -p 14268:14268 \
  -p 9411:9411 \
  jaegertracing/jaeger:2.0.0 \
  --set receivers.otlp.protocols.http.endpoint=0.0.0.0:4318

  2. Go through an AM authentication flow.

    This example authenticates the amAdmin user:

    * curl \
--request POST \
--header "Content-Type: application/json" \
--header "X-OpenAM-Username: amadmin" \
--header 'X-OpenAM-Password: password' \
--header "Accept-API-Version: resource=2.0, protocol=1.0" \
"https://am.example.com:8443/am/json/realms/root/authenticate"*

  3. Navigate to Jaeger’s UI at http://localhost:16686/.

  4. Under Service select am (or the context path where you deployed AM) then click Find Traces.

    jaeger ui

  5. Click the trace for the HTTP POST request to the authenticate endpoint to view the traces for the authentication flow:

    jaeger post request

    Note the forgerock.transaction_id 8156ce8c-c5f4-4f59-ba59-9dd41d654f68-54971. This is the ID you’ll use to correlate the trace with the AM audit logs.

Correlate traces with audits

Distributed tracing doesn’t change the AM audit logs in any way.

However, when you enable distributed tracing, you can enrich the traces to include audit transaction IDs in their metadata. If a request includes a transaction identifier and a span is created for that request, the span metadata is enriched with the transaction identifier. This lets you correlate requests between traces and audit logs to determine which requests are taking longer than anticipated and identify any bottlenecks.

In the previous example, the forgerock.transaction_id was 8156ce8c-c5f4-4f59-ba59-9dd41d654f68-54971 for the authentication request.

The entry in the authentication.audit.json file for that request is as follows:

{
  "_id": "8156ce8c-c5f4-4f59-ba59-9dd41d654f68-54986",
  "timestamp": "2024-11-21T12:30:22.561Z",
  "eventName": "AM-LOGIN-COMPLETED",
  "transactionId": "8156ce8c-c5f4-4f59-ba59-9dd41d654f68-54971",
  "trackingIds": [
    "8156ce8c-c5f4-4f59-ba59-9dd41d654f68-54974"
  ],
  "userId": "id=amadmin,ou=user,dc=openam,dc=forgerock,dc=org",
  "principal": [
    "amadmin"
  ],
 ...
}

Note the correlation between the forgerock.transaction_id in the trace and the transactionId in the log entry.

Change host names

Changing AM host names involves the following high-level steps:

  • Add the new host name to the Realm/DNS Aliases list.

  • Export, edit, then import the configuration.

    This step relies on the ssoadm command, which you install separately from AM as described in Set up administration tools.

  • Stop AM and edit configuration files.

  • Remove the old host name from the Realm/DNS Aliases list.

Before you start, make sure you have an up to date backup of your installation. See Back up configurations for instructions.

Add the new host name as an alias

  1. In the AM admin UI, go to Realms > Realm Name and click Properties.

  2. Add the new host name to the Realm/DNS Aliases list, and save your work.

Export, edit, and import the service configuration

  1. Export the service configuration:

    $ ssoadm \
 export-svc-cfg \
 --adminid uid=amAdmin,ou=People,dc=am,dc=example,dc=com \
 --encryptsecret myEncryptSecretString1234 \
 --password-file /tmp/pwd.txt \
 --outfile config.xml
Service Configuration was exported.

    AM uses the value entered in --encryptsecret to encrypt passwords stored in the backup file. It can be any value, and is required when restoring a configuration.

  2. Edit the service configuration file:

    • Change the fully qualified domain name, such as am.example.com, throughout the file.

    • If you are changing the context path, such as /am, make the following changes:

      • Change the value of com.iplanet.am.services.deploymentDescriptor.

      • Change contextPath in the value of the propertiesViewBeanURL="contextPath/auth/ACServiceInstanceList".

      • Change contextPath in the value of propertiesViewBeanURL="contextPath/auth/ACModuleList".

      • Change the context path in a <Value> element that is a child of an <AttributeValuePair> element.

      • Change the context path where it occurs throughout the file in the full URL to AM, such as https:&#47;&#47;am.example.com:8443&#47;contextPath.

    • If you are changing the port number, change the value of com.iplanet.am.server.port.

      Also change the port number in host:port combinations throughout the file.

    • If you are changing the domain name, change the cookie domain, such as <Value>.example.com</Value>, throughout the file.

  3. Import the updated service configuration:

    $ ssoadm \
 import-svc-cfg \
 --adminid uid=amAdmin,ou=People,dc=am,dc=example,dc=com \
 --encryptsecret myEncryptSecretString1234 \
 --password-file /tmp/pwd.txt \
 --xmlfile config.xml
Directory Service contains existing data. Do you want to delete it? [y\|N] y
Please wait while we import the service configuration…​
Service Configuration was imported.

Edit configuration files for the new host name

  1. Stop AM or the web container where it runs.

  2. Edit the boot properties file, such as /home/user/am/boot.json, changing the fully qualified domain name (FQDN), port, and context path for AM as necessary.

  3. If you are changing the context path, move the folder containing AM configuration, such as /home/user/am/, to match the new context path, such as /home/user/am2/.

  4. If you are changing the location or context path, change the name of the file in the /home/user/.openamcfg folder, such as AMConfig_path_to_tomcat_webapps_am_, to match the new location and context path.

    Also edit the path name in the file to match the change you made when moving the folder.

  5. Restart AM or the web container where it runs.

Remove the old host name alias

  1. In the AM admin UI, go to Realms > Realm Name, and click Properties.

  2. Remove the old host name from the Realm/DNS Aliases list, and save your work.

Tune AM

This page covers key AM tuning strategies to ensure performant access and federation management, and to maximize throughput while minimizing response times.

The recommendations provided here are guidelines for your testing rather than hard and fast rules for every situation. Said another way, the fact that a given setting is configurable implies that no one setting is right in all circumstances.

The extent to which performance tuning advice applies depends to a large extent on your requirements, on your workload, and on what resources you have available. Test suggestions before rolling them out into production.

The suggestions in this page apply to AM deployments with the following characteristics:

  • The deployment has a dedicated DS server for the Core Token Service. The host running this directory server is a high-end server with a large amount of memory and multiple CPUs.

  • The AM server is configured to use server-side sessions.

The following table summarizes the high-level tasks required to tune an AM instance:

Task Resources

Tune general AM settings

Tuning server settings

Tune connectivity to LDAP datastores

Tune LDAP connectivity

Tune the JVM where AM runs

Tune JVM settings

Tune the configuration and user cache

Tune caching

Tuning server settings

AM has a number of settings that can be tuned to increase performance.

Logging settings

The following general points apply:

  • Set debug logging level to error.

  • Set container-level logging to a low level, such as error or severe.

Notification settings

AM has two thread pools used to send notifications to clients. The Service Management Service (SMS) thread pool can be tuned in the AM admin UI, under Configure > Server Defaults > SDK > Data Store:

SMS notification setting
Property Default value Suggestions

Notification Pool Size

1

Specifies the size of the thread pool used to send notifications. A value of 1 causes notifications to be processed sequentially, avoiding any potential out-of-order conditions. In production, where configuration is unlikely to change often, keeping the default of 1 is recommended.

(com.sun.identity.sm.notification.threadpool.size)

The session service has its own thread pool to send notifications to listeners about changes to server-side sessions. This is configured under Configure > Server Defaults > Session > Notification:

Session service notification settings
Property Default value Suggestions

Notification Pool Size

10

This is the size of the thread pool used to send notifications. In production this should be around 25-30.

(com.iplanet.am.notification.threadpool.size)

Notification Thread Pool Threshold

5000

This is the maximum number of notifications in the queue waiting to be sent. The default value should be fine in the majority of installations.

(com.iplanet.am.notification.threadpool.threshold)

Session settings

The Session service has additional properties to tune, which are configured under Configure > Server Defaults > Session > Session Limits. The following suggestion applies to deployments using server-side sessions:

Session Settings
Property Default Value Suggestion

Maximum Session Cache Size

5000

Maximum number of AM sessions to cache on the server.

In production, this value can safely be set into the 100,000s. The maximum session cache size is really controlled by the maximum size of the JVM heap which must be tuned appropriately to match the desired session cache size.

(org.forgerock.openam.session.service.access.persistence.caching.maxsize)

Policy evaluation settings

The AM policy engine places policies for evaluation on a queue in batches. Use ssoadm to optimize performance evaluation throughput by configuring the number of threads available for this queue.

This example increases the number of threads from the default value of 10 to 20:

./ssoadm set-entitlement-conf -u <admin_id> -f <password_file> -a evalThreadSize=20

For more information about this command, refer to set-entitlement-conf.

Tune LDAP connectivity

AM instances use pools of connections when communicating with LDAP datastores. You can tune these connection pools to improve performance and help with load balancing in the case of failover.

AM provides a global timeout setting for connections in a pool. Each store has properties for the maximum pool size, and in some cases, the minimum pool size.

AM attempts to use as few connections to LDAP datastores as possible, down to the minimum pool value, if specified. Under heavy load, AM creates additional connections to the configured datastores, up to the maximum pool value. These connections are made to any of the available LDAP datastores that are configured for the relevant purpose.

When the load begins to drop, some of those connections become idle. If a connection is idle for longer than the configured connection idle time, AM closes the connection, until any specified minimum pool size is reached.

By closing idle connections and recreating them when needed, AM balances connections across all available LDAP servers, rather than keeping the entire pool connected to a single server.

Tuning the connection pool settings can increase performance, or make AM more responsive to LDAP datastore outages.

Configure connection pool timeouts

  1. To configure the timeout used for connections to LDAP stores:

    • Open the bootstrapConfig.properties file in the AM classpath; for example, in /path/to/tomcat/webapps/am/WEB-INF/classes/.

    • Add, or update the following property, and set the idle timeout, in seconds:

      com.sun.am.ldap.connection.idle.seconds=300

  2. You must also configure the setting in the Advanced section of the server defaults, as follows:

    • In the AM admin UI, go to Configure > Server Defaults > Advanced.

    • Add, or edit the following property, and set the idle timeout, in seconds:

      com.sun.am.ldap.connection.idle.seconds=300

  3. Restart AM or the container in which it runs for these changes to take effect.

    After configuring the timeout for the stores, set the pool sizes assigned to the different stores in the AM admin UI:

Tune configuration store LDAP connections

To change LDAP configuration store settings, go to Deployment > Servers > Server Name > Directory Configuration.

LDAP configuration store settings
Setting Default value Details

Minimum Connection Pool

1

Property: minConnectionPool

Maximum Connection Pool

10

The default value of 10 is suitable for most cases; tuning this setting does not affect operational performance, only system startup.

Property: maxConnectionPool

Tune CTS store LDAP connections

You can increase the number of connections used for connecting to the CTS to increase throughput.

The default maximum number of connections to the CTS is 100.

To change the default, go to Deployment > Servers > Server Name > CTS > CTS Token Store, and set the Max Connections property.

You may need to click the Inherit value property to unlock the value for editing.

You can also edit the Max Connections default setting globally under Configure > Server Defaults > CTS > CTS Token Store tab.

If you need to change the default CTS connection timeout, go to Deployment > Servers > Server Name > Advanced and set the org.forgerock.services.datalayer.connection.timeout.cts.async property.

Most CTS requests to the directory server are handled quickly, so the default timeout of 10 seconds is suitable in most cases.

You must restart AM or the container in which it runs for these changes to take effect.

Tune external policy and applications store LDAP connections

To change external policy and application datastore settings, go to Configure > Global Services > External Data Stores > Secondary Configurations > Store Name.

Policy and application data is stored in the configuration datastore if not configured separately. To manage the configuration store connection pool, refer to Tune configuration store LDAP connections.
LDAP policy and application store settings
Setting Default value Information

Minimum Connection Pool Size

1

Must be less than the maximum size to allow reaping to function.

Property: minimumConnectionPool

Maximum Connection Pool Size

10

Property: maximumConnectionPool

Tune identity store LDAP connections

To change LDAP datastore settings, go to Realms > Realm Name > Identity Stores > Identity Store Name. Each store has its own connection pool, so each store needs its own tuning:

LDAP identity store settings
Setting Default value Details

LDAP Connection Pool Minimum Size

1

A good tuning value for this property is 10.

Property: sun-idrepo-ldapv3-config-connection_pool_min_size

LDAP Connection Pool Maximum Size

10

The maximum LDAP connection pool size; a high tuning value for this property is 65, though you might well be able to reduce this for your deployment. Ensure your LDAP server can cope with the maximum number of clients across all the AM servers.

Property: sun-idrepo-ldapv3-config-connection_pool_max_size

Tune UMA store LDAP connections

To increase the number of connections used for UMA-related datastores, go to Deployment > Servers > Server Name > UMA > UMA Store Type, and edit the Max Connections property.

You may need to click the Inherit value property to unlock the value for editing.

You can also edit the Max Connections default settings globally under Configure > Server Defaults > UMA > UMA Store Type.
LDAP UMA store settings
Setting Default value Details

UMA Resource Store > Max Connections

10

Property: org.forgerock.services.resourcesets.store.max.connections

UMA Audit Store > Max Connections

10

Property: org.forgerock.services.umaaudit.store.max.connections

Pending Requests Store > Max Connections

10

Property: org.forgerock.services.pendingrequests.store.max.connections

UMA Resource Labels Store > Max Connections

2

Property: org.forgerock.services.uma.labels.store.max.connections

Tune authentication node LDAP connections

To change connection pool settings for the LDAP Decision node, go to Configure > Authentication > Core Attributes > Global Attributes.

LDAP authentication node settings
Setting Default value Details

Default LDAP Connection Pool Size

1:10

The minimum and maximum LDAP connection pool used by the LDAP authentication node, separated by a colon (:) character.

Use 10:65 for production AM instances.

Property: iplanet-am-auth-ldap-connection-pool-default-size

Tune JVM settings

This section gives some initial guidance on configuring the JVM for running AM when the deployment has a dedicated CTS token store, and AM is configured to use server-side sessions.

These settings provide a strong foundation to the JVM before a more detailed garbage collection tuning exercise, or as best practice configuration for production:

Heap size settings
JVM parameters Suggested value Description

-Xms & -Xmx

At least 1 GB. In production environments, at least 2 to 3 GB. This setting depends on the available physical memory and whether a 32- or 64-bit JVM is used.

-XX:MetaspaceSize & -XX:MaxMetaspaceSize

Set both to 256 MB

Controls the size of the metaspace in the JVM.

-Dsun.net.client.defaultReadTimeout

60000

Controls the read timeout in the Java HTTP client implementation.

This applies only to the Sun/Oracle HotSpot JVM.

-Dsun.net.client.defaultConnectTimeout

High setting: 30000 (30 seconds)

Controls the connect timeout in the Java HTTP client implementation.

When you have hundreds of incoming requests per second, reduce this value to avoid a huge connection queue.

This applies only to the Sun/Oracle HotSpot JVM.
Security settings
JVM parameters Suggested value Description

-Dhttps.protocols

TLSv1.2

Controls the protocols used for outbound HTTPS connections from AM.

Specify one or more of the following values, separated by commas:

  • TLSv1.2

  • TLSv1.3

This setting applies only to Sun/Oracle Java environments.

-Dorg.forgerock.openam.ldap.secure.protocol.version

TLSv1.2

Controls the protocol AM uses to connect to affected external resources.

Specify one or more of the following values, separated by commas:

  • TLSv1.2

  • TLSv1.3

This setting overrides the default server value. Learn more in advanced properties.
Garbage collection settings
JVM parameters Suggested value Description

-verbose:gc

Verbose garbage collection reporting.

-Xlog:gc*

-Xlog:gc=info:file=$CATALINA_HOME/logs/gc-info.log

Logs detailed information about garbage collection. When using the -Xlog:gc option, you can also specify the level, and output file.

-XX:+HeapDumpOnOutOfMemoryError

Out of Memory errors generate a heap dump automatically.

-XX:HeapDumpPath

$CATALINA_HOME/logs/heapdump.hprof

Location of the heap dump.

-XX:+PrintClassHistogram

Prints a heap histogram when the JVM receives a SIGTERM signal.
Other settings
Module flag Suggested value Description

--add-opens

java.xml/com.sun.org.apache.xerces.internal.dom=ALL-UNNAMED

When running AM, SAML Artifact flows (or any flows that use Xerces SOAP libraries) can fail with the following error:

Caused by: java.lang.IllegalAccessError: superclass access check failed: class com.sun.xml.messaging.saaj.soap.SOAPDocumentImpl (in unnamed module @0x774ca796) cannot access class com.sun.org.apache.xerces.internal.dom.DocumentImpl (in module java.xml) because module java.xml does not export com.sun.org.apache.xerces.internal.dom to unnamed module @0x774ca796

Set this module flag to avoid this error.

Tune caching

AM caches data to avoid having to query user and configuration datastores each time it needs the information. By default, AM makes use of LDAP persistent search to receive notification of changes to cached data. For this reason, caching works best when data are stored in a directory server that supports LDAP persistent search.

AM has two kinds of configurable cache on the server side; one for configuration data and one for user data. You can generally use the default settings for configuration data cache. This section covers the configuration choices available for caching user data.

AM implements the global user data cache for its user datastores.

The user datastore also supports a DN cache, used to cache DN lookups that tend to occur in bursts during authentication. The DN cache can become out of date when a user is moved or renamed in the underlying LDAP store, events that are not always reflected in a persistent search result. You can enable the DN cache when the underlying LDAP store supports persistent search and mod DN operations (that is, move or rename DN).

The following diagram depicts the two kinds of cache, and also the two types of caching available for user data:

Servers cache user data and configuration data separately.
Figure 3. Caches

The rest of this page covers settings for global user data cache and for SDK clients. You can find information on datastore cache settings in Tune LDAP connectivity.

Overall server cache settings

By default, AM has caching enabled for both configuration data and user data. This setting is governed by the server property com.iplanet.am.sdk.caching.enabled, which is true by default. If you set this advanced property to false, you can enable caching independently for configuration data and for user data.

Turn off global user data caching

Disabling caching can have a severe negative impact on performance. This is because when caching is disabled, AM must query a datastore each time it needs data.

If, however, you have at least one identity store that does not support LDAP persistent search, then you must disable the global cache for user data. Otherwise, user data caches cannot stay in sync with changes to user data entries:

  1. In the AM admin UI, go to Deployment > Servers > Server Name > Advanced.

  2. Set the value of the com.iplanet.am.sdk.caching.enabled property to false to disable caching overall.

  3. Set the value of the com.sun.identity.sm.cache.enabled property to true to enable configuration data caching.

    All supported configuration datastores support LDAP persistent search, so it is safe to enable configuration data caching.

    You must explicitly set this property to true, because setting the value of the property com.iplanet.am.sdk.caching.enabled to false in the previous step disables both user and configuration data caching.

  4. Save your work.

  5. AM starts persistent searches on user datastores when possible(1) in order to monitor changes.

    With user datastore caching disabled, AM still starts the persistent searches, even though it no longer uses the results.

    Therefore, if you disable user datastore caching, you should also disable persistent searches on identity stores in your deployment to improve performance.

    To disable persistent search on an identity store, go to Realms > Realm Name > Identity Stores > Identity Store Name > Persistent Search Controls and remove the value of the Persistent Search Base DN configuration property (leave it blank).

(1) AM starts persistent searches on user datastores on directory servers that support the psearch control.

Change the maximum size of the global user data cache

With a large user datastore and active user base, the number of user entries in cache can grow large.

  1. In the AM admin UI, go to Configure > Server Defaults > SDK.

  2. Change the value of SDK Caching Maximum Size.

    There is no corresponding setting for configuration data, because the number of configuration entries in a large deployment is not likely to grow nearly as large as the number of user entries.

Clear the configuration cache

When you change configuration property values, the old value remains in effect until the affected service is restarted. To avoid having to restart AM when you change a property value, you can clear the configuration cache and force the new property value to take effect.

To clear the configuration cache, send an empty POST request to the /json/cache endpoint with the clear action, for example:

$ curl \
--request POST \
--header "Content-Type: application/json" \
--header "iPlanetDirectoryPro: kYQVVJ2YFCuAe-b1xjm7evGBDkw.AAJTSQACMDEAAlNLABxQS0ZIYzBPeFVWUzBQOTNLaHV0elVQemdqVU09AAR0eXBlAANDVFMAAlMxAAA.	" \
--header  "Accept-API-Version: resource=1.0" \
--data '{}' \
https://am.example.com:8443/am/json/cache?_action=clear
{}

Only members of a group with the Realm Admin or Cache Admin privilege can run this operation to clear the cache. All attempts to access the endpoint are audited, including information about the user that attempted to clear the cache.

Cache settings

The table below provides a quick reference, primarily for user data cache settings.

Notice that many properties for configuration data cache have sm (for Service Management) in their names, whereas those for user data have idm (for Identity Management) in their names:

Cache properties
Property Description Default Applies to

com.iplanet.am.sdk.cache.maxSize

Maximum number of user entries cached.

10000

Server and SDK

com.iplanet.am.sdk.caching.enabled

Whether to enable caching for both configuration data and also for user data.

If true, this setting overrides com.sun.identity.idm.cache.enabled and com.sun.identity.sm.cache.enabled.

If false, you can enable caching independently for configuration data and for user data using the aforementioned properties.

true

Server and SDK

com.iplanet.am.sdk.remote.pollingTime

How often in minutes the SDK client, such as a web or a Java agent, should poll AM for modified user data entries.

The SDK also uses this value to determine the age of the oldest changes requested. The oldest changes requested are 2 minutes older than this setting. In other words, by default the SDK polls for entries changed in the last 3 minutes.

Set this to 0 or a negative integer to disable polling.

1 (minute)

SDK

com.sun.am.event.notification.expire.time

How long AM stores a given change to a cached entry, so that clients polling for changes do not miss the change.

30 (minutes)

Server only

com.sun.identity.idm.cache.enabled

If com.iplanet.am.sdk.caching.enabled is true, this property is ignored.

Otherwise, set this to true to enable caching of user data.

false

Server and SDK

com.sun.identity.idm.cache.entry.default.expire.time

How many minutes to store a user data entry in the global user data cache.

30 (minutes)

Server and SDK

com.sun.identity.idm.cache.entry.expire.enabled

Whether user data entries in the global user data cache should expire over time.

false

Server and SDK

com.sun.identity.idm.remote.notification.enabled

Whether the SDK client, such as a web or a Java agent, should register a notification listener for user data changes with the AM server.

The SDK client uses the URL specified by com.sun.identity.client.notification.url to register the listener so that AM knows where to send notifications.

If notifications cannot be enabled for some reason, then the SDK client falls back to polling for changes.

true

SDK

com.sun.identity.sm.cache.enabled

If com.iplanet.am.sdk.caching.enabled is true, this property is ignored.

Otherwise, set this to true to enable caching of configuration data. It is recommended that you always set this to true.

false

Server and SDK

sun-idrepo-ldapv3-dncache-enabled

Set this to true to enable DN caching of user data.

false

Server and SDK

sun-idrepo-ldapv3-dncache-size

Sets the cache size.

1500

Server and SDK

Debug logging

AM services capture a variety of information in debug logs. Unlike audit log records, debug log records are unstructured. Debug logs contain different types of information that is useful when troubleshooting AM, including stack traces.

AM uses Logback as the handler for debug logging, making it easily customizable. For example, the level of debug log record output is configurable, as is the storage location and format.

AM lets you enable the debug log level for specific classes in the AM code base. This can be useful when you must turn on debug logging in a production system where you want to avoid excessive logging, but must gather messages when you reproduce a problem.

You can choose the level of logging from the following options:

Off

No debug messages are logged.

Error

Debug messages signifying that an error has occurred are logged.

This is the default level.

Warning

Debug messages signifying potentially harmful situations are logged.

Information

Debug messages that contain coarse-grained information about the status of AM are logged.

Debug

Debug messages that contain fine-grained information useful for troubleshooting AM are logged.

Trace

All debug messages are logged.

Create loggers to specify the debug level for a class, and choose where the output is recorded. The logger used by a feature in AM is hierarchical, based on the class creating the debug messages. The most specific logger is used, which is the logger whose path most closely matches the class that is creating the log messages.

For example, if you knew there was an issue in an authentication node, you could enable trace-level debug logging in org.forgerock.openam.core.rest.authn.trees. If you aren’t sure where the problem lies, you can choose a broader option, for example org.forgerock.openam.core.rest.authn.

The least-specific, catch-all logger is named ROOT.

AM also logs information related to client interactions using the org.apache.http.wire and org.apache.http.headers appenders. The information they collect is useful, for example, when you are developing authentication scripts or when your environment requires STS transformations.

By default, these appenders are always set to the Warning level unless logging is disabled.

Learn more in the org.forgerock.allow.http.client.debug advanced server property.

You can configure debug logging temporarily by using the AM admin UI, or you can create a file in the AM classpath with persistent debug configuration.

Temporarily enable debug logging with Logback.jsp

These steps let you temporarily capture debug messages, until the next time AM or the container in which it runs is restarted.

  1. In the AM admin UI, go to Logback.jsp in the root context of the AM installation, for example https://am.example.com:8443/am/Logback.jsp.

    No links to this page are provided in the AM admin UI.

    Only the amAdmin administrator account can access the Logback.jsp page and alter the debug settings; delegated administrators don’t have access.

    The page displays all the appenders and their associated debug loggers.

    Logback.jsp logger names

    The following lists contain the available logger names ordered by their associated appender:

    Authentication 
    Authentication service, framework, Auth modules, Callbacks, JAAS, API
com.sun.identity.authentication.spi.AMLoginModule,
org.forgerock.openam.core.rest.authn.callbackhandlers,
com.sun.identity.authentication.spi.AMAuthCallBackImpl,
com.sun.identity.authentication.service.AuthContextLookup,
com.sun.identity.authentication.util,
org.forgerock.openam.authentication.service.LoginContextFactory,
com.sun.identity.authentication.server.AuthContextLocal,
com.sun.identity.authentication.service.AMAccountLockout,
com.sun.identity.authentication.service.LoginState,
com.sun.identity.authentication.UI.LoginViewBean,
com.sun.identity.authentication.client,
org.forgerock.openam.core.rest.authn.trees,
com.sun.identity.authentication.spi.FirstTimeLogin,
org.forgerock.openam.auth,
org.forgerock.openam.authentication.service.SessionPropertyUpgrader,
com.sun.identity.authentication.UI.AuthExceptionViewBean,
com.sun.identity.authentication.spi.ReplayPasswd,
com.sun.identity.authentication.config,
com.sun.identity.authentication.share,
org.forgerock.openam.authentication.SessionUpgradeVerifier,
com.sun.identity.authentication.service.DSAMECallbackHandler,
com.sun.identity.authentication.spi.AMModuleProperties,
org.forgerock.openam.utils.MappingUtils,
com.sun.identity.authentication.UI.AuthenticationServletBase,
com.sun.identity.authentication.service.AuthenticationPrincipalDataRetrieverFactory,
com.sun.identity.authentication.UI.LogoutViewBean,
com.iplanet.security,
com.sun.identity.authentication.internal,
com.sun.identity.authentication.AuthContext,
com.sun.identity.policy.plugins.AuthenticatedSharedAgents,
org.forgerock.openam.ldap.LDAPAuthUtils,
com.sun.identity.authentication.UI.AuthViewBeanBase,
org.forgerock.openam.authentication.modules,
com.iplanet.services.cdm,
org.forgerock.openam.authentication.service.AuthUtilsWrapper,
com.sun.identity.policy.plugins.AuthenticatedAgents,
com.sun.identity.authentication.spi.JwtReplayPassword,
com.sun.identity.policy.plugins.AllowedAgents,
com.sun.identity.authentication.service.AuthenticationServiceAttributeCache,
com.sun.identity.authentication.jaas,
com.sun.identity.authentication.service.AuthD,
org.forgerock.openam.core.rest.authn.core,
org.forgerock.openam.scripting.api,
com.sun.identity.common.ISAccountLockout,
org.forgerock.openam.core.rest.authn.RestAuthCallbackHandlerFactory,
org.forgerock.openam.core.rest.authn.RestAuthCallbackHandlerManager,
org.forgerock.openam.webhook,
com.iplanet.services.cdc,
com.sun.identity.authentication.modules,
org.forgerock.openam.core.rest.authn.http.AuthenticationServiceV1,
com.sun.identity.authentication.service.AuthUtils,
com.sun.identity.policy.plugins.AuthenticatedSharedAgentsCondition,
org.forgerock.openam.authentication.service.JAASModuleDetector,
org.forgerock.openam.core.rest.authn.RestAuthenticationHandler
    Configuration 
    Service Configuration, Delegation, SMS Schema, SMS repository, plugins
com.sun.identity.sm.ServiceSchemaManager,
com.iplanet.services.ldap.event.EventService,
com.sun.identity.sm.SMSSchema,
com.sun.identity.tools,
com.sun.identity.sm.SMSUtils,
com.sun.identity.common.configuration.ServerConfigXMLObserver,
com.sun.identity.sm.ServiceSchema,
com.sun.identity.delegation,
com.sun.identity.sm.OrganizationConfigManager,
com.sun.identity.sm.ldap,
com.sun.identity.sm.SMSNotificationManager,
com.sun.identity.sm.PluginSchema,
com.sun.identity.sm.AttributeValidator,
com.sun.identity.sm.ServiceConfigManagerImpl,
com.sun.identity.sm.ServiceConfigImpl,
com.sun.identity.sm.SMSPropertiesObserver,
com.sun.identity.sm.OrganizationConfigManagerImpl,
com.sun.identity.sm.AuthenticationServiceNameProviderImpl,
org.forgerock.openam.xui.XUIFilter,
com.sun.identity.sm.ServiceSchemaImpl,
com.sun.identity.setup,
com.sun.identity.sm.AttributeSchemaState,
com.sun.identity.sm.ServiceInstanceImpl,
org.forgerock.openam.auditors,
com.sun.identity.workflow,
com.sun.identity.sm.ServiceConfigManager,
org.forgerock.openam.sm.validation,
com.sun.identity.common.configuration.SessionSiteNames,
com.sun.identity.sm.ServiceConfig,
com.sun.identity.sm.SMServlet,
com.sun.identity.sm.ServiceManager,
com.sun.identity.common.configuration.ServerPropertyValidator,
com.sun.identity.sm.SMSEntry,
com.sun.identity.sm.PluginConfig,
org.forgerock.openam.utils.OpenAMSettingsImpl,
com.sun.identity.sm.jaxrpc,
com.sun.identity.sm.DNMapper,
com.sun.identity.sm.SMSException,
com.sun.identity.sm.SMSEventListenerManager,
org.forgerock.openam.utils.MapHelper,
com.sun.identity.sm.ServiceInstance,
com.sun.identity.config.util,
com.sun.identity.sm.CachedSubEntries,
com.sun.identity.sm.PluginConfigImpl,
com.sun.identity.authentication.service.ConfiguredSocialAuthServices,
com.sun.identity.sm.ServiceSchemaManagerImpl,
com.sun.identity.sm.CachedSMSEntry,
com.sun.identity.sm.CreateServiceConfig,
com.sun.identity.sm.AttributeSchema,
com.sun.identity.sm.PluginSchemaImpl
    CoreSystem 
    Core infrastructure services, PLL, cookies, naming, logging, upgrade, Scripting
com.sun.identity.monitoring,
com.sun.identity.saml2.idpdiscovery,
com.sun.identity.security.cert.CRLValidator,
org.forgerock.openam.xacml.v3.rest,
org.forgerock.openam.core.rest.SelfServiceUserUiRolePredicate,
org.forgerock.openam.core.rest.cts,
org.forgerock.openam.sm.datalayer.impl.ldap.LdapSearchHandler,
org.forgerock.openam.security,
com.sun.identity.plugin.monitoring.impl,
org.forgerock.openam.sm.datalayer.providers,
com.zaxxer.hikari,
org.forgerock.openam.uma.UmaUserUiRolePredicate,
com.sun.identity.common.RequestUtils,
org.forgerock.openam.entitlement.rest.SubjectAttributesResourceV1,
org.forgerock.openam.services.baseurl,
org.forgerock.openam.core.rest.IdentityRestUtils,
org.forgerock.openam.core.rest.UserGroupsResource,
org.forgerock.openam.oauth2.rest,
com.sun.identity.authentication.UI.taglib,
org.forgerock.openam.core.rest.docs,
com.sun.identity.log,
org.forgerock.openam.core.rest.AllAuthenticatedUsersResource,
org.forgerock.openam.utils.WhitelistObjectInputStream,
org.forgerock.openam.core.rest.dashboard,
com.sun.identity.common.SystemTimerPool,
org.forgerock.openam.core.rest.session.AnyOfAuthzModule,
org.forgerock.openam.rest,
org.forgerock.openam.core.rest.sms,
com.sun.identity.common.admin,
org.forgerock.openam.shared.resourcename,
com.sun.identity.security.AdminTokenAction,
org.forgerock.openam.uma.rest.UmaPolicyResourceAuthzFilter,
org.forgerock.openam.shared.concurrency,
org.forgerock.openam.core.rest.session.SessionResourcePrivilegeAuthzModule,
org.forgerock.openam.entitlement.rest.ResourceTypesResource,
org.forgerock.openam.uma.rest.UmaPolicyServiceImpl,
org.forgerock.openam.entitlement.rest.DecisionCombinersResource,
com.sun.identity.common.HttpURLConnectionManager,
org.forgerock.openam.sm.datalayer.impl.SeriesTaskExecutor,
org.forgerock.openam.network.ipv4.IPv4AddressRange,
org.forgerock.openam.audit,
org.forgerock.audit,
com.sun.identity.common.DNUtils,
org.forgerock.openam.utils.IPRange,
org.forgerock.openam.services.RestSecurity,
org.forgerock.openam.core.rest.IdentityResourceV4,
org.forgerock.openam.core.rest.IdentityResourceV3,
com.sun.identity.security.SecurityDebug,
org.forgerock.openam.backstage,
org.forgerock.openam.core.rest.server,
org.forgerock.openam.utils.ClientUtils,
org.forgerock.openam.core.rest.IdentityResourceV2,
org.forgerock.openam.entitlement.rest.ApplicationV1Filter,
org.forgerock.openam.core.rest.IdentityResourceV1,
org.forgerock.openam.core.rest.devices,
org.forgerock.openam.entitlement.rest.ApplicationsResource,
com.sun.identity.policy.util.Gateway,
com.sun.identity.shared.jaxrpc,
org.forgerock.openam.forgerockrest,
com.iplanet.am.util,
com.iplanet.services.comm,
org.forgerock.openam.core.rest.authn.AuditHelper,
org.forgerock.openam.sm.datalayer.impl.PooledTaskExecutor,
org.forgerock.openam.ldap.LdifUtils,
org.forgerock.openam.core.rest.session.action.LogoutByHandleActionHandler,
org.forgerock.openam.sm.datalayer.impl.ldap.LdapQueryBuilder,
com.sun.identity.shared.search,
org.forgerock.openam.entitlement.rest.SubjectTypesResource,
com.sun.identity.shared.encode.CookieUtils,
com.iplanet.services.naming,
org.forgerock.openam.cors,
com.sun.identity.idsvcs,
com.sun.identity.jaxrpc,
org.forgerock.openam.http,
org.forgerock.openam.shared.guice,
org.forgerock.openam.utils.AMKeyProvider,
org.forgerock.openam.utils.AuthLevelUtils,
org.forgerock.openam.shared.security.whitelist,
org.forgerock.openam.notifications,
com.sun.identity.policy.util.GatewayServletUtils,
org.forgerock.openam.core.sms,
org.forgerock.openam.blacklist,
com.sun.identity.common.configuration.AgentConfiguration,
org.forgerock.openam.entitlement.rest.ApplicationTypesResource,
org.forgerock.openam.monitoring,
com.sun.identity.common.ResourceLookup,
org.forgerock.openam.entitlement.rest.PolicyV1Filter,
com.sun.identity.authentication.server.AuthXMLRequestParser,
org.forgerock.openam.entitlement.rest.wrappers,
com.sun.identity.security.cert.AMCertStore,
org.forgerock.openam.sm.datalayer.impl.SimpleTaskExecutor,
com.sun.identity.shared.locale,
com.sun.identity.shared.whitelist,
org.forgerock.openam.sm.datalayer.impl.ldap.CTSDJLDAPv3PersistentSearch,
com.sun.identity.protocol,
org.forgerock.openam.scripting.rest,
org.forgerock.openam.entitlement.rest.ConditionTypesResource,
org.forgerock.openam.core.rest.record,
com.sun.identity.security.cert.AMCertPath,
org.forgerock.openam.utils.ServiceConfigUtils,
com.sun.identity.authentication.server.AuthXMLRequest
    Federation 
    Federated SSO, protocols (WS-Federation, SAML2), Metadata, Hub, Circle of Trust
com.sun.identity.wsfederation.profile,
com.sun.identity.saml2.servlet,
com.sun.identity.saml2.plugins.SAML2PluginsUtils,
com.sun.identity.plugin.datastore,
com.sun.identity.saml2.logging,
com.sun.identity.saml2.protocol,
com.sun.identity.saml2.common,
com.sun.identity.saml2.plugins.DefaultAccountMapper,
org.forgerock.openam.federation,
com.sun.identity.wsfederation.plugins.DefaultSPAttributeMapper,
com.sun.identity.saml2.plugins.DefaultSPAccountMapper,
com.sun.identity.wsfederation.plugins.whitelist,
com.sun.identity.saml2.profile,
com.sun.identity.wsfederation.plugins.DefaultLibrarySPAccountMapper,
com.sun.identity.saml2.plugins.SAML2IDPProxyFRImpl,
com.sun.identity.wsfederation.key,
com.sun.identity.multiprotocol,
com.sun.identity.saml2.plugins.SAML2IDPProxyImpl,
com.sun.identity.wsfederation.servlet,
com.sun.identity.xacml,
com.sun.identity.plugin.monitoring.MonitorManager,
com.sun.identity.saml2.plugins.DefaultIDPAuthnContextMapper,
com.sun.identity.wsfederation.plugins.DefaultAccountMapper,
com.sun.identity.saml2.plugins.DefaultAttributeMapper,
com.sun.identity.wsfederation.plugins.DefaultAttributeMapper,
org.forgerock.openam.authentication.Saml2SessionUpgradeHandler,
com.sun.identity.saml2.ecp,
org.forgerock.openam.wsfederation,
com.sun.identity.federation,
org.forgerock.openam.saml2,
jsp.saml2,
com.sun.identity.saml2.plugins.DefaultIDPECPSessionMapper,
com.sun.identity.saml2.plugins.DefaultLibrarySPAccountMapper,
com.sun.identity.plugin.log,
com.sun.identity.saml,
com.sun.identity.wsfederation.meta,
com.sun.identity.wsfederation.plugins.DefaultIDPAuthenticationMethodMapper,
com.sun.identity.saml2.plugins.DefaultFedletAdapter,
com.sun.identity.saml2.plugins.DefaultLibraryIDPAttributeMapper,
com.sun.identity.saml2.xmlenc,
com.sun.identity.saml2.plugins.DefaultSPAttributeMapper,
com.sun.identity.saml2.plugins.DefaultSPAuthnContextMapper,
com.sun.identity.saml2.xmlsig,
com.sun.identity.liberty.ws.security,
com.sun.identity.plugin.session.SessionManager,
com.sun.identity.wsfederation.plugins.DefaultIDPAccountMapper,
com.sun.identity.plugin.session.impl.FMSessionProvider,
com.sun.identity.saml2.key,
com.sun.identity.wsfederation.logging,
com.sun.identity.saml2.plugins.DefaultIDPAccountMapper,
com.sun.identity.wsfederation.plugins.DefaultADFSPartnerAccountMapper,
com.sun.identity.saml2.assertion,
com.sun.identity.wsfederation.plugins.DefaultIDPAttributeMapper,
com.sun.identity.plugin.session.impl.FedletSessionProvider,
com.sun.identity.saml2.meta,
com.sun.identity.plugin.configuration,
com.sun.identity.saml2.soapbinding,
com.sun.identity.wsfederation.common,
com.sun.identity.cot
    IdRepo 
    Identity Repositories, Datastores, plugins
com.sun.identity.common.ISResourceBundle,
com.iplanet.am.sdk,
org.forgerock.openam.idrepo.ldap.DJLDAPv3Repo,
org.forgerock.openam.shared.security.crypto,
com.iplanet.sso.SSOTokenManager,
com.iplanet.services.ldap.DefaultDataStoreConfigurationManager,
com.sun.identity.idm,
org.forgerock.openam.idrepo.ldap.helpers.DirectoryHelper,
com.sun.identity.shared.encode.Hash,
org.forgerock.openam.core.realms,
org.forgerock.openam.shared.security.ThreadLocalSecureRandom,
com.iplanet.services.ldap.event.LDAPv3PersistentSearch,
org.forgerock.openam.idrepo.ldap.psearch,
com.sun.identity.security.ServerInstanceAction,
org.forgerock.openam.identity,
org.forgerock.openam.ldap.LDAPUtils
    OAuth2Provider 
    OAuth 2.0 Provider
org.forgerock.openam.oauth2.OpenAMClientRegistrationStore,
org.forgerock.openam.oauth2.secrets,
org.forgerock.openidconnect,
org.forgerock.openam.oauth2.resources.ResourceSetLabelRegistration,
org.forgerock.openam.oauth2.OAuth2GlobalSettings,
org.forgerock.openam.oauth2.OpenAMClientRegistration,
org.forgerock.openam.oauth2.ciba,
org.forgerock.openam.oauth2.requesturis,
org.forgerock.openam.oauth2.OAuth2AuditLogger,
org.forgerock.openam.oauth2.token,
org.forgerock.openam.oauth2.IdentityManager,
org.forgerock.openam.oauth2.IgAgentClientRegistration,
org.forgerock.openam.oauth2.jwks,
org.forgerock.oauth2,
org.forgerock.openam.utils.RealmNormaliser,
org.forgerock.openam.oauth2.AgentClientRegistration,
org.forgerock.openam.oauth2.ClientCredentialsReader,
org.forgerock.openam.oauth2.remoteconsent,
org.forgerock.openam.oauth2.OpenAMScopeValidator,
org.forgerock.openam.oauth2.OAuth2Monitor
    OpenDJ-SDK 
    Directory Server SDK
org.forgerock.opendj.ldif,
org.forgerock.opendj.asn1,
com.forgerock.opendj.util,
com.forgerock.opendj.ldap,
org.forgerock.opendj.ldap,
org.forgerock.opendj.util
    OtherLogging 
    Miscellaneous logs
org.forgerock.openam.secrets.SecretIdChoiceValues,
org.forgerock.am.iot.IntrospectTokenActionHandler,
com.sun.identity.sm.SmsObjectResolver,
org.forgerock.config.resolvers,
org.forgerock.openam.services.datastore,
org.forgerock.openam.utils.JCECipherProvider,
org.forgerock.config.resolvers.SystemPropertyResolver,
com.sun.identity.policy.plugins,
org.forgerock.openam.entitlement.rest,
org.forgerock.openam.services.datastore.DataStoreConsistencyFilter,
org.forgerock.openam.oauth2.saml2,
org.forgerock.secrets.propertyresolver.PropertyResolverSecretStore,
org.forgerock.openam.headers.DisableSameSiteCookiesFilter,
org.forgerock.openam.oauth2.resources,
org.forgerock.openam.uma.rest,
org.forgerock.openam.integration.idm.IdmClientIdRepo,
org.forgerock.am.health.HealthCheckService,
com.sun.identity.shared,
org.forgerock.openam.network.ipv4,
com.forgerock,
org.forgerock.openam.core.rest.session,
org.forgerock.util.encode.Base64url,
org.forgerock.openam.core.rest,
com.iplanet.services.ldap.ServerGroup,
org.forgerock.am.iot.ThingsResource,
org.forgerock.openam.uma,
org.forgerock.openam.secrets.config.GoogleKeyManagementServiceSecretStore,
org.forgerock.api.models.Resource,
org.forgerock.openam.oauth2.saml2.core.Saml2GrantTypeHandler,
com.sun.identity.configuration.ConfigFedMonitoring,
org.forgerock.openam.setup.BootstrapSubstitutionService,
org.forgerock.util.promise,
org.forgerock.config.resolvers.EnvironmentVariableResolver,
org.forgerock.config.util,
org.forgerock.openam.scripting.ScriptEngineConfigurator,
org.forgerock.openam.oauth2.guice,
org.forgerock.openam.scripting.persistence,
org.forgerock.api.models.Items,
org.forgerock.openam.homedirectory.HomeDirectoryUtils,
org.forgerock.openam.selfservice,
com.iplanet.services,
org.forgerock.openam.scripting.ThreadPoolScriptEvaluator,
jsp,
org.forgerock.am.health.ReadinessCheckEndpoint,
io.swagger.models.parameters.AbstractSerializableParameter,
org.forgerock.openam.social,
com.sun.identity.plugin.monitoring,
org.forgerock.openam.services.MailService,
OAuth2Factory,
org.apache.http.headers,
org.forgerock.json,
org.forgerock.openam.oauth2.OAuth2UrisFactory,
com.sun.identity.shared.encode,
org.forgerock.http.swagger,
com.iplanet,
com.sun.identity.common.configuration,
org.forgerock.json.resource.InterfaceCollectionInstance,
org.forgerock.json.resource.http.HttpUtils,
org.forgerock.openam.uma.UmaProviderSettingsFactory,
org.forgerock.openam.utils,
org.forgerock.openam.scripting,
org.forgerock.openam.uma.rest.UmaEnabledFilter,
org.forgerock.openam.sts.publish.rest.RestSTSSetupListener,
org.forgerock.util.encode.Base64,
com.zaxxer,
org.forgerock.openam.oauth2.guice.OAuth2GuiceModule,
org.forgerock.openam.social.idp.SocialIdpJwksSecretsProvider,
org.forgerock.secrets,
org.forgerock.util.promise.Promises,
org.forgerock.secrets.SecretReference,
org.forgerock.openam.sts.publish.common.STSInstanceConfigStoreBase,
io.swagger.models.parameters,
org.forgerock.openam.sts.publish.common,
io.swagger,
org.forgerock.openam.oauth2.pop,
org.forgerock.openam.sm.datalayer,
org.forgerock.openam.social.idp.choiceValues.AllowedJweAlgorithms,
org.forgerock.http,
oauth2,
org.forgerock.openam.service.datastore.LdapDataStoreService,
org.forgerock.http.filter,
org.apache.http.wire,
org.forgerock.http.swagger.OpenApiRequestFilter,
org.forgerock.openam.xui,
org.forgerock.api.models,
com.iplanet.services.ldap.event,
org.forgerock.json.jose.jws.SigningManager,
com.sun.identity.shared.xml.XMLUtils,
org.forgerock.http.oauth2,
org.forgerock.util.promise.PromiseImpl,
org.forgerock.openam.secrets,
org.forgerock.openam.sts.publish.service,
org.forgerock.openam.sm.config.ConsoleConfigHandlerImpl,
org.forgerock.openam.integration.idm,
com.sun.identity.authentication,
io.swagger.models,
org.forgerock.openam.selfservice.SelfServiceRequestHandler,
org.forgerock.am.health.LivenessCheckEndpoint,
com.sun.identity.sm.RootSuffixProvider,
org.forgerock.am.iot,
idRepoAuditor,
org.forgerock.openam.sm.datalayer.impl,
org.forgerock.http.util,
com.sun.identity.plugin.session.impl,
com.sun.identity.common,
org.forgerock.openam.utils.PerThreadCache,
com.sun.identity.shared.xml,
org.forgerock.openam.service.datastore,
com.sun.identity.shared.datastruct,
org.forgerock.json.jose.jws,
com.sun.identity.common.configuration.ConfigurationObserver,
com.sun.identity.configuration,
org.forgerock.http.filter.TransactionIdInboundFilter,
frRest,
org.forgerock.secrets.propertyresolver,
org.apache,
org.forgerock.openam.service,
org.forgerock.openam.secrets.SecretsUtils,
org.forgerock.openam.utils.LogUtils,
ROOT,
com.sun.identity.common.ShutdownManager,
org.forgerock.am.iot.GetAccessTokenActionHandler,
org.forgerock.openam.core.rest.authn,
org.forgerock.openam.scripting.persistence.config.consumer.ScriptTypeAdapter,
com.sun,
org.forgerock.util.i18n,
org.forgerock.openam.entitlement.service.ApplicationServiceImpl,
com.sun.identity.policy.plugins.PrefixResourceName,
com.sun.identity.wsfederation.plugins,
org.forgerock.openam.secrets.config.GoogleSecretManagerSecretStoreProvider,
org.forgerock.api.transform,
org,
org.forgerock.util.encode,
com.sun.identity.sm.SmsWrapperObject,
org.forgerock.openam.sm.config,
org.forgerock.openam.scripting.sandbox,
org.forgerock.openam.shared.security,
org.forgerock.api.transform.OpenApiTransformer,
org.forgerock.http.oauth2.ResourceServerFilter,
org.forgerock.openam.headers,
com.sun.identity,
org.forgerock.openam.core.rest.authn.http,
org.forgerock.openam.errors,
org.forgerock.openam.idrepo.ldap.helpers,
org.forgerock.openam.secrets.config.SecretsPlugin,
org.forgerock.http.protocol.Form,
org.forgerock.json.resource,
org.forgerock.util.i18n.PreferredLocales,
com.iplanet.services.ldap,
com.sun.identity.sm.schema.ParsedSchema,
org.forgerock.openam.scripting.service.ScriptChoiceValues,
org.forgerock.openam.sts.publish.rest.RestSTSInstancePublisherImpl,
org.forgerock.openam.errors.AgentResourceExceptionMappingHandler,
org.forgerock.config.resolvers.FlatFileResolver,
org.forgerock.http.routing,
org.forgerock.openam.oauth2.pop.MutualTlsConfirmationMethod,
org.forgerock.openam.scripting.StandardScriptEvaluator,
org.forgerock.am.iot.IotClientRegistrationStore,
org.forgerock.http.servlet.Servlet3Adapter,
org.forgerock.openam.idrepo,
org.forgerock.config,
ldapUrl,
org.forgerock.json.resource.InterfaceSingletonHandler,
org.forgerock.openam.secrets.config,
org.forgerock.openam.sm.DefaultAnnotatedServiceRegistry,
org.forgerock.am.health,
org.forgerock.caf.authentication.framework,
org.forgerock.am.iot.GetUserTokenActionHandler,
com.sun.identity.authentication.UI.LoginLogoutMapping,
org.forgerock.openam.config,
io,
org.forgerock.caf.authentication,
org.forgerock.openam.sm,
org.forgerock.openam.sm.ServiceSchemaRegistrar,
org.forgerock.api.models.Operation,
org.forgerock.http.protocol,
org.forgerock.util.DirectoryWatcher,
com.sun.identity.security,
org.forgerock.openam.entitlement,
org.forgerock.openam.oauth2.ClientCertificateHeaderFormat,
org.forgerock.am.iot.GetUserCodeActionHandler,
org.forgerock.openam.shared,
org.forgerock.http.servlet,
org.forgerock.api.CrestApiProducer,
org.forgerock.openam.sm.annotations.SchemaBuilder,
org.forgerock.openam.scripting.sandbox.RhinoSandboxClassShutter,
org.forgerock.util.xml,
com.sun.identity.authentication.service.ConfiguredIdentityTypes,
org.forgerock.openam.xacml,
org.forgerock.openam.scripting.service.GlobalScriptChoiceValues,
com.iplanet.services.ldap.Server,
com.sun.identity.sm,
org.forgerock.openam.sts.publish.rest.RestSTSPublishServiceListener,
org.forgerock.secrets.AllowedKeyUsageConstraint,
org.forgerock.openam.oauth2.jar,
org.forgerock.openam.oauth2.OAuth2Utils,
org.forgerock.openam.sm.health.FbcLivenessCheck,
org.forgerock.json.resource.http,
org.forgerock.openam.idrepo.ldap,
com.sun.identity.authentication.UI,
com.iplanet.services.util,
com.sun.identity.liberty.ws,
com.sun.identity.authentication.server,
org.forgerock.util,
com.iplanet.sso,
org.forgerock.openam.sm.health.PluginStartupCheck,
org.forgerock.guice.core.InjectorFactory,
org.forgerock.openam.sm.datalayer.impl.ldap,
org.forgerock.openam.sts.publish,
org.forgerock.macaroons,
org.forgerock.openam.selfservice.SelfServiceTreesResource,
com,
org.forgerock.openam.scripting.service.StandardScriptStoreFactory,
org.forgerock.openam.scripting.persistence.config,
org.forgerock.openam.validation,
com.sun.identity.authentication.service,
com.sun.identity.sm.SMSThreadPool,
org.forgerock.openam.validation.RequestEntitySizeVerificationFilter,
org.forgerock.util.promise.Promises$CompletedPromise,
com.sun.identity.authentication.service.AuthConfigMonitor,
org.forgerock.am,
org.forgerock.openam.scripting.service,
org.forgerock.api,
org.forgerock.http.header.SetCookieHeader,
org.forgerock.macaroons.SerializationFormatV2,
org.forgerock.am.iot.IotService,
org.forgerock.openam.ldap,
com.iplanet.am,
com.sun.identity.plugin,
org.forgerock.macaroons.SerializationFormatV1,
com.sun.identity.plugin.session,
org.forgerock.openam.services,
org.forgerock.util.xml.XMLUtils,
org.forgerock.openam.oauth2.saml2.core,
org.forgerock.openam.social.idp,
org.forgerock.openam.config.ServiceComponentConfigBuilder,
org.forgerock.openam.core.rest.session.action,
com.sun.identity.liberty,
org.forgerock.openam.homedirectory,
org.forgerock.openam.scripting.StandardScriptEngineManager,
org.forgerock.openam.secrets.Secrets,
org.forgerock.caf.authentication.framework.AuthenticationFramework,
org.forgerock.json.jose.utils.Utils,
org.forgerock.openam.social.idp.SocialIdentityProviders,
org.forgerock.openam.core.rest.authn.AuthIdHelper,
org.forgerock.openam.oauth2,
org.forgerock.openam.core.CoreWrapper,
org.forgerock.guice,
org.forgerock.http.protocol.Entity,
org.forgerock.openam.sts.publish.service.RestSTSPublishServiceRequestHandler,
org.forgerock.openam.scripting.persistence.config.consumer,
org.forgerock.openam.network,
org.forgerock.http.header,
org.forgerock.openam.entitlement.service,
org.forgerock.openam.integration,
com.sun.identity.common.SystemTimer,
org.forgerock.openam.core,
com.sun.identity.sm.SmsChangesLogger,
org.forgerock.openam.sm.datalayer.impl.CtsConnectionCheck,
org.forgerock.openam.sts,
com.sun.identity.authentication.server.AuthXMLHandler,
org.forgerock.openam.sm.annotations,
org.forgerock.config.resolvers.PropertyResolvers,
org.forgerock.secrets.SecretsProvider,
com.sun.identity.policy,
com.sun.identity.wsfederation,
org.forgerock.json.resource.http.HttpAdapter,
org.forgerock.http.util.Uris,
com.sun.identity.shared.datastruct.CollectionHelper,
org.forgerock.guice.core,
org.forgerock,
org.forgerock.openam.sts.publish.rest,
org.forgerock.openam.social.idp.choiceValues,
com.iplanet.services.util.Crypt,
com.sun.identity.config,
org.forgerock.json.resource.InterfaceCollectionHandler,
org.forgerock.openam,
jsp.realmSelection,
org.forgerock.openam.service.datastore.SmsDataStoreLookup,
com.sun.identity.authentication.service.AMLoginContext,
com.sun.identity.authentication.spi,
org.forgerock.config.util.JsonValuePropertyEvaluator,
org.forgerock.openam.xacml.v3,
org.forgerock.http.routing.Router,
com.iplanet.services.ldap.LDAPUser,
com.sun.identity.policy.util,
org.apache.http,
com.sun.identity.sm.schema,
org.forgerock.http.servlet.HttpFrameworkServlet,
org.forgerock.openam.setup,
org.forgerock.openam.social.idp.DefaultOpenIdConnectRelyingPartySettings,
org.forgerock.openam.headers.SecureCookieFilter,
com.iplanet.services.util.JCEEncryption,
org.forgerock.json.jose,
org.forgerock.openam.oauth2.OAuth2NotificationPublisher,
com.sun.identity.security.cert,
org.forgerock.json.jose.utils,
org.forgerock.caf,
org.forgerock.openam.oauth2.jar.JarAuthorizeRequestValidator,
org.forgerock.openam.sm.health,
org.forgerock.config.resolvers.ChainedPropertyResolver
    Plugins 
    Plugin Framework
org.forgerock.openam.plugins
    Policy 
    Policy Framework,Subject, Condition, Resource Attributes, XACML, Plugins, API
com.sun.identity.policy.PolicyManager,
com.sun.identity.policy.plugins.Organization,
com.sun.identity.policy.SharedSubject,
com.sun.identity.policy.ActionDecision,
com.sun.identity.policy.ResourceManager,
com.sun.identity.policy.plugins.IDRepoResponseProvider,
com.sun.identity.policy.plugins.AuthSchemeCondition,
com.sun.identity.policy.plugins.LEAuthLevelCondition,
com.sun.identity.policy.PolicyCache,
com.sun.identity.policy.PolicyDecision,
org.forgerock.openam.entitlement.monitoring,
com.sun.identity.policy.ProxyPolicyEvaluatorFactory,
com.sun.identity.policy.Rule,
com.sun.identity.policy.ResourceComparatorValidator,
com.sun.identity.policy.plugins.IPCondition,
com.sun.identity.policy.ProxyPolicyEvaluator,
com.sun.identity.policy.remote,
com.sun.identity.policy.ValidationErrorHandler,
org.forgerock.openam.entitlement.rest.EntitlementsExceptionMappingHandler,
org.forgerock.openam.network.ipv6,
com.sun.identity.policy.Subjects,
com.sun.identity.policy.plugins.PeerOrgReferral,
com.sun.identity.policy.Policy,
com.sun.identity.policy.ActionSchema,
org.forgerock.openam.idrepo.ldap.helpers.ADHelper,
org.forgerock.openam.entitlement.configuration,
com.sun.identity.policy.plugins.SubOrgReferral,
com.sun.identity.policy.plugins.AuthenticateToRealmCondition,
org.forgerock.openam.entitlement.indextree,
com.sun.identity.policy.SubjectEvaluationCache,
org.forgerock.openam.uma.rest.UserPolicyResource,
com.sun.identity.policy.plugins.OrgReferral,
com.sun.identity.policy.plugins.LDAPUsers,
com.sun.identity.policy.plugins.UserSelfCheckCondition,
com.sun.identity.policy.ResponseProviderTypeManager,
com.sun.identity.policy.plugins.LDAPFilterCondition,
com.sun.identity.policy.plugins.SimpleTimeCondition,
com.sun.identity.policy.ResponseProviders,
org.forgerock.openam.xacml.v3.resources,
com.sun.identity.policy.PolicyUtils,
com.sun.identity.policy.plugins.SessionCondition,
org.forgerock.openam.entitlement.CachingEntitlementCondition,
com.sun.identity.policy.plugins.AMIdentitySubject,
com.sun.identity.policy.Referrals,
com.sun.identity.policy.ResourceIndexManager,
com.sun.identity.policy.plugins.AuthLevelCondition,
com.sun.identity.policy.plugins.LDAPConnectionPools,
com.sun.identity.policy.plugins.AuthenticateToServiceCondition,
com.sun.identity.policy.plugins.AuthRoleCondition,
com.sun.identity.policy.plugins.AMIdentityMembershipCondition,
com.sun.identity.entitlement,
com.sun.identity.policy.PolicyEvaluatorFactory,
com.sun.identity.policy.plugins.SessionPropertyCondition,
org.forgerock.openam.entitlement.PolicyConstants,
com.sun.identity.policy.PolicyEvaluator,
com.sun.identity.policy.ServiceTypeManager,
com.sun.identity.policy.ServiceType,
com.sun.identity.policy.ResourceResult,
com.sun.identity.policy.plugins.ResourceEnvIPCondition,
org.forgerock.openam.entitlement.conditions,
com.sun.identity.policy.ConditionTypeManager,
com.sun.identity.policy.PolicyConfig,
com.sun.identity.policy.plugins.LDAPGroups,
org.forgerock.openam.network.ipv4.IPv4Condition,
com.sun.identity.policy.SubjectTypeManager,
org.forgerock.openam.entitlement.utils,
com.sun.identity.policy.util.PolicyDecisionUtils,
org.forgerock.openam.entitlement.PolicySetNotificationConsumer,
com.sun.identity.policy.Conditions,
org.forgerock.openam.core.rest.authn.http.AuthenticationServiceV2,
com.sun.identity.policy.ReferralTypeManager,
org.forgerock.openam.entitlement.rest.PolicyResource,
org.forgerock.openam.entitlement.rest.JsonPolicyParser
    Push 
    Push Notification
org.forgerock.openam.services.push
    Radius 
    RADIUS server
org.forgerock.openam.radius
    Session 
    Session framework, session management, SSOToken, session failover, API
org.forgerock.openam.core.rest.session.action.SetPropertyActionHandler,
org.forgerock.openam.core.rest.session.action.GetPropertyActionHandler,
org.forgerock.openam.core.rest.session.SessionResource,
com.sun.identity.sm.ServerIDValidator,
org.forgerock.openam.cts,
org.forgerock.openam.core.rest.session.action.LogoutActionHandler,
org.forgerock.openam.dpro,
com.iplanet.sso.providers,
org.forgerock.openam.core.rest.session.action.ValidateActionHandler,
org.forgerock.openam.core.rest.session.action.GetSessionPropertiesActionHandler,
org.forgerock.openam.session,
org.forgerock.openam.sm.datalayer.impl.ldap.ExternalLdapConfig,
org.forgerock.openam.core.rest.session.action.UpdateSessionPropertiesActionHandler,
org.forgerock.openam.core.rest.session.SSOTokenPartialSessionFactory,
org.forgerock.openam.sm.SMSConfigurationFactory,
org.forgerock.openam.sm.datalayer.impl.SeriesTaskExecutorThread,
com.iplanet.dpro,
com.sun.identity.plugin.session.impl.FMSessionNotification,
org.forgerock.openam.core.rest.session.action.GetPropertyNamesActionHandler,
org.forgerock.openam.core.rest.session.SessionResourceUtil,
org.forgerock.openam.core.rest.session.SessionResourceV2,
com.sun.identity.sm.SiteIDValidator,
org.forgerock.openam.core.rest.session.action.DeletePropertyActionHandler
    UmaProvider 
    UMA provider
org.forgerock.openam.oauth2.AccessTokenProtectionFilter,
org.forgerock.openam.uma.UmaSettingsImpl,
org.forgerock.openam.uma.icg,
org.forgerock.openam.uma.PendingRequestEmailTemplate,
org.forgerock.openam.uma.rest.UmaPolicyApplicationListener,
org.forgerock.openam.uma.rest.UmaResourceSetRegistrationHook,
org.forgerock.openam.oauth2.resources.labels,
org.forgerock.openam.uma.UmaProviderSettingsImpl,
org.forgerock.openam.uma.UmaGrantTypeHandler,
org.forgerock.openam.uma.rest.UmaLabelResource,
org.forgerock.openam.uma.PendingRequestsService,
org.forgerock.openam.uma.audit
    WebServices 
    Web services security (WSS), STS, Identity Services
com.sun.identity.liberty.ws.paos,
com.sun.identity.liberty.ws.common,
com.sun.identity.policy.plugins.WebServicesClients,
com.sun.identity.liberty.ws.soapbinding,
com.sun.identity.authentication.spi.WSSReplayPasswd
    amUpgrade 
    Upgrade framework
com.sun.identity.sm.ServiceSchemaModifications,
org.forgerock.openam.upgrade,
com.sun.identity.common.configuration.ServerConfiguration,
com.sun.identity.config.upgrade,
com.sun.identity.security.cert.AMCRLStore

    There is an example logback.xml file, which defines similar appenders and loggers.

  2. To set the logging level for all loggers that output to a particular appender:

    1. Select the name of the appender from the Appender drop-down list.

    2. Select the debug level from the Level drop-down list.

    3. Click Apply.

  3. To set the logging level for a class or package:

    1. Select the name of the individual logger from the Logger drop-down list, or select the global ROOT logger to set the level for all loggers.

      The current debug level is shown in the Level field.

      Scripts that create debug messages have their own logger created after the script has executed at least once.

      The name of the logger has the format: scripts.<context>.<uuid>.(<name>).

      For example, scripts.OIDC_CLAIMS.36863ffb-40ec-48b9-94b1-9a99f71cc3b5.(OIDC Claims Script).

    2. Select a new debug level from the Level drop-down list.

    3. Click Apply.

    When you apply any changes to the logger settings, a Logger settings updated message is shown at the top of the Logback.jsp page.

    Changes made in Logback.jsp apply immediately, but aren’t permanently stored. Restarting AM or the container in which it runs resets the levels.

    You can configure the default settings that’ll apply when AM starts up. Learn more in Change the startup debug settings.

  4. As soon as you have reproduced the problem you are investigating, return to the Logback.jsp page and revert the logger levels to the previous settings, to avoid filling up disk space.

Persistent debug logging with logback.xml

Debug logging can be enabled and persisted in AM by configuring a logback.xml file. This file describes the classes for which to capture debug messages, and the destination, or appender, where the output is stored.

You can find more information about configuring Logback in Logback configuration in the Logback Documentation.

Configure basic debug logging

Follow these steps to configure basic persistent debug logging in AM, using a logback.xml file:

  1. Create a logback.xml file in the AM classpath, for example, in /path/to/tomcat/webapps/am/WEB-INF/classes/.

    To view or use an existing file with example loggers and appenders, place the following logback.xml in your classpath and set the paths for your environment.

    Example logback.xml 
    <configuration>
 <!--    amUpgrade  -->
 <appender name="amUpgrade" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/amUpgrade</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="com.sun.identity.sm.ServiceSchemaModifications" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>
 <logger name="com.sun.identity.common.configuration.ServerConfiguration" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>
 <logger name="com.sun.identity.wsfederation.plugins.DefaultIDPAccountMapper" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>
 <logger name="com.sun.identity.saml2.plugins.DefaultIDPAccountMapper" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>
 <logger name="com.sun.identity.config.upgrade" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>
 <logger name="com.sun.identity.saml2.plugins.DefaultLibrarySPAccountMapper" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>
 <logger name="com.sun.identity.wsfederation.plugins.DefaultADFSPartnerAccountMapper" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>
 <logger name="com.sun.identity.wsfederation.plugins.DefaultIDPAttributeMapper" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>
 <logger name="org.forgerock.openam.upgrade" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>
 <logger name="com.sun.identity.wsfederation.plugins.DefaultSPAttributeMapper" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>
 <logger name="com.sun.identity.saml2.plugins.DefaultSPAccountMapper" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>
 <logger name="com.sun.identity.saml2.plugins.DefaultLibraryIDPAttributeMapper" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>
 <logger name="com.sun.identity.security.cert.AMCRLStore" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>
 <logger name="com.sun.identity.saml2.plugins.DefaultSPAttributeMapper" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>
 <logger name="com.sun.identity.wsfederation.plugins.DefaultLibrarySPAccountMapper" level="Error" >
  <appender-ref ref="amUpgrade"/>
 </logger>

 <!--    Authentication  -->
 <appender name="Authentication" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/Authentication</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="com.sun.identity.authentication.spi.AMLoginModule" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.authn.callbackhandlers" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.spi.AMAuthCallBackImpl" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.service.AuthContextLookup" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.util" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.authentication.service.LoginContextFactory" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.server.AuthContextLocal" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.service.AMAccountLockout" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.service.LoginState" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.UI.LoginViewBean" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.client" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.authn.trees" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.spi.FirstTimeLogin" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.auth" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.authentication.service.SessionPropertyUpgrader" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.UI.AuthExceptionViewBean" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.spi.ReplayPasswd" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.config" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.share" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.authentication.SessionUpgradeVerifier" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.service.DSAMECallbackHandler" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.spi.AMModuleProperties" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.utils.MappingUtils" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.UI.AuthenticationServletBase" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.service.AuthenticationPrincipalDataRetrieverFactory"
         level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.UI.LogoutViewBean" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.iplanet.security" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.internal" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.AuthContext" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.AuthenticatedSharedAgents" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.ldap.LDAPAuthUtils" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.UI.AuthViewBeanBase" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.authentication.modules" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.iplanet.services.cdm" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.authentication.service.AuthUtilsWrapper" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.AuthenticatedAgents" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.spi.JwtReplayPassword" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.AllowedAgents" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.service.AuthenticationServiceAttributeCache" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.jaas" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.service.AuthD" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.authn.core" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.scripting.api" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.common.ISAccountLockout" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.authn.RestAuthCallbackHandlerFactory" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.authn.RestAuthCallbackHandlerManager" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.webhook" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.iplanet.services.cdc" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.modules" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.authn.http.AuthenticationServiceV1" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.authentication.service.AuthUtils" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.AuthenticatedSharedAgentsCondition" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.authentication.service.JAASModuleDetector" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.authn.RestAuthenticationHandler" level="Error" >
  <appender-ref ref="Authentication"/>
 </logger>

 <!--    Configuration   -->
 <appender name="Configuration" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/Configuration</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="com.sun.identity.sm.ServiceSchemaManager" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.iplanet.services.ldap.event.EventService" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.SMSSchema" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.tools" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.SMSUtils" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.common.configuration.ServerConfigXMLObserver" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.ServiceSchema" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.delegation" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.OrganizationConfigManager" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.ldap" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.SMSNotificationManager" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.PluginSchema" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.AttributeValidator" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.ServiceConfigManagerImpl" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.ServiceConfigImpl" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.SMSPropertiesObserver" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.OrganizationConfigManagerImpl" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.AuthenticationServiceNameProviderImpl" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="org.forgerock.openam.xui.XUIFilter" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.ServiceSchemaImpl" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.setup" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.AttributeSchemaState" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.ServiceInstanceImpl" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="org.forgerock.openam.auditors" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.workflow" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.ServiceConfigManager" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="org.forgerock.openam.sm.validation" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.common.configuration.SessionSiteNames" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.ServiceConfig" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.SMServlet" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.ServiceManager" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.common.configuration.ServerPropertyValidator" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.SMSEntry" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.PluginConfig" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="org.forgerock.openam.utils.OpenAMSettingsImpl" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.jaxrpc" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.DNMapper" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.SMSException" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.SMSEventListenerManager" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="org.forgerock.openam.utils.MapHelper" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.ServiceInstance" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.config.util" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.CachedSubEntries" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.PluginConfigImpl" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.authentication.service.ConfiguredSocialAuthServices" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.ServiceSchemaManagerImpl" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.CachedSMSEntry" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.CreateServiceConfig" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.AttributeSchema" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>
 <logger name="com.sun.identity.sm.PluginSchemaImpl" level="Error" >
  <appender-ref ref="Configuration"/>
 </logger>

 <!--    CoreSystem  -->
 <appender name="CoreSystem" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/CoreSystem</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="com.sun.identity.monitoring" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.saml2.idpdiscovery" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.security.cert.CRLValidator" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.xacml.v3.rest" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.SelfServiceUserUiRolePredicate" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.cts" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.sm.datalayer.impl.ldap.LdapSearchHandler" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.security" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.plugin.monitoring.impl" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.sm.datalayer.providers" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.zaxxer.hikari" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.uma.UmaUserUiRolePredicate" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.common.RequestUtils" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.rest.SubjectAttributesResourceV1" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.services.baseurl" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.IdentityRestUtils" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.UserGroupsResource" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.rest" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.authentication.UI.taglib" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.docs" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.log" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.AllAuthenticatedUsersResource" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.utils.WhitelistObjectInputStream" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.dashboard" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.common.SystemTimerPool" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.session.AnyOfAuthzModule" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.rest" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.sms" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.common.admin" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.shared.resourcename" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.security.AdminTokenAction" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.uma.rest.UmaPolicyResourceAuthzFilter" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.shared.concurrency" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.session.SessionResourcePrivilegeAuthzModule" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.rest.ResourceTypesResource" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.uma.rest.UmaPolicyServiceImpl" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.rest.DecisionCombinersResource" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.common.HttpURLConnectionManager" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.sm.datalayer.impl.SeriesTaskExecutor" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.network.ipv4.IPv4AddressRange" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.audit" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.audit" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.common.DNUtils" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.utils.IPRange" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.services.RestSecurity" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.IdentityResourceV4" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.IdentityResourceV3" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.security.SecurityDebug" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.backstage" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.server" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.utils.ClientUtils" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.IdentityResourceV2" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.rest.ApplicationV1Filter" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.IdentityResourceV1" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.devices" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.rest.ApplicationsResource" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.policy.util.Gateway" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.shared.jaxrpc" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.forgerockrest" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.iplanet.am.util" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.iplanet.services.comm" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.authn.AuditHelper" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.sm.datalayer.impl.PooledTaskExecutor" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.ldap.LdifUtils" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.session.action.LogoutByHandleActionHandler" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.sm.datalayer.impl.ldap.LdapQueryBuilder" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.shared.search" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.rest.SubjectTypesResource" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.shared.encode.CookieUtils" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.iplanet.services.naming" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.cors" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.idsvcs" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.jaxrpc" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.http" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.shared.guice" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.utils.AMKeyProvider" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.utils.AuthLevelUtils" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.shared.security.whitelist" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.notifications" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.policy.util.GatewayServletUtils" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.sms" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.blacklist" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.common.configuration.AgentConfiguration" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.rest.ApplicationTypesResource" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.monitoring" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.common.ResourceLookup" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.rest.PolicyV1Filter" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.authentication.server.AuthXMLRequestParser" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.rest.wrappers" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.security.cert.AMCertStore" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.sm.datalayer.impl.SimpleTaskExecutor" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.shared.locale" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.shared.whitelist" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.sm.datalayer.impl.ldap.CTSDJLDAPv3PersistentSearch" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.protocol" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.scripting.rest" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.rest.ConditionTypesResource" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.record" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.security.cert.AMCertPath" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="org.forgerock.openam.utils.ServiceConfigUtils" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>
 <logger name="com.sun.identity.authentication.server.AuthXMLRequest" level="Error" >
  <appender-ref ref="CoreSystem"/>
 </logger>

 <!--    Federation  -->
 <appender name="Federation" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/Federation</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="com.sun.identity.wsfederation.profile" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.servlet" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.plugins.SAML2PluginsUtils" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.plugin.datastore" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.logging" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.protocol" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.common" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.plugins.DefaultAccountMapper" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="org.forgerock.openam.federation" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.wsfederation.plugins.whitelist" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.profile" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.plugins.SAML2IDPProxyFRImpl" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.wsfederation.key" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.multiprotocol" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.plugins.SAML2IDPProxyImpl" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.wsfederation.servlet" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.xacml" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.plugin.monitoring.MonitorManager" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.plugins.DefaultIDPAuthnContextMapper" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.wsfederation.plugins.DefaultAccountMapper" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.plugins.DefaultAttributeMapper" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.wsfederation.plugins.DefaultAttributeMapper" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="org.forgerock.openam.authentication.Saml2SessionUpgradeHandler" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.ecp" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="org.forgerock.openam.wsfederation" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.federation" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="org.forgerock.openam.saml2" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="jsp.saml2" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.plugins.DefaultIDPECPSessionMapper" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.plugin.log" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.wsfederation.meta" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.wsfederation.plugins.DefaultIDPAuthenticationMethodMapper" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.plugins.DefaultFedletAdapter" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.xmlenc" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.plugins.DefaultSPAuthnContextMapper" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.xmlsig" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.liberty.ws.security" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.plugin.session.SessionManager" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.plugin.session.impl.FMSessionProvider" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.key" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.wsfederation.logging" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.assertion" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.plugin.session.impl.FedletSessionProvider" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.meta" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.plugin.configuration" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.saml2.soapbinding" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.wsfederation.common" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>
 <logger name="com.sun.identity.cot" level="Error" >
  <appender-ref ref="Federation"/>
 </logger>

 <!--    IdRepo  -->
 <appender name="IdRepo" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/IdRepo</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="com.sun.identity.common.ISResourceBundle" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="com.iplanet.am.sdk" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="org.forgerock.openam.idrepo.ldap.DJLDAPv3Repo" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="org.forgerock.openam.shared.security.crypto" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="com.iplanet.sso.SSOTokenManager" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="com.iplanet.services.ldap.DefaultDataStoreConfigurationManager" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="com.sun.identity.idm" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="org.forgerock.openam.idrepo.ldap.helpers.DirectoryHelper" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="com.sun.identity.shared.encode.Hash" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="org.forgerock.openam.core.realms" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="org.forgerock.openam.shared.security.ThreadLocalSecureRandom" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="com.iplanet.services.ldap.event.LDAPv3PersistentSearch" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="org.forgerock.openam.idrepo.ldap.psearch" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="com.sun.identity.security.ServerInstanceAction" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="org.forgerock.openam.identity" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>
 <logger name="org.forgerock.openam.ldap.LDAPUtils" level="Error" >
  <appender-ref ref="IdRepo"/>
 </logger>

 <!--    OAuth2Provider  -->
 <appender name="OAuth2Provider" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/OAuth2Provider</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="org.forgerock.openam.oauth2.OpenAMClientRegistrationStore" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.secrets" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openidconnect" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.resources.ResourceSetLabelRegistration" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.OAuth2GlobalSettings" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.OpenAMClientRegistration" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.ciba" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.requesturis" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.OAuth2AuditLogger" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.token" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.IdentityManager" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.IgAgentClientRegistration" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.jwks" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.oauth2" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.utils.RealmNormaliser" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.AgentClientRegistration" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.ClientCredentialsReader" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.remoteconsent" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.OpenAMScopeValidator" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.OAuth2Monitor" level="Error" >
  <appender-ref ref="OAuth2Provider"/>
 </logger>

 <!--    OpenDJ-SDK  -->
 <appender name="OpenDJ-SDK" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/OpenDJ-SDK</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="org.forgerock.opendj.ldif" level="Error" >
  <appender-ref ref="OpenDJ-SDK"/>
 </logger>
 <logger name="org.forgerock.opendj.asn1" level="Error" >
  <appender-ref ref="OpenDJ-SDK"/>
 </logger>
 <logger name="com.forgerock.opendj.util" level="Error" >
  <appender-ref ref="OpenDJ-SDK"/>
 </logger>
 <logger name="com.forgerock.opendj.ldap" level="Error" >
  <appender-ref ref="OpenDJ-SDK"/>
 </logger>
 <logger name="org.forgerock.opendj.ldap" level="Error" >
  <appender-ref ref="OpenDJ-SDK"/>
 </logger>
 <logger name="org.forgerock.opendj.util" level="Error" >
  <appender-ref ref="OpenDJ-SDK"/>
 </logger>

 <!--    Plugins     -->
 <appender name="Plugins" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/Plugins</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="org.forgerock.openam.plugins" level="Error" >
  <appender-ref ref="Plugins"/>
 </logger>

 <!--    Policy  -->
 <appender name="Policy" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/Policy</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="com.sun.identity.policy.PolicyManager" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.Organization" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.SharedSubject" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ActionDecision" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ResourceManager" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.IDRepoResponseProvider" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.AuthSchemeCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.LEAuthLevelCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.PolicyCache" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.PolicyDecision" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.monitoring" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ProxyPolicyEvaluatorFactory" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.Rule" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ResourceComparatorValidator" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.IPCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.PolicyContinuousListener" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ProxyPolicyEvaluator" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.remote" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ValidationErrorHandler" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.rest.EntitlementsExceptionMappingHandler" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.network.ipv6" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.Subjects" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.PeerOrgReferral" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.Policy" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ActionSchema" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.idrepo.ldap.helpers.ADHelper" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.configuration" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.SubOrgReferral" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.AuthenticateToRealmCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.indextree" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.LDAPRoles" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.SubjectEvaluationCache" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.uma.rest.UserPolicyResource" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.OrgReferral" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.LDAPUsers" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.UserSelfCheckCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ResponseProviderTypeManager" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.LDAPFilterCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.SimpleTimeCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ResponseProviders" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.xacml.v3.resources" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.PolicyUtils" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.SessionCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.CachingEntitlementCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.AMIdentitySubject" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.Referrals" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ResourceIndexManager" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.AuthLevelCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.LDAPConnectionPools" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.AuthenticateToServiceCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.AuthRoleCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.AMIdentityMembershipCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.entitlement" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.PolicyEvaluatorFactory" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.SessionPropertyCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.PolicyConstants" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.PolicyEvaluator" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ServiceTypeManager" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ServiceType" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ResourceResult" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.ResourceEnvIPCondition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.conditions" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ConditionTypeManager" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.PolicyConfig" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.LDAPGroups" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.network.ipv4.IPv4Condition" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.SubjectTypeManager" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.utils" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.util.PolicyDecisionUtils" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.PolicySetNotificationConsumer" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.Conditions" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.authn.http.AuthenticationServiceV2" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="com.sun.identity.policy.ReferralTypeManager" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.rest.PolicyResource" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>
 <logger name="org.forgerock.openam.entitlement.rest.JsonPolicyParser" level="Error" >
  <appender-ref ref="Policy"/>
 </logger>

 <!--    Push    -->
 <appender name="Push" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/Push</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="org.forgerock.openam.services.push" level="Error" >
  <appender-ref ref="Push"/>
 </logger>

 <!--    Radius  -->
 <appender name="Radius" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/Radius</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="org.forgerock.openam.radius" level="Error" >
  <appender-ref ref="Radius"/>
 </logger>

 <!--    Session  -->
 <appender name="Session" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/Session</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="org.forgerock.openam.core.rest.session.action.SetPropertyActionHandler" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.session.action.GetPropertyActionHandler" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.session.SessionResource" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="com.sun.identity.sm.ServerIDValidator" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.cts" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.session.action.LogoutActionHandler" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.dpro" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="com.iplanet.sso.providers" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.session.action.ValidateActionHandler" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.session.action.GetSessionPropertiesActionHandler"
         level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.session" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.sm.datalayer.impl.ldap.ExternalLdapConfig" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.session.action.UpdateSessionPropertiesActionHandler"
         level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.session.SSOTokenPartialSessionFactory" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.sm.SMSConfigurationFactory" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.sm.datalayer.impl.SeriesTaskExecutorThread" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="com.iplanet.dpro" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="com.sun.identity.plugin.session.impl.FMSessionNotification" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.session.action.GetPropertyNamesActionHandler" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.session.SessionResourceUtil" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.session.SessionResourceV2" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="com.sun.identity.sm.SiteIDValidator" level="Error" >
  <appender-ref ref="Session"/>
 </logger>
 <logger name="org.forgerock.openam.core.rest.session.action.DeletePropertyActionHandler" level="Error" >
  <appender-ref ref="Session"/>
 </logger>

 <!--    UmaProvider     -->
 <appender name="UmaProvider" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/UmaProvider</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="org.forgerock.openam.oauth2.AccessTokenProtectionFilter" level="Error" >
  <appender-ref ref="UmaProvider"/>
 </logger>
 <logger name="org.forgerock.openam.uma.UmaSettingsImpl" level="Error" >
  <appender-ref ref="UmaProvider"/>
 </logger>
 <logger name="org.forgerock.openam.uma.PendingRequestEmailTemplate" level="Error" >
  <appender-ref ref="UmaProvider"/>
 </logger>
 <logger name="org.forgerock.openam.uma.rest.UmaPolicyApplicationListener" level="Error" >
  <appender-ref ref="UmaProvider"/>
 </logger>
 <logger name="org.forgerock.openam.uma.rest.UmaResourceSetRegistrationHook" level="Error" >
  <appender-ref ref="UmaProvider"/>
 </logger>
 <logger name="org.forgerock.openam.oauth2.resources.labels" level="Error" >
  <appender-ref ref="UmaProvider"/>
 </logger>
 <logger name="org.forgerock.openam.uma.UmaProviderSettingsImpl" level="Error" >
  <appender-ref ref="UmaProvider"/>
 </logger>
 <logger name="org.forgerock.openam.uma.UmaGrantTypeHandler" level="Error" >
  <appender-ref ref="UmaProvider"/>
 </logger>
 <logger name="org.forgerock.openam.uma.rest.UmaLabelResource" level="Error" >
  <appender-ref ref="UmaProvider"/>
 </logger>
 <logger name="org.forgerock.openam.uma.PendingRequestsService" level="Error" >
  <appender-ref ref="UmaProvider"/>
 </logger>
 <logger name="org.forgerock.openam.uma.audit" level="Error" >
  <appender-ref ref="UmaProvider"/>
 </logger>

 <!--    WebServices     -->
 <appender name="WebServices" class="ch.qos.logback.core.FileAppender">
  <file>/path/to/debug/WebServices</file>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <logger name="com.sun.identity.liberty.ws.paos" level="Error" >
  <appender-ref ref="WebServices"/>
 </logger>
 <logger name="com.sun.identity.liberty.ws.common" level="Error" >
  <appender-ref ref="WebServices"/>
 </logger>
 <logger name="com.sun.identity.policy.plugins.WebServicesClients" level="Error" >
  <appender-ref ref="WebServices"/>
 </logger>
 <logger name="com.sun.identity.liberty.ws.soapbinding" level="Error" >
  <appender-ref ref="WebServices"/>
 </logger>
 <logger name="com.sun.identity.authentication.spi.WSSReplayPasswd" level="Error" >
  <appender-ref ref="WebServices"/>
 </logger>

 <!--    OtherLogging rotation created so that ROOT could be set without outputting same debug to all files     -->
 <appender name="OtherLogging" class="ch.qos.logback.core.rolling.RollingFileAppender">
  <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
   <file>/path/to/debug/OtherLogging</file>
   <fileNamePattern>/path/to/debug/OtherLogging.%d{yyyy-MM-dd}-%i</fileNamePattern>
   <maxFileSize>1GB</maxFileSize>
  </rollingPolicy>
  <encoder>
   <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
 </appender>
 <root level="Error">
  <appender-ref ref="OtherLogging" />
 </root>
</configuration>

    Download logback.xml.

  2. In your empty logback.xml file, add a top-level element called configuration.

    For example:

    <configuration>
</configuration>

    This element will contain the configuration of the loggers and appenders, added in later steps.

    • To instruct AM to periodically check the logback.xml file for changes, and apply them to the running instance, add both a scan and a scanPeriod attribute to the <configuration> element. For example:

      <configuration scan="true" scanPeriod="30 seconds">
</configuration>

      If AM isn’t configured to scan the logback.xml file for changes, you’ll need to restart the instance to pick up any changes.

      You can set the scanPeriod attribute to a longer time period, for example one hour, so you don’t have to restart a running system when you need to alter the debugging level.

      Learn more in Automatically reloading configuration file upon modification in the Logback Documentation.

    • To troubleshoot issues when configuring debug logging using the logback.xml file, add a debug attribute, set to true, to the <configuration> element. For example:

      <configuration debug="true">
</configuration>

      AM records debug logging status information to the default log file for the container in which it’s running. For example, in Tomcat, status messages about the configuration of logback are recorded in the Catalina.out file.

      Learn more in Status data in the Logback Documentation.

  3. Define one or more appenders in the <configuration> element.

    The following example appender logs messages to a file named debug.out in the default AM debug directory:

    <configuration>
  <appender name="DEBUG.OUT" class="ch.qos.logback.core.FileAppender">
    <file>am/var/debug/debug.out</file>
    <encoder>
      <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
    </encoder>
  </appender>
</configuration>

    The pattern in the above example creates debug log entries that are identical to the output produced by previous versions of AM, including the transaction ID to aid with tracking events as they occur throughout the system.

    You can also define an appender that uses the JsonLayout class to include the transaction ID automatically. Learn more in Format log files.

  4. Define one or more loggers in the <configuration> element.

    Loggers specify which classes to capture debug messages from, including any sub-classes. They also specify the level of debug information to capture, and which appender is used to store the output.

    This example logger applies the Debug level to the scripts.OIDC_CLAIMS.36863ffb-40ec-48b9-94b1-9a99f71cc3b5.(OIDC Claims Script). Script loggers are only created after the script has executed at least once. The output is recorded in the file specified in the debug.out appender, created in an earlier step:

    <configuration>
  <appender name="DEBUG.OUT" class="ch.qos.logback.core.FileAppender">
    <file>am/var/debug/debug.out</file>
    <encoder>
      <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
    </encoder>
  </appender>
  <logger name="scripts.OIDC_CLAIMS.36863ffb-40ec-48b9-94b1-9a99f71cc3b5.(OIDC Claims Script)" level="Debug" >
    <appender-ref ref="DEBUG.OUT" />
  </logger>
</configuration>

  5. Define a single <root> catch-all element in the <configuration> element, to specify the global logging level for all classes that don’t match any of the loggers defined in the logback.xml file.

    <configuration>
  <appender name="DEBUG.OUT" class="ch.qos.logback.core.FileAppender">
    <file>am/var/debug/debug.out</file>
    <encoder>
      <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
    </encoder>
  </appender>
  <logger name="scripts.OIDC_CLAIMS.36863ffb-40ec-48b9-94b1-9a99f71cc3b5.(OIDC Claims Script)" level="Debug" >
    <appender-ref ref="DEBUG.OUT" />
  </logger>
  <root level="Error">
    <appender-ref ref="DEBUG.OUT" />
  </root>
</configuration>

  6. Save your changes.

    The changes are applied the next time you restart AM, or the container in which it runs.

    If you are editing an existing logback.xml that AM has already loaded, and contains the scan="true" attribute, you don’t need to reboot.

    Instead, wait for the amount of time specified in the scanPeriod attribute, and the new configuration will be loaded into AM.

  7. To confirm the configuration from the logback.xml file has loaded, go to the Logback.jsp file, for example at https://am.example.com:8443/am/Logback.jsp, which reflects the configuration found:

    Logback.jsp reflecting the configuration in logback.xml

    Any changes made in the Logback.jsp are temporary and aren’t persisted to the logback.xml file.

Output to stdout

Configure logback.xml to send logging to standard output. For example, for Apache Tomcat deployments, console output is typically redirected to the Tomcat logging file, catalina.out.

This example configuration captures all debug-level logging using the default <root> element, and redirects it to the STDOUT appender:

<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> (1)
    <encoder>
      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>
  <root level="Debug">                                                 (2)
    <appender-ref ref="STDOUT" />
  </root>
</configuration>

  1. To configure this example, create the following elements:

    1 An <appender> that uses the ch.qos.logback.core.ConsoleAppender class.
    2 A <logger>, or a <root> element as shown here, referencing the STDOUT appender.

  2. Save your changes as described in Configure basic debug logging.

  3. Check that debug logging is now output to stdout. For example:

    tail -f $TOMCAT_HOME/logs/catalina.out

Output to multiple locations

You can direct debug logging to more than one output location by defining multiple appenders and loggers. However, you can’t define more than one root element.

This example defines loggers for the com.sun.identity.sm.ServiceInstance and org.forgerock.openam.utils.MapHelper classes that output debug logging to file using the DEBUG.OUT appender.

All warning-level logging is also directed to standard output using the STDOUT appender.

<configuration>
  <appender name="DEBUG.OUT" class="ch.qos.logback.core.FileAppender"> (1)
    <file>am/var/debug/debug.out</file>
    <encoder>
      <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
    </encoder>
  </appender>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> (2)
    <encoder>
      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>
  <logger name="com.sun.identity.sm.ServiceInstance" level="Debug"> (3)
      <appender-ref ref="DEBUG.OUT" />
    </logger>
  <logger name="org.forgerock.openam.utils.MapHelper" level="Debug">  (3)
    <appender-ref ref="DEBUG.OUT" />
  </logger>
  <root level="Warning">                                                 (4)
    <appender-ref ref="STDOUT" />
  </root>
</configuration>

  1. To configure this example, create the following elements:

    1 An <appender> that uses the ch.qos.logback.core.FileAppender class.
    2 An <appender> that uses the ch.qos.logback.core.ConsoleAppender class.
    3 A <logger> for each script, referencing the DEBUG.OUT appender.
    4 A <logger>, or a <root> element as shown here, referencing the STDOUT appender.

  2. Save and verify your changes as described in Configure basic debug logging.

Format log files

The org.forgerock.openam.logback.JsonLayout class extends Logback JSON layout functionality by adding the transaction ID to the JSON output.

This example shows how you can include the JsonLayout class to format your log files:

<appender name="JSON" class="ch.qos.logback.core.rolling.RollingFileAppender"> (1)
  <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
    <fileNamePattern>am/var/debug/debugLog.%d{yyyy_MM_dd}.json</fileNamePattern>
    <maxHistory>7</maxHistory>
  </rollingPolicy>
  <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">          (2)
    <layout class="org.forgerock.openam.logback.JsonLayout">                   (3)
      <jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter"> (4)
        <prettyPrint>true</prettyPrint>
      </jsonFormatter>
      <timestampFormat>yyyy-MM-dd' 'HH:mm:ss.SSS</timestampFormat>
      <appendLineSeparator>true</appendLineSeparator>
    </layout>
  </encoder>
</appender>

  1. To configure this example, create the following elements:

    1 An <appender> that uses the ch.qos.logback.core.rolling.RollingFileAppender class.
    2 An <encoder> that uses the ch.qos.logback.core.encoder.LayoutWrappingEncoder class.
    3 A <layout> element that uses the org.forgerock.openam.logback.JsonLayout class.
    4 A <jsonFormatter> element that uses the ch.qos.logback.contrib.jackson.JacksonJsonFormatter class.

  2. Save and verify your changes as described in Configure basic debug logging.

    The use of the JsonLayout class results in the addition of a transactionId at the top level of the log entry.

    For example:

    {
  "timestamp" : "2024-12-16 15:39:44.562",
  "level" : "ERROR",
  "thread" : "http-nio-8080-exec-6",
  "mdc" : {
    "transactionId" : "eb0664cc-4615-461e-973a-64a1fc4f659a-34695"
  },
  "logger" : "org.forgerock.openam.core.rest.authn.trees.AuthTrees",
  "message" : "Exception in processing the tree",
  "context" : "default",
  "transactionId" : "eb0664cc-4615-461e-973a-64a1fc4f659a-34695"
}

Rotate debug logs

Logback provides built-in support for a number of log file rotation schemes, including time- and-size based rotation. If you have configured AM with a logback.xml file, you can configure log file rotation in the appenders, as follows:

  1. In the <configuration> element, create an appender that uses the ch.qos.logback.core.rolling.RollingFileAppender class, for example:

    <appender name="DAILYLOG" class="ch.qos.logback.core.rolling.RollingFileAppender">
  <encoder>
    <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
</appender>

    Within the appender, specify whether to rotate based on time, and optionally also size, as follows:

    • To rotate the log files based only on time, add a <rollingPolicy> element to the appender, which uses the ch.qos.logback.core.rolling.TimeBasedRollingPolicy class.

      Include a <fileNamePattern> element that defines when the log files should roll over, and the naming convention.

      For example, the following appender rolls the log file over at midnight each day, and includes the date in the filename:

      <appender name="DAILYLOG" class="ch.qos.logback.core.rolling.RollingFileAppender">
  <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
    <fileNamePattern>am/var/debug/dailyLog.%d{yyyy-MM-dd}.log</fileNamePattern>
  </rollingPolicy>
  <encoder>
    <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
</appender>

    • To rotate the log files based on both time and size, add a <rollingPolicy> element to the appender, which uses the ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy class.

      Include a <fileNamePattern> element that defines when the log files should roll over, and where the counter for rolling over based on size occurs, specified by including %i. You must also include a <maxFileSize> element to define the maximum size of the log files.

      For example, the following appender rolls the log file over at midnight each day, but earlier if the file reaches 2 gigabytes in size, and includes the date in the filename:

      <appender name="DAILYLOG2GB" class="ch.qos.logback.core.rolling.RollingFileAppender">
  <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
    <fileNamePattern>am/var/debug/dailyLog2GB.%d{yyyy-MM-dd}-%i.log</fileNamePattern>
    <maxFileSize>2GB</maxFileSize>
  </rollingPolicy>
  <encoder>
    <pattern>%lo{5}: %d{ISO8601}: Thread[%t]: TransactionId[%X{transactionId}]%n%level: %m%n%ex</pattern>
  </encoder>
</appender>

  2. Save and verify your changes as described in Configure basic debug logging.

    Debug log files will roll over each night, and also if they reach the 2GB size limit. The file names will contain the date, and a counter to signify the order in which they were written.

Change the startup debug settings

You can configure the settings that are applied when AM starts up and there is no logback.xml file present.

The settings specified as defaults will be reflected in the Logback.jsp file, for example at https://am.example.com:8443/am/Logback.jsp. However, they won’t override the configuration contained with a custom logback.xml file.

Set the default debug level

These steps set the default debug level used by all loggers, when AM starts up:

  1. In the AM admin UI, go to Deployment > Servers > Server Name > General > Debugging.

  2. Select an option from the Debug Level field.

    The default level for debug logging is Error. This level is appropriate for normal production operations, in which case no debug log messages are expected.

    Setting the debug log level to Warning increases the volume of messages. Setting the debug log level to Message dumps detailed trace messages.

    Unless told to do so by Ping Identity support, don’t use Warning or Message levels as a default in production. Instead, set the levels on a per-class basis.

  3. Save your changes.

    Changes are applied immediately.

Set the default debug directory

These steps set the default directory used to store debug log files:

  1. In the AM admin UI, go to Deployment > Servers > Server Name > General > Debugging.

  2. Enter a directory in which to store log files in the Debug Directory field.

    The default value is as follows:

    • Unix/Linux

    • Windows

    %BASE_DIR%/var/debug

    BASE_DIR is the local PingAM directory; for example /path/to/am.

    %BASE_DIR%\var\debug

    BASE_DIR is the local PingAM directory; for example /path/to/am.

    Make sure the specified folder can be written to by the account that is running AM or the container in which it runs.

  3. Save your changes.

    The changes are applied the next time you restart AM, or the container in which it runs.

Combine log messages in a single file

These steps log all debug messages to a single debug.out file:

  1. In the AM admin UI, go to Deployment > Servers > Server Name > General > Debugging.

  2. Set the Merge Debug Files property to On.

  3. Save your changes.

    Changes are applied immediately.

    All debug log messages are written to a single debug file named debug.out. The file is located in the directory specified in the Debug Directory property. Learn more in Set the default debug directory.

Capture troubleshooting information

The AM recording facility lets you initiate events to monitor AM while saving output that is useful when performing troubleshooting.

AM recording events save four types of information:

  • AM debug logs

  • Thread dumps, which show you the status of every active thread, with output similar to a JStack stack trace

  • Important runtime properties

  • The AM configuration

You initiate a recording event by invoking the ssoadm start-recording command with a JSON file, or through a REST call with a JSON payload. The file or payload controls the amount of information AM records, the duration of the recording, and the location of recording output files.

Recording control file/payload reference
Record control file configuration properties
issueID

Type: Number

Required. The issue identifier—a positive integer stored internally as a Java long data type. A case number is a good choice for the issueID value.

The issueID is a component of the path at which recorded information is stored.

See Retrieving Recording Information for more information.

referenceID

Type: String

Required. A second identifier for the recording event. Use this property to segregate multiple recording events for the same issue.

The referenceID is a component of the path at which recorded information is stored.

Spaces are not allowed in the referenceID value.

See Retrieving Recording Information for more information.

Description

Type: String

Required. A textual description of the recording event.

zipEnable

Type: Boolean

Required. Whether to compress the output directory into a zip file when recording has stopped.

configExport

Type: Object

Required. An object containing the following properties:

enable

Type: Boolean

Required. Whether to export the AM configuration upon completion of the recording event. Exporting the AM configuration is a best practice, because it is extremely useful to have access to the configuration when troubleshooting.

password

Type: String

Required if enable is true. A key required to import the exported configuration. The key is used the same way that the ssoadm export-svc-cfg command uses the -e argument.

sharePassword

Type: Boolean

Required if enable is true. Whether to show the password value in the ssoadm start-recording, ssoadm get-recording-status, and ssoadm stop-recording output, and in the info.json file, which is output during recording events, and which contains runtime properties.

debugLogs

Type: Object

Required. An object containing the following properties:

debugLevel

Type: String

Required. The debug level to set for the recording event. Set the value of debugLevel to MESSAGE to get the most troubleshooting information from your recording period. Other acceptable but less commonly used values are ERROR and WARNING.

autoStop

Type: Object

Optional. Contains another object used to specify an event that automatically ends a recording period. For time-based termination, specify a time object; for termination based on uncompressed file size, specify a fileSize object. If you specify both time and fileSize objects, the event that occurs first causes recording to stop.

Specifying fileSize and time objects is a best practice, because it ensures that the recorded output does not occupy a larger than expected amount of space on your file system, and that recording events end in a timely fashion.

time

Type: Object

Optional; must be specified in the autoStop object if fileSize is not specified. Configures a recording period to terminate recording after this amount of time.

timeUnit:

Type: String

Required. Acceptable values are MILLISECONDS, SECONDS, MINUTES, HOURS, and DAYS.

value:

Type: Numeric

Required. Values in MILLISECONDS are rounded down to the second. The minimum acceptable value for autoStop is one second.

fileSize

Type: Object

Optional; must be specified in the autoStop object if time is not specified. Configures a recording period to terminate after the aggregate size of uncompressed debug logs has reached this size.

sizeUnit:

Type: String

Required. Acceptable values are B, KB, MB, and GB.

value:

Type: Numeric

Required.

threadDump

Type: Object

Required. An object containing the following properties:

enable

Type: Boolean

Required. Whether to dump threads during the recording event. Thread dumps are especially useful when troubleshooting performance issues and issues with unresponsive servers.

delay

Type: Object

Required if enable is true. Contains another object used to specify an interval at which thread dumps are taken. The initial thread dump is taken at the start of the recording event; subsequent thread dumps are taken at multiples of the delay interval.

timeUnit

Type: String

Required. Acceptable values are MILLISECONDS, SECONDS, MINUTES, HOURS, and DAYS.

value

Type: Numeric

Required. The minimum acceptable value is one second. Time units that are smaller than seconds, such as MILLISECONDS, are rounded to the closest second.

Recording control file/payload example 
{
  "issueID": 103572,
  "referenceID": "policyEvalFails",
  "description": "Troubleshooting artifacts in support of case 103572",
  "zipEnable": true,
  "configExport": {
    "enable": true,
    "password": "5x2RR70",
    "sharePassword": false
  },
  "debugLogs": {
    "debugLevel": "MESSAGE",
    "autoStop": {
      "time": {
        "timeUnit": "SECONDS",
        "value": 15
      },
      "fileSize": {
        "sizeUnit": "GB",
        "value": 1
      }
    }
  },
  "threadDump": {
    "enable": true,
    "delay": {
      "timeUnit": "SECONDS",
      "value": 5
    }
  }
}

The recording control file properties in the preceding example affect the recording output as follows:

Recording control file example properties and effects on recording behavior
Recording Control File Property Value Effect

issueID, referenceID

103572, policyEvalFails

Recording output is stored at the path debugFileLocation/record/103572/policyEvalFails_timestamp.zip. For more information about the location of recording output, see Retrieve recording information.

Description

Troubleshooting artifacts in support of case 103572

No effect.

zipEnable

true

Recording output is compressed into a .zip file.

configExport / enable

true

The AM configuration is exported at the start of the recording event.

configExport / password

5x2RR70

Knowledge of this password will be required to access the AM configuration that was saved during recording.

configExport / sharePassword

false

The password is not displayed in output messages displayed during the recording event or in the info.json file.

debugLogs / debugLevel

MESSAGE

Recording enables message-level debug logs during the recording event.

debugLogs / autoStop / time

SECONDS, 15

Because both the time and fileSize properties are set, recording stops after 15 seconds, or after the size of the debug logs exceeds 1 GB, whichever occurs first.

debugLogs / autoStop / fileSize

GB, 1

Because both the time and fileSize properties are set, recording stops after 15 seconds, or after the size of the debug logs exceeds 1 GB, whichever occurs first.

threadDump / enable

true

Thread dumps are taken throughout the recording event.

threadDump / delay

SECONDS, 5

The first thread dump is taken when the recording event starts. Additional thread dumps are taken every five seconds hence.

The following table shows different tasks related to recording troubleshooting information:

Task or Requirement Resources

Start and stop recording information

Use the ssoadm command or REST calls to start and stop recording information. You can also check if there are active recording events using REST (Get recording status (REST)).

Retrieve information

AM stores the troubleshooting information you gathered, so it is ready to be sent to Ping Identity Support representatives.

Start and stop recording (ssoadm)

Start AM recording with the ssoadm start-recording command. For example:

$ ssoadm \
start-recording \
--servername https://am.example.com:8443/am \
--adminid uid=amAdmin,ou=People,dc=am,dc=example,dc=com \
--password-file /tmp/pwd.txt \
--jsonfile recording.json
{
  "recording": true,
  "record": {
    "issueID": 103572,
    "referenceID": "policyEvalFails",
    "description": "Record everything",
    "zipEnable": false,
    "threadDump": {
      "enable": true,
      "delay": {
        "timeUnit": "SECONDS",
        "value": 5
      }
    },
    "configExport": {
      "enable": true,
      "password": "admin password",
      "sharePassword": true
    },
    "debugLogs": {
      "debugLevel": "message",
      "autoStop": {
        "time": {
          "timeUnit": "MILLISECONDS",
          "value": 15000
        },
        "fileSize": {
          "sizeUnit": "KB",
          "value": 1048576
        }
      }
    },
    "status": "RUNNING",
    "folder": "/path/to/am/var/debug/record/103572/policyEvalFails/"
  }
}

The ssoadm command output in the preceding example is shown in indented format for ease of reading. The actual output is not indented.

In the preceding ssoadm start-recording command example, the recording.json file specifies the information to be recorded and under what conditions recording automatically terminates.

An active recording event stops when:

  • To explicitly tell AM to stop recording, use the ssoadm stop-recording command.

    See the ssoadm for details about this command.

  • Another ssoadm start-recording command is sent to AM that specifies an issue ID that differs from the active recording event’s issue ID. In this case, the initial recording session terminates and the new recording event starts. Note that you can determine whether an AM recording event is active by using the ssoadm get-recording-status command.

  • A timer configured in the recording control file determines that the maximum amount of time for the recording event has been reached.

  • A file size monitor configured in the recording control file determines that the maximum amount of information in debug logs has been reached.

Start and stop recording (REST)

To start a recording event:

  1. Send an HTTP POST request to the /json/records endpoint with the _action=start parameter.

  2. Specify a JSON payload identical in format to the input file for the ssoadm start-recording command.

    For details of the file contents, and an example file, refer to Capture troubleshooting information.

You must authenticate to AM as an administrative user to obtain an SSO token prior to calling the /json/records REST endpoint. You then pass the SSO token in the iPlanetDirectoryPro header as proof of authentication.

$ curl \
--request POST \
--header "Content-Type: application/json" \
--header "iPlanetDirectoryPro: AQIC5…​" \
--header "Accept-API-Version: resource=1.0" \
--data ' {
  "issueID": 103572,
  "referenceID": "policyEvalFails",
  "description": "Troubleshooting artifacts in support of case 103572",
  "zipEnable": true,
  "configExport": {
   "enable": true,
   "password": "5x2RR70",
   "sharePassword": false
  },
  "debugLogs": {
   "debugLevel": "MESSAGE",
   "autoStop": {
    "time":  {
     "timeUnit": "SECONDS",
     "value": 15
    },
    "fileSize": {
     "sizeUnit": "GB",
     "value": 1
    }
   }
  },
  "threadDump" : {
   "enable": true,
   "delay" :  {
    "timeUnit": "SECONDS",
    "value": 5
   }
  }
 }' \
https://am.example.com:8443/am/json/records?_action=start
{
    "recording":true,
    "record":{
        "issueID":103572,
        "referenceID":"policyEvalFails",
        "description":"Troubleshooting artifacts in support of case 103572",
        "zipEnable":true,
        "threadDump":{
            "enable":true,
            "delay":{
                "timeUnit":"SECONDS",
                "value":5
            }
        },
        "configExport":{
            "enable":true,
            "password":"xxxxxx",
            "sharePassword":false
        },
        "debugLogs":{
            "debugLevel":"message",
            "autoStop":{
                "time":{
                    "timeUnit":"MILLISECONDS",
                    "value":15000
                },
                "fileSize":{
                    "sizeUnit":"KB",
                    "value":1048576
                }
            }
        },
        "status":"RUNNING",
        "folder":"/path/to/am/var/debug/record/103572/policyEvalFails/"
    }
}

The curl command output is indented for ease of reading. The actual output is not indented, and the actions available from the /json/records endpoint do not support the _prettyPrint parameter.

To stop a recording event, send an HTTP POST request to the /json/records endpoint, specifying the _action=stop parameter in the URL:

$ curl \
--request POST \
--header "iPlanetDirectoryPro: AQIC5…​" \
--header "Accept-API-Version: resource=1.0" \
https://am.example.com:8443/am/json/records?_action=stop

If there is no active recording event, AM returns a 400 error code.

If there is an active recording event, output similar to the following appears:

{
  "recording": false,
  "record": {
    "issueID": 103572,
    "referenceID": "policyEvalFails",
    "description": "Troubleshooting artifacts in support of case 103572",
    "zipEnable": true,
    "threadDump": {
      "enable": true,
      "delay": {
        "timeUnit": "SECONDS",
        "value": 5
      }
    },
    "configExport": {
      "enable": true,
      "password": "xxxxxx",
      "sharePassword": false
    },
    "debugLogs": {
      "debugLevel": "message",
      "autoStop": {
        "time": {
          "timeUnit": "MILLISECONDS",
          "value": 15000
        },
        "fileSize": {
          "sizeUnit": "KB",
          "value": 1048576
        }
      }
    },
    "status": "STOPPED",
    "folder": "/path/to/am/var/debug/record/103572/policyEvalFails/"
  }
}

Get recording status (REST)

To get the status of a recording event, perform an HTTP POST using the /json/records endpoint, specifying the _action=status parameter in the URL:

$ curl \
--request POST \
--header "iPlanetDirectoryPro: AQIC5…​" \
--header "Accept-API-Version: resource=1.0" \
https://am.example.com:8443/am/json/records?_action=status

If there is no active recording event, the following output appears:

{
    "recording":false
}

If there is an active recording event, output similar to the following appears:

{
    "recording":true,
    "record":{
        "issueID":103572,
        "referenceID":"policyEvalFails",
        "description":"Troubleshooting artifacts in support of case 103572",
        "zipEnable":true,
        "threadDump":{
            "enable":true,
            "delay":{
                "timeUnit":"SECONDS",
                "value":5
            }
        },
        "configExport":{
            "enable":true,
            "password":"xxxxxx",
            "sharePassword":false
        },
        "debugLogs":{
            "debugLevel":"message",
            "autoStop":{
                "time":{
                    "timeUnit":"MILLISECONDS",
                    "value":15000
                },
                "fileSize":{
                    "sizeUnit":"KB",
                    "value":1048576
                }
            }
        },
        "status":"RUNNING",
        "folder":"/path/to/am/var/debug/record/103572/policyEvalFails/"
    }
}

Retrieve recording information

Information recorded by AM is stored in debugFileLocation/record/issueID/referenceID. For example, if the debug file location is /path/to/am/var/debug, the issue ID 103572, and the reference ID policyEvalFails, the path containing recorded information is /path/to/am/var/debug/record/103572/policyEvalFails.

When there are multiple recording events with the same issueID and referenceID, AM appends a timestamp to the referenceID of the earliest paths. For example, multiple recording events for issue ID 103572 and reference ID policyEvalFails might be stored at the following paths:

  • Most recent recording: debugFileLocation/record/103572/policyEvalFails

  • Next most recent recording: debugFileLocation/record/103572/policyEvalFails_2015-10-24-11-48-51-902-PDT

  • Earliest recording: debugFileLocation/record/103572/policyEvalFails_2015-08-10-15-15-10-140-PDT

AM compresses the output from recording events when you set the zipEnable property to true. The output file can be found at the path debugFileLocation/record/issueID/referenceID_timestamp.zip. For example, compressed output for a recording event for issue ID 103572 and reference ID policyEvalFails might be stored at the following path: debugFileLocation/record/103572/policyEvalFails_2015-08-12-12-19-02-683-PDT.zip.

Use the referenceID property value to segregate output when reproducing the same problem multiple times. For example, while troubleshooting case 103572, you notice that you only have a problem when evaluating policy for members of the Finance realm. You could trigger two recording events as follows:

Segregate recording output using the referenceID
AM behavior referenceIDValue Recording output path

Policy evaluation behaves as expected for members of the Engineering realm.

policyEvalSucceeds

debugFileLocation/record/103572/policyEvalSucceeds

Policy evaluation unexpectedly fails for members of the Finance realm.

policyEvalFails

debugFileLocation/record/103572/policyEvalFails

Reference

This reference provides information related to maintaining an AM instance, and covers the following topics:

Monitoring

amster service name: Monitoring

Configuration

The following settings appear on the Configuration tab:

Monitoring Status

Enable the monitoring system in AM.

Default value: false

amster attribute: enabled

Monitoring HTTP Port

Port number for the HTTP monitoring interface.

This attribute is deprecated.

Default value: 8082

amster attribute: httpPort

Monitoring HTTP interface status

Enable / Disable the HTTP access to the monitoring system.

This attribute is deprecated.

Default value: false

amster attribute: httpEnabled

Monitoring HTTP interface authentication file path

Path to the monitoring system authentication file

The openam_mon_auth file contains the username and password of the account used to protect the monitoring interfaces. The default username is demo with a password of changeit. Use the ampassword command to encrypt a new password.

This attribute is deprecated.

Default value: %BASE_DIR%/security/openam_mon_auth

amster attribute: authfilePath

Monitoring RMI Port

Port number for the JMX monitoring interface

This attribute is deprecated.

Default value: 9999

amster attribute: rmiPort

Monitoring RMI interface status

Enable / Disable the JMX access to the monitoring system

This attribute is deprecated.

Default value: false

amster attribute: rmiEnabled

Policy evaluation monitoring history size

Size of the window of most recent policy evaluations to record to expose via monitoring system. Valid range is 100 - 1000000.

This attribute is deprecated.

Default value: 10000

amster attribute: policyHistoryWindowSize

Session monitoring history size

Size of the window of most recent session operations to record to expose via monitoring system. Valid range is 100 - 1000000.

This attribute is deprecated.

Default value: 10000

amster attribute: sessionHistoryWindowSize

Secondary configurations

This service has the following secondary configurations.

crest

Enabled

Default value: false

amster attribute: enabled

graphite

Hostname

The hostname of the Graphite server to which metrics should be published.

amster attribute: host

Port

The port of the Graphite server to which metrics should be published.

Default value: 2004

amster attribute: port

Frequency

The frequency (in seconds) at which metrics should be published.

Default value: 30

amster attribute: frequency

prometheus

Enabled

Default value: false

amster attribute: enabled

Authentication Type

The type of authentication determines whether Prometheus needs to authenticate: None or HTTP Basic.

Default value: HTTP Basic

amster attribute: authenticationType

Username

If the authentication type is HTTP Basic, specify a username for Prometheus to use when accessing the endpoint.

Default value: prometheus

amster attribute: username

Password

If the authentication type is HTTP Basic, specify a password for Prometheus to use when accessing the endpoint.

If you set a Secret Label Identifier and AM finds a matching secret in a secret store, the Password is ignored.

amster attribute: password

Secret Label Identifier

AM uses this identifier to create a specific secret label, using the template am.services.monitoring.prometheus.identifier.secret where identifier is the value of Secret Label Identifier.

The identifier can only contain alphanumeric characters a-z, A-Z, 0-9, and periods (.). It can’t start or end with a period.

If you set a Secret Label Identifier and AM finds a matching secret in a secret store, the Password is ignored.

amster attribute: password

Monitoring metrics

This section describes the monitoring metrics for common REST, JMX, or Graphite interfaces as well as the Prometheus monitoring metrics.

Metric types

The following metric types are available.

Summary

The summary metric samples observations, providing a count of observations, sum total of observed amounts, average rate of events, and moving average rates across sliding time windows.

  • Fields

  • Prometheus fields

Field Description

_id

The metric ID.

_type

The metric type.

count

The number of events recorded for this metric.

total

The sum of the values of events recorded for this metric.

As the increment is always 1, the total and the count are always equal.

m1_rate

The one-minute average rate.

m5_rate

The five-minute average rate.

m15_rate

The fifteen-minute average rate.

mean_rate

The average rate.

units

A description of the units the metric is presented in.

Example

{
  "_id" : "authentication.success",
  "_type" : "summary",
  "count" : 2,
  "total" : 2.0,
  "m1_rate" : 3.2668341885586836E-14,
  "m5_rate" : 7.794695663154025E-5,
  "m15_rate" : 0.01377545747021923,
  "mean_rate" : 8.238608027596704E-4,
  "units" : "events/second"
}

The Prometheus endpoints don’t provide rate-based statistics because rates can be calculated from the time-series data.

Field Description

# TYPE

The metric ID and type formatted as a comment.

_count

The number of events recorded.

_sum

The sum of the number of events recorded.

This metric is output as _total for the deprecated /metrics/prometheus endpoint

Example

# TYPE am_authentication summary
am_authentication_count{outcome="success"} 2.0
am_authentication_sum{outcome="success"} 2.0

Timer

The timer metric combines rate and duration information.

  • Fields

  • Prometheus fields

Field Description

_id

The metric ID.

_type

The metric type.

count

The number of events recorded for this metric.

total

The sum of the durations recorded for this metric.

min

The minimum duration recorded for this metric.

max

The maximum duration recorded for this metric.

mean

The mean average duration recorded for this metric.

stddev

The standard deviation of durations recorded for this metric.

duration_units

The units used for measuring the durations in the metric.

p50

50% of the durations recorded are at or below this value.

p75

75% of the durations recorded are at or below this value.

p95

95% of the durations recorded are at or below this value.

p98

98% of the durations recorded are at or below this value.

p99

99% of the durations recorded are at or below this value.

p999

99.9% of the durations recorded are at or below this value.

m1_rate

The one-minute average rate.

m5_rate

The five-minute average rate.

m15_rate

The fifteen-minute average rate.

mean_rate

The average rate.

rate_units

The units used for measuring the rate of the metric.

Duration-based values, such as min, max, and p50, are weighted towards newer data. By representing approximately the last five minutes of data, the timers make it easier to see recent changes in behavior, rather than a uniform average of recordings since the server was started.

Example

{
  "_id" : "cts.connection.success",
  "_type" : "timer",
  "count" : 486,
  "total" : 80.0,
  "min" : 0.0,
  "max" : 1.0,
  "mean" : 0.1905615495053855,
  "stddev" : 0.39274399467782056,
  "duration_units" : "milliseconds",
  "p50" : 0.0,
  "p75" : 0.0,
  "p95" : 1.0,
  "p98" : 1.0,
  "p99" : 1.0,
  "p999" : 1.0,
  "m1_rate" : 0.1819109974890356,
  "m5_rate" : 0.05433445522996721,
  "m15_rate" : 0.03155662103953588,
  "mean_rate" : 0.020858521722211427,
  "rate_units" : "calls/second"
}

The Prometheus endpoints don’t provide rate-based statistics because rates can be calculated from the time-series data.

Field Description

# TYPE

The metric ID, and type. Formatted as a comment.

The Timer metric type is reported as a Summary type.

_count

The number of events recorded.

_sum

The sum of the number of events recorded.

This metric is output as _total for the deprecated /metrics/prometheus endpoint

{quantile="0.5"}

50% of the durations are at or below this value.

{quantile="0.75"}

75% of the durations are at or below this value.

{quantile="0.95"}

95% of the durations are at or below this value.

{quantile="0.98"}

98% of the durations are at or below this value.

{quantile="0.99"}

99% of the durations are at or below this value.

{quantile="0.999"}

99.9% of the durations are at or below this value.

Duration-based quantile values are weighted towards newer data. By representing approximately the last five minutes of data, the timers make it easier to see recent changes in behavior, rather than a uniform average of recordings since the server was started.

Example

# TYPE am_cts_connection_seconds summary
am_cts_connection_seconds{outcome="success",quantile="0.5",} 0.0
am_cts_connection_seconds{outcome="success",quantile="0.75",} 0.0
am_cts_connection_seconds{outcome="success",quantile="0.95",} 0.001
am_cts_connection_seconds{outcome="success",quantile="0.98",} 0.001
am_cts_connection_seconds{outcome="success",quantile="0.99",} 0.001
am_cts_connection_seconds{outcome="success",quantile="0.999",} 0.001
am_cts_connection_count{outcome="success",} 492.0
am_cts_connection_seconds_sum{outcome="success",} 0.081

Gauge

The gauge metric is a numerical value that can increase or decrease. The value for a gauge is calculated when requested, and represents the state of the metric at that specific time.

  • Fields

  • Prometheus fields

Field Description

_id

The metric ID.

_type

The metric type.

value

The current value of the metric.

Example

{
  "_id" : "jvm.used-memory",
  "_type" : "gauge",
  "value" : 2.13385216E9
}
Field Description

# TYPE

The metric ID, and type. Formatted as a comment.

{Metric ID}

The current value. Large values may be represented in scientific E-notation.

Example

# TYPE am_jvm_used_memory_bytes gauge
am_jvm_used_memory_bytes 2.13385216E9

Distinct counter

Metric providing an estimate of the number of unique values recorded.

For example, this could be used to estimate the number of unique users who have authenticated, or unique client IP addresses.

The DistinctCounter metric is calculated per instance of AM, and can’t be aggregated across multiple instances to get a site-wide view.

  • Fields

  • Prometheus fields

Field Description

_id

The metric ID.

_type

The metric type. Note that the distinctCounter type is reported as a gauge type. The output formats are identical.

value

The calculated estimate of the number of unique values recorded in the metric.

Example

{
  "_id" : "authentication.unique-uuid.success",
  "_type" : "gauge",
  "value" : 3.0
}
Field Description

# TYPE

The metric ID, and type. Note that the distinctCounter type is reported as a gauge type. The output formats are identical. Formatted as a comment.

{Metric ID}

The calculated estimate of the number of unique values recorded in the metric.

Example

# TYPE am_authentication_unique_uuid gauge
am_authentication_unique_uuid{outcome="success"} 3.0

Authentication metrics

AM exposes the following authentication-related monitoring metrics:

  • Authentication metrics

  • Prometheus authentication metrics

Name Type Description

authentication.outcome

Summary

Rate of successful/unsuccessful/timed-out authentication flows.

The count of successful authentications is incremented when an authentication journey completes successfully. Likewise, the authentication count for failure outcomes is incremented for failed authentication journeys.

For example, the authorization code flow requires a user session to exist and could redirect the user to a journey for authentication. The completion of this authentication step would then update the count.

The client credentials grant, however, doesn’t use a journey for authentication and, therefore, doesn’t increment the count.

authentication.module.outcome

Summary

This metric was used only for authentication with modules and chains and is no longer documented.

authentication.unique-uuid.success

DistinctCounter

Count of unique identities that have successfully logged in.
The deprecated /metrics/prometheus endpoint uses _total in metric names. The /metrics/prometheus/0.0.4 endpoint uses _sum in metric names, which conforms with the latest Prometheus version. This table shows only the /metrics/prometheus/0.0.4 endpoint metrics.
Name Type Description

am_authentication_count{outcome=outcome,}

Summary

Rate of successful/unsuccessful/timed-out authentication flows (count).

The count of successful authentications is incremented when an authentication journey completes successfully. Likewise, the authentication count for failure outcomes is incremented for failed authentication journeys.

For example, the authorization code flow requires a user session to exist and could redirect the user to a journey for authentication. The completion of this authentication step would then update the count.

The client credentials grant, however, doesn’t use a journey for authentication and, therefore, doesn’t increment the count.

am_authentication_module_count{module=module,outcome=outcome,}

Summary

This metric was used only for authentication with modules and chains and is no longer documented.

am_authentication_module_sum{module=module,outcome=outcome,}

Summary

This metric was used only for authentication with modules and chains and is no longer documented.

am_authentication_sum{outcome=outcome,}

Summary

Rate of successful/unsuccessful/timed-out authentication flows (total).

am_authentication_unique_uuid{outcome=outcome,}

Gauge

Count of unique identities which have successfully logged in.
Authentication metrics labels
Label Values

outcome

  • success

  • failure

  • timeout

Authorization metrics

AM exposes the following authorization-related monitoring metrics after a policy evaluation takes place:

  • Authorization metrics

  • Prometheus authorization metrics

Name Type Description

authorization.policy-set.policy-set.evaluate.action.action.outcome

Summary

Rate of policy evaluation allowed/denied actions returned under a given policy set

authorization.policy-set.policy-set.evaluate.advice.advice-type

Summary

Rate of policy evaluation advice types returned under a given policy set.

authorization.policy-set.evaluate.subject-cache.size

Gauge

Number of cached subject membership relationships.

authorization.policy-set.policy-set.evaluate.outcome

Timer

Rate of successful/unsuccessful policy evaluation calls under a given policy set and time taken to perform this operation.

authorization.policy-set.policy-set.policy.policy-operation

Gauge

Number of policies created/updated/deleted under a given policy set since this AM instance started.
The deprecated /metrics/prometheus endpoint uses _total in metric names. The /metrics/prometheus/0.0.4 endpoint uses _sum in metric names, which conforms with the latest Prometheus version. This table shows only the /metrics/prometheus/0.0.4 endpoint metrics.
Name Type Description

am_authorization_policy_set_policy_count{operation=policy-operation,policy_set=policy-set,}

Summary

Number of policies created/updated/deleted under a given policy set since this AM instance was started. (Summary)

am_authorization_policy_set_policy_sum{operation=policy-operation,policy_set=policy-set,}

Summary

Number of policies created/updated/deleted under a given policy set since this AM instance was started. (Summary)

am_authorization_policy_set_evaluate_subject_cache_size

Summary

Number of cached subject membership relationships.

am_authorization_policy_set_evaluate_seconds{outcome=outcome,policy_set=policy-set,quantile=quantile,}

Summary

Rate of successful/unsuccessful policy evaluation calls under a given policy set and time taken to perform this operation. (Timer)

am_authorization_policy_set_evaluate_count{outcome=outcome,policy_set=policy-set,}

Summary

Rate of successful/unsuccessful policy evaluation calls under a given policy set and time taken to perform this operation. (Timer)

am_authorization_policy_set_evaluate_seconds_sum{outcome=outcome,policy_set=policy-set,}

Summary

Rate of successful/unsuccessful policy evaluation calls under a given policy set and time taken to perform this operation. (Timer)

am_authorization_policy_set_evaluate_action_sum{action_type=action,outcome="allow",policy_set=policy-set,}

Summary

Rate of policy evaluation allowed/denied actions being returned under a given policy set (total).

am_authorization_policy_set_evaluate_action_count{action_type=action,outcome="allow",policy_set=policy-set,}

Summary

Rate of policy evaluation allowed/denied actions being returned under a given policy set (count).

am_authorization_policy_set_evaluate_advice{policy_set=policy-set,advice-type=advice-type,}

Summary

Rate of policy evaluation advice types being returned under a given policy set.

am_authorization_policy_set_evaluate_advice_count{policy_set=policy-set,advice-type,}

Summary

Rate of policy evaluation advice types being returned under a given policy set (count).

am_authorization_policy_set_evaluate_advice_sum{policy_set=policy-set,advice-type=advice-type}

Summary

Rate of policy evaluation advice types being returned under a given policy set (total).
Authorization metrics labels
Label Values

action

Name of the action as specified in the policy, for example:

  • GET

  • GRANT

  • MODIFY

  • DELEGATE

  • READ

advice-type

Name of the policy condition advice, for example:

  • AuthSchemeConditionAdvice

  • AuthenticateToServiceConditionAdvice

  • AuthLevelConditionAdvice

outcome

  • success

  • allow

policy-operation

Type of operation performed on the policy, for example:

  • create

  • delete

  • update

policy-set

Name of the policy set, for example:

  • iPlanetAMWebAgentService

  • oauth2Scopes

quantile

Refer to Timer for quantile values.

Denylisting metrics

AM exposes the following denylisting monitoring metrics:

  • Denylisting metrics

  • Prometheus denylisting metrics

Name Type Description

denylist type.blacklist.bloomfilter.check.outcome

Summary

Rate of bloom filter denylist checks.

denylist type.blacklist.cts.search.outcome

Timer

Tracks time to search CTS for denylist entries

denylist type.blacklist.cts.search.result

Summary

Rate of denylist entries returned by searches.

denylist type.blacklist.cache.check.cache outcome

Summary

Rate of cache hits/misses of the denylist cache layer.

denylist type.blacklist.check.check outcome

Summary

Rate of denylist checks.
The deprecated /metrics/prometheus endpoint uses _total in metric names. The /metrics/prometheus/0.0.4 endpoint uses _sum in metric names, which conforms with the latest Prometheus version. This table shows only the /metrics/prometheus/0.0.4 endpoint metrics.
Name Type Description

am_blacklist_cts_search_count{blacklist_type=denylist type,outcome=outcome,}

Summary

Tracks time to search CTS for denylist entries (count).

am_blacklist_cts_search_result_count{blacklist_type=denylist type,}

Summary

Rate of denylist entries returned by searches (count).

am_blacklist_cts_search_result_sum{blacklist_type=denylist type,}

Summary

Rate of denylist entries returned by searches (total).

am_blacklist_cts_search_seconds_sum{blacklist_type=denylist type,outcome=outcome,}

Summary

Tracks time to search CTS for denylist entries (count).

am_blacklist_cts_search_seconds{blacklist_type=denylist type,outcome=outcome,quantile=quantile,}

Summary

Tracks time to search CTS for denylist entries.

am_blacklist_bloomfilter_check{blacklist_type=denylist type,outcome=outcome}

Summary

Rate of bloom filter denylist checks.

am_blacklist_cache{blacklist_type=denylist type,outcome=cache outcome}

Summary

Rate of cache hits/misses of the denylist cache layer.

am_blacklist_check{blacklist_type=denylist type,outcome=check outcome}

Summary

Rate of denylist checks.
Denylisting metrics labels
Label Values

denylist type

  • session_client_based

  • oauth2

outcome

  • success

  • failure

cache outcome

  • hit

  • miss

check outcome

  • true The token is denylisted

  • false The token is not denylisted

quantile

Refer to Timer for quantile values.

CTS metrics

AM exposes the following CTS-related monitoring metrics:

  • CTS metrics

  • Prometheus CTS metrics

Name Type Description

cts.connection.outcome

Timer

Rate of successful/unsuccessful CTS connections to DS and time taken to obtain the connection.

cts.connection.state.status

Counter

The number of connections in each state.

cts.reaper.cache.size

Gauge

Number of entries in the token reaper cache.

cts.reaper.cache.cts reaper token type.deletion.outcome

Summary

Rate of successful/unsuccessful token deletions from cache by token type.

cts.reaper.search.cts reaper token type.deletion.outcome

Summary

Rate of successful/unsuccessful token deletions from search by token type.

cts.reaper.search.outcome

Timer

Rate of successful/unsuccessful searches and time taken to perform this operation.

cts.task.cts reaper token type.operation.outcome

Summary

Rate of successful/unsuccessful CTS operation types, by token type and time taken to perform them.
The deprecated /metrics/prometheus endpoint uses _total in metric names. The /metrics/prometheus/0.0.4 endpoint uses _sum in metric names, which conforms with the latest Prometheus version. This table shows only the /metrics/prometheus/0.0.4 endpoint metrics.
Name Type Description

am_cts_connection_count{outcome=outcome,}

Summary

Rate of successful/unsuccessful CTS connections to DS and time taken to obtain the connection.

am_cts_connection_seconds_sum{outcome=outcome,}

Summary

Rate of successful/unsuccessful CTS connections to DS and time taken to obtain the connection (total).

am_cts_connection_seconds{outcome=outcome,quantile=quantile,}

Summary

Rate of successful/unsuccessful CTS connections to DS and time taken to obtain the connection.

am_cts_connection_state{status=status,}

Counter

The number of connections in each state.

am_cts_reaper_cache_size

Gauge

Number of entries in the token reaper cache.

am_cts_reaper_deletion_sum{outcome=outcome,reaper_type="cache",token_type=cts reaper token type,}

Summary

Number of token deletions from cache by token type (total).

am_cts_reaper_deletion_count{outcome=outcome,reaper_type="cache",token_type=cts reaper token type,}

Summary

Number of token deletions from cache by token type (count).

am_cts_reaper_search_count{outcome=outcome,}

Summary

Time taken to perform searches by CTS reaper (count).

am_cts_reaper_search_seconds_sum{outcome=outcome,}

Summary

Time taken to perform searches by CTS reaper (total).

am_cts_reaper_search_seconds{outcome=outcome,quantile=quantile,}

Summary

Time taken to perform searches by CTS reaper.

am_cts_task_count{operation=operation,outcome=outcome,token_type=token-type,}

Summary

Rate of successful/unsuccessful CTS operation types, by token type and time taken to perform them.

am_cts_task_pending{operation=operation,}

Counter

Tracks number of active create operations.

am_cts_task_seconds_sum{operation=operation,outcome=outcome,token_type=token-type,}

Summary

Rate of successful/unsuccessful CTS operation types, by token type and time taken to perform them (total).

am_cts_task_seconds{operation=operation,outcome=outcome,token_type=token-type,quantile=quantile,}

Summary

Rate of successful/unsuccessful CTS operation types, by token type and time taken to perform them.
CTS metrics labels
Label Values

cts reaper token type

  • cluster-notification

  • session

operation

  • create

  • delete

  • partial-query

  • patch

  • query

  • read

  • update

  • upsert

outcome

  • success

  • failure

quantile

Refer to Timer for quantile values.

status

  • out

  • pending

token_type

  • authentication-whitelist

  • back-channel-authentication-state

  • cluster-notification

  • logout-user

  • oauth2-blacklist

  • oauth2-csrf-protection

  • oauth2-grant-set

  • oauth2-stateless-grant

  • oauth2-stateless

  • oauth2

  • push-notification

  • request-uri-object

  • resource-set

  • rest

  • saml2

  • session-blacklist

  • session

  • sts

  • suspended-auth-session

  • transaction

  • uma-audit-entry

  • uma-pending-request

  • uma-permission-ticket

  • uma-requesting-party

  • unknown

JVM metrics

AM exposes the JVM-related monitoring metrics covered in this section.

These metrics may depend on the JVM version and configuration. In particular, garbage-collector-related metrics depend on the garbage collector that the server uses. The garbage-collector metric names are unstable, and can change even in a minor JVM release.

The following JVM metrics are all Gauge metrics.

  • JVM metrics

  • Prometheus JVM metrics

Name Description

jvm.available-cpus

Number of processors available to the Java virtual machine.

jvm.class-loading.loaded

Number of classes loaded since the Java virtual machine started.

jvm.class-loading.unloaded

Number of classes unloaded since the Java virtual machine started.

jvm.free-used-memory

Amount of free memory.

jvm.used-memory

Amount of used memory.

jvm.max-memory

Maximum amount of memory.

jvm.garbage-collector.Copy.count

Number of collections performed by the "copy" garbage collection algorithm.

jvm.garbage-collector.Copy.time

Approximate accumulated time taken by the "copy" garbage collection algorithm.

jvm.garbage-collector.MarkSweepCompact.count

Number of collections performed by the "mark sweep" garbage collection algorithm.

jvm.garbage-collector.MarkSweepCompact.time

Approximate accumulated time taken by the "mark sweep" garbage collection algorithm.

jvm.memory-usage.heap.init

Amount of heap memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.heap.max

Maximum amount of heap memory the Java virtual machine attempts to use.

jvm.memory-usage.heap.committed

Amount of heap memory committed for the Java virtual machine to use.

jvm.memory-usage.heap.used

Amount of heap memory used by the Java virtual machine.

jvm.memory-usage.heap.usage

Percentage of heap memory used out of the maximum available.

jvm.memory-usage.non-heap.init

Amount of non-heap memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.non-heap.max

Maximum amount of non-heap memory the Java virtual machine attempts to use.

jvm.memory-usage.non-heap.committed

Amount of non-heap memory that is committed for the Java virtual machine to use.

jvm.memory-usage.non-heap.used

Amount of non-heap memory used by the Java virtual machine.

jvm.memory-usage.non-heap.usage

Percentage of non-heap memory used out of the maximum available.

jvm.memory-usage.pools.Compressed-Class-Space.init

Amount of "compressed class space" memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.Compressed-Class-Space.max

Maximum amount of "compressed class space" memory the Java virtual machine attempts to use.

jvm.memory-usage.pools.Compressed-Class-Space.committed

Amount of "compressed class space" memory committed for the Java virtual machine to use.

jvm.memory-usage.pools.Compressed-Class-Space.used

Amount of "compressed class space" memory used by the Java virtual machine.

jvm.memory-usage.pools.Compressed-Class-Space.usage

Percentage of "compressed class space" memory used out of the maximum available.

jvm.memory-usage.pools.CodeHeap-'non-nmethods'.init

Amount of CodeHeap "non-nmethods" memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.CodeHeap-'non-nmethods'.max

Maximum amount of CodeHeap "non-nmethods" memory the Java virtual machine attempts to use.

jvm.memory-usage.pools.CodeHeap-'non-nmethods'.committed

Amount of CodeHeap "non-nmethods" memory committed for the Java virtual machine to use.

jvm.memory-usage.pools.CodeHeap-'non-nmethods'.used

Amount of CodeHeap "non-nmethods" memory used by the Java virtual machine.

jvm.memory-usage.pools.CodeHeap-'non-nmethods'.usage

Percentage of CodeHeap "non-nmethods" memory used out of the maximum available.

jvm.memory-usage.pools.CodeHeap-'non-profiled-nmethods'.init

Amount of CodeHeap "non-profiled-nmethods" memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.CodeHeap-'non-profiled-nmethods'.max

Maximum amount of CodeHeap "non-profiled-nmethods" memory the Java virtual machine attempts to use.

jvm.memory-usage.pools.CodeHeap-'non-profiled-nmethods'.committed

Amount of CodeHeap "non-profiled-nmethods" memory committed for the Java virtual machine to use.

jvm.memory-usage.pools.CodeHeap-'non-profiled-nmethods'.used

Amount of CodeHeap "non-profiled-nmethods" memory used by the Java virtual machine.

jvm.memory-usage.pools.CodeHeap-'non-profiled-nmethods'.usage

Percentage of CodeHeap "non-profiled-nmethods" memory used out of the maximum available.

jvm.memory-usage.pools.CodeHeap-'profiled-nmethods'.init

Amount of CodeHeap "profiled-nmethods" memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.CodeHeap-'profiled-nmethods'.max

Maximum amount of CodeHeap "profiled-nmethods" memory the Java virtual machine attempts to use.

jvm.memory-usage.pools.CodeHeap-'profiled-nmethods'.committed

Amount of CodeHeap "profiled-nmethods" memory committed for the Java virtual machine to use.

jvm.memory-usage.pools.CodeHeap-'profiled-nmethods'.used

Amount of CodeHeap "profiled-nmethods" memory used by the Java virtual machine.

jvm.memory-usage.pools.CodeHeap-'profiled-nmethods'.usage

Percentage of CodeHeap "profiled-nmethods" memory used out of the maximum available.

jvm.memory-usage.pools.Metaspace.init

Amount of "metaspace" memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.Metaspace.max

Maximum amount of "metaspace" memory the Java virtual machine attempts to use.

jvm.memory-usage.pools.Metaspace.committed

Amount of "metaspace" memory committed for the Java virtual machine to use.

jvm.memory-usage.pools.Metaspace.used

Amount of "metaspace" memory used by the Java virtual machine.

jvm.memory-usage.pools.Metaspace.usage

Percentage of "metaspace" memory used out of the maximum available.

jvm.memory-usage.pools.Eden-Space.init

Amount of "eden space" memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.Eden-Space.max

Maximum amount of "eden space" memory (young generation) the Java virtual machine attempts to use.

jvm.memory-usage.pools.Eden-Space.committed

Amount of "eden space" memory (young generation) committed for the Java virtual machine to use.

jvm.memory-usage.pools.Eden-Space.used-after-gc

Amount of "eden space" memory (young generation) after the last time garbage collection recycled unused objects in this memory pool.

jvm.memory-usage.pools.Eden-Space.used

Amount of "eden space" memory (young generation) used by the Java virtual machine.

jvm.memory-usage.pools.Eden-Space.usage

Percentage of "eden space" memory (young generation) used out of the maximum available.

jvm.memory-usage.pools.Tenured-Gen.init

Amount of "tenured generation" (old generation) memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.Tenured-Gen.max

Maximum amount of "tenured generation" (old generation) memory the Java virtual machine attempts to use.

jvm.memory-usage.pools.Tenured-Gen.committed

Amount of "tenured generation" memory (old generation) committed for the Java virtual machine to use.

jvm.memory-usage.pools.Tenured-Gen.used-after-gc

Amount of "tenured generation" memory (old generation) after the last time garbage collection recycled unused objects in this memory pool.

jvm.memory-usage.pools.Tenured-Gen.used

Amount of "tenured generation" memory (old generation) used by the Java virtual machine.

jvm.memory-usage.pools.Tenured-Gen.usage

Percentage of "tenured generation" memory (old generation) used out of the maximum available.

jvm.memory-usage.pools.Survivor-Space.init

Amount of "survivor space" memory (young generation) the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.Survivor-Space.max

Maximum amount of "survivor space" memory (young generation) the Java virtual machine attempts to use.

jvm.memory-usage.pools.Survivor-Space.committed

Amount of "survivor space" memory (young generation) committed for the Java virtual machine to use.

jvm.memory-usage.pools.Survivor-Space.used-after-gc

Amount of "survivor space" memory (young generation) after the last time garbage collection recycled unused objects in this memory pool.

jvm.memory-usage.pools.Survivor-Space.used

Amount of "survivor space" memory (young generation) used by the Java virtual machine.

jvm.memory-usage.pools.Survivor-Space.usage

Percentage of "survivor space" memory (young generation) used out of the maximum available.

jvm.memory-usage.total.committed

Amount of memory committed for the Java virtual machine to use.

jvm.memory-usage.total.init

Amount of memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.total.max

Maximum amount of memory the Java virtual machine attempts to use.

jvm.memory-usage.total.used

Amount of memory used by the Java virtual machine.

jvm.thread-state.blocked.count

Number of threads in the BLOCKED state.

jvm.thread-state.count

Number of live threads including both daemon and non-daemon threads.

jvm.thread-state.daemon.count

Number of live daemon threads.

jvm.thread-state.new.count

Number of threads in the NEW state.

jvm.thread-state.runnable.count

Number of threads in the RUNNABLE state.

jvm.thread-state.terminated.count

Number of threads in the TERMINATED state.

jvm.thread-state.timed_waiting.count

Number of threads in the TIMED_WAITING state.

jvm.thread-state.waiting.count

Number of threads in the WAITING state.
Name Description

am_jvm_available_cpus

Number of processors available to the Java virtual machine.

am_jvm_class_loading_loaded

Number of classes loaded since the Java virtual machine started.

am_jvm_class_loading_unloaded

Number of classes unloaded since the Java virtual machine started.

am_jvm_free_used_memory_bytes

Amount of free memory.

am_jvm_used_memory_bytes

Amount of used memory.

am_jvm_max_memory_bytes

Maximum amount of memory.

am_jvm_garbage_collector_copy_count

Number of collections performed by the "copy" garbage collection algorithm.

am_jvm_garbage_collector_copy_time

Approximate accumulated time taken by the "copy" garbage collection algorithm.

am_jvm_garbage_collector_markSweepCompact_count

Number of collections performed by the "mark sweep" garbage collection algorithm.

am_jvm_garbage_collector_markSweepCompact_time

Approximate accumulated time taken by the "mark sweep" garbage collection algorithm.

am_jvm_memory_usage_heap_init

Amount of heap memory the Java virtual machine initially requested from the operating system.

am_jvm_memory_usage_heap_max

Maximum amount of heap memory the Java virtual machine attempts to use.

am_jvm_memory_usage_heap_committed

Amount of heap memory committed for the Java virtual machine to use.

am_jvm_memory_usage_heap_used

Amount of heap memory used by the Java virtual machine.

am_jvm_memory_usage_heap_usage

Percentage of heap memory used out of the maximum available.

am_jvm_memory_usage_non_heap_init

Amount of non_heap memory the Java virtual machine initially requested from the operating system.

am_jvm_memory_usage_non_heap_max

Maximum amount of non_heap memory the Java virtual machine attempts to use.

am_jvm_memory_usage_non_heap_committed

Amount of non_heap memory that is committed for the Java virtual machine to use.

am_jvm_memory_usage_non_heap_used

Amount of non_heap memory used by the Java virtual machine.

am_jvm_memory_usage_non_heap_usage

Percentage of non_heap memory used out of the maximum available.

am_jvm_memory_usage_pools_compressed_Class_Space_init

Amount of "compressed class space" memory the Java virtual machine initially requested from the operating system.

am_jvm_memory_usage_pools_compressed_Class_Space_max

Maximum amount of "compressed class space" memory the Java virtual machine attempts to use.

am_jvm_memory_usage_pools_compressed_Class_Space_committed

Amount of "compressed class space" memory committed for the Java virtual machine to use.

am_jvm_memory_usage_pools_compressed_Class_Space_used

Amount of "compressed class space" memory used by the Java virtual machine.

am_jvm_memory_usage_pools_compressed_Class_Space_usage

Percentage of "compressed class space" memory used out of the maximum available.

am_jvm_memory_usage_pools_codeheapnon_nmethodsinit

Amount of CodeHeap "non_nmethods" memory the Java virtual machine initially requested from the operating system.

am_jvm_memory_usage_pools_codeheap__non_nmethods\__max

Maximum amount of CodeHeap "non_nmethods" memory the Java virtual machine attempts to use.

am_jvm_memory_usage_pools_codeheap__non_nmethods__committed

Amount of CodeHeap "non_nmethods" memory committed for the Java virtual machine to use.

am_jvm_memory_usage_pools_codeheap__non_nmethods\__used

Amount of CodeHeap "non_nmethods" memory used by the Java virtual machine.

am_jvm_memory_usage_pools_codeheap__non_nmethods\__usage

Percentage of CodeHeap "non_nmethods" memory used out of the maximum available.

am_jvm_memory_usage_pools_codeheap__non_profiled_nmethods\__init

Amount of CodeHeap "non_profiled_nmethods" memory the Java virtual machine initially requested from the operating system.

am_jvm_memory_usage_pools_codeheap__non_profiled_nmethods\__max

Maximum amount of CodeHeap "non_profiled_nmethods" memory the Java virtual machine attempts to use.

am_jvm_memory_usage_pools_codeheap__non_profiled_nmethods\__committed

Amount of CodeHeap "non_profiled_nmethods" memory committed for the Java virtual machine to use.

am_jvm_memory_usage_pools_codeheap__non_profiled_nmethods\__used

Amount of CodeHeap "non_profiled_nmethods" memory used by the Java virtual machine.

am_jvm_memory_usage_pools_codeheap__non_profiled_nmethods\__usage

Percentage of CodeHeap "non_profiled_nmethods" memory used out of the maximum available.

am_jvm_memory_usage_pools_codeheap__profiled_nmethods\__init

Amount of CodeHeap "profiled_nmethods" memory the Java virtual machine initially requested from the operating system.

am_jvm_memory_usage_pools_codeheap__profiled_nmethods\__max

Maximum amount of CodeHeap "profiled_nmethods" memory the Java virtual machine attempts to use.

am_jvm_memory_usage_pools_codeheap__profiled_nmethods\__committed

Amount of CodeHeap "profiled_nmethods" memory committed for the Java virtual machine to use.

am_jvm_memory_usage_pools_codeheap__profiled_nmethods\__used

Amount of CodeHeap "profiled_nmethods" memory used by the Java virtual machine.

am_jvm_memory_usage_pools_codeheap__profiled_nmethods\__usage

Percentage of CodeHeap "profiled_nmethods" memory used out of the maximum available.

am_jvm_memory_usage_pools_metaspace_init

Amount of "metaspace" memory the Java virtual machine initially requested from the operating system.

am_jvm_memory_usage_pools_metaspace_max

Maximum amount of "metaspace" memory the Java virtual machine attempts to use.

am_jvm_memory_usage_pools_metaspace_committed

Amount of "metaspace" memory committed for the Java virtual machine to use.

am_jvm_memory_usage_pools_metaspace_used

Amount of "metaspace" memory used by the Java virtual machine.

am_jvm_memory_usage_pools_metaspace_usage

Percentage of "metaspace" memory used out of the maximum available.

am_jvm_memory_usage_pools.Eden_Space_init

Amount of "eden space" memory the Java virtual machine initially requested from the operating system.

am_jvm_memory_usage_pools.Eden_Space_max

Maximum amount of "eden space" memory (young generation) the Java virtual machine attempts to use.

am_jvm_memory_usage_pools.Eden_Space_committed

Amount of "eden space" memory (young generation) committed for the Java virtual machine to use.

am_jvm_memory_usage_pools.Eden_Space_used_after_gc

Amount of "eden space" memory (young generation) after the last time garbage collection recycled unused objects in this memory pool.

am_jvm_memory_usage_pools.Eden_Space_used

Amount of "eden space" memory (young generation) used by the Java virtual machine.

am_jvm_memory_usage_pools.Eden_Space_usage

Percentage of "eden space" memory (young generation) used out of the maximum available.

am_jvm_memory_usage_pools_tenured_Gen_init

Amount of "tenured generation" (old generation) memory the Java virtual machine initially requested from the operating system.

am_jvm_memory_usage_pools_tenured_Gen_max

Maximum amount of "tenured generation" (old generation) memory the Java virtual machine attempts to use.

am_jvm_memory_usage_pools_tenured_Gen_committed

Amount of "tenured generation" memory (old generation) committed for the Java virtual machine to use.

am_jvm_memory_usage_pools_tenured_Gen_used_after_gc

Amount of "tenured generation" memory (old generation) after the last time garbage collection recycled unused objects in this memory pool.

am_jvm_memory_usage_pools_tenured_Gen_used

Amount of "tenured generation" memory (old generation) used by the Java virtual machine.

am_jvm_memory_usage_pools_tenured_Gen_usage

Percentage of "tenured generation" memory (old generation) used out of the maximum available.

am_jvm_memory_usage_pools.Survivor_Space_init

Amount of "survivor space" memory (young generation) the Java virtual machine initially requested from the operating system.

am_jvm_memory_usage_pools.Survivor_Space_max

Maximum amount of "survivor space" memory (young generation) the Java virtual machine attempts to use.

am_jvm_memory_usage_pools.Survivor_Space_committed

Amount of "survivor space" memory (young generation) committed for the Java virtual machine to use.

am_jvm_memory_usage_pools.Survivor_Space_used_after_gc

Amount of "survivor space" memory (young generation) after the last time garbage collection recycled unused objects in this memory pool.

am_jvm_memory_usage_pools.Survivor_Space_used

Amount of "survivor space" memory (young generation) used by the Java virtual machine.

am_jvm_memory_usage_pools.Survivor_Space_usage

Percentage of "survivor space" memory (young generation) used out of the maximum available.

am_jvm_memory_usage_total_committed

Amount of memory committed for the Java virtual machine to use.

am_jvm_memory_usage_total_init

Amount of memory the Java virtual machine initially requested from the operating system.

am_jvm_memory_usage_total_max

Maximum amount of memory the Java virtual machine attempts to use.

am_jvm_memory_usage_total_used

Amount of memory used by the Java virtual machine.

am_jvm_thread_state_blocked_count

Number of threads in the BLOCKED state.

am_jvm_thread_state_count

Number of live threads including both daemon and non_daemon threads.

am_jvm_thread_state_daemon_count

Number of live daemon threads.

am_jvm_thread_state_new_count

Number of threads in the NEW state.

am_jvm_thread_state_runnable_count

Number of threads in the RUNNABLE state.

am_jvm_thread_state_terminated_count

Number of threads in the TERMINATED state.

am_jvm_thread_state_timed_waiting_count

Number of threads in the TIMED_WAITING state.

am_jvm_thread_state_waiting_count

Number of threads in the WAITING state.

OAuth 2.0 metrics

AM exposes the following OAuth 2.0 monitoring metrics:

  • OAuth 2.0 metrics

  • Prometheus OAuth 2.0 metrics

Name Type Description

oauth2.grant.grant-type

Summary

Rate of OAuth 2.0 grant completion by grant type.

oauth2.grant.revoke

Summary

Rate of OAuth 2.0 grant revocation

oauth2.token.token-type.issue

Summary

Rate of OAuth 2.0 token issuance by token type.

oauth2.token.access-token.revoke

Summary

Rate of OAuth 2.0 access token revocation.

oauth2.token.read-as-jwt.outcome

Timer

Rate of successfully/unsuccessfully reading OAuth 2.0 JSON Web Tokens (JWT).
The deprecated /metrics/prometheus endpoint uses _total in metric names. The /metrics/prometheus/0.0.4 endpoint uses _sum in metric names, which conforms with the latest Prometheus version. This table shows only the /metrics/prometheus/0.0.4 endpoint metrics.
Name Type Description

am_oauth2_grant_count{grant_type=grant-type,}

Summary

Rate of OAuth 2.0 grant completion by grant type (count).

am_oauth2_grant_revoke_count{grant_type="unknown",}

Summary

Rate of OAuth 2.0 grant revocation for unknown grant types (count).

am_oauth2_grant_revoke_sum{grant_type="unknown",}

Summary

Rate of OAuth 2.0 grant revocation for unknown grant types (total).

am_oauth2_grant_sum{grant_type=grant-type,}

Summary

Rate of OAuth 2.0 grant completion by grant type (total).

am_oauth2_token_issue_count{token_type=token-type,}

Summary

Rate of OAuth 2.0 token issuance by token type (count).

am_oauth2_token_issue_sum{token_type=token-type,}

Summary

Rate of OAuth 2.0 token issuance by token type (total).

am_oauth2_token_read_as_jwt_count{outcome=outcome,}

Summary

Rate of successfully/unsuccessfully reading OAuth 2.0 JSON Web Tokens (JWT) (count).

am_oauth2_token_read_as_jwt_seconds_sum{outcome=outcome,}

Summary

Rate of successfully/unsuccessfully reading OAuth 2.0 JSON Web Tokens (JWT) (total).

am_oauth2_token_read_as_jwt_seconds{outcome=outcome,quantile=quantile,}

Summary

Rate of successfully/unsuccessfully reading OAuth 2.0 JSON Web Tokens (JWT).

am_oauth2_token_revoke_count{token_type="access-token",}

Summary

Rate of OAuth 2.0 access token revocation (count)

am_oauth2_token_revoke_sum{token_type="access-token",}

Summary

Rate of OAuth 2.0 access token revocation (total)
OAuth 2.0 metrics labels
Label Values

grant-type

  • authorization-code

  • back-channel

  • client-credentials

  • device-code

  • implicit

  • jwt-bearer

  • refresh

  • resource-owner-password

  • saml2

  • token-exchange

  • uma2

outcome

  • success

  • failure

token-type

  • access-token

  • authorization-code

  • device-code

  • id-token

  • ops

  • permission-ticket

  • refresh-token

Scripting metrics

AM exposes the following metrics for monitoring the scripting threadpool:

  • Scripting metrics

  • Prometheus scripting metrics

Name Type Description

scripting.threadpool.max.threads.count.script-context

Gauge

The maximum number of threads in the pool.

scripting.threadpool.min.threads.count.script-context

Gauge

The minimum number of threads in the pool.

scripting.threadpool.max.queue.size.script-context

Gauge

The maximum number of script executions that can be queued before the pool starts rejecting them.

scripting.threadpool.rejected.count.script-context

Counter

The number of script executions that have been rejected by the pool.

scripting.threadpool.threads.count.state.script-context

Gauge

The number of threads that are actively executing scripts.
Name Type Description

am_scripting_threadpool_max_threads_count{context=script-context,}

Gauge

The maximum number of threads in the pool.

am_scripting_threadpool_min_threads_count{context=script-context,}

Gauge

The minimum number of threads in the pool.

am_scripting_threadpool_max_queue_size{context=script-context,}

Gauge

The maximum number of script executions that can be queued before the pool starts rejecting them.

am_scripting_threadpool_rejected_count{context=script-context,}

Counter

The number of script executions that have been rejected by the pool.

am_scripting_threadpool_threads_count{context=script-context,state=state,}

Gauge

The number of threads that are actively executing scripts.
Scripting 2.0 metrics labels
Label Values

script-context

A supported script context value; for example, AUTHENTICATION_TREE_DECISION_NODE or OAUTH2_MAY_ACT.

state

  • active

  • blocked

Session metrics

AM exposes the following session-related monitoring metrics:

  • Session metrics

  • Prometheus session metrics

Name Type Description

session.session-type.session-operation.outcome

Timer

Rate of successful/unsuccessful outcomes for this particular operation type and the time taken to perform this operation.

session.session-type.lifetime

Timer

Rate of session lifetimes.

session.authentication-in-memory.store.size

Gauge

Number of journey sessions stored in the in-memory authentication session store.

session.cts-based.cache.eviction

Summary

Rate of evictions from the session cache. (Summary)

session.cts-based.cache.session-outcome

Summary

Rate of cache hits/misses for the session cache.

session.cts-based.cache.size

Gauge

Number of sessions in the session cache.
The deprecated /metrics/prometheus endpoint uses _total in metric names. The /metrics/prometheus/0.0.4 endpoint uses _sum in metric names, which conforms with the latest Prometheus version. This table shows only the /metrics/prometheus/0.0.4 endpoint metrics.
Name Type Description

am_session_count{operation=session-operation,outcome=session-outcome,session_type=session-type,}

Summary

Rate of successful/unsuccessful sessions for idle/max timeout and time taken to perform this operation (count).

am_session_cts_based_cache_count{outcome=session-outcome,}

Summary

Number of cache hits/misses in the session cache (count).

am_session_cts_based_cache_eviction_count

Summary

Rate of evictions from the session cache.

am_session_cts_based_cache_eviction_sum

Summary

Rate of evictions from the session cache (total).

am_session_cts_based_cache_size

Gauge

Number of sessions in the session cache.

am_session_cts_based_cache_sum{outcome=session-outcome,}

Summary

Number of cache hits/misses in the session cache (total).

am_session_lifetime_count{session_type=session-type,}

Summary

Rate of session lifetimes (count).

am_session_lifetime_seconds_sum{session_type=session-type,}

Summary

Lifetime of session, by session type (total).

am_session_lifetime_seconds{session_type=session-type,quantile=quantile,}

Summary

Lifetime of session, by session type.

am_session_seconds_sum{operation=session-operation,outcome=outcome,session_type=session-type,}

Summary

Rate of OAuth 2.0 grant completion by grant type (count).

am_session_seconds{operation=session-operation,outcome=outcome,session_type=session-type,quantile=quantile,}

Summary

Tracks service time for successful/unsuccessful sessions by operation and session type.

am_session_store_size{session_type="authentication-in-memory",}

Gauge

Number of journey sessions stored in the in-memory authentication session store.

am_session_sum{operation=session-operation,session_type=session-type,}

Summary

Rate of successful/unsuccessful sessions for idle/max timeout and time taken to perform this operation (total).
Session 2.0 metrics labels
Label Values

operation

  • check-exists

  • create

  • dereference-restricted-token-id

  • destroy

  • get-matching-sessions

  • get-restricted-token-id

  • get-valid-sessions

  • is-applicable

  • logout

  • refresh

  • register-listener

  • register-pll-listener

  • resolve

  • set-external-property

  • set-property

  • validate

outcome

  • success

  • failure

`session-operation

  • idle-timeout

  • max-timeout

session-outcome

  • hit

  • miss

session-type

  • authentication-client-based

  • authentication-cts-based

  • authentication-in-memory

  • client-based

  • cts-based