PingIntelligence

Extract user information from JWT in sideband mode

The API Security Engine (ASE) supports the decoding of transparent JSON Web Token (JWT) received as part of application programming interface (API) requests. ASE extracts the user information from the JWT and logs it in ASE access logs. The API Behavioral Security (ABS) AI Engine analyzes these access logs to generate reports and detect attacks.

The following diagram shows the traffic flow when ASE is in sideband mode.

Diagram of traffic flow when the ASE is in sideband mode

A JWT consists of a header, a payload, and a signature. They are concatenated with periods(.). The following is a sample JWT structure.

Sample JWT structure

ASE decodes the payload to extract user information from a JWT. It can decode JWTs received as part of request headers or query strings. In sideband mode, ASE supports only Bearer scheme in the Authorization header.

ASE does not validate JWTs. It just decodes the JWTs and extracts the user information.

ASE supports a list of usernames in JWT. When the username claim in the payload is an array with multiple elements, ASE extracts the first element of the array. The elements in the array can be strings or numbers and the array should be a valid JavaScript Object Notation (JSON) array.

Sample JWT

ASE supports arrays only for username claims in the payload. It does not support arrays in clientid or location claims.

When deployed in sideband mode, ASE receives the API request information from the gateway policy and extracts the metadata. The user_info object contains the user information along with other metadata. The following is an example snippet of information received by ASE from API gateway:

{
 "source_ip": "127.0.0.1",
 "source_port": 12345,
 "method": "GET",
 "url": "/api3?query=eyJ0eXAiOiJKV1QiLCJhbGciHuDXOyfQqAnoXC4bA&abc=xyz",
 "http_version": "1.1",
 "user_info":[{"username":"abc","client_id":"cabfsghhbsag"}],
  "headers": [ { "host": "shop.com" },
     	     { "content-type": "application/xml" },
               { "content-length": "100" },
               { "x-forwarded-for": "dev.pxy.com" },
               { "user-agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.110 Safari/537.36"
                }
             ]
}

ASE extracts the user information from the user_info object, JWT or both. The following scenarios explain the different ways in which ASE extracts user information:

  • If the gateway policy sends the user_info object with username and clientid, ASE does not decode the JWT. It extracts the user information from the user_info object.

  • If the gateway policy sends the user_info object without username and clientid, ASE decodes the JWT to extract the information.

  • If the gateway policy sends the user_info object without a username, but with clientid, ASE decodes the JWT and extractsusername from the JWT and client identifier from the user_info object.

  • If the gateway policy sends the user_info object with a username, but without a clientid, ASE decodes the JWT to extract clientid and captures theusername from the user_info object.

  • If the gateway policy does not send user_info object or sends an invalid user_info object, ASE decodes the JWT to extract theusername and clientid information if available.

If the JWT decoding fails, the API request is not blocked. ASE logs the information got from the gateway policy in the access logs.

Configure API JSON files

This topic discusses what API JSON files are, and how they are configured to secure the APIs in your environment.

API JSON files are used to configure the behavior and properties of your APIs in ASE. The parameters in API JSON files help ASE to uniquely identify the APIs in your environment. Each API has a unique API JSON file in ASE. ASE ships with sample JSON files located in the /config/api directory.

The parameters configured in an API JSON file help ASE extract metadata from API traffic, set decoys to trap intruding attacks, perform health checks on backend servers, and so on. The API JSON parameters also help the ABS AI Engine to build AI models to detect any Indicators of Attacks (IoAs) on APIs. For more information on the parameters in API JSON files, see the following:

Adding API JSON file to ASE

You can manually configure the JSON file with the required parameters and add them to ASE.

The sample JSON file has an extension of .example. If you are customizing the example file, then save the file as a .json file.

Manually add API JSON to ASE

After configuring an API JSON file, add it to ASE to activate ASE processing. To add an API, execute the following CLI command.

/<ASE_Installation path>/pingidentity/ase/bin/cli.sh –u admin -p admin add_api {file_path/api_name}

You can also use the Create API in ASE Admin APIs to add an API JSON file to ASE. Here is a sample curl command for it.

curl --location --request POST '{{API}}=<API Name>' \
--header '{{Access_Key_Header}}: {{Access_Key}}' \
--header '{{Secret_Key_Header}}: {{Secret_key}}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "api_metadata": {
        "protocol": "https",
        "url": "/patmapp",
        "hostname": "*",
        "oauth2_access_token": false,
        "apikey_qs": "",
        <<Request body continues...>>

List API JSON files

You can check the addition of an API JSON file to ASE by executing the following CLI command.

/<ASE_Installation path>/pingidentity/ase/bin/cli.sh –u admin -p admin list_api

You can also use List API in ASE Admin APIs to verify. Here is a samplencurl command for it.

curl --location --request GET '{{List_API}}' \
--header '{{Access_Key_Header}}: {{Access_Key}}' \
--header '{{Secret_Key_Header}}: {{Secret_key}}'

Update API JSON files

After activation, an API JSON definition can be updated in real time. Edit the API JSON file located in the /config/api directory and make the desired changes. Save the edited API JSON file and execute the following CLI command.

/<ASE_Installation path>/pingidentity/ase/bin/cli.sh –u admin -p admin update_api <api_name>

For example:

/opt/pingidentity/ase/bin/cli.sh –u admin -p admin update_api shop
api shop updated successfully

You can also use Update API in ASE Admin APIs to update the JSON. Here is a sample curl command for it.

curl --location --request PUT '{{API}}=<API Name>' \
--header '{{Access_Key_Header}}: {{Access_Key}}' \
--header '{{Secret_Key_Header}}: {{Secret_key}}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "api_metadata": {
        "protocol": "https",
        "url": "/pubatmapp",
        "hostname": "*",
        "oauth2_access_token": false,
         <<Request body continues...>>

Sideband ASE configuration using the ase.conf file

API Security Enforcer (ASE) system-level configuration entails modifying parameters in the config/ase.conf file. Some values have default settings that you can modify to support application requirements.

The following table provides parameter values and descriptions.

Parameter Description

ASE mode

mode

Change the mode to sidebandfor ASE to work in a sideband mode. The default value is inline.

ASE time zone

timezone

Sets ASE’s time zone. The values can be local or UTC. Default value is UTC. If ASE is deployed in a cluster, configure the same time zone on each cluster node manually.

enable_sideband_keepalive

When set to true, ASE sends a keep-alive in response header for the TCP connection between API gateway and ASE. With the default false value, ASE sends a connection close in response header for connection between API gateway and ASE.

This parameter is applicable only when mode is set to sideband.

enable_sideband_authentication

This parameter only applies in the ASE sideband mode. Set it to true to enable authentication with a shared secret between an API gateway and ASE. After setting it to true, generate a sideband authentication token using ASE create_sideband_token command.

enable_mtls

When set to true, mutual TLS (MTLS) is enabled for sideband communication between ASE and the Apigee API Gateway. The default is false.

This feature requires ASE version 5.1.3 or later.

ASE ports

http_ws_port

Data port used for HTTP or WebSocket protocol.

The default value is 8000.

https_wss_port

Data port used for HTTPS or Secure WebSocket (wss).

The default value is 8443.

management_port

Management port used for command-line interface (CLI) and REST API management.

The default value is 8010.

ASE administration and audit

admin_log_level

The level of log detail captured. Options include:

Fatal – 1, Error – 2, Warning – 3, Info – 4, Debug – 5

enable_audit

When set to true, ASE logs all actions performed in ASE in the audit log files.

The default value is true.

syslog_server

Syslog server hostname or IPv4 address:port number.

Leave this parameter blank for no syslog generation.

hostname_refresh

N/A

auth_method

Authentication method used for administrator access.

  • ase::db (Default - Native authentication)

  • pam::ldap(Linux-PAM authentication with script)

ase_health

When true, enables load balancers to perform a health check using the following URL: http(s)://<ASE Name>/ase, where <ASE Name> is the ASE domain name

The default value is false.

Do not configure the /ase URL in an API JSON file.

enable_1G

N/A

http_ws_process

The number of HTTP processes. It is set to 1. Do not change this value.

https_wss_process

The number of HTTPS or processes. It is set to 1. Do not change this value.

enable_access_log

When true, log client traffic request and response information. Default value is true.

flush_log_immediate

When true, log files are immediately written to the file system. When false, log files are written after a time interval. The default value is true.

attack_list_memory

The amount of memory used for maintaining allow and deny lists. The default value is 128 MB.

keystore_password

Password for the key store. For more information on updating the key store password, see Updating Keystore Password.

enable_hostname_rewrite

N/A

ASE cluster

enable_cluster

When true, run setup in cluster mode.

The default value is false, run in standalone mode.

Security

enable_sslv3

When true, enable SSLv3. Default value is false.

server_ca_cert_path

N/A

enable_xff

N/A

enable_firewall

When true, activates the ASE firewall.

The default value is true.

enable_strict_request_parser

When true, ASE blocks client http requests with invalid headers start.

The default value is true.

Real-time API security

enable_ase_detected_attack

When true, activates the real-time security in ASE.

The default value is false.

API deception

decoy_alert_interval

The time interval between decoy API email alerts.

The default value is 180 minutes.

Maximum value is 1440 minutes (24 hours).

AI-based API Behavioral Security (ABS)

enable_abs

When true (default), send access log files to ABS AI Engine for generating API metrics and detecting attacks using machine learning algorithms. Make sure it is set to true when ASE is connected to PingOne.

enable_abs_attack

When true (default), ASE fetches attack list from ABS AI Engine and blocks access by clients in the attack list.

When false, attack list is not downloaded.

abs_attack_request_minute

Time interval in minutes at which ASE fetches ABS attack list. The default value is 10 minutes.

Google Pub/Sub configuration

enable_google_pubsub

Set it to true if you want ASE to push metrics data to Google cloud. The default value is false.

ASE must be in the sideband mode for Google Pub/Sub configuration to take effect.

google_pubsub_topic

The path to your topic for publishing and subscribing the messages. For example,/pingidentity/topic/<your_topic>, such as /viatests/topics/ping_incoming.

google_pubsub_concurrency

The number of concurrent connection between ASE and Google Pub/Sub. The maximum value is 1024 connections. Default value is 1000 connections.

google_pubsub_qps

The number of messages per second that ASE can publish to the topic. Maximum value is 10,000. The default value is 1000.

google_pubsub_apikey

The API Key to establish connection between ASE and Google Pub/Sub. Configuring API Key for Google Pub/Sub is optional.

cache_queue_size

The number of messages that are buffered in cache when ASE is not able to publish to Google Pub/Sub. Maximum size of the queue is 10,000 messages. The default value is 300 messages.

google_pubsub_timeout

The time in seconds for which ASE tries to publish messages to Google Pub/Sub. In case of failure to publish, ASE makes three attempts to publish the message, after which it writes the message to the google_pubsub_failed.log file.

API Publish (ABS)

enable_abs_publish

When true, ASE polls ABS to get list of published APIs and list of non-discovered APIs and decide whether APIs received will be added, deleted or updated. When false, the published list will not be downloaded.

The default value is false.

abs_publish_request_minutes

This value determines how often ASE will get published API list from ABS. The default value is 10 minutes.

Alerts and reports

enable_email

When true, send email notifications. The default value is false.

For more information, see Email alerts and reports.

email_report

Time interval in days at which ASE sends reports. Minimum value is one day and the maximum is seven days.

The default value is 1.

smtp_host

Hostname of SMTP server.

smtp_port

Port number of SMTP server.

smtp_ssl

Set to true if you want email communication to be over SSL. Make sure that the SMTP server supports SSL. If you set smtp_ssl to true and the SMTP server does not support SSL, email communication falls back to the non-SSL channel. The default value is true.

Set it to false if email communication is over a non-SSL channel. The email communication will fail if you set the parameter to false, but the SMTP server only supports SSL communication.

smtp_cert_verification

Set to true if you want ASE to verify the SMTP server’s SSL certificate. The default value is true.

If you set it to false, ASE does not verify SMTP server’s SSL certificate; however, the communication is still over SSL.

If you have configured an IP address as smtp_host and set smtp_cert_verification to true, then make sure that the certificate configured on the SMTP server has the following:

X509v3 extensions:
           X509v3 Key Usage:
              Key Encipherment, Data Encipherment
           X509v3 Extended Key Usage:
              TLS Web Server Authentication
           X509v3 Subject Alternative Name:
               IP Address: X.X.X.X

sender_email

Email address for sending email alerts and reports.

sender_password

Password of sender’s email account.

You can leave this field blank if your SMTP server does not require authentication.

receiver_email

Email address to notify about alerts and reports

See email alerts for more information.

ASE server resource utilization

cpu_usage

Percentage threshold value of CPU utilization.

See email alerts for more information.

memory_usage

Percentage threshold value of memory usage.

email alerts alerts for more information.

filesystem_size

Percentage threshold value of filesystem capacity.

See email alerts for more information.

buffer_size

Customizable payload buffer size to reduce the number of iterations required for reading and writing payloads.

Default value is 16KB. Minimum is 1KB and maximum is 32KB.

Example

The following is a sample ase.conf file:

; This is API Security Enforcer's main configuration file. This file is in the standard .ini format.
; It contains ports, firewall, log, ABS flags. The comments start with a semicolon (;).

; Defines running mode for API Security Enforcer (Allowed values are inline or sideband).
mode=inline

; Defines http(s)/websocket(s) ports for API Security Enforcer. Linux user should have the privilege to bind to these ports.
; If you comment out a port, then that protocol is disabled.
http_ws_port=8000
https_wss_port=8443

; REST API
management_port=8010

; For controller.log and balancer.log only
; 1-5 (FATAL, ERROR, WARNING, INFO, DEBUG)
admin_log_level=4

; Defines the number of processes for a protocol.
; The maximum number of allowed process for each protocol is 6 (1 master + 5 child). The
; following defines 1 process for both http/ws and https/wss protocol.
http_ws_process=1
https_wss_process=1

; Enable or disable access logs to the filesystem (request/response).
; WARNING! It must be set to true for sending logs to ABS for analytics.
enable_access_log=true
; To write access log immediately to the filesystem, set to true.
flush_log_immediate=true

; Setting this value to true will enable this node to participate in an API Security Enforcer
; cluster. Define cluster configurations in the cluster.conf
enable_cluster=false

; Current API Security Enforcer version has 3 firewall features: API Mapping, API Pattern
; Enforcement, and Attack Types.
enable_firewall=true

; X-Forwarded For
enable_xff=false

; SSLv3
enable_sslv3=false

; enable Nagle's algorithm (if NIC card is 1G).
enable_1G=true

; tcp send buffer size in bytes(kernel)
tcp_send_buffer_size=65535
; tcp receive buffer size in bytes(kernel)
tcp_receive_buffer_size=65535

; buffer size for send and receive in KBs (user)
buffer_size=16KB

; Set this value to true, to allow API Security Enforcer to send logs to ABS. This
; configuration depends on the value of the enable_access_log parameter.
enable_abs=true

; Set this value to true, to allow API Security Enforcer to fetch attack list from ABS.
enable_abs_attack=true

; This value determines how often API Security Enforcer will get attack list from ABS.
abs_attack_request_minutes=10

; Set this value to true, to allow API Security Enforcer to fetch published API list from ABS.
enable_abs_publish=false

; This value determines how often API Security Enforcer will get published API list from ABS.
abs_publish_request_minutes=10

; Set this value to true, to allow API Security Enforcer to block auto detected attacks.
enable_ase_detected_attack=false

; Set this value to true to enable email for both alerts and daily reports.
enable_email=false

; Defines report frequency in days [0=no reports, 1=every day, 2=once in two days and max is 7 ; days]
email_report=1
; Specify your email settings
smtp_host=smtp://<smtp-server>
smtp_port=587
; Set this value to true if smtp host support SSL
smtp_ssl=true
; Set this value to true if SSL certificate verification is required
smtp_cert_verification=false
sender_email=
sender_password=
receiver_email=

; Defines threshold for an email alert. For example, if CPU usage is 70%, you will get an
; alert.
cpu_usage=70
memory_usage=70
filesystem_size=70

; Authentication method. Format is <auth_agent>::<auth_service>
; Valid values for auth_agent are ase and pam
; ase agent only supports db auth_service
; pam agent can support user configured pam services
; For example ase::db, pam::passwd, pam::ldap etc
auth_method=ase::db

; Enable auditing. Valid values are true or false.
enable_audit=true

; Decoy alert interval in minutes. [min=15, default=3*60, max=24*60]
decoy_alert_interval=180

; Interval for a hostname lookup (in seconds). [min=10, default=60, max=86400]
hostname_refresh=60

; Syslog server settings. The valid format is host:port. Host can be an FQDN or an IPv4
; address.
syslog_server=

; Attack List size in MB or GB. [min=64MB, max=1024GB]
; ASE will take 3*(configured memory) internally. Make sure that the system has at least
; 3*(configured memory) available
; If you are running ASE inside a container, configure the container to use 3*(configured
; memory) shared memory.
attack_list_memory=128MB

; Enable or Disable health check module. ASE uses '/ase' url for both http and https. This is
; useful if ASE is deployed behind a load balancer.
enable_ase_health=false

; Location for server's trusted CA certificates. If empty, Server's certificate will not be
; verified.
server_ca_cert_path=

; enable client side authentication. This setting is applicable only in sideband mode. Once enabled
; request will be authenticated using authentication tokens.
enable_sideband_authentication=false

; enable connection keepalive for requests from gateway to ase.
; This setting is applicable only in sideband mode.
; Once enabled ase will add 'Connection: keep-alive' header in response
; Once disabled ase will add 'Connection: close' header in response
enable_sideband_keepalive=false

; keystore password
keystore_password=OBF:AES:sRNp0W7sSi1zrReXeHodKQ:lXcvbBhKZgDTrjQOfOkzR2mpca4bTUcwPAuerMPwvM4

; enable hostname rewrite for inline mode. ASE will rewrite the host header in request
; to the server's hostname
enable_hostname_rewrite=false

; enable strict parsing checks for client requests
; If enabled, ASE will block request with invalid header start
; If disabled, it will allow requests
; default value = true
enable_strict_request_parser=true

; Set the timezone to utc or local. The default timezone is utc.
timezone=utc

; Google Pub Sub Configuation
enable_google_pubsub=false

google_pubsub_topic=/topic/apimetrics

; Number of concurrent connections to Google Pub/Sub
; Minimum: 1, Default: 1000, Maximum: 1024
google_pubsub_concurrency=1000

; Number of messages published per second.
; Minimum: 1, Default: 1000, Maximum: 10000
google_pubsub_qps=1000

; Google service account API key (Optional)
google_pubsub_apikey=

; Maximum number of messages buffered in memory
; If queue is full, messages are written to logs/google_pubsub_failed.log
; Minimum: 1, Default: 300, Maximum: 10000
cache_queue_size=300

; Timeout in seconds to publish a message to Google Pub/Sub.
; Minimum: 10, Default: 30, Maximum: 300
google_pubsub_timeout=30