

Security Guidelines for Production Deployment

Given below are the common security guidelines for deploying a MWARE ESB in a production environment.

In addition, see the production deployment checklist and any other product-specific guidelines in the respective product's documentation.

• Runtime-level security
• OS-level security
• Network-level security

Runtime-level security

Given below are the security guidelines for the API-M and Micro Integrator runtimes. Note that some of these guidelines are common to both runtimes, whereas some guidelines are runtime-specific.

API-M runtime security

Given below are the security guidelines for the API-M runtime.

Guideline	Details
Apply security updates	Apply all the security patches relevant to your product version. Note the following: MWARE releases security patch notifications monthly via the Support Portal. However, MWARE issues patches immediately to customers if there are highly critical issues. MWARE does not issue patches publicly for older product versions. Community users are encouraged to use the latest product version to receive all the security issues resolved until that particular product release.
Change default keystores	Change the default key stores and create new keys for all the cryptographic operations. MWARE products, by default, come with a self-signed SSL key. As these keys are public, it is recommended to configure your own keys for security purposes. Consider the following guidelines when creating the keystores: Select a key size of at least 2048 bits. Use an SHA256 certificate. Make sure that ESB default certificates do not exist in any of the keystores in your production environment. For example, be sure to delete the default public certificate in the default trust store that is shipped with the product. For more information on recommendations for using keystores in ESB products, see About Asymmetric Cryptography. For information on how to create and configure your own keys and keystores, see Creating New Keystores.
Encrypt passwords in configuration files	MWARE products use a tool called Secure Vault to encrypt the plain-text passwords in configuration files. See Encrypting Passwords in Configuration Files for instructions.
Change default ports	For information on all the default ports used by MWARE ESB, see Default Product Ports. For example, the default HTTPS port is 9443 and the HTTP port is 9763. In addition, Axis2 services are exposed over the following ports: 8243 and 8280. For information on changing a default port, see Changing the Default Ports with Offset.
Enable read-only access to external user stores (LDAPs etc.)	If your product runtimes are connecting to an external user store, such as Microsoft Active Directory for the purpose of reading and retrieving user information, be sure to enable read-only access to that user store. See Configuring a Read-Only LDAP User Store for the API-M runtime.
Always communicate (with external user stores) over TLS	All connections from your MWARE ESB to external databases, userstores (LDAP), or other services, should be over TLS to ensure adequate network-level protection. Therefore, be sure to use external systems (user stores, databases) that are TLS-enabled.
Connect to data stores using a less privileged user	When connecting the server to external databases or user stores (LDAP), be sure to go through a user who does not have permission to change the data store's schema. Be sure not to use the root user of the data store because all permissions are generally granted to the root user.
Configure strong HTTP(S) security	To have strong transport-level security, disable SSL protocol versions and enable only TLS protocol versions: TLS 1, TLS 1.1, and TLS 1.2. This can be done by replacing the sslProtocol = "TLS" property with sslEnabledProtocols="TLSv1,TLSv1.1,TLSv1.2" under [transport.https.sslHostConfig .properties] in the deployment.toml file. In addition, configure strong ciphers for ThriftAuthenticationService, Tomcat transport, and PassThrough transport in the deployment.toml file. See the following links for instructions: Configuring Transport Level Security Supported Cipher Suites Note : When deciding on the TLS protocol and the ciphers, consider the compatibility with existing client applications. Imposing maximum security might cause functional problems with client applications. Apply ciphers with 256 bit key length if you have applied the "Unlimited" strength policy. Note that an unlimited strength policy is recommended. Also, consider the following factors when deciding on the ciphers: DES/3DES are deprecated and should not be used. MD5 should not be used due to known collision attacks. RC4 should not be used due to analytical cryptographic attacks. DSS is limited to a small 1024 bit key size. Cipher-suites that do not provide Perfect Forward Secrecy/ Forward Secrecy (PFS/FS). GCM based ciphers are recommended over CBC ciphers.
Remove weak ciphers for PassThrough transport	Remove any weak ciphers from the PassThrough transport and ensure that the server does not accept connections using those weak ciphers. For this, PreferredCiphers should be configured for the PassThrough transport in the deployment.toml file (stored in the <PRODUCT_HOME>/repository/conf/ directory). For more information, see Configuring Transport Level Security.
Update the HTTP Response header "Server" value	By default, all MWARE products pass "MWARE Carbon Server" as the server value in HTTP headers when sending HTTP responses. This means that information about the MWARE product stack is exposed through HTTP responses. It is recommended to change this by configuring the server name for relevant connectors via deployment.toml. For more information, see Configuring Transport Level Security.
Enabling HTTP Strict Transport Security Headers (HSTS)	Be sure that HTTP Strict Transport Security (HSTS) is enabled for all the applications deployed in your server. This includes the management console, and any other web applications and/or Jaggery applications. Note that (for MWARE products based on Carbon 4.4.11 or later versions, which implies API-M 2.1.0 and newer) HSTS is disabled for the applications with which the product is shipped by default. This is because HSTS validation can interrupt the development processes by validating signatures of self-signed certificates.
Preventing browser caching	If there are dynamic pages in your application with sensitive information, you need to prevent browser caching. This can be done by making sure that the applications deployed in your server return the relevant HTTP response headers. Note that cache prevention headers are enabled for the applications with which the product is shipped by default. Therefore, you need to manually enable cache prevention headers only for all the new applications that you deploy in your server.
Increase Ephemeral Diffie-Hellman Key size	Before starting the server, open the product startup script (api-manager.sh in Linux and api-manager.bat in Windows) and enter the following with the other Java properties: -Djdk.tls.ephemeralDHKeySize=2048 \
Disable client-initiated renegotiation	Before starting the server, open the product startup script (api-manager.sh in Linux and api-manager.bat in Windows) and enter the following with the other Java properties: -Djdk.tls.rejectClientInitiatedRenegotiation=true \
Enable HostName Verification	If your product is using Carbon Kernel 4.4.17 or a later version (which implies API-M 2.2.0 and newer), make sure that hostname verification is enabled in the product startup script (api-manager.sh in Linux and api-manager.bat in Windows) with the Strict mode. If it is not, you need to enable it as below: -Dhttpclient.hostnameVerifier=Strict \ In Carbon versions prior to 4.4.17, be sure that hostname verification is enabled by setting the following property to false. -Dorg.wso2.ignoreHostnameVerification=false \ For instructions, see Enabling HostName Verification.
Increase JSESSIONID length	If required, increase the session ID length by changing the sessionIDLength attribute of the session manager in the context.xml file (stored in the <PRODUCT_HOME>/repository/conf/tomcat/context.xml directory) as shown below. The default value is 16 bytes. <Manager className="org.wso2.carbon.webapp.mgt.CarbonTomcatSessionManager" sessionIdLength="16"></Manager>
Change default admin credentials	The Administrator account is configured by default. The default user name and password of the administrator account is "admin". To change the administrator credentials, you need to first sign in to the management console of the API-M server as "admin", and then use the Change Password option under Home->Configure->User Management->Users in the navigator. For more information on how to change the password of the administrator in the API-M server, see Changing the super admin credentials.
Restrict access to the management console	Make sure that the permission for signing into the management console is granted only to the users that need to use the management console. For example, the majority of users only need to login to the connected service providers via the MWARE product. Therefore, such users should not have permission to log into the management console. You need to make sure that only administrative users have access to the product's management console. Further, instead of granting all permission to one administrator, you can distribute the responsibilities among administrators by assigning different permissions for conducting various tasks. For instructions, see Managing User Roles.
Enable log rotation and monitoring	Ensure that you have a relevant log rotation scheme to manage logs. Log4J properties for MWARE products can be configured in the <PRODUCT_HOME>/repository/conf/log4j2.properties file. Rollover based on a time period can be configured by changing the below configuration (Default value is 1 day). appender.CARBON_LOGFILE.policies.time.interval = 1 You can also configure rollover based on log file size, and also it is possible to limit the number of backup files. For details on how to configure log rotation and manage log growth details in the API-M runtime, see Managing log growth.
Prevent log forging	Log forging can be identified by appending a UUID to the log message. The conversion character '%u' can be used in the pattern layout to log a UUID. For example, the log pattern can be set as following for AUDIT logs so that the UUID is printed at the beginning of each log record. appender.AUDIT_LOGFILE.layout.pattern = [%u] TID: [%tenantId] [%d] %5p {%c} - %m%ex%n For more information on configuring logging, see Setting up logging in API Manage.
Set appropriate JVM parameters	The recommended JDK version is JDK 8 or 11. For more information, see Tested Operating Systems and JDKs. You do not need to set the Heap and Permgen values for the JVM from JDK 1.8 onwards as the MaxPermSize value has been removed from Hotspot JVM.
Restrict outbound connections of Publisher node	In an API-M deployment, it is recommended to restrict outbound connections of the Control Plane node (which contains the Publisher) and only allow access to the internal nodes (only to the nodes that the Publisher portal is intended to communicate with) of the deployment. Therefore, even if a situation arises where privileged user credentials are exposed to a user with malicious intent, such users will not be able to exploit and perform any unintended network interactions. See the API-M deployment documentation for details.
Use a separate admin user account to login into the system	MWARE recommends that you use two separate admin user accounts in production - one account for logging into the system and the other one as the system user doing configurations (for internal service communications). For more information regarding admin user accounts, see super admin configurations.
Defining callback URL regular expression	For password recovery, you can define a regular expression to validate the callback URL. The default configuration allows any callback URL. Note that if you are using the recovery option, it is highly recommended to define the regular expression that validates and only allows access to specific callback URLs. See the Callback URL Regular Expressions documentation for details.
Configure client authentication	Client authentication is used to identify the application or client making a request to the MWARE ESB REST APIs. By default, web applications provided with MWARE ESB use a set of default credentials for authentication. However, it is recommended to change these default credentials to enhance security. For more details see, Configure client authentication

Micro Integrator runtime security

Given below are the security guidelines for the Micro Integrator runtime.

Guideline	Details
Apply security updates	Apply all the security patches relevant to your product version. Use MWARE Updates to get the latest security patches. See MWARE Updates for details. Note the following: MWARE releases security patch notifications monthly via the Support Portal. However, MWARE issues patches immediately to customers if there are highly critical issues. MWARE does not issue patches publicly for older product versions. Community users are encouraged to use the latest product version to receive all the security issues resolved until that particular product release. MWARE Docker repository releases docker images with security fixes. Users with a subscription can fetch these docker images.
Change default keystores	Change the default key stores and create new keys for all the cryptographic operations. MWARE products, by default, come with a self-signed SSL key. Since these keys are public, it is recommended to configure your own keys for security purposes. Consider the following guidelines when creating the keystores: Select a key size of at least 2048 bits. Use an SHA256 certificate. Make sure that ESB default certificates do not exist in any of the keystores in your production environment. For example, be sure to delete the default public certificate in the default trust store that is shipped with the product. See Creating New Keystores for information on how to create and configure your own keys.
Encrypt passwords in configuration files	MWARE products use a tool called Secure Vault to encrypt the plain-text passwords in configuration files. See Securing Passwords in Configuration Files for instructions.
Change default ports	For information on all the default ports used by MWARE ESB, see Default Product Ports. For information on changing a default port, see Changing the Default Ports with Offset.
Enable read-only access to external user stores (LDAPs etc.)	If your product runtimes are connecting to an external user store for the purpose of reading and retrieving user information, be sure to enable read-only access to that user store. See Configuring a User Store for the Micro Integrator runtime.
Always communicate (with external user stores) over TLS	All connections from your server to external databases, userstores (LDAP), or other services, should be over TLS, to ensure adequate network-level protection. Therefore, be sure to use external systems (user stores, databases) that are TLS-enabled.
Connect to data stores using a less privileged user	When connecting the server to external databases or user stores (LDAP), be sure to go through a user who does not have permission to change the data store's schema. Be sure not to use the root user of the data store because all permissions are generally granted to the root user.
Configure strong HTTP(S) security	To have strong transport-level security, use TLS 1.2 and disable SSL, TLS 1.0 and 1.1. The TLS protocol and strong ciphers are configured for the passthrough transport in the deployment.toml file. See the following links for instructions: Configuring Transport-Level Security Note the following: When deciding on the TLS protocol and the ciphers, consider the compatibility with existing client applications. Imposing maximum security might cause functional problems with client applications. Apply ciphers with 256 bit key length if you have applied unlimited strength policy. Note that unlimited strength policy is recommended. Also, consider the following factors when deciding on the ciphers: DES/3DES are deprecated and should not be used. MD5 should not be used due to known collision attacks. RC4 should not be used due to crypto-analytical attacks. DSS is limited to a small 1024 bit key size. Cipher-suites that do not provide Perfect Forward Secrecy/ Forward Secrecy (PFS/FS). GCM based ciphers are recommended over CBC ciphers.
Remove weak ciphers for PassThrough transport	Remove any weak ciphers from the PassThrough transport and ensure that the server does not accept connections using those weak ciphers. The PassThrough transport is configured using the deployement.toml file. See Disabling Weak Ciphers for instructions.
Increase Ephemeral Diffie-Hellman Key size	Before starting the server, open the product startup script ( micro-integrator.sh in Linux and micro-integrator.bat in Windows) and enter the following with the other Java properties: -Djdk.tls.ephemeralDHKeySize=2048 \
Disable client-initiated renegotiation	Before starting the server, open the product startup script ( micro-integrator.sh in Linux and micro-integrator.bat in Windows) and enter the following with the other Java properties: -Djdk.tls.rejectClientInitiatedRenegotiation=true \
Enable HostName Verification	Make sure that hostname verification is enabled in the product startup script ( micro-integrator.sh in Linux and micro-integrator.bat in Windows) with the Strict mode. That is, you need to enable the following parameter: -Dhttpclient.hostnameVerifier=Strict \
Verify super admin credentials	The user name and password of the super administrator of your Micro Integrator (the first administrator) is created by adding the following configuration to the deployment.toml file. When you go into production, be sure to manually check your user store and ensure that unwanted super admin records are removed. [super_admin] username = "admin" password = "admin" admin_role = "admin" create_admin_account = true Note that you can easily use the management API to add, update, and delete admins and regular users in the user store. However, the super admin users created from the deployment.toml file should be managed manually. See the following topics for instructions to correctly create your administrators and other users in the Micro Integrator. Configuring a User Store for the Micro Integrator. Managing Users for the Micro Integrator.
Enable log rotation and monitoring	Ensure that you have a relevant log rotation scheme to manage logs. Log4J properties for Micro Integrator can be configured in the <MI_HOME>/conf/log4j2.properties file. To roll the wso2carbon.log based on size, this guide can be used. See Monitoring Logs for details on how to configure logging details in MWARE products.
Prevent Log Forging	Log forging can be prevented by appending a UUID to the log message. Read about configuring logs in the Micro Integrator.
Set appropriate JVM parameters	The recommended JDK version is JDK 11. See the installation pre-requisites for more information. Tip: To run the JVM with 2 GB (-Xmx2048m), you should ideally have about 4GB of memory on the physical machine.

OS-level security

This section provides the list of OS-level security guidelines for your production environment.

Guideline	Details
Run ESB processes with a specific OS-level user	Use a separate OS-level user to run MWARE products. Make sure that the user is only granted the permissions required for running the product for that particular user. Do not use the root/administrator user of your OS because the root/administrator is granted all privileges by default.
Minimize software to avoid vulnerability	Make sure that you only install the software/packages that are relevant to your MWARE product's deployment. Also, continuously monitor the software that you install. For information on the minimum software that your MWARE product will need, see Installation Prerequisites.
Enable the Firewall	Enable a firewall at the OS level (for example, iptables). This will provide protection for inbound and outbound connections of your MWARE product. Make sure that you only open the required outbound and inbound ports from the OS-level firewall.
Restrict access to TCP ports used for clustering	Apply a firewall at the host-level to disallow access to TCP ports used for clustering (port 4000, 4001, … by default) from unrecognized hosts. These ports should be accessible only from other members of the MWARE product cluster.
Use Secure Shell(SSH)	Use Secure Shell (SSH) when interacting with servers and executing commands. Adhere to the following best practices when you configure SSH: Change the default SSH port to a higher value. Disable the root or administrator. Enable login with user keys. Display a legal banner or a security banner with security warnings before SSH authentication.
Keep the system up-to-date	If there are security updates available for the packages installed in your OS (including the Java runtime), be sure to perform the necessary testing in a staging environment, and then proceed to install them for your OS.
Monitor user activities	Monitor the activities of your OS users. You can do this by enabling OS-level logs and by reviewing them regularly. You can also set up a centralized log monitoring system for this purpose.
Session Data Cleanup	Note: This security guideline is only applicable to MWARE ESB. In a production environment, there is a possibility for a deadlock or a database lock to occur when running a session data cleanup task in high load scenarios. To mitigate this, configure the following property to clean data in chunks. Configure this property in the file under session_data with the required chunk size. This value is in the number of records and depends on the database type and server capacity. It also depends on the amount of load generated by single sign-on (SSO). A higher value increases the chances of deadlocks and a lower value increases the time it takes for a cleanup. [session_data] cleanup.clean_expired_session_data_in_chunks_of = 8192 For more information on configuring sessions in production, see Authentication Session Persistence in the MWARE ESB documentation.
Make regular backups	Make sure to back up important files and archive them continuously. For more information, see Backup and Recovery Recommendations.

Network-level security

This section provides a list of security guidelines for configuring the network.

Guideline	Details
Create a failover setup	In the cloud native deployment, high-availability should be achieved via the container orchestration system (e.g., Kubernetes ). In a VM deployment, there should be at least two nodes with the failover configuration. When your servers are clustered, you need to regularly monitor the health of your server instances. For example, you need to monitor resource-level factors such as the server's resource utilization, response time anomalies, and the number of incoming network connections. Server monitoring will help you identify when additional server instances (failover instances) are required. You can also make decisions about network routing changes that you need to do in order to avoid server downtime.
Maintain network-level logging	Be sure to maintain and monitor logs for your proxy servers, load balancers, and other network devices.
Check open ports and services	Periodically check for open ports using port scanning tools and make sure that only the necessary ports are open to both internal and external networks. Be sure that only the ports relevant to your MWARE products are open for communication. If there are other ports started, be sure to monitor them. For the full list of ports in all MWARE products, see Default Product Ports.
Configure device-level security	All your network devices should be periodically checked for anomalies. For example, you need to verify routing tables and firewall rules. Also, make sure that the default credentials are changed before the first use of those devices.
Apply firmware updates	Firmware updates for your network devices should be applied regularly.
Note:This security guideline is applicable to the API-M runtime. Block the /services and /carbon contexts from the DMZ	Access to the "/services" and "/carbon" contexts should be blocked from the DMZ level (i.e., from the proxy server, load balancer and/or firewall). The "/services" context is used in MWARE products to expose admin services. These admin services are used for performing administrative operations using SOAP requests. The "/carbon" context is used in MWARE products to expose the management console (administration console) of the product. The management console is a user interface for performing some of the administrative operations of a product. In addition to the "/services" and "/carbon" contexts, be sure to expose only the required applications in your product to users beyond the DMZ level in your network. Note: It is recommended to use an allowlisting approach when allowing access to resources in your product from the DMZ level.

Configure client authentication

Client authentication is used to identify the application or the client that is making the request. The web applications provided out of the box use a set of default credentials to authenticate with MWARE ESB REST APIs that are marked as secure under the ResourceAccessControl tag of the <APIM_HOME>/repository/conf/identity/identity.xml file.

Follow the steps below to change the default credentials.

1. Shut the server down in case you have already started it.

2. Add the following configuration changes to the <APIM_HOME>/repository/conf/deployment.toml file.

   • Add the app_password property and enter a preferred password as the value.

     [identity.auth_framework.endpoint] 
     app_password="<value of preferred password>"

   • Add the hash property and enter the SHA-256 hash value of the app_password as the property value.

     [account_recovery.endpoint.auth]
     hash="<SHA-256 hash of the newly added app_password property value>"

   • If the authenticationendpoint web app is hosted externally, follow the instructions given below.

     a. Open the EndpointConfig.properties file found in the root of the authenticationendpoint folder.

     b. Change the app.password property value to the value added as app_password in the deployment.toml file.

     c. Do the same changes to the EndpointConfig.properties file located in the <APIM_HOME>/repository/deployment/server/webapps/authenticationendpoint/WEB-INF/classes directory.

   • If the accountrecoveryendpoint web app is hosted externally, follow the instructions given below.

     a. Open the RecoveryEndpointConfig.properties file found in the root of the accountrecoveryendpoint folder.

     b. Change the app.password property value to the value added as app_password in the deployment.toml file.

     c. Do the same changes to the RecoveryEndpointConfig.properties file located in the <APIM_HOME>/repository/deployment/server/webapps/accountrecoveryendpoint/WEB-INF/classes directory.

3. Once these changes are configured, restart the server.

   • Linux/Unix : sh wso2server.sh
   • Windows : wso2server.bat

Top