Using HashiCorp Secrets¶
Info
This feature is available as a product update from 27/10/2020 onwards. If you don't already have this update, you can get the latest updates now.
By default, the Micro Integrator is configured to use ESB secure vault for encrypting secrets. However, you may encounter certain limitations with ESB secure vault if you use secrets with a large number of characters. You can overcome this issue by using HashiCorp secrets.
Note
HashiCorp secrets are only applicable to synapse configurations. For server configurations, you need to use ESB secure vault.
Before you begin¶
- Generate the required secrets in your HashiCorp vault.
-
Select the server authentication method that you want to use when connecting the Micro Integrator with HashiCorp. There are three authentication methods available for HashiCorp:
- Static Token authentication
- AppRole Pull authentication
- LDAP authentication
If you select Static Token authentication, you need to generate a tokenID from HashiCorp. If you select AppRole Pull authentication, you need to generate a secret ID and role ID. See the HashiCorp documentation for details and instructions.
Copy the vault-java-driver to the lib folder of your Micro Integrator distribution (<MI_HOME>/lib).
Note
Vault driver version 5.1.0 is tested with the Micro Integrator.
Connecting MI to the HashiCorp server¶
Once you have set up the HashiCorp server, you can configure the Micro Integrator to connect with HashiCorp.
Add the following configurations to the deployment.toml file (stored in the <MI_HOME>/conf folder) of the Micro Integrator.
Tip
Be sure to apply the security tokens relevant to the authentication method you are using.
[[external_vault]]
name = "hashicorp" # required
address = "http://127.0.0.1:8200" # required
rootToken = "ROOT_TOKEN" # required
cacheableDuration = "15000"
engineVersion = "2"
# If namespace is used, apply the namespace value:
namespace = "NAMESPACE"
# If HashiCorp vault server is hosted in HTTPS protocol, apply below fields
trustStoreFile = "${carbon.home}/repository/resources/security/client-truststore.jks"
keyStoreFile = "${carbon.home}/repository/resources/security/wso2carbon.jks"
keyStorePassword = "KEY_STORE_PASSWORD"[[external_vault]]
name = "hashicorp" # required
address = "http://127.0.0.1:8200" # required
roleId = "ROLE_ID" # required
secretId = "SECRET_ID" # required
cacheableDuration = "15000"
engineVersion = "2"
# If namespace is used, apply the namespace value:
namespace = "NAMESPACE"
# If HashiCorp vault server is hosted in HTTPS protocol, apply below fields
trustStoreFile = "${carbon.home}/repository/resources/security/client-truststore.jks"
keyStoreFile = "${carbon.home}/repository/resources/security/wso2carbon.jks"
keyStorePassword = "KEY_STORE_PASSWORD"| Parameter | Description | |
|---|---|---|
| name | Specify 'hashicorp' as the external vault name. | |
| address | The address URL of the HashiCorp server where the secrets are stored. | |
| rootToken | Only applicable if static token authentication is used when connecting the Micro Integrator to the HashiCorp server. Use the root token generated from the HashiCorp server. See the HashiCorp documentation for instructions on generating this token. | |
| roleId | Only applies if AppRole Pull authentication is used when connecting the Micro Integrator to the HashiCorp server. Instead of specifying the rootToken, specify the role ID and secret ID generated from HashiCorp. The secret ID and role ID will internally generate a token and authenticate the HashiCorp server connection. See the HashiCorp documentation for instructions on generating these values. Note: The secret ID you generate in HashiCorp may expire. If that happens, you can renew the security token. | |
| secretId | ||
| cacheableDuration | All resources fetched from the HashiCorp vault are cached for this number of milliseconds. The default value is 15000. | |
| engineVersion | The version of the HashiCorp secret engine. | |
| namespace | Namespace support is available only in the Enterprise edition of HashiCorp. The namespace value specified here applies globally to HashiCorp secrets in all synapse configurations. | |
| trustStoreFile | The keystore file (trust store) that is used to store the digital certificates that the Micro Integrator trusts for SSL communication. To use the default truststore file in the Micro Integrator, specify '${carbon.home}/repository/resources/security/client-truststore.jks' as the value. | |
| keyStoreFile | The keystore used for SSL handshaking when the Micro Integrator communicates with the HashiCorp server. To use the default keystore file in the Micro Integrator, specify '${carbon.home}/repository/resources/security/wso2carbon.jks' as the value. | |
| keyStorePassword | The password of the keystore file that is used for SSL communication. If you are using the default keystore file in the Micro Integrator, the default password is 'wso2carbon'. | |
Note
For information on enabling AppRole auth method in HashiCorp secure vault, see the ACL rules and configuration sections.
Accessing HashiCorp secrets in synapse configurations¶
Once your Micro Integrator is connected to HashiCorp, you can point to the secrets stored in the HashiCorp vault from your synapse configurations.
Given below is a sample synapse property that uses a HashiCorp secret:
<property name="HashiCorpSecret" expression="hashicorp:vault-lookup('path-name', 'field-name') />Accessing HashiCorp secrets using environment variables in synapse configurations¶
You can point to the secret stored in the HashiCorp vault from your synapse configurations using environment variables.
Given below is a sample synapse property to retrieve a HashiCorp secret using environment variables.
<property name="HashiCorpSecret" expression="hashicorp:vault-lookup('$env:path-name', '$env:field-name')" />Renewing security token (AppRole Pull authentication)¶
When you generate the secret token from HashiCorp (for enabling AppRole Pull authentication), you have the option of limiting the number of times the secret ID token can be used. This is done using the secret_id_num_uses parameter in HashiCorp. In this case, the secret ID will expire after it is used for the specified number of times.
In such situations, you need to regenerate a secret token from HashiCorp and apply it to the Micro Integrator. You can directly apply the new secret token to the deployment.toml file and restart the Micro Integrator. However, note that there will be a downtime due to server restart.
If you want to update the secret token dynamically without restarting the server, you can use the management API of the Micro Integrator. As shown below, you can send a request to the given URL with the new secret ID (specified in the sample payload).
- Management API URL:
https://HOST:9164/management/external-vaults/hashicorp - Payload:
{ "secretId" : "new_secret_id" }
Tip
You can use the ESB API Controller (APICTL) to update the secret token dynamically. For more information, see Update HashiCorp AppRole Pull secret ID.
Using Namespaces for the HashiCorp connection¶
Namespace support is available only in the Enterprise edition of HashiCorp.
You can add a global namespace value in the deployment.toml file. See the instructions on connecting the Micro Integrator with HashiCorp. You can also use namespace values per request in synapse configurations as shown below.
<property name="HashiCorpSecret" expression="hashicorp:vault-lookup('namespace', 'path-name', 'field-name') />