Secure Data Storage
Secure Data Storage (SDS) is a service that helps you securely control access to your data. SDS provides storage for sensitive data and lets you control who is authenticated (signed in) and authorized (has permissions) to use data.
This document explains how you can use SDS to store sensitive data and control access to the stored data. It also explains client-end functionality, i.e., how to decrypt the data post-reading.
To store App data in SDS for security purposes, register the App by calling the SDS Create App endpoint. You must specify a name that is unique within the Tenant and a base64 encoded RSA public key of any size.
A Tenant can register as many Apps with SDS as is required. On successful registration of the App, SDS returns a unique App ID in the response. You can use the App ID to read the application configuration or update the RSA public key.
🚧 It is the application's responsibility to create an RSA key pair and maintain the private key with itself. SDS only takes the public key to encrypt the data.
To register an App in the SDS service, call the SDS Create App endpoint.
Request bodies are specified in JSON format. The following example shows a request body to register an App:
Member | Description | Validation | Data type |
name | Name of the application which has to be registered. | pattern: ^[a-zA-Z0-9-_]{3,16}$ example: Participant | string |
key | Base-64 encoded public key generated by the application. | | string |
To update an App in the SDS service, call the SDS Update App endpoint.
Request bodies are specified in JSON format. The following example shows a request body to Update an App:
The X-SDS-SIGNATURE signature identifies which SDS application is making the request and is used for the authentication and authorization of that application.
For example:
- Only the owner application can modify an existing vault.
- Verify that the application has permission to store/read data to or from a vault.
The format of the signature is: <base64encoded_signature>.<base64encoded_appName>
Generate the signature by using the RSA private key which was generated by the App and SHA1WITHRSA as the algorithm. An RSA signature can only be verified using the public key of that pair and the same algorithm which was used to generate it.
Encode the signature in base64 and appended the base64 encoded App name with the dot sign.
❗CAUTION
The keys and signature might include some non-printable characters. We recommend that you generate it programmatically rather than using an online utility.
A Vault is a secure storage mechanism that is designed to protect your confidential data against unauthorized access. Tenants can create as many Vaults as needed to satisfy their business needs.
On successful creation of the Vault, the SDS service returns a unique ID for the vault which can be used to modify the Vault or read the existing configurations. After the Vault is created only the owner application can read or modify its configuration. The owner of the Vault has access to modify permissions for any other application to that Vault.
Vault permission is set by setting the App name and including permission flags. To set the Vault permissions the value of the appName member by specifying the name of the application for which the permission is required and setting the permission1 member to the three-digit binary string permission string. For example: appName= “Application123” and permission= “101”.
The three characters in the permission specify the write_permssion, decrypted_read_permission, encrypted_read_permission respectively. The allowed permissions for apps are:
- 110: Can read decrypted data from the vault and write data in the vault.
- 101: Can read data in encrypted form from the vault and write data in the vault.
- 100: Can write data in the vault.
- 010: Can read decrypted data from the vault.
- 001: Can read data in encrypted form from the vault.
- 000: No permissions assigned.
Only the applications which have permission to a Vault are able to write or read the data from that Vault. The owner of the Vault is assigned with permission 101 by default which can be later updated through the update-vault API.
📘 NOTE
Owner permission cannot be changed from default while creating the Vault but can be updated later.
Other than App permissions, Vaults have a read limit associated with them. This value specifies the maximum number of records that are returned in a single request. The minimum number of records is 1 and the maximum is 50.
Vault names must be unique to the tenant.
To create a Vault, call the Create Vault endpoint.
Request bodies are specified in JSON format. The following example shows a request body to create a Vault.
Member | Description | Validation | Data type |
name* | Vault name. | pattern: ^[a-zA-Z0-9-_]{3,16}$ example: phone-number | string |
owner* | Vault owner. | | string |
readLimit (Optional) | Read limit. | default: 1 minimum: 1 maximum: 50 | integer($int64) |
uniqueData | | default: false | boolean |
permissions (Optional) | List of applications allowed to read/write data in the vault according to the permission specified. | | |
app* | | example: Participant | string |
permission* | Application permission. | Valid values: [110, 101, 100, 010, 001, 000] 110: application with this permission can read decrypted data from the vault and write data in the vault. 101: application with this permission can read data from the vault and write data in the vault. 100: application with this permission can write data in the vault. 010: application with this permission can read decrypted data from the vault. 001: application with this permission can read data from the vault. 000: the application doesn't have any permission. | string |
enable (Optional) | | | boolean |
📘NOTE
The items marked (*) are mandatory attributes required while creating the Participants via batch processing.
To disable a Vault, call the Update Vault endpoint and set the enable member of the request body to False. A Vault can be disabled only if no data exists in that Vault.
Data is confidential information that is stored in a Vault to protect it. This data can be stored, updated, deleted, or read depending on the permissions that the application has:
- The data can be stored in any of the Vaults by the applications which have write permission to that Vault.
- The existing data in a Vault can be updated by any application which has write permission to that Vault.
- The existing data can be read by any application which has any of the read permissions to that Vault.
An application can store data in a Vault if it has the write permission, i.e., any of the following: 100, 101, or 110. The data should be base64 encoded while storing it in the vault.
Along with the data, you can include metadata information. The metadata can be of any type (string/number/list/JSON). On successfully storing the data in the vault, SDS returns a unique data ID in the response which you can use to read or update the data.
To write data to a Vault, call the Create Data endpoint.
Request bodies are specified in JSON format. The following example shows a request body to write data to a Vault.
Member | Description | Data type |
data | Data to be stored. | string |
meta | Metadata for data stored in the vault. It is a key value pair. | |
vault | Data vault. | string |
An application can update data in a Vault by its ID if it has the write permission, i.e., any of the following: 100, 101, or 110. With this permission, the data, the metadata stored with it, or even the Vault can be updated.
To update the Vault, the requesting app must have the write permission to both the existing and the modified Vault.
To update data in a Vault, call the Update Data endpoint.
Request bodies are specified in JSON format. The following example shows a request body to update data in a Vault.
An application can read data from a Vault by its ID if it has any of the read permissions, i.e., any of the following: 101, 110, 001, or 010.
Apps that have any of the decrypted read permissions (110 or 010) retrieve the data in the same form as it was stored.
Apps that have any of the encrypted read permissions (101 or 001) retrieve the data in the encrypted form. SDS uses hybrid encryption to provide a high level of security to data. Also, it allows for storing more data as RSA is limited to encrypting the data of a maximum of 512 bytes if RSA-2048 is used to generate the key pair.
The data from the Vault is encrypted with a secret (AES-128 bit key) of length 16 which is different every time and initialization-vector of 16 bytes ({0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}. The algorithm used to encrypt the data is AES/CBC/PKCS5Padding. The same should be used to decrypt the data.
The secret is encrypted with the application’s public key which was shared at the time of registration using the algorithm RSA/ECB/PKCS1Padding.
Both the encrypted secret and the encrypted data are sent in the response by appending with the dot sign. For example: <encrypted_secret>.<encrypted_data>
📘 INFO
Both the secret and the data are base64 encoded after encryption is applied. Base64 decoding has to be applied before decrypting them.
To decrypt the encrypted secret, first, apply base64 decoding to the secret and then use the private key of the same RSA key-pair generated at the time of registration.
The algorithm used to encrypt the secret is RSA/ECB/PKCS1Padding. The same should be used to decrypt the secret. The decrypted secret should have a length of 16 which is actually a 128-bit AES key.
To decrypt the data part, first, apply base64 decoding to the data string and then use the decrypted secret generated in the above step as the AES key. The algorithm used to encrypt the data is AES/CBC/PKCS5Padding. The same should be used to decrypt the data.
📘 NOTE
Since the keys may include some nonprintable characters. Hence, it is recommended to decrypt it programmatically rather than using any online utility. Also, do verify the length of the secret after decryption.
Apps can retrieve data from a Vault for more than one data ID at a time. When reading data using multiple data IDs, the rules around permissions and encryption-decryption of data are followed in the same way as for the single read. Note that the data read limit still applies when retrieving data for multiple data IDs.
The SDS service allows you to store confidential files within Vaults. These files can be read, downloaded, or searched based on the permission(s) that the application has.
You can store files or URLs of files which can be any file format: e.g. text, pdf, doc, audio, and video. Within SDS, you can also store tags with files that allow you to search through them. A tag consists of a tag key and a tag value. Tag keys and tag values are both required, but tag values can be empty (null) strings. For example, The key might be CONSIGNEE, and the value might be ADDRESS.
The SDS service allows you to upload files in two ways:
- File upload: By sending the file in the multipart-form-request or providing file data in binary format.
- URL upload: By sending a URL where the document is originally stored.
To upload a file, call the SDS Upload File endpoint.
Request bodies are specified in JSON format. The following example shows a request body to upload a file to a Vault.
Member | Description | Data type |
name | Name of the file. | string |
url | URL for the file. | string |
description | Description of the file. | string |
tags | Includes key and value. | string |
vault | Data vault | string |
type | File types. Valid values: FILE, URL. | string |
callback | CallbackURL to send file upload status. | string($uri) |
The download of files from the SDS service is performed in two ways:
- If the file is stored in the SDS service then you are sent a pre-signed link that has a configurable expiration time.
- If only the file URL is stored then the store URL is returned.
When you upload a file, you have the option to add a tag along with it. These tags can be used to search and filter uploaded files. During the search, matching files or URL metadata is exposed to you. After that, you can download the file based on your ID.
For one tag there can be multiple docs available. During the search, the metadata for all matching documents is sent. After searching based on a tag, you are sent the metadata but you'll not be able to view the file content yet.
To view the content of the file, you need to make a separate request to download the file. After making a download request, you are checked for download access. If you have access, the file is downloaded and can be viewed.