Signing management requests

Few management APIs require requests signed by the ECT for authentication and authorization.

These APIs include:

• JWT issuing APIs
• Client enumeration APIs

Verifying that the request was sent by ECT

To verify that a request came from the ECT service, we:

• Check the request signature to verify the authenticity of the request.
  You must sign all management requests.
• Check the request timestamp to make sure that the request is not an old one being sent as part of a replay attack

Management request headers

Fetching the certificate

Certificate URL validation

To check and validate the request signature, we verify the URL in the SignatureCertChainUrl header to make sure that it matches the format that ECT uses.

This can help to protect against requests that attempt to make our web service download malicious files.

Make sure that the URL meets all of the following criteria:

• The protocol is https (not case sensitive).
• The hostname is the FQDN of the ECT. (not case sensitive).
• The path begins with /ect.api/ (case sensitive).
• If a port is specified in the URL, the port is 443.

The following are examples of correctly formatted URLs:

• https://subdomain.ect.com/ect.api/ect-api-cert.pem
• https://subdomain.ect.com:443/ect.api/ect-api-cert.pem
• https://subdomain.ect.com/ect.api/../ect.api/ect-api-cert.pem

The following are examples of invalid URLs:

• http://subdomain.ect.com/ect.api/ect-api-cert.pem (invalid protocol)
• https://notect.com/ect.api/ect-api-cert.pem (invalid hostname)
• https://subdomain.ect.com/EcT.aPi/ect-api-cert.pem (invalid path)
• https://subdomain.ect.com/invalid.path/ect-api-cert.pem (invalid path)
• https://subdomain.ect.com:563/ect.api/ect-api-cert.pem (invalid port)

We download the PEM-encoded X.509 certificate chain using the URL specified in the SignatureCertChainUrl header in the request.

This chain can be provided at runtime so that it can be updated periodically.

This certificate chain we downloaded in the preceding stage is composed of the following, in order:

• The ECT signing certificate.
• One or more additional certificates that create a chain of trust to a root certificate of a certificate authority (CA).

Certificate ID validation

If you choose to utilize a self-signed certificate, we will generate and return a certificate UUID for the same.

Requests signed with the said certificate need to provide the SignatureCertUUID parameter.

Upon the receipt of a management request, we validate the UUID provided and fetch the corresponding certificate from our data store.

Certificate verification

CA signed certificate

We confirm the validity of the signing certificate by completing the following stages:

• Check the Not Before and Not After dates of the signing certificate to make sure that it is not expired.
• Make sure the ECT FQDN is present in the Subject Alternative Names (SANs) section of the signing certificate.
• Validate that all certificates in the chain combine to create a chain of trust to a trusted root CA certificate.

Self-signed certificate

We confirm the validity of the signing certificate by completing the following stages:

• Validate that a certificate with the said ID exists with us.
• Check the Not Before and Not After dates of the signing certificate to make sure that it is not expired.
• Make sure the domain name ECT FQDN is present in the Subject Alternative Names (SANs) section of the signing certificate.

If certificate verification fails, we discard the request.

Signature verification

• After confirming the validity of the signing certificate, we extract the public key from it.
• We base64-decode the value of the Signature header in the request to obtain the encrypted signature.
• Use the public key extracted from the signing certificate to decrypt the encrypted signature to produce the asserted hash value.
• We generate an SHA-1 hash value from the full HTTP request body to produce the derived hash value.
• Compare the asserted hash value and derived hash values to ensure that they match.
  If they do not match, we discard the request.

Timestamp verification

• Every request that your web service sends must include a timestamp in the request body.

• The timestamp is part of the signed portion of the request, so it cannot be modified without also invalidating the request signature.

• We use this timestamp to verify the freshness of the request before sending a response.

• This can help protect our web service from attempted replay attacks in which an attacker acquires a properly signed request and then repeatedly resends it to disrupt the our service.

We allow a tolerance of no more than 150 seconds (two and a half minutes). This means that our service should only process requests in which the timestamp is within 150 seconds of the current time.

If the timestamp differs from the current time by more than 150 seconds, we discard the request.

The timestamp is part of the request object in the JSON body of the request. For example:

POST /jwt/issue
{
 "fqdn": "awesome.ect.com",
 "client_id": "86f7e437faa5a7fce15d1ddcb9eaeaea377667b8",
 "timestamp": "2019-05-13T12:34:56Z",
}

We return HTTP error code 400 Bad Request, to reject requests in which the timestamp falls outside the tolerance.

Sample python snippet for signed requests

#!/usr/bin/env python3
import datetime
import requests
import json
import pem
import base64
from OpenSSL import crypto
keyfile_path = 'host.key' # PEM encoded RSA/ECDSA private key
keyfile = pem.parse_file(keyfile_path)[0]
private_key = crypto.load_privatekey(crypto.FILETYPE_PEM, keyfile.as_bytes())
url = 'https://<base_url>/tract/management/token/issue/' # base URL is pre-shared with you
payload = {
 'client_id': '', # client ID for which you are generating the token
 # ID can be fetched from the client enumeration API
 'fqdn': '', # your FQDN pre-shared with us
 'timestamp': datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%SZ')
}
# optional minification stage
# doesn't really make a difference as long as the body sent and the body hashed+signed are the same
minified_json = json.dumps(payload, separators=(',', ':'))
headers = {
 'Content-Type': 'application/json',
 'Signature': base64.b64encode(crypto.sign(private_key, bytes(minified_json, 'utf-8'), 'sha1')),
 'SignatureCertUUID': '' # This is shared with you when you pre-share a cert with us.
 # WARNING: Make sure the UUID is the correct UUID for the certificate used for this request.
 # Mismatching UUID will cause the request to fail.
}
response = requests.request('POST', url, headers=headers, data=minified_json)
# verify data sent and received
print(response.request.headers, end='\n\n')
print(response.request.body, end='\n\n')
print(response.text.encode('utf8'))

Verifying hook signatures

All outgoing hook requests from TRACT to ECTs are signed to authenticate that the request actually originated from TRACT.

Verifying that the request was sent by TRACT

To verify that a request came from TRACT, do:

• Check the request signature to verify the authenticity of the request.

We sign all outgoing hook requests.

• Check the request timestamp to make sure that the request is not an old one being sent as part of a replay attack.

Requests that TRACT sends to your web service includes two HTTP headers that should be used to check the request signature

Management request headers

Parameter

Fetching the certificate

Certificate URL validation

To check and validate the request signature, verify the URL in the signature-certificate-url header to make sure that it matches the format that TRACT uses.

This can help to protect against requests that attempt to make your web service download malicious files.

Make sure that the URL meets all of the following criteria:

• The protocol is HTTPS (not case sensitive).
• The hostname is the FQDN of the TRACT server you are communicating with. It is of the format *.haptikapi.com for production, and *.hellohaptik.com for staging. (not case sensitive).
• The path is /tract/hooks/certificate/.
• If a port is specified in the URL, the port is 443.

The following are examples of correctly formatted URLs:

• https://subdomain.haptikapi.com/tract/hooks/certificate/
• https://subdomain.haptikapi.com:443/tract/hooks/certificate/
• https://subdomain.haptikapi.com:443//tract/hooks/certificate/

The following are examples of invalid URLs:

• http://subdomain.haptikapi.com/tract/hooks/certificate/ (invalid protocol)
• https://nothaptikapi.com/tract/hooks/certificate/ (invalid hostname)
• https://subdomain.haptikapi.com/tract.api/hooks/certificate/ (invalid path)
• https://subdomain.haptikapi.com/invalid.path/tract-api-cert.pem (invalid path)
• https://subdomain.haptikapi.com:563/ect.api/ect-api-cert.pem (invalid port)

Download the PEM-encoded X.509 certificate using the URL specified in the signature-certificate-url header in the request.

This is provided at runtime so that it can be updated periodically.

This certificate chain we downloaded in the preceding stage is composed of the following, in order:

• The ECT signing certificate.
• Zero or more additional certificates create a chain of trust to a root certificate of a certificate authority (CA).

Certificate verification

To confirm the validity of the signing certificate, complete the following stages:

• Check the Not Before and Not After dates of the signing certificate to make sure that it is not expired.
• Make sure the domain name of the TRACT server is present in the Subject Alternative Names (SANs) section of the signing certificate.

If certificate verification fails, discard the request.

Signature verification

• After confirming the validity of the signing certificate, extract the public key from it.
• base64-decode the value of the signature header in the request to obtain the encrypted signature.
• Use the public key extracted from the signing certificate to decrypt the encrypted signature to produce the asserted hash value.
• Generate an SHA-256 hash value from the full HTTP request body to produce the derived hash value.
• Compare the asserted hash value and derived hash values to ensure that they match.

If they do not match, discard the request. 

Timestamp verification

• Every request that our web service sends includes a signature_timestamp in the request body.

• The timestamp is part of the signed portion of the request, so it cannot be modified without also invalidating the request signature.

• We use this timestamp to verify the freshness of the request.

• This can help protect your web service from attempted replay attacks in which an attacker acquires a properly signed request and then repeatedly resends it to disrupt the web service.

We allow a tolerance of no more than 120 seconds (two minutes). This means that your service should only process requests in which the timestamp is within 120 seconds of the current time.

If the timestamp differs from the current time by more than 120 seconds, discard the request.

The timestamp is part of the request object in the JSON body of the request. For example:

{
    "user_id": "qUgNn5epx_QBQYL_.UrSrmMXE-_QmBZ-8pwglUWAJ8OcqhPK0yKjb",
    "conversation_number": 10,
    "message_body": "{\\"text\\": \\"7\\", \\"type\\": \\"TEXT\\", \\"data\\": {\\"quick_replies\\": []}}",
    "message_id": 83607,
    "sender": "ceu",
    "timestamp": "2021-08-05T10:59:14Z",
    "metadata": {},
    "signature_timestamp": "2021-08-06T08:42:39Z"
}

You should HTTP error code 400 Bad Request to reject requests in which the timestamp falls outside the tolerance.

Sample python (django) snippet for verifying hook signatures  

import base64
import json
import pem
import requests
import time
from OpenSSL import crypto
from dateutil import parser
from django.core.handlers.wsgi import WSGIRequest
from url_normalize import url_normalize
from urllib.parse import urlparse

def request_handler(request: WSGIRequest):
    headers = request.META
    try:
        signature = headers['HTTP_' + 'signature'.upper()]
        certificate_url = headers['HTTP_' + 'signature-certificate-url'.upper()]
        body = request.body
        return VerifyIncomingHookSignature(certificate_url, signature, body).verify()
    except Exception as err:
        print(err)
        return False

class VerifyIncomingHookSignature(object):
    """docstring for VerifySelfSignedCert


    Attributes:
        arg (TYPE): Description
    """


    TOLERANCE = 60 * 2  # 2 minutes


    def __init__(self, signature_certificate_url: str, signature: bytes, body: bytes):
        super(VerifyIncomingHookSignature, self).__init__()


        print()
        print('signature_certificate_url: %s | %s' % (signature_certificate_url, type(signature_certificate_url)))
        self.signature_certificate_url = signature_certificate_url
        print('signature: %s | %s' % (signature, type(signature)))
        self.signature = signature
        print('body: %s | %s' % (body, type(body)))
        self.body = body


        self.certificate = None
        self.normalized_url = None


    def fetch_certificate(self):
        print(f'[INFO] begin fetch_certificate')
        try:
            self.certificate = json.loads(requests.get(self.signature_certificate_url).content)['certificate']
        except Exception as err:
            print(f'[FAIL] fetch_certificate: {err}')


    def normalize_url(self):
        print(f'[INFO] begin normalize_url')
        try:
            self.normalized_url = url_normalize(self.signature_certificate_url)
        except Exception as err:
            print(f'[FAIL] normalize_url: {err}')


    def validate_signature_cert_url(self):
        print(f'[INFO] begin validate_signature_cert_url')
        self.normalize_url()
        try:
            o = urlparse(self.normalized_url)
            assert o.scheme == 'https'
            assert o.hostname == 'devqa.hellohaptik.com'
            assert o.path == '/tract/hooks/certificate/'
            if o.port:
                assert o.port == 443
            self.fetch_certificate()
            return True
        except AssertionError:
            print('[FAIL] validate_signature_cert_url')
            return False
        except Exception as err:
            print(f'[FAIL] validate_signature_cert_url {err}')
            return False


    def load_certificate(self):
        print(f'[INFO] begin load_certificate')
        try:
            self.certificate = crypto.load_certificate(
                crypto.FILETYPE_PEM,
                pem.parse(bytes(self.certificate, 'utf-8'))[0].as_bytes()
            )
        except Exception as err:
            print(f'[FAIL] load_certificate: {err}')
        return True


    def verify_hash(self):
        print(f'[INFO] begin verify_hash')
        try:
            crypto.verify(self.certificate,
                          base64.b64decode(self.signature),
                          self.body,
                          'sha256')
            return True
        except crypto.Error as err:
            print(f'[FAIL] verify_hash: {err}')
            return False


    def validate_timestamp(self):
        print(f'[INFO] begin validate_timestamp')
        try:
            timestamp = parser.parse(json.loads(self.body)['signature_timestamp']).timestamp()
            now = time.time()
            if not now - self.TOLERANCE < timestamp < now + self.TOLERANCE:
                print(f'[FAIL] validate_timestamp')
                return False
        except Exception as err:
            print(f'[FAIL] validate_timestamp: {err}')
            return False
        return True


    def verify(self):
        result = (
                self.signature_certificate_url
                and self.signature
                and isinstance(self.signature, bytes)
                and self.body
                and isinstance(self.body, bytes)
                and self.validate_signature_cert_url()
                and self.load_certificate()
                and self.verify_hash()
                and self.validate_timestamp()
        )
        if result:
            print('ALL OK')
        return result