Survival Guide - Encryption, Authentication

This is a survival guide covering the mind-numbing topics of Cryptography, Encryption, Authorization and Authentication. For the mathematically challenged (and who is not) the maths involved in cryptography are gruesome in the extreme and are covered, if at all, at the level of 'stuff happens'. The guide concentrates on system functionality and consists almost exclusively of descriptions and explanations rather than specific commands or implementations. Much of this stuff is background for SSL/TLS and X.509 certificates.

Contents:

• Terminology
• Cryptographic Overview
• Symmetric Encryption
• Asymmetric Encryption
• Digests and Hashes
• Message Authentication code (MAC)
• Digital Signatures

Terminology

A non-exhaustive list of terms used in security. Many of these terms are explained further in the text others are left dangling either to create a sense of tension in the reader or, more likely, because we have no idea what they mean.

Authentication	The process or procedures used to verify that data or information asserting that it originates from a source can only have come from that source. Techniques used include digital signatures, MACs and even good 'ole passwords.
Authorization	When a user has Authenticated they are typically Authorized, based on their login credentials or account properties to access, or deny access to, certain systems resources such as files, applications or network resources. The term privilege is sometimes used synonymously for Authorization, thus a user may have enough privileges to access resource X but not resource Y. They are Authorized to access X but not Authorized to access Y.
Cipher	A cipher a.k.a. an encryption algorithm. To cipher a.k.a. the process of encryption.
Clear Text	A.k.a. plain text. A block of data to which no encryption process has been applied or a block of data that results from a decryption or deciphering operation.
Decipher	Apply an decryption algorithm to an encrypted block of text and, given the appropriate keys turn it into plain text. Most, but not all, ciphers use the same algorithm to encrypt and decrypt.
Encryption	The process of transforming data using a cipher (or encryption algorithm). Techniques used may be symmetric or asymmetric.
Hash	a.k.a. digest or one-way hash. A algorithm for reducing an infinitely large block to a unique octect string of a fixed and smaller size - typically 64 octets. Hashes are used to ensure message integrity and are used in MACs or digital signatures.
Plain Text	A.k.a. clear text. A block of data to which no encryption has been applied or a block of data that results from a decryption or deciphering operation.

Cryptographic Overview

Cryptography according to Webster is "the enciphering and deciphering of messages in secret code or cipher; also : the computerized encoding and decoding of information".

It is the process of transforming (enciphering or encrypting) data (called clear or plain text) using some process (a cipher or encryption algorithm) into some gobbledygook that can only be transformed back (deciphered or decrypted) to plain text if the recipient has some some secret knowledge such as a key or a set of keys.

Historically the ciphers or encryption algorithms used formed the secret. For example - shift every character one position left (the cipher) - that we used as kids when sending secret messages to our friends. The weakness here is that if the method or encryption algorithm (the cipher) is discovered all communication sent using that algorithm (or cipher) can be converted into plain text (deciphered). A new algorithm has to be created and propagated to all parties.

Modern cryptography assumes that the bad guys will discover the cryptographic algorithm, indeed, for reasons we will see later, the algorithms themselves are widely published. Instead the secret lies with a unique key or keys which are used by the algorithm to transform the data. If the key is exposed or compromised (discovered by a bad guy) then by simply replacing the key, but keeping the algorithm the same, we can recommence confidential communication. The bad guy has to start all over again to discover the key with no greater knowledge than before but with, hopefully, more careful key maintenance procedures in place.

Cryptographic algorithms are not provably secure. Instead they are widely published and exposed to repeated attack by dedicated researchers and specialists (black hat testers) who love this kind of stuff. Only having resisted repeated and sustained attacks are the algorithms used operationally. Since research into the cryptographic algorithms is ongoing it can occasionally mean that apparently robust, mature algorithms need to be replaced when weaknesses are discovered. A recent example here relates to theoretical weaknesses being discovered in the MD5 digest algorithm. While it is always possible to use a brute force attack to find a key, cryptographic systems use a concept known as computationally infeasible which simply means that it would cost too much or take too long to mount such a brute force attack. Computationally infeasible is based on todays's technology and is therefore relative not absolute and does change over time - thus for example in some algorithms the key size is increased over time as computational capacity increases.

Cryptography may be used for three purposes:

1. Confidentiality: Only the parties to the communication can understand the messages or data sent between the parties.

2. Authentication: The data could only have come from a known source.

3. Integrity: The data received by one party was the data sent by the other party and was not manipulated or compromised during transmission.

One or more of the above may be provided by a single algorithm or may be provided by a combination of algorithms and methods.

First the basic techniques. Modern cryptographic techniques are either symmetric or asymmetric.

Symmetric Cryptography

Symmetric encryption algorithms, also called single-key, shared-secret, or even, confusingly, private-key systems, use a single key (or set of keys) to encrypt and decrypt the data. This single key - the shared secret - must be securely exchanged between the parties that will use it prior to the actual secure communication. The limitations of shared-secret systems are twofold. First, the key must be distributed securely using a process called key management, which itself is not trivial. Second, the method of securing the key once distributed lies with all the parties to the communication: "I trust myself but do I trust all the other parties to keep the key secret?". If a shared-secret key is compromised at any of the parties then it is compromised for all parties that use it. Symmetric algorithms use significantly less computational resources than their asymmetric counterparts. They are, generally, the only viable method for encrypting bulk data streams.

Examples of common symmetric key algorithms are DES, AES, IDEA, and RC4, and typical key sizes are 64, 128, or 192 bits. Figure 1 shows the operational use of a shared secret for classic confidential communications. Note: The term shared secret, which describes a single key (or set of keys) used, or shared, by both ends of the communication should not be confused with secret sharing, which describes a process whereby the shared, or single, secret key is broken up into parts and shared between multiple persons to make it more secure.

Figure 1 - Symmetric Cryptography

Asymmetric Cryptography

Asymmetric encryption algorithms use a pair of keys - a public and a private key - and are generally referred to as public-key cryptographic systems or sometimes as nonsecret encryption (a slight oxymoron). In these systems, data (called plain-text in the jargon) that is encrypted with one key can only be decrypted with the paired key. Given one key, it is computationally infeasible to derive the paired key. Asymmetric encryption works by making one key, called the public key, widely available, while maintaining the other key, surprisingly called the private key, a secret. This process has an interesting side effect. If a message is encrypted with a private key and can be decrypted with its paired public key, then only the owner of the private key could have done it. This property is used in digital signatures and is described later. Asymmetric algorithms use significant computational resources in comparison with their symmetric counterparts and therefore are generally not used to encrypt bulk data streams.

The most widely used public-key encryption systems are RSA (after the inventors Rivest, Shamir, and Adelman) and epileptic curves. Typical key sizes for public-key systems are 512 bits, 1,024 and increasingly 2048 bits, or even higher. The public keys of a private/public key pair can be safely stored in a public service such as DNS, while the private key must be maintained securely in a private location. Figure 2 illustrates the use of public-key cryptography for classic confidential communications.

Figure 2 - Asymmetric Cryptography

To achieve confidentiality, a message to be sent from Host2 to Host1 is encrypted with the public key of Host1. Only the private key of Host1 can decrpyt this message. If Host1 wishes to send a confidential message to Host 2 then it must obtain the public key of Host2 (not shown in diagram 2).

Public-key systems have one significant limitation. They rely on knowing, or trusting, that the public key which will be used in communications with a person or organization really is the public key of the person or organization and has not been spoofed by a malicious third party. There are two broad methods by which this is usually accomplished. A Public Key Infrastructure (PKI), or more commonly by the use of a trusted third party. The third party securely manages, and attests to the authenticity of, public keys. If the third party (a.k.a a Certificate Authority in the context of X.509 certificates) is requested to provide the public key of X, they are trusted to provide the correct public key. The third party is trusted to have satisfied themselves by some process - attestation, notarization, or some other process - that X is the one and only, or globally unique, X. The most common method for making available public keys that have been verified by a third party is to embed them in an X.509 (or SSL) certificate which has been digitally signed by the the issuer (typically a Certificate Authority).

Message Digests (Hashes)

To provide data integrity, any message could be simply encrypted. In order to modify the data content of a message the attacker would have to be in possesion of the single key (in symmetric systems) or the private key (in asymmetric systems). However, encryption systems use complex mathematical functions, and are therefore big users of CPU resources. To encrypt all messages may incur unacceptably high overheads and especially frustrating where data confidentiality is not a requirement. Fortunately, other techniques can be used to reduce this load. The most common is a lightweight procedure called a one-way hash, simply a hash, or more commonly a message digest. The hash or digest algorithm creates a unique and relatively small fixed-size block of data (irrespective of the original message length) that cannot be reversed. The messages being sent typically include both the plain text (unencrypted) and a digest of the message. The hash algorithm is applied to the received plain text and if the result matches the message digest, this means the received data was not altered. The message digest is, in some senses, similar in concept to a checksum but has significantly different mathematical properties.

The most common forms of message digest are MD5 and SHA-1 and increasingly SHA-256 (both part of the SHA family of digests). Figure 3 shows the message digest in action.

Figure 3 - Message Digest or One-Way Hash

Message Authentication Code (MAC)

In the symmetric or shared-secret world, the process of authentication and data integrity uses what is called a Message Authentication Code (MAC). The MAC combines the message digest with the shared key. The key part authenticates the sender, and the hash (or digest) part ensures data integrity.

The most common forms of MACs are HMAC-MD5 and HMAC-SHA-1 and increasingly HMAC-SHA-256. Figure 4 shows how the MAC is used. Note: The MD5 hash algorithm, and by implication any algorithm that uses it, such as HMAC-MD5 or RSA-MD5, has been moved to a "not recommended" status in most IETF documents, due to some theoretical weaknesses published in early 2005. These weaknesses do not invalidate the use of the algorithm.

Figure 4 - Message Authentication Code (MAC)

Digital Signatures

In the asymmetric or public-key world, the process of authentication and data integrity uses what is called a digital signature. The message being sent is again hashed to create a message digest using, say, MD5, SHA-1 or SHA-256 to ensure data integrity. The resulting message digest is then encrypted using the private key of the sender. Both the plain-text message and the encrypted digest are sent to the other party. The receiver decrypts the message digest using the public key of the sender, applies the hash algorithm to the plain-text data, and if the results match, then both the authenticity of the sender and the integrity of the data are assured.

Typical key sizes for digital signature systems are 768 bits, 1,024 bits, or increasingly higher values. The most common digital signature algorithms are RSA-MD5, RSA-SHA-1, RSA-SHA-256 and Digital Signature Architecture (DSA; a US Government standard). Figure 5 shows how the digital signature is used. Note: The MD5 hash algorithm, and by implication any algorithm that uses it, such as RSA-MD5, has been moved to a "not recommended" status in most IETF documents, due to some theoretical weaknesses published in early 2005. These weaknesses do not invalidate the use of the algorithm.

Figure 5 - Digital Signature

Note: In the above diagram while anyone can decrypt the message and recover the digest using the universally available public key only the possessor of the private key can encrypt it - thus proving the authenticity of the source. The underlying digest provides the message integrity. This use is in contrast to confidentiality in which the sender encrypts the data using the public key of the recipient.

---

Problems, comments, suggestions, corrections (including broken links) or something to add? Please take the time from a busy life to 'mail us' (at top of screen), the webmaster (below) or info-support at zytrax. You will have a warm inner glow for the rest of the day.