Random String and Token Generator

There are two ways to make a random token, and most frameworks use the second. One picks each character independently from an alphabet. The other draws random bytes and encodes them, as Python’s secrets.token_hex(32), Node’s crypto.randomBytes(32).toString('hex') and openssl rand -hex 32 do. For hex the two are equivalent: 32 random bytes are 64 hex characters and 256 bits either way. Base64url is not quite: 32 bytes encode to 43 characters, but 256 bits fill only 42 of them and four bits of the last, so the final character takes only 16 of the 64 values, while 43 characters chosen one at a time carry 258 bits. Random bytes mode shows the equivalent OpenSSL, Python and Node.js call.

Picking characters is where generators go wrong. The obvious code, a random byte modulo 62, is biased, because 256 = 4 × 62 + 8: bytes 248 to 255 wrap round onto the first eight characters of the alphabet, 0 to 7, which then come up five times in 256 instead of four, 25% more often than the rest. The average loss is small, yet a 32-character string made only of those eight characters is about 450 times likelier than it should be, and the min-entropy, the figure that matters to an attacker who guesses the likeliest values first, falls from 190.5 to 181.7 bits. This generator uses rejection sampling instead: a byte at or above 248 is discarded and another drawn, which costs about 3% more bytes. Base58 discards 24 of every 256 bytes and digits discard 6; hex and base64url, whose sizes divide 256, discard none, which is one reason frameworks prefer them. The bytes come only from crypto.getRandomValues.

Entropy is length × log2(alphabet size), which holds only because every position is uniform and independent. A prefix such as sk_test_ adds nothing, since it is identical on every key. Providers add one anyway so that a leaked key can be found: GitHub’s secret scanning partner programme asks for a unique prefix, high-entropy random data and a checksum, and notifies the provider when a matching string appears in a public repository so the key can be revoked. GitHub’s own tokens start ghp_ and similar, with an underscore so a double-click selects the whole token. This tool adds the prefix but not a checksum. With Secret on, it warns below 128 bits, the minimum RFC 6749 sets for OAuth tokens. It never removes repeats from a batch, because that would bias it; it warns when repeats are likely instead.