Encrypting Every Packet — AES-GCM, Replay Detection & Rate Limiting

In this article:

• Why AES-256-GCM and not AES-CBC, AES-CTR, or ChaCha20-Poly1305
• How to construct a nonce that survives both bad RNGs and replay attempts
• What goes into AAD, and why every byte of it must be deterministic
• A sliding-window replay detector that tolerates BLE's out-of-order delivery
• Rate limiting and audit logging as the last line of defense

This is Part 3 of a three-part series. Part 1 covered the BLE transport; Part 2 built the authenticated channel that produces the session key we use here.

AES-256-GCM

GCM (Galois/Counter Mode) turns AES from a raw block cipher into an authenticated encryption with associated data (AEAD) construction. In one operation it gives you all three properties you actually want from a packet cipher:

• Confidentiality — the plaintext is encrypted under a counter-mode keystream.
• Integrity — a 128-bit MAC tag fails verification if a single bit was flipped.
• Authenticity — only someone who knows the key could have produced a valid tag.

The alternatives are all worse for our use case:

On modern phones with AES-NI / ARMv8 crypto extensions, AES-GCM is hardware-accelerated and runs faster than ChaCha20-Poly1305. We picked AES-GCM for that reason. On older devices or constrained microcontrollers, ChaCha20-Poly1305 would be the better default — both are AEADs and the protocol around them is identical.

Encrypted packet format

Every encrypted packet on the wire has the same shape:

┌──────────┬────────────────┬──────────────────┬──────────────┐
│  Type    │     Nonce      │    Ciphertext    │   MAC tag    │
│  1 byte  │   12 bytes     │    N bytes       │   16 bytes   │
│  (0x30+) │ (random‖ctr)   │                  │              │
└──────────┴────────────────┴──────────────────┴──────────────┘

The fixed overhead is just 29 bytes per packet — small enough to fit a meaningful payload even at MTU 23. The encrypt path looks like:

Future<Uint8List> _encrypt(
  Uint8List plaintext,
  String keyHex,
  String deviceId,
) async {
  final key = SecretKey(hex.decode(keyHex));
  final nonce = nonceManager.generateNonce(deviceId);   // 12 bytes
  final aad = aadManager.buildAAD(
    packetType: plaintext[0],
    sessionKey: keyHex,
    counter: nonceManager.extractCounter(nonce),
  );

  final box = await AesGcm.with256bits().encrypt(
    plaintext,
    secretKey: key,
    nonce: nonce,
    aad: utf8.encode(aad),
  );

  return Uint8List.fromList([
    plaintext[0],     // type
    ...nonce,         // 12 B
    ...box.cipherText,
    ...box.mac.bytes, // 16 B
  ]);
}

Nonces — random + counter

The single most dangerous mistake you can make with AES-GCM is reusing a nonce with the same key. If you do, the attacker can recover the authentication key and forge tags at will. This isn't a theoretical concern; it has caused real-world breakage in shipped systems.

The nonce is 12 bytes. We split it deliberately:

┌─────────────────────────┬──────────────────┐
│      Random part        │      Counter     │
│        8 bytes          │      4 bytes     │
│  (entropy: prevents     │  (monotonic:     │
│   pattern analysis)     │   replay window) │
└─────────────────────────┴──────────────────┘
                12 bytes (AES-GCM requirement)

Two design choices, two reasons:

The counter is bound to the session and incremented on every encryption. The random part is generated from the OS CSPRNG (SecureRandom on Android, SecRandomCopyBytes on iOS — both backed by hardware entropy on modern devices).

Uint8List generateNonce(String deviceId) {
  final counter = _incrementCounter(deviceId);   // 4 B big-endian
  final random  = _csprng.nextBytes(8);          // 8 B random
  return Uint8List.fromList([...random, ...counter]);
}

Additional Authenticated Data

AAD is data that travels with the packet, is not encrypted, but is covered by the MAC tag. If anyone tampers with it, decryption fails. AAD is where you put context that should be cryptographically bound to the message but does not need to be hidden:

String buildAAD({
  required int packetType,
  required String sessionKey,
  required int counter,
}) {
  final keyHash = _hashSessionKey(sessionKey).substring(0, 8);
  return [
    'BLE-v2',           // protocol version
    'type=$packetType', // bind to message type
    'key=$keyHash',     // bind to session
    'ctr=$counter',     // bind to position in sequence
  ].join(':');
}
// → "BLE-v2:type=30:key=a1b2c3d4:ctr=7"

What each component blocks:

Just like HKDF inputs in Part 2, every byte of AAD must be derivable identically by both parties — or decryption fails for legitimate traffic too. Hardcode the protocol string, derive the key hash from the session key both sides already share, and lift the counter directly out of the nonce field of the incoming packet.

Replay detection

The MAC tag protects against modification, but not against an attacker recording a valid encrypted packet and replaying it later. Replay detection is the receiver's responsibility, and it must coexist with BLE's habit of delivering packets out of order.

The strategy: a sliding window of accepted counters per device.

bool validateNonce(Uint8List nonce, String deviceId) {
  final counter = _extractCounter(nonce);
  final state   = _devices[deviceId] ??= _State();

  if (counter > state.highest) {
    // shift window forward
    state.highest = counter;
    state.seen.add(counter);
    state.seen.removeWhere((c) => c < counter - _windowSize);
    return true;
  }

  if (counter < state.highest - _windowSize) return false; // too old
  if (state.seen.contains(counter))         return false; // duplicate
  state.seen.add(counter);
  return true;
}

A worked example with window size 1000 and current state highest=51, seen={45,46,47,48,50,51}:

The "skipped too far ahead" check protects against an attacker who pre-mints a packet with a huge counter to permanently shift the window past anything legitimate.

Auth packets bypass encryption

The four handshake messages from Part 2 (0x11–0x14) cannot be encrypted — there is no shared key yet to encrypt them with. The encryption middleware short-circuits on these types:

static const Set<int> _authPacketTypes = {0x11, 0x12, 0x13, 0x14};

@override
SecurityHandler process(SecurityHandler next) {
  return (packet) async {
    if (_authPacketTypes.contains(packet.type)) {
      return next(packet); // pass through
    }
    return _encrypt(packet, sessionKey, deviceId);
  };
}

This is safe because the handshake itself only carries:

• Public keys, which are designed to be transmitted in the clear.
• The terminal's certificate, which is anyway signed by the root and verifiable offline.
• A signature over the ephemeral key plus transcript, which an MITM cannot forge without the terminal's static private key.

The actual transaction data — payment tokens, amounts, results — only flows after both sides have a session key, and from that moment on every packet is encrypted.

Rate limiting

Even with all of the above, an attacker can still try to exhaust the terminal: open thousands of half-finished handshakes, each forcing the terminal to do an expensive ECDH + signature verification. Without a budget on the receiver, this is a cheap DoS.

The fix is a per-device, per-minute budget enforced before any cryptographic work is done:

class RateLimiter {
  RateLimiter({this.maxRequestsPerMinute = 30});

  bool allow(String deviceId) {
    final now = DateTime.now();
    final history = _history.putIfAbsent(deviceId, () => []);
    history.removeWhere(
      (t) => now.difference(t) > const Duration(minutes: 1),
    );
    if (history.length >= maxRequestsPerMinute) return false;
    history.add(now);
    return true;
  }
}

For the DH handshake we set the budget to 30 requests per device per minute — far more than any legitimate customer would ever generate, far less than an attacker would need to be effective. After the limit is hit, the middleware refuses to process new auth packets from that device until the window slides forward.

Security audit logging

Every security-relevant event is logged with structured context:

• Handshake started / completed / failed (and which check failed).
• Decryption failure (MAC mismatch, AAD mismatch).
• Replay detected (counter, current window).
• Rate limit triggered (device id, count in window).
• Session created / expired / wiped.

Logs are buffered locally and uploaded opportunistically when the terminal next has internet. They never contain key material, only event metadata. This is the difference between "we got attacked once" and "we have a baseline of normal behaviour and an alert when it deviates."

Takeaways

1. Pick an AEAD. AES-GCM if you have AES-NI, ChaCha20-Poly1305 if you do not. Never roll CBC + HMAC by hand.
2. Nonce uniqueness is non-negotiable. Random + counter gives you defence in depth against both bad RNGs and replay.
3. AAD binds context to ciphertext. Use it for protocol version, packet type, session, and position — anything an attacker might want to lie about.
4. Replay detection needs a sliding window because BLE re-orders. A strict "must be greater than last" check will drop legitimate traffic.
5. Bypass encryption only for handshake packets that carry public material the protocol is already designed to expose.
6. Rate limit before crypto. Cheap pre-checks protect expensive operations from DoS.
7. Log security events. A breach you cannot see is a breach you cannot fix.

Three articles in, you have an end-to-end picture of how a real BLE-based payment channel actually works — from the radio to the cipher. The same patterns apply to any sensitive BLE protocol: medical devices, smart locks, IoT credentials. The transport changes; the principles do not.