๐ŸฅฆNdiro
Chapter 10 of 13

Storage: DynamoDB, S3, and caches

This is the chapter closest to your home turf, so it moves fast and focuses on the web-specific parts: how a web app shapes its storage around access patterns, and how HTTP caching turns a slow photo store into a fast one.

Four tables, no GSIs, scans on purpose

DynamoDB is a key-value/wide-column store: you get O(1) access by partition key, ordered range reads within a partition by sort key, and (almost) nothing else. So you design the keys from the queries โ€” the same access-pattern-first thinking as any HBase/Bigtable schema:

TablePKSKDesigned for
usersuser_id (Google sub)โ€”Point reads on every request (the auth guard)
mealsuser_idsk = date#meal_idAll range queries below
sharesshare_tokenโ€”Point read per share-link visit
invitesinvite_tokenโ€”Point read + conditional claim

The meals sort key is the whole data model in one string: 2026-09-03#183042-a1b2c3 โ€” date, then a meal ID whose prefix is the client-supplied time of day. Lexicographic order of the SK is chronological order, so:

db.py โ†—
begins_with(sk, '2026-09-')            # one month  = ONE Query
between(sk, '2026-08-28#', '2026-09-03#~')   # any day window = ONE Query

(The #~ trick: ~ sorts after the digits, so the range is inclusive of the end day's meals.) A side effect worth knowing because it shows up in the UI: the time is part of the key, so editing a meal's date or time would be a delete+create, not an update โ€” which is why edit mode disables those two inputs.

Everything that isn't one of those patterns โ€” admin user listing, share/invite listing, the monitor stats โ€” is a scan, and the code refuses to pretend otherwise. At โ‰ค100 users a scan is microscopic; CLAUDE.md explicitly warns contributors not to "fix" it with GSIs. Complexity is spent where the access pattern is hot, and nowhere else.

โš ๏ธ The Decimal discipline

boto3 refuses floats (binary floats can't round-trip through DynamoDB's decimal wire type) โ€” a raw float into put_item throws. The other edge fails quietly, which is worse: Flask's jsonify serializes a raw Decimal as a JSON string ("12.5"), and the first symptom is JavaScript arithmetic misbehaving. Hence the rule stamped through the codebase: Decimal(str(x)) going in, float() coming out โ€” in review, check both edges of every new numeric field.

Photos: private bucket, proxied bytes

Photos live in S3 at users/{user_id}/meals/{date}/{meal_id}.jpg โ€” keys built server-side only, never accepted from a client. The bucket is private, and the app deliberately does not hand out presigned S3 URLs. Instead the browser fetches /photo/<date>/<meal_id> and the app streams the bytes itself, because putting the app in the path buys three things: every photo read passes the same auth guards as everything else (session for owners, token scope for shares), a revoked share stops being served immediately โ€” a presigned URL keeps working until it expires, wherever it was forwarded, while here only a recipient's already-cached bytes linger, bounded by the one-day max-age below โ€” and there's one choke point to rate-limit and cache.

The cache hierarchy

Proxying means paying an S3 round trip per photo view โ€” unless you cache. Ndiro stacks two layers, and the versioning idea that makes them safe is the transferable lesson:

browser cache (per user)             in-process LRU (shared)          S3
    โ”‚                                        โ”‚                         โ”‚
    โ”‚  <img src="/photo/โ€ฆ?v=9f8aโ€ฆ">         โ”‚                         โ”‚
    โ”‚  hit: 0 requests (immutable, 1y)       โ”‚                         โ”‚
    โ”‚  miss โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ถ byte-budgeted dict              โ”‚
    โ”‚                                  hit: no S3 call                 โ”‚
    โ”‚                                  miss โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ถ get_object
๐Ÿ˜ From your world

Version-the-key-instead-of-invalidating is immutable versioned naming โ€” the same discipline as content addressing (immutable HDFS block IDs, hash-keyed build systems), though here the version derives from a change timestamp rather than the bytes: you get the no-invalidation property without hashing every photo, at the cost of the version not being a content proof. The ETag/304 exchange is the same version tag used as a short-circuit read โ€” "still on v9f8a? no bytes for you".

Deletion, ordered like a systems engineer would

Account deletion has no transaction spanning S3 and four tables, so order substitutes for atomicity: photos โ†’ meals โ†’ shares โ†’ invites โ†’ the users row last. Every prefix of that sequence is retryable, and the user row acts as the commit record โ€” while it exists, deletion can be re-run. The admin monitor even counts "orphans" (meal rows and photo objects whose user row is gone) as a detector for the sequence having stopped part-way: a reconciliation check, not a hope.

๐Ÿ”ฌ Try it

On the review page with photos visible, open Network and reload twice. First load: photo requests return 200 with sizes. Second: they say (memory cache) or (disk cache) โ€” zero requests hit the server. Now edit a meal's photo and watch its ?v= change in the new <img> URL. Then read get_photo_bytes in db.py and find the line that refuses keys outside the owner's prefix.

๐Ÿ