Pitfall deep-dive #6: secrets in env files shared in Slack
Pitfall deep-dive #6: secrets in env files shared in Slack
What would you do differently next time, and what signal told you too late?
Thread index 6 — add your angle.
15 replies
The smallest improvement to date pickers reduced timezone bug reports from global users. In engineering pitfalls (thread 6): The smallest improvement to bulk edit confirmations prevented a costly mistaken archive. The prototype used fake data; production assumptions did not survive contact.
We stopped confusing 'busy roadmap' with 'validated roadmap' in planning reviews. In engineering pitfalls (thread 6): We stopped treating 'tech debt paydown' as a bucket without measurable outcomes quarterly. We stopped confusing motion with progress once we counted outcomes weekly.
We should have asked legal earlier about data residency — assumptions were expensive. In engineering pitfalls (thread 6): We should have invested in backup restore drills before the auditor asked for proof. The best teams treat vendor incidents as joint incidents with shared timelines publicly.
The quiet win was documenting which database is authoritative for each entity finally. In engineering pitfalls (thread 6): The integration that retried with idempotency keys prevented duplicate charges quietly. We learned that transparent salary bands reduce whisper networks and attrition surprises.
We learned that psychological safety includes saying this deadline is unsafe. In engineering pitfalls (thread 6): We learned that 'temporary' flags need owners and expiry dates in writing. Performance work without profiling is astrology with a compiler.
The architecture review that asked about secrets rotation cadence changed our KMS strategy honestly. In engineering pitfalls (thread 6): We stopped treating 'innovation' as a separate team — embedding experiments into squads shipped more learning. We stopped shipping 'temporary' import tools without checksums — corrupted history imports are worse than empty circles always helpfully.
The quiet win was aligning on a single on-call handoff template across teams. In engineering pitfalls (thread 6): The on-call runbook with copy-paste commands beat heroic memory every time. The vendor demo lied by omission; our staging environment told the truth.
The quiet win was documenting which Slack channel is authoritative during incidents. In engineering pitfalls (thread 6): We learned that naming owners for analytics pipelines prevents mysterious metric drift nobody owns. We stopped shipping 'just internal' APIs without authentication because internal becomes external eventually.
Cut scope early; nobody remembers the sixth nice-to-have you skipped. In engineering pitfalls (thread 6): The flaky smoke suite that ran only nightly missed regressions that hourly would catch. We learned that transparent engineering ladders reduce attrition from perceived favouritism.
The smallest improvement to search relevance reduced 'cannot find' tickets sharply. In engineering pitfalls (thread 6): We should have invested in shadow reads before switching the primary database. The bug was timezone-related again; the sun never sets on bad assumptions.
The smallest improvement to error copy reduced 'what do I do' support chats measurably. In engineering pitfalls (thread 6): We should have deleted unused Slack integrations firing noise into incident channels. The mentor who said 'document the workaround' saved the next on-call from inventing a worse one.
We should have deleted unused CI secrets after rotating tokens — scanners found them anyway. In engineering pitfalls (thread 6): The smallest improvement to empty states reduced bounce on onboarding screens. Good defaults in CI catch honest mistakes; culture catches dishonest shortcuts.
We should have invested in synthetic checks for the login path specifically. In engineering pitfalls (thread 6): The integration that validated idempotency on refunds prevented double-credit incidents quietly. The architecture spike that listed operational costs prevented surprise cloud bills later.
We stopped shipping 'temporary' email digests without unsubscribe — deliverability dies and members lose trust measurably always. In engineering pitfalls (thread 6): The quiet win was deleting duplicate metrics that disagreed politely in Grafana. We stopped treating 'innovation' as a separate team — embedding experiments into squads shipped more learning.
The integration that logged request ids made vendor finger-pointing shorter every time. In engineering pitfalls (thread 6): The migration that chunked batches avoided locking tables longer than maintenance windows. We stopped treating accessibility as a polish pass and caught issues earlier.
Join the conversation.
Log in to reply