Feedback Loop Field Guide Here I’ve gathered as many feedback loops as I could think of and lay my hands on, with their latency in application. For increasing the quality of AI output, it helps to feed back deterministic signals. Starting from the basics, you can reduce errors with strong, pervasive typing, linting and automated testing at multiple levels, but that only covers building a thing initially, not all the other questions: is it the right thing, is it deployed correctly, and so on. There are a few basic guidelines and conclusions at the bottom of the page.
How to use feedback loops, and how they relate ↓ What loops to use for what decisions →
high fidelity medium fidelity low fidelity latency range (min to max)◆ human-mediated ⚡ AI can shorten this loop hover a pill to trace its proxy chain & balancing metrics
machine time build time product time strategy time 100ms 1s 1min 1hr 1day 1wk 1mo 1qtr 1yr 3yr
machine time build time product time strategy time 100ms 1s 1min 1hr 1day 1wk 1mo 1qtr 1yr 3yr Build · Can we make the thing? Correctness IDE inline feedback Syntax / parser Hot reload / REPL Type system Lint / static analysis ◆ Pair / mob programming · XP Runtime schema validation Unit tests Coverage / mutation testing Visual regression testing ⚡ AI code review Accessibility checks Integration tests E2E tests ⚡ Error monitoring ◆ Code review ⚡ ◆ Bug reports ⚡ Reliability Synthetic monitoring / uptime checks Canary / progressive rollout APM / distributed tracing ◆ Chaos engineering / game days ◆ Incident post-mortems ⚡ SLO / error budget burn Performance Bundle size / asset weight Lighthouse / synthetic performance Memory / resource utilization Benchmark tests ◆ Load / stress testing Core Web Vitals (field / RUM) Capacity / scaling signals Security Secrets detection SAST / code security scan ⚡ Runtime security detection SCA / dependency vulnerabilities ⚡ Cloud misconfiguration / CSPM ◆ Threat modeling ⚡ DAST / automated web scanning Open vulnerability age / patch SLA ⚡ ◆ Bug bounty / VDP ⚡ ◆ Pen test / red team Maintainability Unused code / dead exports Code complexity & coupling metrics Build & CI duration Rework rate / code churn Hotspot analysis (churn × complexity) Dependency freshness (libyear) ◆ Tech debt accumulation ⚡ Delivery flow ◆ Daily standup · Scrum ◆ App store review gate WIP · Kanban Work item age · Kanban Sprint predictability (say:do ratio) · Scrum Throughput · Kanban Lead time for changes · DORA Deployment frequency · DORA Mean time to restore · DORA Change failure rate · DORA Task cycle time Flow efficiency · Kanban PR cycle time Feasibility ◆ Feasibility spike / throwaway prototype · Cagan product risks ◆ Technology / model evaluation · Cagan product risks ◆ Design / RFC review ⚡ Data Data quality tests Pipeline freshness / data SLAs Data anomaly detection AI & model Eval suite / golden-set regression Model latency / time-to-first-token Token / cost per request LLM-as-judge scoring ◆ AI red-teaming / adversarial evals ⚡ Model & output drift Value · Is it worth making? Usability testing First-click / findability Clicks / interaction cost ◆ Task ease (SEQ / SUS) ◆ Dogfooding ◆ Sprint review / stakeholder demo · Scrum ◆ Beta / early-access program ◆ Usability tests ⚡ ◆ Task success rate ◆ Time on task Usability signals AI synthesis of qualitative streams Session recordings / rage clicks ⚡ Thumbs up/down on AI output ◆ Public reviews (G2 / app stores / Reddit) ⚡ Funnel / drop-off analysis ◆ Support tickets ⚡ Support ops Time to first response ⚡ Resolution time ⚡ Contact rate / tickets per customer First contact resolution Ticket CSAT Desirability ◆ Opportunity scoring (importance vs satisfaction) · ODI ◆ Willingness-to-pay research · Van Westendorp ◆ Feature request board / upvotes ⚡ ◆ Weekly discovery interviews ⚡ Feature adoption / activation Fake door / prototype tests Sean Ellis PMF survey · Sean Ellis ◆ Kano survey · Kano A/B experiments NPS / CSAT Solves the problem ◆ Churn interviews / exit surveys ⚡ Trial / freemium conversion Stickiness (DAU/MAU) Expansion / referral revenue Retention curves Better than alternatives ◆ Win/loss analysis ⚡ Reach · Can we get it to people? Paid CPM / auction pressure Cost per click (CPC) Paid ads CTR / CPA Landing conversion rate (CVR) Quality / relevance score Ad-spend liquidation Frequency / creative fatigue Incrementality / lift tests ROAS / channel CAC Attribution Last-click attribution Multi-touch / data-driven attribution ◆ Self-reported attribution Marketing mix modeling Email deliverability Spam / reject rate Delivered / inbox placement Bounce rate Click-through rate Open rate Unsubscribe / list churn Sender reputation Outbound ◆ Cold outreach reply rates ◆ Meetings booked / pipeline Pipeline stage conversion Sales cycle length Content Content engagement SEO ◆ Keyword research (difficulty vs. potential) ⚡ Technical SEO / indexation Keyword rankings / SERP position AI search visibility (AEO / GEO) Domain authority / backlink profile Organic traffic & CTR Content decay / refresh signal App stores App store ratings & ASO Short-form video Views / impressions Audience retention / watch time Completion / loop rate Hook rate / swipe-aways Follows-per-view Shares & saves Organic social Engagement rate Follower / subscriber growth Viral / referral Referral / K-factor Brand ◆ PR / press coverage ◆ Brand awareness / recall Partnerships ◆ Partnership pipeline Events ◆ Events / webinar performance Community Team · Can we sustain the people doing the thing? Hiring Onboarding ramp / time-to-first-commit ◆ Sourcing response rate ◆ Interview signal ◆ Offer acceptance rate ◆ Quality of hire / ramp time ◆ Time-to-hire ◆ Compensation benchmarking Engagement ◆ 1:1 sentiment Pulse surveys ⚡ ◆ Team retrospectives ◆ Developer experience survey eNPS / engagement surveys ⚡ Capacity On-call load / pages per person Meeting load / focus time Bus factor / knowledge concentration ◆ Attrition / regretted departures Growth ◆ Performance reviews / 360 feedback ◆ Growth / promotion readiness Alignment ◆ OKR check-ins / goal scoring · OKR Risk & Moat · Do the economics allow success to continue? Capital Cloud spend / burn alerts ◆ 13-week cash flow forecast AI / inference spend ◆ Budget vs actuals (monthly close) ◆ Investor / fundraising feedback Pricing realization / discount rate Cash collection / DSO ◆ Revenue forecast accuracy Runway / burn multiple Revenue concentration Revenue churn / GRR CAC payback period MRR / ARR growth Gross margin Blended CAC LTV:CAC ratio LTV Platform ◆ Platform / API dependency changes ◆ Vendor / provider concentration Compliance ◆ Compliance audits (SOC2 / ISO) ⚡ ◆ Regulatory & legal signals ⚡ Trust & safety Fraud / abuse / chargeback rate Moat Churn by tenure / lock-in strength ◆ Competitor launches ⚡ ◆ Pricing power Anticipatory ◆ Pre-mortems ⚡
Each pill marks indicatively how long you have to wait to get feedback from that loop: the time between doing something and finding out whether it worked. The line behind it shows the range of time it typically takes.
How to work with feedback loops?
It sounds obvious until the first time but: start with your goals. These tell you the questions you need to ask , which then make obvious the feedback loops to pay attention to. Each of those loops may have faster proxies that let you fix an issue earlier, and complementary measures that tell you if you are damaging something elsewhere because of your work on the primary issue. For example:
our goal: stop the weekly outages our questions: how do we keep production up? how do we keep shipping safely? what to pay attention to: error monitoring , alerts / on-call pages , uptime checks , change failure rate fast proxies: unit tests , e2e tests , canary / progressive rollout complementary: deployment frequency , lead time for changes A signal is only useful if you can act on it
Can you turn what a loop tells you into a decision and an action before it goes stale? And will you have the capacity for that action when it arrives?
Slow loops are more difficult to work with
Slower loops are easier to miss. Some cannot be sped up at all: how a page climbs in search, whether customers stay, whether a new hire works out. These take weeks or months by nature. They are less automatable, and less able to be shortened by AI. Many slow loops don’t have a fast proxy. It is typically the job of people more senior in the org to pay attention to them, and judgement and intuition help enormously. And of course if people have their intuition tuned from inappropriate examples in their past, they risk reacting incorrectly.
Many metrics aren’t easily balanced
Only some of these feedback loops have an exact complement, which means it is very easy to do better in one area and make something else worse.
Dataset v0.23.1 · CC BY 4.0 · cite as: Konrad Bloor, Feedback Loop Field Guide, konradbloor.com/loops · also available as
markdown and
JSON All the prose is mine. I used AI to gather and organise the data, and to build the thing that displays it.