Re-reading this thread from the top changes the question. Two parts of a turn are bounded in count — history, held to two turns for most built-in roles, and suggestions, held to three. An earlier post in this thread set one fixed budget for the prompt, with headroom, as the point where its compaction pass fires; that bounds the prompt’s total at the moment the pass fires. What nothing in the thread does is watch the parts as they change between passes, or cap a single piece: a single piece inside one turn has no cap of its own in anything the thread describes. What can grow from turn to turn could be the workflow state itself, and the search store it is ingested into: every turn, both the current workflow version and the history are ingested into that store, and nothing in the thread describes the store replacing or de-duplicating what it is given. Whether it keeps them all is an inference — the log below is built to test it, and one reading would overturn it on its own: the store’s item count holding flat while cycles accumulate. So put the cycle control where the growth can be, and let it drive the window. One separation is worth stating plainly, because the thread touches it twice: the KV a request uses is, on the engines as described here, the KV of that request’s own prompt and the tokens it generates, whether computed in the request or reused from an identical opening; what a slot holds can be more than that — state from an earlier prompt can still sit in it — but held state is not context the new request carries, and the size that matters is the used side, governed by keeping each single prompt, with room for its output, inside its bound; the cross-request reuse that reuses prompt computation is the matching-prefix reuse cache_prompt provides, which reuses computation on identical opening tokens rather than carrying context across — what persists and could grow between cycles is the state and the store; slot prompt state and model residency can persist too, without growing on their own. That separation is an inference from the engines as the thread describes them, and it carries its own overturn reading: if prompt tokens climb while the turn’s own carried content — pieces, suggestions, state — holds flat on an unchanged slot, held state is entering the used side and the separation falls; the log’s per-cycle prompt lines are where that would show. The builds below are named now so the log’s branches can point at them; none of them is built before its branch lands.
For the state branch, the build is the Planner’s own pass, changed two ways: it fires on a count or a bound instead of when the window is nearly full, and it moves closed material out instead of summarizing open material. After a set count of cycles — a cycle being one full pass of one workflow’s loop, an input taken in, acted on and written back — or when the workflow’s serialized size passes a set bound, compact the live state without summarizing it. Set the count from the recent growth rate — the change, per cycle, in what the workflow’s serialized size costs in prompt tokens, measured across the most recent run of cycles: roughly the cycles the state takes to grow by a quarter of the size bound. Where the measured growth is zero, the count has nothing to derive from; there, set it at twenty cycles — the same fallback the run below uses when growth is near zero. A quarter is a starting fraction, chosen so compaction lands well before the bound alone would fire; tune it from the log, and re-derive the count when the growth rate changes. Set the size bound by what the state costs in the prompt, its tokens counted against num_ctx, with headroom left for the compaction pass itself and the next action — the headroom sized from the largest compaction pass and the longest next action the log has seen — not when the window is already full. The state bound is one part of the whole-turn budget: the system prefix, the pin, the history turns, the suggestions, the incoming piece and room for the output are summed against num_ctx, and the pass fires on that sum, so no turn is bounded only in its parts. The sum fires the pass, not the moving: inside the pass, only closed or finished material moves out, so a transient piece can call the pass early without touching live state. And the sum binds the prompt itself, not only the pass: any part that would take the turn past the sum is held out that turn by its exact reference and fetched on demand — the treatment the caps below give a returned result and an oversize piece — whether or not its branch build has landed.
What moves out: closed TODOs and closed Comments go to the version history, and are reached again through it by their exact references, as a superseded version is reached through version control; a Unit moves out only when it is finished and every Connection joining it joins only finished Units — finished meaning whatever completion marker your build keeps for a Unit or a Connection, and if it keeps none, adding that marker is part of this build — a finished Unit still joined to a live one stays live with it; a Connection moves when both Units it joins are finished; Environments and Layout move out only when nothing live depends on them — no live item references them, is keyed to them, or sits inside them, whichever way the link runs in your build; and nothing at all moves out while an open TODO, an open Comment, a decision or an exact reference still points to it. What stays: open TODOs, open Comments, decisions and exact references, exactly as written.
Drop superseded workflow versions from the search store — they leave the store, not the record: version control keeps them — keep only the current version available as current state, and bound the history side in its own terms: one copy of each history turn — a turn known by its absolute place in the conversation — its ordinal from the conversation’s start, not its place in the current window — so the same turn ingested again adds no second entry — and turns older than a set span of cycles fold into a running summary entry — the originals move out of the searched store into a cold archive, and each summary entry carries the index of its originals — that index is the route back to them — with a real recall cost: a question nobody anticipated, over a folded turn, is answered from the summary unless the original is fetched back through the index — the span set the same way the count above is set, from the history side’s own measured growth — the change per cycle in its entry count and in what its turns cost as suggestion tokens — as roughly the cycles that growth takes to add a quarter of what the history side may cost the prompt, the count’s own starting fraction applied to this side, and where that growth measures zero the span falls back to twenty cycles, the count’s own fallback, and nothing folds before it — and the standing guard is an index of the folded turns plus spot questions whose answers live in them — a summary cannot be checked the exact way the working state is, and that limit is named rather than hidden. The conversation is the part the earlier posts set apart for summarizing; it is the working state that must never be summarized. Keep current-state chunks out of the returned suggestions: the pin below already carries the current version, so a returned copy of it is a second copy in the prompt. The current version itself stays in the store while it is current — not pruned, not folded — as a single entry per version: each turn’s ingestion of the current version replaces the entry that version already has, rather than adding another, so a long-lived current version cannot accumulate copies of itself; keeping it out of the returns is its other treatment, and when a newer version supersedes it, it leaves under the superseded-version rule above. A cycle that finds nothing closed or finished simply passes — and if the bound lands while everything is still open, the overflow route is to split the live state by role or by subgraph into separately pinned parts, so no single prompt has to carry all of it — Connections crossing the split are carried as exact references in both parts, and the list-back below runs over the union of the parts, so inventory exactness is kept; what the split gives up, by design and only in this overflow case, is one prompt holding the whole graph at once — the crossing content is fetched on demand through its exact reference.
The pinned block from earlier earns its place for one sharper reason: for any role whose route to the workflow is the search, that route is the ranked suggestions — the cap the thread puts on its suggestions, three results in total, carried onto this route because its results are those same suggestions; read that carry as an inference, with its overturn reading in the log below: returns on that route arriving uncapped, or above three, overturn it — so pin the whole current workflow, in the compact form it is already built in — Comments, Units, Connections, Environments and Layout all present — with its TODOs, decisions and exact references word for word inside it. To be precise about my earlier “inject it whole” line: the expanded listing adds detail on demand. One check keeps “whole” honest: count the items in the compact form against the expanded listing and compare every item field by field; any piece the listing holds that the compact form lacks joins the pin. Two things are not branch builds: the pin, and the Planner trigger change from the earlier post. Both are corrections our own earlier posts proposed and this one stands behind, and both hold whichever branch lands; the branch builds are what the Planner then moves, the pruning, the cap and the bounds. The pin also has its own overflow treatment, and it is the pin’s, not a branch build, so it too holds whichever branch lands: if the block itself would take a turn past the whole-turn sum, it is pinned in parts by role or by subgraph, the same split the state branch uses for bound overflow, with the crossing Connections carried as exact references in each part and the list-back running over the union. Retrieval should not be the thing that decides whether a live item exists. It could also let one agent monitor another against the same current state, instead of each reading ranked suggestions of it — the block carries the workflow’s version marker — and if a version carries no marker in your build, adding one is part of this build, with the same standing as the completion markers above, since the stale-version reading below classifies suggestions by it — and the log below records that marker for each role’s block as well as for the suggestions, so two roles holding blocks from different moments would show it. The workflow stays the governing copy — the block is only ever generated from it and never edited in place, so a difference between the two is settled by regenerating the block, not by merging them. And place the changing suggestions after the system prefix and the workflow block, so llama.cpp’s prompt cache — the cache_prompt you just switched on — can reuse the prompt up to the first token that changed. Where the server keeps per-slot prompt state, keep one workflow’s turns for one role on the same server slot, so the prefix is compared against that workflow-and-role’s own previous prompt and not another pairing’s. The Ollama keep_alive beside it was added in the reuse step, but as a setting it reads as residency rather than prompt reuse — residency is what it buys, on the thread’s own presentation of the setting; that reading has its test in the log: on cycles that engine serves after an idle gap, reuse would show as first-token time falling with no reload and no change in the prompt, and if it shows, the reading falls — so the prefix ordering is a llama.cpp lever.
The order matters: measure before rebuilding, and build only what the branch, or combination of branches, you land on names; the rest waits. On your longest Telegram workflow, log per cycle, in the first run and in every confirmation run after a build lands:
- the prompt tokens, and the tokens carried by the returned suggestions
- the size of the largest single piece carried into the prompt that cycle, the count of pieces, and how many of them sat under the single-piece bound the piece branch below sets
- the workflow’s serialized size, and the number of items in the search store — split by kind, workflow versions and history turns, with the number of entries carrying the current version counted separately
- the times: search, the turn’s own ingestion into the store, cycle start to first output token, and the cycle’s total time
- the output tokens, and whether the cycle completed its task — so cost per successful cycle can be computed, not just watched
- the number of workflows and roles active in that cycle, the idle interval before it, and whether the model was reloaded inside that interval
- which engine served the cycle
- on the llama.cpp side, the share of the prompt reused from its cache, the length of the opening stretch unchanged from the previous turn, and which slot served the turn
- whether the suggestions name the current workflow version or an older one, the version marker carried by each role’s pinned block, and whether any result was set aside by the per-result cap below
Hold the cache settings fixed throughout. Run it for at least twice the compaction count of cycles — where growth is near zero and the count fell back to twenty, that is forty cycles — and call a branch only when its reading’s direction holds across the run, not on a single cycle — with one exception: the item-count reading — the single check on the whole account — can overturn the store account at any point in the run. The branches carry their own overturn readings under it: the state account falls if what the workflow costs in prompt tokens holds flat across the run, and the piece account falls if the largest piece and the piece count both hold flat. The readings can overlap, so treat them as combinations, not as exclusive boxes. If prompt tokens and suggestion tokens stay flat while the time to first token climbs: a climbing store count with climbing search time points to the store behind the search; a climbing ingestion time points to the per-turn write, read beside the workflow’s size, since a larger version also takes longer to write; a falling reused share points to the cache, read beside the unchanged-opening length, since an earlier first change shortens reuse on its own — and on cycles the other engine serves, the cache reading is the reload line instead, since residency is that side’s lever. If none of those move, the climb could sit outside this account: if it tracks the number of active workflows and roles, contention on one server is the driver — stagger the loops’ cycles, by at least the longest cycle time the log has measured, so the heavy stages stop landing together, or cap how many workflows run against one server at once, at the active count where the log shows the climb beginning; if it tracks idle gaps ending in a reload, set the residency window from the longest measured gap instead of a round number; if it tracks neither, split the turn into stages — search, prompt build, first token, generation — and the stage that climbs names the next place to look. If suggestion tokens climb while the workflow’s size holds steady, retrieval is carrying growth into the prompt — prune superseded versions, and cap the tokens any single returned result can carry — the cap sized from the whole-turn sum: one result may carry what leaves the system prefix, the pin, the history turns, the incoming piece, the other returned results and the output room inside that sum, a number the log’s largest in-budget turn makes concrete. A result over the cap is not cut inside and does not vanish: it is carried by its exact reference and fetched on demand — and when fetched, it enters under the same whole-turn budget, in place of other content, not on top of it — and its absence from the prompt shows in the log with the result named, so if a later cycle reaches for it, the log shows exactly which cap event set it aside. If prompt tokens climb while suggestion tokens and the workflow’s size both hold steady, the growth is arriving inside the turn itself: if the largest piece is what climbs, the lever is a bound on what a single piece can carry in — the bound sized the same way, from the whole-turn sum: one piece may carry what leaves the rest of that turn inside the sum. A piece over the bound is held out of the prompt and carried by a reference — the piece is first stored and given that reference, and if the build keeps no store for incoming pieces, adding one is part of this build — its content fetched on demand in parts, each part under the same bound, so a piece that cannot fit whole is still reachable without ever entering whole — the trade named plainly: no single prompt ever holds the whole piece at once, only the part that turn fetched; if the count of pieces under that bound is what climbs — the mid-size count, in the log’s terms — the lever is a bound on pieces per turn, sized from the same whole-turn sum: the count whose combined carried size, at the sizes the log measures, leaves the rest of the turn inside the sum. Pieces past that bound get the same treatment: held out, carried by exact reference, fetched on demand, and the holding-out shows in the log.
If suggestion tokens and the workflow’s size climb together, the stale-version line separates the candidate sources: suggestions naming an older workflow version point to stale duplication feeding the prompt; suggestions naming the current version point to duplication of the current copy when the entry count says the store holds it more than once, and to the state reaching the prompt through search when it holds it exactly once; suggestions naming neither point to growth in the history side of the store, with the state’s own climb read by the state branch below. If the workflow’s size and the prompt tokens climb together while suggestion tokens hold steady, that points to the state as the growth.
Each branch then confirms on its own measure flattening after its build lands, with one caution kept separate: in the state branch, the bound alone does not promise the search time flattening — that part waits on the store pruning also running. And the run closes by recomputing cost per successful cycle from the logged totals — that number is the verdict the whole log exists to move, not just one more input. And whichever branch lands, no fix counts as working until the list-back of TODOs and references stays exact after it — run over the live state and the version history together once closed material has moved there, as it runs over the union of the parts in the split route: a fix that flattens a number by losing or altering a live item has not worked.
A note on my own last reply: it answered the final lines of your post and missed the earlier questions. I had not read the whole thread back before answering. Reading it back first is where my best work in this thread comes from, and I believe it is where yours does too. The search disclosure in those final lines still matters — it is why the store’s item count is in the log at all. Run the log first; the branch, or combination of branches, it lands on names the fix, or fixes, to build.