Thinking Progress Vectors: a loading bar for reasoning models, read from the paper and the code

What the paper claims

The paper is by Roy Eisenstadt, Itamar Zimerman and Lior Wolf, listed with Tel-Aviv University (Zimerman also with IBM Research). The version I read is v1, dated 8 June 2025, marked as a preprint under review. Its starting point is a known problem: a reasoning model that thinks too little misses the complexity of the task, and one that thinks too long can loop, doubt itself and run out of budget.

It makes two claims. First, the model keeps an internal estimate of how far along its thinking phase it is, and a simple linear projection of the final hidden layer recovers that estimate. The authors call the projection vector a thinking progress vector. Second, that signal is causal: if you push the hidden state along the vector during generation, the model finishes its thinking sooner. They call this overclocking.

Everything is tested on two distilled DeepSeek-R1 models, a 32B model built on Qwen and an 8B model built on Llama, and on two math benchmarks, Math-500 and GSM-8K. That scope matters for everything below.

The pipeline, step by step

The README lays the work out as four scripts. This is what each one does, as I read the code.

1. generate_tpv_data.py. Loads a model in half precision, builds the prompt “{problem}, please reason step by step, and put your final answer within \boxed{}”, ends it with an opening think tag, and samples answers. Defaults match the paper: 30 Math-500 problems, 5 samples each, temperature 0.6, top-p 0.95, seed 42. It asks Hugging Face generate for all hidden states and saves the last-layer vector of every generated token.
2. prepare_tpv_dataset.py. Drops samples with errors or missing files, then looks for the closing think tag inside the generated tokens. A sample that never closes its thinking is skipped. For the rest, token number j of N gets the label j divided by N, so the label climbs from near 0 to exactly 1. The split into train and test (80/20, seed 42) is made by problem, so one problem never appears on both sides.
3. train_tpv.py. Fits a linear model with the closed-form solution of H transpose H plus a tiny epsilon (1e-10 by default) times the identity, against the labels. There is no bias term. It reports mean squared error on train and test and draws a scatter plot. The saved weight vector is the TPV.
4. tpv_intervention.py. Loads the model and the vector, registers a hook, and generates with greedy decoding. The default is the Llama 8B model, alpha 100 and 2048 new tokens.

The hook: what the intervention really does

The whole intervention is one forward hook on model.model.norm, the final normalization layer of a Hugging Face decoder model. After that layer runs, the hook copies its output and adds alpha times the TPV to the last token position. The paper writes it as h_alpha = h + alpha times theta, so the predicted progress theta times h rises by alpha times the squared length of theta.

Where it sits is deliberate. The paper explains that the final hidden layer is downstream of the attention layers, so the keys and values that were cached for earlier tokens are untouched. The nudge affects only the choice of the next token. But that token then becomes input to the next step, so the effect compounds over the whole thinking phase. The hook is removed in a finally block when generation ends.

Two practical notes from the code. Alpha is not portable: the size of the vector depends on how the regressor was trained, so 100 means something different for every model and every training set. And the vector is plain float32 added to half-precision states, which I would want to check in a real run.

What the “loading bar” is in this repo

The paper presents an interactive progress bar and links a project page for examples. The repository itself has no such interface. The monitoring side ends at a regression plot: a scatter of predicted against true position for up to 1000 test tokens, with an exponential smoothing line (factor 0.1) drawn over samples that are sorted by true position. That plot checks the regressor. It is not a live bar over a real generation.

If you want a bar in a UI, you have to build it: run the model locally, take the last hidden state at every step, multiply by the vector, smooth the series, and draw it. The paper adds that a small recurrent network (a single-layer GRU) can replace exponential smoothing and gets lower loss in both the learned and zero-shot settings. That variant is described in the paper, and I did not look for it in the repo.

The numbers the paper reports

The paper compares TPV with three baselines under fixed token budgets: the base model, the shortest of five samples at temperature 0.6, and a prompt that tells the model to be concise and stop. It counts correct answers. The table below copies the Math-500 correct counts for five rows, as printed in the paper. I did not recompute any of them.

How to read those numbers

On the 32B model at a 512-token budget, TPV with alpha 100 raises correct answers from 36 for the base model to 100, and beats both baselines there (54 and 53). That is the headline result, and it is a large one. The authors also report that adding TPV to the concise prompt works best of all, up to 148 at 512 tokens. Their reading is that the two methods are complementary.

The picture is less even elsewhere. At 2048 tokens on the same model, the concise prompt alone gets 316 and TPV alone gets 284, and the authors say so themselves. On the 8B model, TPV with alpha 100 alone beats the base model at 512 and 1024 tokens, but it trails both baselines at every Math-500 budget, and at 2048 tokens it gets 102 against 112 for the base model. Alpha 5 does almost nothing on that model.

The efficiency claim is smaller and clearer. With alpha 100 on the 8B model plus the concise prompt on GSM-8K, the paper says the average thinking length drops from about 500 tokens to under 350, a 30% saving. A worked example in the paper shows a seating problem where the original thinking runs past 2048 tokens and the nudged run answers 720 in 806 tokens.

Where the evidence is thin

These are my own judgments from reading the paper, not findings from running anything. The training set is small, 30 problems for the monitoring study and 80 Math-500 problems for the evaluation regressor, and every training sample is a thinking phase that finished. Long runs that never closed are not in the data, which are exactly the runs a progress bar would most need to handle.

The target is also relative position, not remaining work. Two problems with a 400-token and a 4000-token solution both end at 1.0. A bar at 50% tells you the model is halfway through what it will end up writing, not what it will take. And the paper’s own limitations section lists the rest: math only, no study of side effects on coherence or factual accuracy, a need for hidden-state access, and no look at breadth or uncertainty.

One more caution. Faster is not the same as better for hard problems. A model that is pushed to stop doubting itself will also stop checking itself. The paper reports an unchanged error rate, but its own printed tables show more wrong answers (answered minus correct) in some rows, for example 2 wrong for the base 32B model on Math500 at 512 tokens against 6 with TPV at alpha 100, as well as far more answered problems. I did not run it, so treat the flat-error claim as the authors’ reading, not something I confirmed.

Compared with ordinary progress indicators

Today’s progress cues for reasoning models are mostly outside the model: a spinner, an elapsed-time counter, a token count, or a short summary of the thoughts so far. The table compares them with TPV on what each one can honestly tell the user. It is my own analysis, and it describes kinds of indicator, not particular products.

The licence and what it costs to run

The README and the LICENSE file give CC BY-NC-SA 4.0, and the code headers add that commercial use needs a separate licence from the rights holder named in the README. GitHub’s own metadata calls the licence “Other”. In plain terms: study it, reproduce it and share changes under the same terms, but do not build a product on it without a deal.

The paper says each experiment used a single A100 GPU and took at most two days. requirements.txt is a list of conda and pip commands for Python 3.10, PyTorch 2.3.1 with CUDA 12.1, accelerate, bitsandbytes, datasets and a few more. It does not list transformers, which every script imports, so a clean install will need that added. The last commit is from 7 July 2025, and the repo had one contributor, 56 stars and one open issue when I looked.

Who it is for

It suits researchers and tinkerers who run open-weight reasoning models locally and want to study progress signals or test the “overclocking” claim on their own tasks. It also suits people building a local inference UI who want a starting point for a progress bar and are fine with a non-commercial licence.

It does not suit anyone who uses a hosted reasoning model through an API, since the method needs hidden states. It does not suit a team that wants a commercial feature without a licence. And it does not suit anyone expecting a polished download: there is no trained vector, no UI and no tests.

Hands-on: not done

I did not install this or run a single script, so there are no timings, plots or outputs in this review. If I were to test it, I would train a vector on the 8B model, plot the bar on prompts that are not math, and compare alpha 0, 5 and 100 on tasks where the answer can be checked. That is a plan, not a result, and I will say so plainly if a later edition reports one.