I built a sidecar that takes the user’s prompt and does one forward pass, predicting whether the model’s parametric knowledge is sufficient to answer the question correctly (no answer tokens are generated).
So far I’m getting good results, but I’m wondering if there is a better way to do this. Thoughts?
