🚨 Your RL only improves 𝗽𝗮𝘀𝘀@𝟭, not 𝗽𝗮𝘀𝘀@𝗸? 🚨
That’s not a bug — it’s a 𝗳𝗲𝗮𝘁𝘂𝗿𝗲 𝗼𝗳 𝘁𝗵𝗲 𝗼𝗯𝗷𝗲𝗰𝘁𝗶𝘃𝗲 you’re optimizing.
You get what you optimize for. If you want better pass@k, you need to optimize for pass@k at training time.
🧵 How?
Of course we can do this: looks like the most obvious way to try out. But that’s one of many combinations we can play with, for example, logsumexp as the “softmax” to bridge the 2 objectives.
