How to test creatives without paying for an answer you never get

$50 a day buys under two conversions per creative.

i went looking for what a creative test actually has to cost before it can tell you anything, and counted 73 people asking some version of that question across 55 separate threads. ten of them name a figure at all, and in almost every case it is the budget they already have, asked as a doubt — is five dollars a day enough, is fifty, how long can ten euros a day be left to run. nobody works it the other way round, from what the test needs back to what it costs. that is the calculation, and it is four minutes long.

the question arrives in three shapes and they are all the same question:

"also what is the right budget to start ?"

"how many ad sets/creatives per ad set in the abo testing phase, and what daily budget would you start at, $50/day?"

"with a budget of around $100 per day, i'm struggling to build a consistent testing and scaling system."

i have some sympathy for the non-answers, because the real answer is arithmetic and the arithmetic is unflattering. most people asking cannot afford the test they are describing, and nobody wants to be the one to say it.

so here is the whole calculation, in the order to run it. three numbers you already have, about four minutes, and at the end of it you will know whether your budget can buy an answer or whether you are about to spend a month generating random numbers. nothing is held back and there is no gate on any of it.

if you want to see the thing i'm building while you're here, it's the only ask in this piece: https://clear-cortex.com/early-access?src=x-test-floor-top

The number your test has to clear

start with the one piece of platform mechanics everything else hangs off: an ad set needs roughly fifty conversions a week to optimise properly. below that, delivery does not have enough signal to learn from and it says so, in the interface, by parking you in a state it calls limited.

that fifty is for the ad set. it is not for each creative inside it.

every creative you add is another thing the same budget has to be split across. an ad set getting sixty conversions a week with ten creatives in it is giving each creative six conversions a week, and six conversions is not a result. it is a rounding error with a thumbnail attached. you will look at it, you will see one creative on four and another on nine, and you will conclude the second one is better. it is not. you have measured noise and promoted it.

working backwards from that gives the floor:

under about ten conversions per creative per week, you are not running a test.

ten is not a statistical threshold and i am not going to dress it up as one. it is the point below which the gaps between your creatives are smaller than the week-to-week wobble of the same creative against itself. above it you can start believing the order they came in. below it you cannot, and no amount of staring will change that.

The arithmetic, on your own cost per result

three numbers, all of which you already have.

one: your cost per result. whatever it currently costs you to get the thing you are optimising for. if you do not have one yet because nothing has converted, use what the account was doing before, or the cheapest week you have ever had — and know that you are being optimistic.

two: how many creatives are in the test. count them honestly. four variations of one idea is four creatives as far as the budget is concerned, even though it is one idea as far as you are concerned.

three: ten. the per-creative floor from above.

multiply them, and you have the weekly spend the test needs:

creatives × 10 × your cost per result = the week

divide by seven for the daily number.

worked on a real figure from somebody describing their own account — cost per result of $46:

• 2 creatives — 20 conversions — $920 a week — $131 a day
• 3 creatives — 30 conversions — $1,380 a week — $197 a day
• 4 creatives — 40 conversions — $1,840 a week — $263 a day
• 6 creatives — 60 conversions — $2,760 a week — $394 a day

and on somebody else's account, at $64:

• 2 creatives — 20 conversions — $1,280 a week — $183 a day
• 3 creatives — 30 conversions — $1,920 a week — $274 a day
• 4 creatives — 40 conversions — $2,560 a week — $366 a day
• 6 creatives — 60 conversions — $3,840 a week — $549 a day

those are not big-brand numbers. those are two ordinary accounts, and the four-creative test that everybody describes as a sensible starting point needs somewhere between $263 and $366 a day to return an answer.

What fifty dollars a day actually buys

now run it the other way, because this is the direction that hurts.

$50 a day is $350 a week. at a $46 cost per result that is seven and a half conversions in the week, in total, across everything in the ad set. put four creatives in it and each one is holding fewer than two.

two.

the floor was ten. you are at under two, which is not "a small test" or "an early read". it is a week in which no creative could possibly have distinguished itself from any other, bought at full price. and at the end of it you will have four rows on a screen with different numbers in them, because four rows always have different numbers in them, and you will make a decision.

one person put the outcome in the plainest terms available: "wasted $480 + 2 weeks of cycle time." another, further into it: "now i've had 3 orders in two weeks and cpa is around $64." three orders in two weeks cannot rank anything. it can barely establish that the account works.

this is why the honest answer to "what budget should i start at" is not a number. it is a question back: how many things are you trying to learn about at once, because that is what sets the price.

Illustration only: constant CPA and equal allocation assumed. Check actual results per creative; a count alone does not establish a winner.

The four honest moves when you are under the floor

most people reading this will be under it. that is not a failure and it is not a reason to stop advertising — it is a reason to stop calling what you are doing a test. there are four moves and only four, and three of them are about arithmetic rather than effort.

one: test fewer things. this is the cheapest move and the one nobody makes, because putting six creatives live feels like progress and putting two live feels like timidity. but the maths is brutal and linear: at $46 a result, two creatives need $131 a day and six need $394. if your budget is fixed, the number of creatives is the only variable you control, and cutting it from six to two does not slow you down — it is the difference between getting an answer this month and not.

two: raise the budget to meet the test you want. the sums above tell you exactly what that costs. what makes this a real decision rather than an obvious one is that the money is spent whether the test resolves or not. going from $50 to $263 a day to read a four-way test is $1,491 extra in the week, and you should say that number out loud before you do it.

three: pool instead of splitting. two ad sets each on thirty conversions a week are two ad sets that both fail the fifty. one ad set on sixty clears it. splitting the same money across more places is the single most common way accounts end up with nothing to read, and consolidating costs nothing but the loss of granularity you were never able to read anyway.

four: accept that you cannot test yet, and spend the money on making fewer, better things. this is a legitimate position and almost nobody says it. below a certain budget the correct strategy is not a better testing framework, it is judgement — pick the idea you believe in, run it, and put the money you would have spent proving yourself wrong into making it good. that stops being right the moment you can afford the floor, and it is right until then.

the order matters. one is free, three is free, two costs money, four costs pride.

The whole thing on one account

somebody with a $100 a day budget, wanting to test four new creatives, cost per result $46.

• the floor: 4 × 10 × $46 = $1,840 a week, or $263 a day.
• what they have: $100 a day, $700 a week, about fifteen conversions in total.
• across four creatives: under four each. the floor is ten. they are at a third of it.

so the four-way test is not available at $100 a day. what is available:

• two creatives needs 20 conversions, $920 a week, $131 a day. still above $100, but close enough that a fortnight's read gets there.
• two creatives over two weeks is 20 conversions against roughly 30 bought. that clears it.

which gives the actual answer for this account: run two creatives, read them after two weeks, and pick the two on the largest difference between the ideas rather than the two you like best. two genuinely different selling ideas resolve at this budget. four variations of one idea never will.

and if they come back and say they cannot wait two weeks, the honest reply is that they can either have the answer or have it quickly, and at $100 a day they cannot have both.

then the question the whole exercise exists for: what happens when one of the two wins.

the floor does not disappear once you start scaling, it moves. the winner you are putting money behind is still one creative, and the campaign carrying it still has to clear fifty a week for delivery to keep learning on it. what changes is that you are no longer splitting a budget across four unknowns, so the same spend buys far more certainty. that is the real argument for testing two things instead of six — not that it is more rigorous, but that it gets you into a scaling campaign sooner, behind a creative you have an actual reason to believe in.

the mistake described most often is running the test campaign and the scaling campaign as one campaign. they have opposite jobs. one is buying information at a deliberately bad price; the other is buying results at the best price it can find. a single campaign asked to do both will quietly stop doing the first, and it will not tell you that it has.

The part i cannot prove

the fifty-conversions figure is platform mechanics and the ten-per-creative figure is arithmetic on top of it. neither is my measurement, and i am not going to present them as one. i do not run an account at this spend and if i pretended otherwise the first person to ask which one would end it.

the costs per result in here are other people's, taken from what they wrote about their own accounts. they are real and they are two accounts, which is enough to work an example and not enough to call a benchmark. put your own number in — it is the only one that matters, and the calculation is three multiplications.

and there is one thing i genuinely do not know, which i have now asked twice and nobody has answered: below what spend does isolating a test stop being worth doing at all? somewhere under the floor there is a point where the disciplined approach costs more than it returns and you should simply back your judgement. the fourth move above assumes that point exists. i cannot tell you where it is, and if you have found it on your own account i would rather have your number than be right.

if any of the above is wrong, say so — that is more use to me than agreement.

i'm building the version of this where the whole test plan sits on one canvas and the brief never has to be retyped: https://clear-cortex.com/early-access?src=x-test-floor-end

the campaign side of it is here: https://clear-cortex.com/use-cases/ai-ad-creative