CHEATSHEET: DOs and DON'Ts of Optimisation
Reprinted from STM forum
Optimisation. It's hard, but that's because it's key.
Here are 14 tips and a bunch of resources to help you get from "I think this campaign has promise" to "I'm buying my next Lamborghini now".
Optimisation: Dos and Don'ts
1) DON'T just go with your gut (unless you're a stone-cold expert). Unless you're a Stackman- or MrGreen-level expert, statistical math is MASSIVELY more accurate than going on feel for what to cut when.
2) DO run tests to statistical significance. Cutting placements, landers, banners and campaigns too early is one of the surest ways to fail at AM. Doesn't matter if it LOOKS significant, do the math!
3) DON'T run tests beyond statistical significance. There's no point spending $100 on a $0.20 payout that doesn't convert. Once you know for certain that an ad won't work or a lander's not the best of the bunch, stop and test something else.
4) DO use Bayesian testing wherever you can. Here's our guide to Bayesian testing for landers. It's both cheaper and doesn't build up errors compared to conventional split-test calculators.
5) DO split-test offers against each other. 80% of AM success is finding the right offer / traffic source combination.
6) DO split-test the same offer across different networks. Yes, I know it sounds weird, but trust us - testing the same offer on several networks is usually one of the biggest optimisations you can do.
7) DON'T Cut On CTR. Usually. CTR isn't a very good predictor of how successful a landing page or ad will be. Here's Finch's guide to why. You should almost always cut on CVR.
8) DO eliminate ads on CTR if they can't possibly generate a profit. If the maximum likely CTR from your data would give you a CPC so high that you can't make a profit (here's how to calculate that), there's no point continuing to run the ad. No need to wait for CVR.
9) DO run a bot test. Cylons won't sign up for your life insurance product. A simple bot test can save you a lot of time waiting for placements to reach statistical significance.
10) DON'T freak out if you hit a bad day. Campaigns go up and down. A single day's bad ROI might just mean there was a national holiday or a good TV show on.
11) DO try to run 24/7 to get representative data across the day and week. Conversion rates vary a lot by time of day and day of week. Try to get as broad a sample as you can, and don't make decisions based on a single day's data.
12) DO analyze the data closely and look for possible profit pockets defined by device, OS, carrier ... So your campaign's not in profit overall - but that doesn't mean it doesn't have pockets of profit. Look for individual placements, OSes, carriers - anything that's making money.
13) DON'T avoid testing ideas because they're weird. If it's a super-strange idea, but there's no reason it definitely won't work, test it. Weird ideas often lead to great campaigns.
14) DON'T fall into a rut with your creatives. It's very easy to do the same thing over and over again and think you're testing something new. Vary your design, your fonts, your image sources, your image framing, your copy, your CTAs, everything. Every so often, review all your recent creatives to be sure you're not repeating the same mistakes.
15) DON'T EVER STOP TESTING. Ever. EVER. There's always more improvements to be made.