The remote config trap
For most studios, Remote Config is a safety net. It’s a way to fix a broken level or turn off a buggy feature without a store update. But if that’s all you’re using it for, you’re sitting on a gold mine and refusing to dig.
The problem with traditional Remote Config is the “Blind Update”. You change a value—say, the cost of a ‘Starter Pack’—and then you wait. You check your dashboard 3 days later. Revenue is up! Success? Maybe not. What if your D7 retention just dropped by 5% because the game became too ‘Pay-to-Win’?
In a modern game, every metric is interconnected. You can’t optimize one without affecting the others.
The problem: statistical slowness
Traditional tools like Firebase or Google Analytics require thousands of users to reach “statistical significance.” For a mid-sized studio, waiting for an A/B test to conclude can take weeks. In that time, the market has moved, and your players have moved on.
[!IMPORTANT] Traditional A/B testing is a ‘One-Size-Fits-All’ solution. It finds a winner for the average player, but there is no such thing as an average player.
The solution: ML-driven optimization
Instead of manual toggles, we use Multi-Armed Bandits (MAB). Here is how it changes the game:
- Efficiency on Small Samples: While Firebase needs 1,000+ users, XP Lab starts optimizing with as few as 50 active users.
- Complexity Handling: Humans can’t manually test 100 variables at once. Our ML can. It identifies the “Top 10” critical variables out of 1000+ and finds the “sweet spot” within hours, not days.
- Safety Boundaries: You don’t hand over control to the AI. Designers set the Safe Ranges (e.g., “Difficulty can be between 0.8 and 1.2”). The ML then finds the optimal value within those bounds.
Why you should care
By moving from static configs to dynamic optimization, we’ve seen real-world IAP revenue increases of 13% to 16%.
Stop treating Remote Config as a remote control. Start treating it as your game’s brain.