iOS engine A/B testing #8
Description
Description
The Chromium project relies on in-the-wild A/B testing to evaluate the real-world impact of any change we make. It's common, for example, that we find a feature that improves Chrome UI causes performance degradation in an unexpected way, or that a change not intended to be user-visible results in significantly increased crashes. Chrome launches nearly all major features on up to a random 1% sample of our users before enabling it for everyone, and we've found this to be essential in product quality. In addition, the Chromium open source project (on which Chrome is based) has a strict policy of "flag guarding" major and risky changes such that they can be disabled in the wild without pushing a new version of the application.
Apple's proposed design for alternative browser engines in the EU includes a rule that your app must:
Be a separate binary from any app that uses the system-provided web browser engine
Leaving all product issues aside and focusing just on the engineering perspective, this appears to effectively prohibit the use of A/B testing and flag guarding, which are Chrome's primary tools for ensuring product quality. Without such a mechanism, it is essentially guaranteed that a Chrome product using an alternative browser engine would be significantly lower quality and more frustrating to users than one using only the system-provided engine.
Outcome
iOS to support a mechanism for Chrome in the EU to switch transparently (at least on application startup) between an alternative browser engine and the system provided one.