Why task-specific features are fused for the final object detection？

[HB-X]

The goal of this paper is to decouple the classification and localization tasks of object detection, but why not use the two aligned features directly but instead use the fused feature for final detection?