This is more correct than you may know. The Kaaba and even its apocryphal space-rock significantly pre-date Islam.
The location was dedicated to any number of gods through time, oftentimes several. In pre-Islamic times, Mecca grew from basically being a 24/7 trade-hub of the numerous semi-nomadic tribespeople of the region, and generally speaking the Kaaba (and numerous similar structures in similar multi-cultural locales) were proclaimed as something of a holy neutral ground where conflict was forbidden and all were to assume it was equally holy to everyone's gods, no matter what form or name they may have.
Mecca's Kaaba was supposedly one of the few that was built as a permanent structure (however small) and the local powers of Mecca increasingly marketed how awesome it was that they had one of the few stone trade-hub temples and what a good job they did keeping it secure. It was a pan-religious tourist trap centuries before Islam, the central location of prayer for all caravans and trade representatives who would arrive along the regular cycle of seasons. Adding new features like the alleged meteorite of the Black Stone over time served to just strengthen its mystic, and by consequence, economic lure through the region.
So once Islam showed up and began retroactively taking credit for all regional history, the lords of Mecca certainly didn't want to give up on their most reliable tourism engine, so they were more than happy to collude with the early Muslim leaders to rewrite the regular cycle of trade-tourism into a "tradition" of mandatory pilgrimage, not merely retaining the economic value, but strengthening it greatly.