The same thing happened to me, but in a different way

When I started to develop the Jo Engine, I used a lot of SGL functions and many people says that it's not an homebrew, just an SGL wrapper.
I don't agree with that, because if you used the SGL, you are in the same conditions as the developers in 1995.
Do you think the developers had time to rewrite everything ?
Of course not. It is already hard enough to develop a game.
In computer science, this has a name: Reinventing the wheel.

Furthermore, my primary goal is not to replace the SGL, but help people to create games on the Sega Saturn.
Otherwise, what is the purpose of replacing SGL ?
On the other hand, if you are interested in programming closer to the machine, there is an excellent library : libyaul
Today, I have enough backwards to replace the SGL not because it's not homebrew, but because I want a lightweight engine with only useful features.
That's the reason why I made an option on the Makefile (JO_COMPILE_USING_SGL) to let developers choose whether or not to use SGL features
