If anyone's wondering, I used to be a physicist and gravity was essentially my area of study, OP is right assuming an ideal system, and some of the counter arguments I've seen here are bizarre.
If this wasn't true, then gravity would be a constant acceleration all the time and everything would take the same amount of time to fall towards everything else (assuming constant starting distance).
You can introduce all the technicalities you want about how negligible the difference is between a bowling ball and a feather, and while you'd be right (well actually still wrong, this is an idealised case after all, you can still do the calculation and prove it to be true) you'd be missing the more interesting fact that OP has decided to share with you.
If you do the maths correctly, you should get a=G(m+M)/r^2 for the acceleration between the two, if m is the mass of the bowling ball or feather, you can see why increasing it would result in a larger acceleration. From there it's just a little integration to get the flight time. For the argument where the effect of the bowling ball/feather is negligible, that's apparent by making the approximation m+M≈M, but it is an approximation.
I could probably go ahead and work out what the corrections are under GR but I don't want to and they'd be pretty damn tiny.