It's not the number of paddles, it's the WSYNC. Most games will use the time MARIA isn't doing DMA for calculations. But the pseudo kernal uses all of this time to read the paddles. One option would be to make it a DLI, which would reduce resolution but free up the time.
