All of my URLs on my Magento installation require index.php in them, like:
http://example.com/index.php/admin/
http://example.com/index.php/customer/account/login/
The problem is that the system by default links to URLs like
http://example.com/admin/
http://example.com/customer/account/login/
Which look prettier anyway. I assume this is a rewrite issue in .htaccess, but as tinkering with that in the past has given me 500s, I'd like to ask you guys first.
Changing the SEO settings, flushing the configuration cache, and reindexing URLs did not work as suggested here.