Drupal very slow in Vagrant environment

Question

I've begun migrating a lot of our development environments to Vagrant. So far, this has been great for almost everything, but our first Drupal migration is unusable. It's unbelievably slow. Our Wordpress, CakePHP and Node.js sites all perform very adequately or better, but not Drupal. This think is just awful.

The box is a Veewee-created Ubuntu 12.04 64bit machine. It's the same base box we use for all of our web-based projects so nothing unique there. In my sites directory, I have a canonical directory (sites/my-site/) with all of the site resources and a symlink to that canonical directory with the domain name (sites/dev.mysite.com -> /vagrant/www/sites/my-site) that is evidently required for some module that the team is using.

This is a mixed Windows/OSX dev team and it's slow across both platforms. The only semi-unconventional snippet from my Vagrantfile is this:

config.vm.forward_port 80, 8080

config.vm.share_folder( "v-root", "/vagrant", ".", :extra => 'dmode=777,fmode=777' )

# Allows symlinks to the host directory.
config.vm.customize ["setextradata", :id, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/v-root", "1"]

Vagrant::Config.run do |config|
  config.vm.provision :shell, :path => "provision.vm.sh"
end

My shell provisioner only does a couple of things:

• Installs drush
• Creates the aforementioned symlink to the canonical site directory
• Writes out an Nginx server block
• If necessary, creates a settings.php file.

Is there anything I can do to improve performance? Like, a lot?

UPDATE

I've narrowed this down to a point where it looks like the issue is the remote database. To compare apples to apples with no project baggage, I downloaded a fresh copy of Drupal 7.21 and performed a standard install from the Vagrant web server against 3 different databases:

• A new database created on the same Vagrant VM as the webserver (localhost)
• A new database created on the shared dev server used in the original question (dev)
• A new database created on an EC2 instance (tmp)

Once that was done, I logged in to the fresh Drupal install and loaded the homepage (localhost:8080) 5 times. I then connected to each database and loaded the same page, the same way. What I found was that the page loaded 4-6x slower when Drupal was connected to the remote database.

Remember, this is a fresh (standard) install. There is no project baggage.

Alva

Are you connecting to the DB via a host name, or ip-address? And is the DB running on an IPv4 or IPv6? Also serverfault.com/questions/495914/… – Kampmeier

Answer 1

I just was trying to solve this issue myself. I tried the suggestions here and at Rails Windows Vagrant very slow response time. No real luck, I shaved 200 ms off 1800 ms response time on a warm request with no real data rendered. This with Ruby on Rails, not Drupal. The problem is the same, though.

Switching the shared folder to Rsync gave me a response time of ~280ms on that same request.

Vagrantfile:

  config.vm.synced_folder '.', '/vagrant', type: 'rsync',
                                       rsync__exclude: '.git/'

Usage:

$ vagrant up
$ vagrant rsync-auto

The latter command will watch your working dir and sync changed automatically.

See https://www.vagrantup.com/docs/synced-folders/rsync.html and https://www.vagrantup.com/docs/cli/rsync-auto.html

Obviate

I also got really annoyed at the slow sync frequency using rsync. I often have to hit page refresh twice to check out a minor CSS adjustment, since the sync was only halfway through on the first. – Obviate

Answer 2

I am hit by similar problem, too. It seems that VirtualBox shared folder can be very slow for project tree with +1000 files.

Switching to NFS might be the solution.

Iceland

I tried that out of curiosity, but I didn't get any kind of massive performance boost. I also have some developers on Windows, so it's useless for them. The biggest boost I've found is making the database local. That's no good for actual development, but as a test case, that increased speed significantly. – Alva

While developing a solution for a Drupal development VM in Vagrant, I've run into similar issues. Allocating more RAM to the VM and to the local database certainly helped in my experience, though for the sake of those that follow I'll post my findings on switching to NFS for projects with many files. – Coverup

I was having serious speed issues. For me it was a combination of increasing my apc.shm_size and switching to an NFS file system. Together this shaved 5-6 seconds off page loads in Drupal. – Parenthesis

Just following up, switching to NFS made things a full order of magnitude faster. Undine (the project I maintain on d.o) now provides a stock configuration for it on boxen that support it. Note that this isn't much of a solution for Windows—though I've heard you can achieve it through some gymnastics, I haven't confirmed personally. – Coverup

This solution improved the site install speed tremendous. It took more than 10 minutes for our default profile to install when using Shared folders. Via NSF it decreased to one minute. – Umber

the problem with NFS is that you cannot set the owner/group for that shared folder, if needed. – Camorra

Answer 3

The issue is almost certainly either skip_name_resolve (being needed in my.cnf) or VirtualBox's poor handling of shared directories with large numbers of files. Both are easy to track down with strace -c, but you may find it easier just to correct them one at a time and see which one fixes your performance issues.

If you're still seeing slowness after both of these changes, let me know and we can debug it further.

Noel

Well, it's not skip_name_resolve. I added that a long time ago. Even if it's the large number of files (I assume it is), I wouldn't know how to fix that. – Alva

@RobWilkerson Easy way to test—pull the files directory local, instead of leaving it on the host computer. If you are currently at DrupalCon, I'm in the core room and am happy to assist. – Noel

I believe the keyword is "skip-name-resolve", as opposed to "skip_name_resolve" no? – Vietnamese

@Vietnamese MySQL will accept either in the config file, but requires dashes on the command line. Canonical form is to use underscores in my.cnf and dashes on the command line, but it's really one of those "whatever floats your boat" situations. – Noel

Answer 4

I got here via google for similar, so I'm replying in the hopes others find this useful.

If you're using the precise32 vagrant box as your starting point, it's worth noting that the box by default has only 360MB of RAM.

Up the ram (at least in Vagrant V2 with VirtualBox) like so

config.vm.provider :virtualbox do |vb|
    vb.customize ["modifyvm", :id, "--memory", "1024"]
end

This made Drupal much more responsive for me.

Mu

I was using a custom box with 512MB, but have applied this change to up the memory and will be trying it soon. Thanks. – Alva

Answer 5

It's just a PHP/MySQL app so there's not much special about Drupal besides how it has been customized. You may have done some of this, but here are some suggestions to isolate the issue.

• Check the Drupal dblog for errors.
• Check your nginx & php logs for errors.
• Consider how many active modules you are running (over 100? That would be a very heavy install)
• Install a fresh Drupal instance & compare. This may isolate the problem to your instance and not Drupal in general.

If you find that it is your instance of Drupal

• Install the devel module and enable memory reporting so you know how much memory is being used per page load, as well as to have a base line for improvement.
• Make sure you have APC or another PHP opcache installed, and make sure the hit rate is good. If you weren't running it before, note the memory usage difference reported by devel.
• run something like xhprof or disable suspicious modules till you find the major offenders.
• enable mysql slow & index log to find potential issues, then add indexes or take other action appropriately

If your other apps are running fine, I suspect there is a problem with a particular module, or you have a fat Drupal install in general that needs some optimizing or more memory.

Amye

Did you find out how to resolve this problem? My research seems to point to the shared folders being slow, however I haven't found a solution that speeds them up. – Glaser

@JeroenCoupé - I'm still working on it. It hasn't been a top priority. May know more by the end of the week. Can you provide any details on your "shared folders" slowness? – Alva

I found a link that seems to indicate it is due to slow network mappings groups.drupal.org/node/270828, however the solution provided did not help me. – Glaser

I've been working on this as time permits and have isolated what appears to be the primary source. I've updated my question with what I found. – Alva

Answer 6

I tried pretty much everything to get my slow Vagrant to speed up and finally stumbled on this in the Issues tracker of the project.

config.vm.provider "virtualbox" do |v|
    v.memory = 1024
    v.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
    v.customize ["modifyvm", :id, "--natdnsproxy1", "on"]
end

I had previously tried NFS to no avail; this happened to be the silver bullet.

Ado

Interesting. I found this too and haven't noticed much difference. The silver bullet for me was using a local database. – Alva

I got a 200ms reduction in response time out of 1800ms doing this. With Ruby on Rails – Obviate

Answer 7

Since Vagrant 1.5 you can use rsync as a mechanism to sync a folder to the guest machine. Because rsync copies the files directly onto the remote filesystem, performance is noticeably better than NFS and VM shared folders.

You can read more about it here: http://www.vagrantup.com/blog/feature-preview-vagrant-1-5-rsync.html.

Winger

Answer 8

Latency is a big issue with database connections in any server environment. Even just running encryption on the DB connections is going to be a substantial performance issue, though it's presumably needed under these conditions.

What's your ping time to the database? If you've got at least one round trip for each query you run, then that's going to add up. Plus a bit of time for encryption. Worse again. if you don't use persistent database connections.

I'd think about where you do your caching. Eg cache in memcached on the VM instead of in the DB.

Penang

Answer 9

I run into the same problem. These advises will be especially helpful for those who uses Windows host machine. You will not be able to get decent performance without NFS supoort (for windows it is a big issue), so:

1. Do not use synced folder at all.

   config.vm.synced_folder "../data", "/vagrant", disabled: true
   

2. Setub samba server in the guest VM + network drive on Windows host. There are a lot of articles how to do it, e.g.: https://www.liberiangeek.net/2014/07/ubuntu-tips-create-samba-file-server-ubuntu-14-04/

Samovar

Answer 10

If the NFS shares with Vagrant are still too slow for you, you can do the contrary:

• Instead of installing a NFS server on your host machine, you can install it on the VM guest: http://guillaumeduveau.com/en/drupal-lightning-fast-synced-folders-in-vagrant-virtualbox/

Ibanez

Answer 11

I started to get slow performance on a drupal site once I installed nodejs and gulp. I had to do this because the drupal bootstrap 4 barrio sass subtheme requires nodejs/gulp. Then I ran into issues with vagrant on Windows and npm install commands. All npm install commands fail because they create sym links and Windows OS does not recognize these links. I had to create a sym link to the sites node_modules folder over to my vagrant home directory. npm install comands work after doing this. But then I started noticed the very slow response on this website. My other site run fast.

Seedman