How to make the most of Symfony on Heroku
We love Symfony. We also love Heroku. If you've been listening to my podcast, Sound of Symfony, you probably know that I have complained a lot about the difficulties of making Symfony and Heroku play nicely together.
We've recently launched a client's app on Heroku, and as opposed to our internal apps, where we don't mind if there's downtime or performance issues, this time we did some original research to make it as smooth as possible.
A lot of this is also applicable to dockerizing Symfony apps, but we'll probably make a separate special entry on this sometime in the future.
How Heroku works
I'm not gonna go into too much detail here, because Heroku has really quite excellent documentation on the matter, but there are some key concepts that you need to understand.
Whenever you push your code to Heroku, that triggers the compilation of a slug. A slug is (according to the Heroku docs) a "compressed and pre-packaged [copy] of your application optimized for distribution to the dyno manager". The piece of code that is run to compile your application resides in a buildpack. One Heroku application can have multiple buildpacks, which are run sequentially.
HTTP requests are served by web dynos. When a web dyno is started, it gets a copy of the slug, and starts the process defined in the Procfile. The filesystem on the dyno is ephemeral, which means that changes written to it during the lifecycle of the dyno will not be shared between dynos or stored anywhere.
By convention configuration, like usernames and passwords for services, hosts, etc. is stored in environment variables. When configuration is changed the dyno is restarted.
Any questions on Dynos? Go read this: Dynos and the Dyno Manager.
The problem
The problem here lies with the Symfony cache. We want the Symfony cache to be a part of the slug, so that everything is ready to go when a dyno boots up, but Symfony stores configuration in the cache, configuration we'd much rather get from environment variables.
If e.g. there is a problem with your Redis database you may find that Heroku automatically migrates your application to a new server. When this happens, they will change your configuration, and thus restart your dynos, but they will not rebuild your slug. As such, storing configuration in the slug and hoping for the best is not an option.
Heroku has an excellent article about running Symfony on Heroku where they suggest clearing the cache when the dyno boots. This works, but I do not like it. It slows down dyno booting, and also running one-off commands. We offer an alternate solution below. The article is however worth a read.
What we did
Now, I want to preface this by saying that there is also an article on deploying to Heroku in the Symfony cookbook. It gives some good advice, in fact I would agree with everything it says with the exception on how to handle multiple buildpacks (which is a little outdated, it's easier now, I should probably PR that). Follow the instructions in the article.
Things you should be doing that you can read about in those docs that I have no further comment on include
- Setting up your Procfile
- Logging to stdout/stderr
- Setting the SYMFONY_ENV environment variable
This blog post deals with some of the things where I either differ, have something to add, or have entirely new things to tell you.
Multiple buildpacks
If you're anything like us at Fervo, you're probably using gulp and bower to handle your frontend assets. That means you have a package.json file. This causes your application to be detected as a Node.js application. Simply run the following to add the PHP buildpack:
heroku buildpacks:add heroku/php
This should give an output similar to this:
$ heroku buildpacks:add heroku/php
=== nameless-brushlands-4859 Buildpack
1. heroku/nodejs
2. heroku/php
Your application will now first run the Node.js buildpack and then the PHP buildpack.
Making sure your Bower components are cached
In order to ensure fast builds, the Heroku Node.js buildpack caches Node modules and Bower components. To take advantage of this, make sure you install your Bower components in a bower_components directory, as a part of your npm install. Simply add the following to your package.json:
"scripts": {
"postinstall": "bower install"
}
Running gulp
The best way to run gulp is probably to add it as a postinstall script in your composer.json. That way you can still e.g. incorporate the output from FOSJsRoutingBundle. We also find that it's useful to run bower (possibly again) from there. This does nothing for Heroku, but it's nice to only have one command to run after a git update.
Add something like the following to your composer.json:
"scripts": {
"post-install-cmd": [
"node_modules/.bin/bower install",
"node_modules/.bin/gulp"
]
}
Other things to remember in composer.json
Heroku picks up your required PHP extensions from composer.json (and also the version of PHP to use), so remember to set requirements for those:
"require": {
"php": "^5.4|^7.0",
"ext-gd": "*",
"ext-exif": "*"
}
Making Symfony trust Heroku
The Heroku router is basically a big proxy server. If you want to know the real IP of a request, or whether or not the client used SSL, you need to trust the Heroku router. Sadly the Heroku router does not have a fixed IP address, or even a fixed subnet. As is detailed in Heroku's Symfony article, you need to trust any IP address. However, your dyno is only accessible through the Heroku router, and the Heroku router takes full responsibility to strip out evil forged headers, so as long as we know we're behind it, we should be fine.
Still, making your application trust everything willy-nilly seemed scary to us. How do we actually know we're running on Heroku? We decided to just set an config (environment) variable:
if (getenv('HEROKU') === 'yes') {
Request::setTrustedProxies(array($request->server->get('REMOTE_ADDR')));
}
Changing ini settings
PHP has a little known feature called per user ini-files. Heroku supports these. In order to set ini directives, just chuck a .user.ini file in your web directory.
Using environment variables in your dev environment
There's a lovely library called vlucas/phpdotenv. Create a .env.dist file which has example values for your environment variables, and a .env file which has your real values. Add .env to your .gitignore.
As for how to activate phpdotenv, I find it best to do in the AppKernel:
public function boot()
{
if (file_exists(__DIR__.'/../.env')) {
$this->dotenv = new \Dotenv\Dotenv(__DIR__.'/..');
$this->dotenv->load();
}
return parent::boot();
}
Using environment variables as configuration
As we said, Heroku uses environment variables for configuration. Your app should too. Now, Symfony has some support for environment variables. There's the fact that environment variables starting with SYMFONY__ are imported as kernel parameters. The Symfony Standard Edition ships with Incenteev's ParameterHandler, allowing you to map environment variables to entries in parameters.yml. Sadly both of those options means that the parameters will become compiled in to the slug, and thus set in stone.
There has been a long standing request to make Symfony play nicely with these dynamic parameters, but thus far, nothing has been merged. I made a PR for the solution we're using here, in symfony/symfony#18155, which I'd recommend you to read to grasp the problem, but that is not yet merged, and even if it were to be merged, it wouldn't be released until 3.1 at the earliest.
In the meantime we've also released FervoEnvironmentBundle which does exactly the same thing, just as a bundle.
Sadly there is a problem with our approach. It required bundle author opt-in. The bundle author must explicitly allow for environment variables in a particular configuration node.
In e.g. our own FervoPubnubBundle, this makes for using environment variables very simple:
fervo_pubnub:
publish_key: "@=env('PUBNUB_PUBLISH_KEY')"
subscribe_key: "@=env('PUBNUB_SUBSCRIBE_KEY')"
Using the FervoEnvironmentBundle is just as easy for the bundle author. Just have your extension use the ConfigurationResolverTrait, and you can do something like this:
$def = $container->getDefinition('fervo_pubnub');
$def->replaceArgument(0, $this->resolve($config['publish_key']));
$def->replaceArgument(1, $this->resolve($config['subscribe_key']));
That's easy, right? Well, for simple cases like this, and with bundle author opt-in, it really is this simple. Without bundle author opt-in, things become slightly more cludgy:
snc_redis:
clients:
default:
type: predis
alias: default
dsn: redis://localhost
services:
snc_redis.connection.default_parameters:
class: Predis\Connection\Parameters
public: false
factory: [Predis\Connection\Parameters, create]
arguments:
- "@=env('REDIS_URL')"
This is not as nice. It's not nice to look at, and it relies on internal knowledge (namely the service name). This is fragile. We also tried using PHPRedis. Even less nice. For Doctrine we even had to build a factory service. How do you know what exactly to do? You read dissect the extension and configuration files for each bundle, and hope that it's not too complicated. Some bundles at least allow you to pass in an already constructed service (like MonologBundle). That's pretty okay.
If your want the nice way to be more prevalent, you could voice your concern at an aforementioned PR, just sayin' :)