Adding tenancy-awareness to Laravel queues and artisan commands

Multi-tenant applications are a great way of offering a customised experience for multiple users, from a single, shared codebase. There are a number of different ways through which a "tenant mode" can be accessed. Some of the more common ways are via a custom domain or subdomain, a URL parameter, or a value in the user session. When the application boots, it checks for this value, and then configures the application internally - loading the correct database, applying the correct assets, and so on. This all works fine when we've got these values coming in via requests to our application, but how do we handle tenancy in parts of the application where there is no domain, URL parameter or user session?

Queues

Laravel has a great queue implementation. These are processes which run continually in the background, handling either more intensive or less time-critical jobs. The challenge here is that queues are not invoked through a HTTP request, so don't have any of the tenancy indicators we mentioned above.

Can we have a queue per tenant? One option is to have a dedicated queue for each tenant. php artisan queue:listen --tenant=tenant_a

A challenge here is that it will lead to a lot of queue runners to manage on a production environment. It also needs a new queue runner to be started for each new tenant added, which may be a challenge to guarantee on production.

Another approach is to make queued jobs tenancy-aware. In our multi-tenant application, we need a way to make sure our application is tenant-aware. Let's say we have a TenancyProvider, which has been registered in the providers array in config/app.php:

'providers' => [
   ...
   App\Providers\TenancyProvider::class,
],

Within our boot function, we're going to attempt configuration for each of our environments.

public function boot()
{
    if ($this->app->runningInConsole()) {
        $this->configureConsole();
        $this->configureQueues();
    } else {
        // detectTenancy is where we're identifying the tenant based
        // on domain, URL param, session, etc
        $this->setEnvironmentForTenant($this->detectTenancy());
    }
}

Inside our TenancyProvider, we will likely have a lot of logic already to detect which tenancy we're running under via URL/user session, and set the environment accordingly. We're now going to be calling this logic from multiple contexts (HTTP request, queues, and console), so we can bundle it up in it's own function.

private function setEnvironmentForTenant($tenant)
{
    // Set database for tenant
    // Set cache for tenant
    ...
    // Stamp the tenant name on each queued job.
    Queue::createPayloadUsing(function () use ($tenant) {
        return ['tenant' => $tenant];
    });
}

A key addition here is the createPayloadUsing() function. After invoking this, whenever a queued job is created in the application, the current tenant will be added as an attribute to the queue job. When we are running in console mode, we can set up our queue listener to check for this parameter, and set the environment accordingly:

private function configureQueues()
{
    // When a job is processed, check for tenant name to
    // set correct environment.
    $this->app['events']->listen(\Illuminate\Queue\Events\JobProcessing::class, function ($event) {
        if (isset($event->job->payload()['tenant'])) {
            $this->setEnvironmentForTenant($event->job->payload()['tenant']);
        }
    });
}

Long-running process

A critical thing to note about queues is the difference in behaviour, depending on how they're invoked. There are a couple of common ways to invoke queue handlers:

• queue:listen: The application is restarted on every invocation. Any tenancy changes will be applied on boot. Code changes are taken into consideration on the next run.
• queue:work: This is a long-running job, where the application environment is booted, and new jobs loaded into it. Code changes aren't taken into consideration until the queue listener is restarted.

The queue:work approach is more efficient in terms of memory usage, as the application is only invoked once. However, if this is the way your queues are running, beware that swapping between tenancies may require a little extra work. Because of the way Laravel will bind services on application boot, in many cases it's not sufficient to alter a config file and then have the service work on the new value. The database is a common example of this. If a queue listener starts to work, loads tenancy A for the first job, then loads a job for tenancy B next, the database will still be on the connection for tenancy A.

For example, if you're using a connections.tenant as the main database connection, and updating the database name per tenant, then there are additional steps beyond just updating the config:

config(['database.connections.tenant.database' => 'tenant_b']);
DB::purge('tenant');
DB::reconnect('tenant');

The risk here is that you're processing a job for one tenant, but in a different tenant's environment. This can manifest as sending the wrong email template out for a receipt, for example. For instances which are resolved on first usage, it's possible as part of the tenancy boot to force Laravel to forget about the currently-resolved instance. This will mean that when we try to use something like a Mail facade, we force it to resolve under the hood, but now it will resolve to our current tenant configuration.

private function setEnvironmentForTenant($tenant)
{
    ...
    // Clear current mail bindings, so next usage will re-bind them
    Mail::clearResolvedInstances();
    App::forgetInstance('mail.manager');
}

This is also a concern if there are singletons being bound with tenant-specific settings, or singletons which latch on to the database connection at the time of their creation. Typically these will need to be re-bound, using something like the below:

$this->app->singleton('square1\ExcitingPackage\SettingsManager', function ($app) {
    return new \square1\ExcitingPackage\SettingsManager($app);
});

This re-binding can also impact testing, if you're relying heavily on Laravel's mocks, and built-in fakes. For example, your test may create a Notification::fake to emulate a particular behaviour for your test, but if the setEnvironmentForTenant call is triggered afterwards, and you're re-binding the Notification facade, it can mean that your fake gets lost. In this scenario, conditionally rebinding based on test mode can help to avoid this problem.

Artisan commands

When running an artisan command, we will often want to run a command in a tenant-specific environment. The simplest way to do this is to add a --tenant=X parameter to each artisan command. Rather than doing this on individual commands, we can enable this method globally.

Coming back to our TenancyProvider, we can set this up when we are configuring our console mode:

private function configureConsole()
{
    // Add --tenant option to any artisan command.
    $this->app['events']->listen(ArtisanStarting::class, function ($event) {
        $definition = $event->artisan->getDefinition();
        $definition->addOption(
            new InputOption('--tenant', null, InputOption::VALUE_OPTIONAL, 'The tenant subdomain the command should be run for. Use * or all for every tenant.')
        );
        $event->artisan->setDefinition($definition);
        $event->artisan->setDispatcher($this->getConsoleDispatcher());
    });
    ....
}

protected function getConsoleDispatcher() : \Symfony\Component\EventDispatcher\EventDispatcher
{
    // Hook into the app's console dispatcher event
    if (!$this->consoleDispatcher) {
        $this->consoleDispatcher = app(EventDispatcher::class);
    }

    return $this->consoleDispatcher;
}

The key bit here is the InputOption('--tenant'..) line. This is how we add an optional argument to artisan commands. Doing it in this way means that we are binding it to every artisan command in the application. Setting it as InputOption::VALUE_OPTIONAL means that it isn't compulsory - commands which don't need a tenant to run will not be broken by this change.

Now that we're adding a tenant identifier to each command, we need to close the loop, by having that command run under the tenant mentioned in the command argument. In the same way that we've automatically enabled all of our artisan commands to accept this parameter, we can also automatically apply it.

private function configureConsole()
{
    ...
    // Check for tenant option on artisan command, setting environment if it exists.
    $this->getConsoleDispatcher()->addListener(ConsoleEvents::COMMAND, function (ConsoleCommandEvent $event) {
        $tenant = $event->getInput()->getParameterOption('--tenant', null);

        if (!is_null($tenant)) {
            $this->setEnvironmentForTenant($tenant);
        }
    });
}

This block will add a listener to each console command, checking for the --tenant option, and, if found, will apply the environment we need for that tenant.