Don't be afraid of using Laravel's Jobs and Queues to scale your app

You might already know that Laravel's Job and Queue system is quite powerful and that it can speed a request up by offloading heavy tasks – like sending an email – to a queue worker. This post might be old news for you, but you might learn something new anyway.

I recently had the opportunity to upgrade a Laravel 5.0 app to version 5.7. I don't want to go into the details, what the app does … let's say it's a personalized "coupon code collection" app. Users can create an account and can see a list of coupon codes or receive new coupon codes via a personalized newsletter.

The app is fairly simple, but has quite a lot of traffic. One bottleneck was the tracking of "page views". When a user visits the details page of a coupon, the app would increment a 'views' counter in a MySQL database table. It's not the best tracking in the world, but it solves a business problem in a very simple way:

public function show(Coupon $coupon)
{
    $coupon->increment('views');

    return view('coupon.show')->withCoupon($coupon);
}

Why might this a bottleneck you might ask? Well, MySQL can handle only a certain amount of requests/statements per second. If there are too many requests, the requests pile up and will be worked through, when MySQL is less busy. This might take a couple of seconds depending on the load.

If the server is under high load, each request the app makes will feel slow, as the app has to wait that the "views" columns is incremented before showing the view. Not cool.

This can be solved with queues: I've created a simple 'IncrementViewsOnCoupon'-job which does exactly what's on the tin:

namespace App\Jobs;

use App\Coupon;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class IncrementViewsOnCoupon implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $coupon;

    public function __construct(Coupon $coupon)
    {
        $this->coupon = $coupon;
    }

    public function handle()
    {
        $this->coupon->increment('views');
    }
}

In the controller, the job is dispatched and put on a "views" queue.

public function show(Coupon $coupon)
{
    IncrementViewsOnCoupon::dispatch($coupon)->onQueue('views');

    return view('coupon.show')->withCoupon($coupon);
}

The app runs its queue workers with Laravel Horizon and the mentioned "views" queue has been setup by using only 1 process.

// config/horizon.php

'environments' => [
    'production' => [
        // ...
        'supervisor-2' => [
            'connection' => 'redis',
            'queue'      => ['views'],
            'balance'    => 'simple',
            'processes'  => 1,
            'tries'      => 3,
        ],
    ],
],

Now, when 100 requests are made to the app and the "views" column has to be incremented 100 times, the actual work of incrementing the column is offloaded to the queue. The request will feel super fast again.

After we applied this change to the app, we saw a huge decrease in server usage and we even could switch to a cheaper and smaller Digital Ocean droplet 🤑.

This method of improving the performance of the app is obviously not perfect for every app and won't scale indefinitely! If you're dealing with 1000s of requests per second you have to apply a different solution … but if your traffic is that big, you will (hopefully) have the resources to do so.

For our use case, this worked out very well and will keep the app fast and responsive for the next months or even years.