Implementing Mail/Message Queue Systems in Laravel

Sending bulk emails efficiently is a common challenge in web applications. Whether you’re sending newsletters, notifications, or marketing campaigns, processing large volumes of emails can overwhelm your server and create poor user experiences. Laravel’s queue system provides an elegant solution for batch email processing that keeps your application responsive while reliably delivering messages.

Why Use Queues for Email Processing?

Email processing can be time-consuming, especially when dealing with:

• External SMTP server delays
• Large recipient lists
• Email template rendering
• Potential network timeouts

Without queues, users would wait for all emails to send before seeing a response. Queues solve this by:

• Improving user experience – Immediate response to users
• Preventing timeouts – Long-running processes don’t block requests
• Enabling scalability – Process emails across multiple workers
• Providing reliability – Failed jobs can be retried automatically

Setting Up Laravel Queues

First, configure your queue driver in .env:

QUEUE_CONNECTION=database

Create the jobs table:

php artisan queue:table
php artisan migrate

Creating Email Jobs

Generate a job class for processing individual emails:

php artisan make:job SendEmailJob

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Mail;

class SendEmailJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $tries = 3;
    public $timeout = 60;

    public function __construct(
        public string $email,
        public string $subject,
        public string $message,
        public array $data = []
    ) {}

    public function handle()
    {
        Mail::raw($this->message, function ($mail) {
            $mail->to($this->email)
                 ->subject($this->subject);
        });
    }
}

Implementing Batch Processing

For sending emails to multiple recipients, create a batch job:

<?php

namespace App\Jobs;

use Illuminate\Bus\Batchable;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Bus;

class ProcessEmailBatch implements ShouldQueue
{
    use Batchable, Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function __construct(
        public array $emails,
        public string $subject,
        public string $message
    ) {}

    public function handle()
    {
        $jobs = collect($this->emails)->map(function ($email) {
            return new SendEmailJob($email, $this->subject, $this->message);
        });

        Bus::batch($jobs)
            ->name('Email Campaign')
            ->dispatch();
    }
}

Dispatching Email Jobs

In your controller, dispatch the batch job:

<?php

namespace App\Http\Controllers;

use App\Jobs\ProcessEmailBatch;
use Illuminate\Http\Request;

class EmailController extends Controller
{
    public function sendCampaign(Request $request)
    {
        $emails = ['[email protected]', '[email protected]', '[email protected]'];
        
        ProcessEmailBatch::dispatch(
            $emails,
            $request->subject,
            $request->message
        );

        return response()->json(['message' => 'Email campaign queued successfully']);
    }
}

Advanced Batch Processing with Progress Tracking

Laravel’s batch feature provides built-in progress tracking:

public function sendCampaignWithTracking(Request $request)
{
    $emails = User::pluck('email')->toArray();
    
    $jobs = collect($emails)->map(function ($email) use ($request) {
        return new SendEmailJob($email, $request->subject, $request->message);
    });

    $batch = Bus::batch($jobs)
        ->name('Newsletter Campaign')
        ->then(function () {
            Log::info('Email campaign completed');
        })
        ->catch(function () {
            Log::error('Email campaign failed');
        })
        ->finally(function () {
            Log::info('Email campaign finished');
        })
        ->dispatch();

    return response()->json([
        'message' => 'Campaign started',
        'batch_id' => $batch->id
    ]);
}

Monitoring Batch Progress

Create an endpoint to check batch progress:

public function getBatchStatus($batchId)
{
    $batch = Bus::findBatch($batchId);
    
    if (!$batch) {
        return response()->json(['error' => 'Batch not found'], 404);
    }

    return response()->json([
        'total_jobs' => $batch->totalJobs,
        'processed_jobs' => $batch->processedJobs(),
        'pending_jobs' => $batch->pendingJobs,
        'failed_jobs' => $batch->failedJobs,
        'progress' => $batch->progress(),
        'finished' => $batch->finished(),
        'cancelled' => $batch->cancelled()
    ]);
}

Running Queue Workers

Start processing queued jobs:

php artisan queue:work

php artisan queue:work --queue=default --tries=3

For production environments, use Supervisor to manage queue workers:

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /path/to/your/app/artisan queue:work --sleep=3 --tries=3
autostart=true
autorestart=true
numprocs=3
user=www-data

Rate Limiting and Throttling

Prevent overwhelming email services with rate limiting:

class SendEmailJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function handle()
    {
        Redis::throttle('emails')->allow(60)->every(60)->then(function () {
            Mail::raw($this->message, function ($mail) {
                $mail->to($this->email)->subject($this->subject);
            });
        }, function () {
            $this->release(10);
        });
    }
}

Best Practices

1. Chunk Large Lists: Break massive email lists into smaller batches

User::chunk(100, function ($users) {
    ProcessEmailBatch::dispatch($users->pluck('email')->toArray(), $subject, $message);
});

2. Use Delays for Spacing: Add delays between batches to prevent overwhelming

ProcessEmailBatch::dispatch($emails, $subject, $message)->delay(now()->addMinutes(5));

3. Monitor Failed Jobs: Set up alerts for failed job monitoring

4. Use Proper Queue Names: Separate email queues from other background tasks

5. Implement Circuit Breakers: Stop processing if too many failures occur

Conclusion

Laravel’s queue system transforms email processing from a blocking operation into an efficient, scalable background task. By implementing batch processing with proper monitoring and error handling, you can reliably send thousands of emails while maintaining excellent application performance.

The combination of jobs, batches, and queue workers provides a robust foundation for any email-intensive application, ensuring your users never wait for bulk operations to complete while maintaining the reliability your business demands.