MongoDB multi-document transactions in Symfony 4 with Doctrine and MongoDB ODM bundle

In MongoDB, regardless of its version, an operation on a single document is atomic. Because you can use embedded documents and arrays to capture relationships between data in a single document structure instead of normalizing across multiple documents and collections, this single-document atomicity obviates the need for multi-document transactions for many practical use cases.

With the release of version 4.0, we now have multi-document ACID transactions in MongoDB.

In MongoDB 4.0 however (including current - at the time of writing this post - version 4.0.9) transactions work across a replica set only, and MongoDB 4.2 will extend support to transactions across a sharded deployment.

Which means that they are not supported in standalone server installations.

Transaction properties

• Multi-document transactions can be used across multiple operations, collections, databases, and documents.
• Multi-document transactions provide an “all-or-nothing” proposition.
• When a transaction commits, all data changes made in the transaction are saved.
• If any operation in the transaction fails, the transaction aborts and all data changes made in the transaction are discarded without ever becoming visible.
• Until a transaction commits, no write operations in the transaction are visible outside the transaction.

Transaction constraints

• You can execute CRUD operations only on existing collections.
• A multi-document transaction cannot include an insert operation that would result in the creation of a new collection.
• Transactions always are associated with a session.
• At any given time, you can have at most one open transaction for a session.
• To associate read and write operations with an open transaction, you need to pass the session to the operation.
• Transactions with write conflicts are aborted.

Examples

mongo shell

The following mongo shell example highlights the key components of using transactions:

// Start a session.
session = db.getMongo().startSession( { readPreference: { mode: "primary" } } );

employeesCollection = session.getDatabase("hr").employees;
eventsCollection = session.getDatabase("reporting").events;

// Start a transaction
session.startTransaction( { readConcern: { level: "snapshot" }, writeConcern: { w: "majority" } } );

// Operations inside the transaction
try {
   employeesCollection.updateOne( { employee: 3 }, { $set: { status: "Inactive" } } );
   eventsCollection.insertOne( { employee: 3, status: { new: "Inactive", old: "Active" } } );
} catch (error) {
   // Abort transaction on error
   session.abortTransaction();
   throw error;
}

// Commit the transaction using write concern set at transaction start
session.commitTransaction();

session.endSession();

Pure PHP

If you're using the PHP library that wraps the driver, after creating an instance of Client e.g. called $client, you can do the following:

$session = $client->startSession();
$session->startTransaction();
try {
    // Perform actions.
    $session->commitTransaction();
} catch(Exception $e) {
    $session->abortTransaction();
}

Symfony with Doctrine and DoctrineMongoDBBundle

This example is based on Symfony 4 with DoctrineMongoDBBundle (doctrine/mongodb-odm-bundle) installed, which integrates Doctrine2 MongoDB Object Document Mapper (ODM) library (doctrine/mongodb-odm) into Symfony.

composer require doctrine/mongodb-odm-bundle alcaeus/mongo-php-adapter

The problem with this ODM is that it does not support multi-document transaction out of the box:

As per the documentation, MongoDB write operations are "atomic on the level of a single document".

Even when updating multiple documents within a single write operation, though the modification of each document is atomic, the operation as a whole is not and other operations may interleave.

As stated in the FAQ, "MongoDB does not support multi-document transactions" and neither does Doctrine MongoDB ODM.

But even with transaction support missing from Doctrine MongoDB ODM library, we can still use MongoDB multi-document transactions with few little tricks.

Step-by-step guide

Because Doctrine MongoDB ODM library does not support transactions, will will use native PHP MongoDB driver instead. First though we need to dig it out of the depths of Doctrine.

Let's start with creating new TransactionalDocumentManager which will extend Doctrine ODM's native DocumentManager:

// src/Manager/MongoDB/TransactionalDocumentManager.php
namespace App\Manager\MongoDB;

use Doctrine\ODM\MongoDB\DocumentManager;

/**
 * Class TransactionalDocumentManager
 *
 * Extends Doctrine2 MongoDB Object Document Mapper (ODM) library
 * to add support for sessions and multi-document transactions.
 *
 * @link https://www.doctrine-project.org/projects/mongodb-odm.html
 * @link https://docs.mongodb.com/master/core/transactions/
 */
class TransactionalDocumentManager extends DocumentManager
{

}

What we will need to be able to use sessions and transactions is native PHP's \MongoDB\Client, which we can easily dig out of Doctrine:

/** @var \Doctrine\MongoDB\Connection $mongoConnection */
$mongoConnection = $this->getConnection();

/** @var \MongoClient $mongoClient */
$mongoClient = $mongoConnection->getMongoClient();

/** @var \MongoDB\Client $phpClient */
$phpClient = $mongoClient->getClient();

Converting it into constructor for our TransactionalDocumentManager class:

// src/Manager/MongoDB/TransactionalDocumentManager.php
class TransactionalDocumentManager extends DocumentManager
{
    /**
     * @var \MongoDB\Client
     */
    protected $phpClient;

    /**
     * {@inheritDoc}
     */
    public function __construct(
        Connection $conn = null,
        Configuration $config = null,
        EventManager $eventManager = null
    ) {
        parent::__construct($conn, $config, $eventManager);

        $this->phpClient = $this->getConnection()
            ->getMongoClient()
            ->getClient();
    }
}

To have all services injected into our TransactionalDocumentManager we need to define them in the services.yaml:

// config/services.yaml
services:
    App\Manager\MongoDB\TransactionalDocumentManager:
        public: true
        arguments:
            - "@doctrine_mongodb.odm.default_connection"
            - "@doctrine_mongodb.odm.default_configuration"
            - "@doctrine_mongodb.odm.default_connection.event_manager"

Now, having access to \MongoDB\Client we can start a \MongoDB\Driver\Session:

use MongoDB\Driver\ReadPreference;

/** @var \MongoDB\Driver\Session $session */
$session = $this->phpClient->startSession([
    'readPreference' => new ReadPreference(ReadPreference::RP_PRIMARY),
]);

And having session we can start a new transaction:

use MongoDB\Driver\ReadConcern;
use MongoDB\Driver\WriteConcern;

$session->startTransaction([
    'readConcern' => new ReadConcern('snapshot'),
    'writeConcern' => new WriteConcern(WriteConcern::MAJORITY),
]);

Adding all that (and few extra helper methods) to our TransactionalDocumentManager class we get its final version:

TransactionalDocumentManager

// src/Manager/MongoDB/TransactionalDocumentManager.php
namespace App\Manager\MongoDB;

use Doctrine\Common\EventManager;
use Doctrine\MongoDB\Connection;
use Doctrine\ODM\MongoDB\Configuration;
use Doctrine\ODM\MongoDB\DocumentManager;
use MongoDB\Driver\Exception\RuntimeException;
use MongoDB\Driver\ReadConcern;
use MongoDB\Driver\ReadPreference;
use MongoDB\Driver\Session;
use MongoDB\Driver\WriteConcern;

/**
 * Class TransactionalDocumentManager
 *
 * Extends Doctrine2 MongoDB Object Document Mapper (ODM) library
 * to add support for sessions and multi-document transactions.
 *
 * @link https://www.doctrine-project.org/projects/mongodb-odm.html
 * @link https://docs.mongodb.com/master/core/transactions/
 */
class TransactionalDocumentManager extends DocumentManager
{
    /**
     * @var \MongoDB\Client
     */
    protected $phpClient;

    /**
     * @var \MongoDB\Driver\Session
     */
    private $session;

    /**
     * {@inheritDoc}
     */
    public function __construct(
        Connection $conn = null,
        Configuration $config = null,
        EventManager $eventManager = null
    ) {
        parent::__construct($conn, $config, $eventManager);

        $this->phpClient = $this->getConnection()
            ->getMongoClient()
            ->getClient();
    }

    /**
     * Start a transaction.
     *
     * @link http://php.net/manual/en/mongodb-driver-session.starttransaction.php
     * @link https://docs.mongodb.com/manual/reference/method/Session.startTransaction/
     * @link https://docs.mongodb.com/master/core/transactions/
     * @link https://docs.mongodb.com/manual/reference/read-concern-snapshot/
     * @link https://docs.mongodb.com/manual/reference/write-concern/#writeconcern._dq_majority_dq_
     *
     * @param array $options
     *
     * @return void
     */
    public function startTransaction($options = [])
    {
        $this->startSession();

        $options = [
            'readConcern' => new ReadConcern('snapshot'),
            'writeConcern' => new WriteConcern(WriteConcern::MAJORITY),
        ] + $options;

        $this->session->startTransaction($options);
    }

    /**
     * Commit current transaction.
     *
     * @link http://php.net/manual/en/mongodb-driver-session.committransaction.php
     * @link https://docs.mongodb.com/manual/reference/method/Session.commitTransaction/
     * @link https://docs.mongodb.com/manual/reference/command/commitTransaction/
     * @link https://docs.mongodb.com/master/core/transactions/
     *
     * @return void
     */
    public function commitTransaction()
    {
        $this->session->commitTransaction();
    }

    /**
     * Abort current transaction.
     *
     * @link http://php.net/manual/en/mongodb-driver-session.aborttransaction.php
     * @link https://docs.mongodb.com/manual/reference/method/Session.abortTransaction/
     * @link https://docs.mongodb.com/manual/reference/command/abortTransaction/
     * @link https://docs.mongodb.com/master/core/transactions/
     *
     * @return void
     */
    public function abortTransaction()
    {
        $this->session->abortTransaction();
    }

    /**
     * Start a new client session.
     *
     * @link http://php.net/manual/en/mongodb-driver-manager.startsession.php
     * @link https://docs.mongodb.com/manual/reference/server-sessions/
     * @link https://docs.mongodb.com/manual/reference/method/Session/
     * @link https://docs.mongodb.com/manual/reference/read-preference/#primary
     *
     * @param array $options
     *
     * @return void
     */
    public function startSession($options = [])
    {
        if (!$this->session instanceof Session) {
            $options = [
                'readPreference' => new ReadPreference(ReadPreference::RP_PRIMARY),
            ] + $options;

            $this->session = $this->phpClient->startSession($options);
        } else {
            throw new RuntimeException('Session already started.');
        }
    }

    /**
     * Get current session.
     *
     * @link https://docs.mongodb.com/manual/reference/server-sessions/
     * @link https://docs.mongodb.com/manual/reference/method/Session/
     *
     * @return \MongoDB\Driver\Session
     */
    public function getSession()
    {
        return $this->session;
    }

    /**
     * End current session.
     *
     * @link http://php.net/manual/en/mongodb-driver-session.endsession.php
     * @link https://docs.mongodb.com/manual/reference/server-sessions/
     * @link https://docs.mongodb.com/manual/reference/method/Session/
     *
     * @return void
     */
    public function endSession()
    {
        if ($this->session instanceof Session) {
            $this->session->endSession();
        } else {
            throw new RuntimeException('Session not found.');
        }
    }
}

Now we just need to use it.

ProductManager

Let's say in our fictional ProductManager we have a method publishing all unpublished products. Before introducing our new TransactionalDocumentManager it might have looked something like this:

// src/Manager/ProductManager.php
namespace App\Manager;

use App\Document\Product;
use Doctrine\ODM\MongoDB\DocumentManager;

/**
 * Class ProductManager
 *
 * @package App\Manager
 */
class ProductManager
{
    /**
     * @var \Doctrine\ODM\MongoDB\DocumentManager
     */
    protected $documentManager;

    /**
     * @var \Doctrine\Common\Persistence\ObjectRepository
     */
    protected $productRepository;

    /**
     * ProductManager constructor.
     *
     * @param \Doctrine\ODM\MongoDB\DocumentManager $documentManager
     */
    public function __construct(DocumentManager $documentManager)
    {
        $this->documentManager = $documentManager;
        $this->productRepository = $documentManager->getRepository(Product::class);
    }

    /**
     * Publish all unpublished products.
     */
    public function publishProducts()
    {
        /** @var \App\Document\Product[] $products */
        $products = $this->productRepository->findBy(['published' => false]);

        foreach ($products as $product) {
            $product->setPublished(true);
        }

        $this->documentManager->flush();
    }
}

In this scenario each update is atomic, meaning that if we have 5 unpublished products (so 5 documents to update), and the process fails on the 3rd product for whatever reason, we are left with 2 updated products and 3 that are not updated. Not good at all.

Our new TransactionalDocumentManager to the rescue!

After switching from native DocumentManager to our new TransactionalDocumentManager we can add all new shiny transaction-related code, which will make our ProductManager look something like this:

// src/Manager/ProductManager.php
namespace App\Manager;

use App\Document\Product;
use App\Manager\MongoDB\TransactionalDocumentManager;

/**
 * Class ProductManager
 *
 * @package App\Manager
 */
class ProductManager
{
    /**
     * @var \App\Manager\MongoDB\TransactionalDocumentManager
     */
    protected $documentManager;

    /**
     * @var \Doctrine\Common\Persistence\ObjectRepository
     */
    protected $productRepository;

    /**
     * ProductManager constructor.
     *
     * @param \App\Manager\MongoDB\TransactionalDocumentManager $documentManager
     */
    public function __construct(TransactionalDocumentManager $documentManager)
    {
        $this->documentManager = $documentManager;
        $this->productRepository = $documentManager->getRepository(Product::class);
    }

    /**
     * Publish all unpublished products.
     */
    public function publishProducts()
    {
        /** @var \App\Document\Product[] $products */
        $products = $this->productRepository->findBy(['published' => false]);

        $this->documentManager->startTransaction();

        foreach ($products as $product) {
            $product->setPublished(true);
        }

        $this->documentManager->flush(null, [
            'session' => $this->documentManager->getSession(),
        ]);

        $this->documentManager->commitTransaction();
    }
}

In this case if the update fails at any point during the process, the transaction will be automatically aborted and all changes made in the transaction discarded without ever becoming visible.

We might also wrap that publishProducts() code in a try-catch block and call $this->documentManager->abortTransaction(); in its catch section, but if a transaction is not committed, it will be automatically aborted (rolled back), so there is no real need for that here.

Sources:

• MongoDB documentation for Transactions
• MongoDB Multi-Document ACID Transactions are GA
• Mastering MongoDB — Introducing multi-document transactions in v4.0