Command and Query Responsibility Segregation (CQRS) Pattern

Command and Query Responsibility Segregation (CQRS) Pattern

The Command and Query Responsibility Segregation (CQRS) Pattern is a solution to the problems that are inherent to the Create, Read, Update and Delete (CRUD) approach to data handling. We use Haskell to explore the problem scope and proposed solution described by the pattern.

Posted on August 28, 2016 by Ernesto Garbarino

Introduction

The Command and Query Responsibility Segregation (CQRS) Pattern is described by Homer et al. (2014, p. 42) and Fowler (2011), based on the work by Young (2010), as a solution to the problems1 that are inherent to the Create, Read, Update and Delete (CRUD)2 approach to data handling.

Unlike many patterns that answer to general business or engineering problems, the CQRS pattern is an alternative to the another pattern, CRUD, due its various claimed shortcomings. In more concrete terms, it is CRUD in the way it is commonly implemented in Object-Oriented Languages (OOP), involving the use of Data Transfer Objects (DTOs) (Homer et al., 2014, p. 42) and Object Relational Mapping (ORM) technology (Young, 2010, p. 6) such as Hibernate3 or Entity Framework4.

In this article, we will examine whether the problem identified by the pattern as well as the suggested solution is applicable to a functional programming language like Haskell. For simplicity, we ignore Event Sourcing (Fowler, 2005) as the default “backend” implementation for CQRS.

Problem

Functional Problems

This pattern is based on the shortcomings that emerge from what Young (2010, p. 2) calls a “Stereotypical Architecture” which typically implements the CRUD paradigm using Data Transfer Objects (DTOs) to represent both retrieved data as well as data to be changed.

A stereotypical architecture using the CRUD/DTO approach.

A stereotypical architecture using the CRUD/DTO approach.

The main issue, though, is conceptual (the use of a single model for both read and write operations) and not necessarily exclusive to imperative OOP languages. Let’s consider an example by representing an order consisting of line items using the record syntax in Haskell.

type OrderId     = Int
type Date        = String
type ItemId      = Int
type Name        = String
type Description = String
type Qty         = Int
type Price       = Float

data Order = Order {
  orderId        :: Maybe OrderId  
 ,lastUpdate     :: Maybe Date     
 ,customerName   :: Name           
 ,lineItems      :: [LineItem]    
} deriving (Show)

data LineItem = LineItem {
  parentOrderId  :: Maybe OrderId    
 ,itemId         :: Maybe ItemId     
 ,description    :: Description
 ,qty            :: Qty 
 ,price          :: Price
} deriving (Show)

The idea under the CRUD/DTO metaphor is that we can “consistently” manipulate the data using the same entity representation—“DTO up/down interaction” (Young, 2010, p. 4)—for both read and write actions so we have a series of functions that implement the CRUD verbs against the above defined data types:

orderCreate    :: Order      -> IO OrderId
orderRead      :: OrderId    -> IO Order
orderUpdate    :: Order      -> IO ()
orderDelete    :: OrderId    -> IO ()

lineItemCreate :: LineItem   -> IO ItemId
lineItemRead   :: ItemId     -> IO LineItem
lineItemUpdate :: LineItem   -> IO ()
lineItemDelete :: ItemId     -> IO ()

So far so good. Let’s look at the case of reading an existing order, say order#456:

-- Read Code
order <- orderRead 456
putStr "Order #"
putStrLn $ case orderId order of
             Just orderId -> show $ orderId
             Nothing      -> "Order Id missing"
putStr "Last Updated: "
putStrLn $ case lastUpdate order of
             Just date    -> date 
             Nothing      -> "Never updated"
putStrLn $ concat ["Customer: ",customerName order] 
putStr "Line Items: "
mapM_ print $ lineItems order

Why is the code so verbose? Because the orderId and lastUpdate fields are of type Maybe since they may be potentially empty. One can argue that the orderRead function should guarantee that an Order value is always “correctly” populated; an exception should be thrown if this not the case. Likewise, we already know the value of orderId (456).

The point is, though, that simply applying fromJust to an Order’s field, in the belief that it was generated by the right function, is an accident waiting to happen. Thus, bullet-proof code should contemplate the possibility that the value under consideration may not have originated from the correct function.

If we now look at the act of creating a new order, we will find new issues. The first has to do with consistency. If entities are treated atomically, then we need to perform our CRUD actions within some sort of transactional context. For instance:

-- Write Code
transactionWrapper $ do
  let order = Order {
                 orderId      = Nothing
                ,lastUpdate   = Nothing
                ,customerName = "Ernie" 
                ,lineItems    = [] 
              } 
  orderId <- orderCreate order
  let item1 = LineItem {
                 parentOrderId = Just orderId
                ,itemId        = Nothing
                ,description   = "Garlic Bread"
                ,qty           = 2
                ,price         = 3.5
              }
  itemId <- lineItemCreate item1 
  return ()

In the above example, we first create an Order value and obtain its orderId, and then use it to create a LineItem value separately. Note in this case, unlike that of read code we had contemplated before, we have to set the orderId and lastUpdate fields to Nothing. The entire procedure is wrapped by the transactionWrapper function that creates the appropriate context for the underlying data store.

This is just a way of implementing the creation of a complete order. Another possibility is to populate Order with all its children LineItem values so that both are persisted as part of a single create action:

-- Write Code (Alternative version) 
let item1 = LineItem {
              parentOrderId = Nothing 
             ,itemId        = Nothing
             ,description   = "Garlic Bread"
             ,qty           = 2
             ,price         = 3.5
            }
let order = Order {
              orderId      = Nothing
             ,customerName = "Ernie"
             ,lastUpdate   = Nothing
             ,lineItems    = [item1] 
            } 
orderCreate order

Note in the above code that the parentOrderId is now set to Nothing as opposed to the previous version.

In a nutshell, the Order and LineItem values must be handled in different ways depending on the context. This situation may lead to bugs and unintended behaviour. In the case of the Order type in particular, we can use the following table to see how different values are expected under different contexts:

Context orderId lastUpdate lineItems
Create Nothing Nothing []
Read Just x Just x [], ..., [x:n]
Update Just x Nothing [], ..., [x:n]
Delete - - -

This is, in a nutshell, the main issue surrounding the CRUD/DTO paradigm; however, this example may not be sufficient to discredit it yet. Let’s look at a more real-world use case in which we are asked to produce a report of customer names and their purchased items based on a given price. Let’s say that the manager wants to know who are the “stingy” customers who buy items priced at £3.50.

For example:

> stingyCustomersReport 3.50
Customer Name: Ernie - Item: Garlic Bread
Customer Name: Ernie - Item: Mozzarella Sticks
Customer Name: Giovanna - Item: Potato Wedges

By trying to abide to the CRUD/DTO metaphor, we could produce a list of orders and populate just the fields that are relevant to the report:

...
   [Order {
       orderId      = Nothing
      ,lastUpdate   = Nothing
      ,customerName = "Ernie"
      ,lineItems    = [LineItem {
                         parentOrderId = Nothing
                        ,itemId        = Nothing
                        ,description   = "Garlic Bread"
                        ,qty           = 0
                        ,price         = 0
                       }
                      ]
   }
...

In order to produce the above report, a function is required to “navigate” the result produced by readPrice.

stingyCustomersReport :: Price -> IO ()
stingyCustomersReport price = do
  orders <- readByPrice price
  mapM_ (\order -> do
    putStr "Customer Name: "
    putStr $ customerName order
    putStr " - Item: "
    case length (lineItems order) of 
        1 -> putStrLn $ description ((lineItems order)!!0)
        0 -> putStrLn "No items found"
        _ -> putStrLn "Multiple items found"
  ) orders 

Again, we come across verbose code for something which should be fairly trivial. The above code iterates through a list of orders and extracts the customer’s name and the item’s description from each LineItem value associated with an Order value.

Under this new context, the cardinality of lineItems is exactly 1, since exactly one line item per customer record is being used to associate a customer name with an item description. This results in a new context for the Order entity, so we add ReadByPrice to the context table we have recently introduced:

Context orderId lastUpdate lineItems
Create Nothing Nothing []
Read Just x Just x [], ..., [x:n]
Update Just x Nothing [], ..., [x:n]
Delete - - -
ReadByPrice Nothing Nothing [x]

Let’s suppose that the manager asks us to add the suffix “Stingy” to the first customer of the list returned by £3.50 and that we want to implement the requirement by leveraging our CRUD/DTO superpowers:

orders <- readByPrice 3.50
case orders of
  (o:_) -> orderUpdate o {
             customerName = (customerName o) ++ " Stingy"
           }
  _     -> putStrLn "No orders found"

Because different invariants hold for Order under different contexts, the above will fail. For instance, the orderUpdate function expects orderId to be populated. To make the above code work, we would need to choose, broadly speaking, one of the following two strategies:

  1. Populate the Order value with all the fields that make it compatible with the Create context. This can only work if LineItem values are updated independently from Order since the readByPrice function only returns one LineItem value per Order value.

  2. Modify readByPrice so that it returns at least orderId and then obtain a new complete Order value using the orderRead function.

For the second strategy, this is what the code would like like—please note the dangerous fromJust application on orderId and how easy it is to mistake o' (primed version) for o.

  orders <- readByPrice 3.50
  case orders of
    (o:_) -> do o' <- orderRead (fromJust $ orderId o)
                orderUpdate o' {
                  customerName = (customerName o) ++ " Stingy"
                }
    _     -> putStrLn "No orders found"

In conclusion, reusing DTOs to create views or projections, like in the case of readByPrice, further aggravates the problems that the CRUD/DTO paradigm intrinsically exhibits. Homer et al. (2014, p. 43) also note that the CRUD approach “can make managing security and permissions more cumbersome because each entity is subject to both read and write operations, which might inadvertently expose data in the wrong context.”

Non-Functional Problems

In the typical CRUD/DTO approach, read and write operations share the same pipe.

In the typical CRUD/DTO approach, read and write operations share the same pipe.

The CRUD/DTO strategy does not necessarily dictate an underlying implementation, but if the implementation consists of a monolithic data access layer (for example, Hibernate in Java or Persistent5 in Haskell) that points to a single physical data store (for example, MySQL), then the following issues may arise:

Solution

Functional

As the pattern’s name suggests, the solution is to segregate the write (command) from the read (query) operations.

An example of a controller interacting with Command and Query interfaces.

An example of a controller interacting with Command and Query interfaces.

Let’s start with the query functions. In the below example we return views as tuples which make it easier to create ad-hoc projections. For example, the orderQuerySummarised function returns a succinct summary in which the line items have been summed to produce a total price whereas orderQueryFull returns the full order with a list of line items.

type Total = Price
 
orderQuerySummarised :: OrderId 
                     -> IO (Name,Date,Total) 
orderQueryFull       :: OrderId 
                     -> IO (Name,Date,Total
                           ,[(Description,Qty,Price)]
                           )
orderQueryLineItems  :: OrderId 
                     -> IO (
                           [(ItemId,Description,Qty,Price)]
                           ) 
orderQueryItem       :: ItemId 
                     -> IO (Description,Qty,Price)
orderQueryByPrice    :: Price  
                     -> IO ([(OrderId,Name,Description)])

We can immediately see how this model is more efficient than the CRUD one since it eliminates the redundancies associated with DTOs. However, Homer et al. (2014, p. 43) note: “one disadvantage is that, unlike CRUD designs, CQRS code cannot automatically be generated by using scaffold mechanisms”. Indeed, a framework like Persistence, could not anticipate what specific queries the programmer had in mind.

We now turn our attention to the Command aspect. The idea here is that we have specialised commands to alter the persisted state at the right level of granularity and in a way that is independent from the read (query) model:

data OrderCommand = 
    OrderCreate     Name    [(Description,Qty,Price)]
  | OrderUpdateName OrderId Name
  | OrderUpdateFull OrderId Name [(Description,Qty,Price)]
  | OrderUpdateItem OrderId ItemId Description Qty Price
  | OrderDelete     OrderId
     deriving (Show)

For example, the OrderUpdateName command only changes the order’s customer name whereas OrderUpdateFull also replaces the order’s line items.

An interesting property of the command approach is that we can easily process a list of commands rather than creating a specific function for each command since they are disconnected from the query model.

orderCommand :: [OrderCommand] -> IO [OrderEvent]

Note: An ideal CQRS implementation would return the commands’ events in an asynchronous, non-blocking fashion. This model simply aims to illustrate the pattern’s functional implication. Note also that these events represent the commands’ outcomes and are not to be confused with events for the purpose of implementing the Event Sourcing pattern.

These are the possible events that may arise from sending the defined commands:

data OrderEvent = 
    OrderCreated          OrderId
  | OrderCreateFailed     Name
  | OrderUpdated          OrderId String
  | OrderUpdateFailed     OrderId 
  | OrderItemUpdated      OrderId ItemId
  | OrderItemUpdateFailed OrderId ItemId
  | OrderDeleted          OrderId
  | OrderDeleteFailed     OrderId 
     deriving (Show)

Now that we are equipped with our CQRS version of our order handling framework, let’s reimplement the stingy customers report requirement. However, since we had spent too much time coding, the manager has just come up a new requirement:

“By the way, if you find Herbert among the stingy customers, please delete his orders! He is my naughty cousin who has been ordering low priced starters just to annoy me.”

Fair enough. We first get a list of stingy customers by applying the orderQueryByPrice function which returns a clean list of tuples (OrderId,Name,Description) from orderQueryByPrice. Right after, we write a function called managerRequest which implements the requirement: if the name is “Herbert”, we issue an OrderDelete command, otherwise, we just append the word “Stingy” to the customer’s name:

stingyCustomerList <- orderQueryByPrice 3.50
let managerRequest (orderId,name,_) = 
     case name of
       "Herbert" -> OrderDelete     orderId 
       _         -> OrderUpdateName orderId 
                                    (name ++ " Stingy")

All we have to do now is simply apply the function managerRequest to each tuple in stingyCustomerList and pass the result (which is a list of commands) to the orderCommand function:

events <- orderCommand $ 
             map managerRequest stingyCustomerList  

That’s it. The consumer code is actually less verbose and more meaningful than the CRUD/DTO version we had recently exemplified. But what about errors? We can examine the events list and produce a report as follows:

let report = map 
      (\event -> case event of
         OrderDeleteFailed orderId -> "Could not delete: " 
                                      ++ show orderId
         OrderUpdateFailed orderId -> "Could not update: " 
                                      ++ show orderId
      ) events
case report of
   [] -> putStrLn "Success!"
   xs -> mapM_ putStrLn report  

Non-Functional

The CQRS pattern not only suggests that data should be handled using a different set of application-level functions but that it also should be segregated in terms of different read and write physical stores (Homer et al., 2014, pp. 44–45).

For example, there may be a single “write” node which replicates to four “read” nodes that are used by the query functions. In more complex scenarios, nodes may be deployed to host only specific subsets of read data.

An example showing a potential physical implementation of the Command and Query Interfaces.

An example showing a potential physical implementation of the Command and Query Interfaces.

Considerations

Based on the points mentioned by Homer et al. (2014, p. 44):

We also have to note that the tuple-based approach used in this article’s example comes with a significant drawback: a single change to the logical entity handled by both the queries and commands results in several changes both on the provider and consumer sides.

Discussion and Conclusion

The problem identified by the CQRS pattern, embodied by the traditional CRUD/DTO approach to data handling, seems to be also relevant to a functional programming like Haskell when representing entities using the record syntax.

The “command” solution can be effectively implemented in Haskell using sum types which allows to perform multiple state changes in a single invocation. Arguably, queries may also be implemented in a “command” fashion so that it is not necessary to perform queries one-by-one:

data OrderQuery = 
    OrderQuerySummarised OrderId
  | OrderQueryFull       OrderId
  | OrderQueryLineItems  OrderId
  | OrderQueryItem       ItemId
  | OrderQueryByPrice    Price
     deriving (Show)

data OrderQueryResult =
    OrderQuerySummarisedResult (Name,Date,Total)
  | OrderQueryFullResult       (Name,Date,Total
                               ,[(ItemId,Description,Qty,Price)]
                               )
  | OrderQueryLineItemsResult  (ItemId,Description,Qty,Price)
  | OrderQueryItemResult       (Description,Qty,Price)
  | OrderQueryByPriceResult    [(OrderId,Name,Description)]
     deriving (Show)

queryOrder :: [OrderQuery] -> IO [OrderQueryResult]

The author’s final conclusion is that the CQRS pattern does not “replace” CRUD/DTO, but offers, instead, an alternative when scalability and complex update and query scenarios render said traditional pattern inappropriate.

Implementation

This article uses the minimum amount of Haskell syntax required to illustrate the CQRS model. It does not propose a concrete implementation. For actual implementations, refer to the following:

Please also note that in this article we ignore Event Sourcing (ES) which is commonly used in conjunction with CQRS. The pattern is in fact many times referred to as CQRS/ES.

References

Fowler, M., 2011. CQRS [WWW Document]. URL http://martinfowler.com/bliki/CQRS.html

Fowler, M., 2005. Event Sourcing [WWW Document]. URL http://martinfowler.com/eaaDev/EventSourcing.html

Homer, A., Sharp, J., Brader, L., Narumoto, M., Swanson, T., 2014. Cloud Design Patterns: Prescriptive Architecture Guidance for Cloud Applications. Microsoft.

Young, G., 2010. CQRS Documents [WWW Document]. URL https://cqrs.files.wordpress.com/2010/11/cqrs_documents.pdf


  1. https://msdn.microsoft.com/en-us/library/ms978509.aspx

  2. https://en.wikipedia.org/wiki/Create,_read,_update_and_delete

  3. http://hibernate.org

  4. https://msdn.microsoft.com/en-gb/data/ef.aspx

  5. https://www.stackage.org/package/persistent