TRUNGTQ

Think Big, Act Small, Fail Fast and Learn Rapidly

NAVIGATION - SEARCH

Polyglot Persistence Using DDD and Repository Unit of Work

Working with different data stores (SQL, NoSQL ..) in high performance enterprise application using DDD and Repository unit of work pattern

Introduction

Traditional business applications usually use a relational database to store data and use this database as an integration point. That entails designing sets of relational tables and accessing them with a single data access layer in code, as well as using an ORM to convert relational tables to an OOP structure .

Nowadays, that might not be optmized for each business use case scenario, considering some challenges encountered with relational database design. It's unnecessary here to list all the disadvantages of such design we can find a more articles about. But let's give a hint:

Agility: Mismatch between data structure in the database and the application object model has an adverse productivity.

Performance:Relational models were designed to take minimum disk space and to consume less resources, which has some side effects and performance limitations.

Availability and price:Increasing availability in relational model complicates the consistency and has its price.

Flexibility: Not all developers are familiar with relational models and schemas, in some situations, complex business models require a thorough knowledge of SQL language and relational models.

In a number of enterprise applications that capture a huge amount of data and numbers of concurrent requests, the relational model did not provide the requirements. So looking for other alternatives to solve a specific set of persistence problems leads many designers to select a non-relational systems referred to as NoSQL databases.

Non-relational systems or NoSQL databases can be categorized by a set of functional areas: Key/Value, document databases, column family databases, graph databases. Each category can fit one or more business use case scenarios and it's for a designer to choose the most appropriate system. Some NoSQL databases use filesystem for persistence while some of them are in memory.

Background

The advent of various NoSQL databases and the abundance of cheap storage spaces (disk and memory) will be a momentum for designers and developers to switch to non-relational models.

However, an obvious question will occupy the mind. How to design an application so that it will dialog with different data stores?

What Are We Going To Achieve At The End?

We are going to design and build a simple application using DDD and Repository Unit of Work pattern. This application will persist data in relational (SQL) and non relational (NoSQL) data stores.

As a real world example, we will use an enterprise product store, let's say an enterprise (customer) with many organizations, departments and employees managing a huge amount of products and each employee can search, list, add, update and remove products. Also, the application needs to store the users' traffic to help administrators to address possible performance issues. To meet the requirement, the application will be deployed on the cloud (Windows Azure) and on-premises. Finally, for demos and testing purposes, the application will store all the data in memory.

Selecting Databases

For our example, products store application we can recognize different types of data requirements. For each requirement, we will use a specific database type as follows:

  • Customers, organizations, departments and users necessitate a safe reliable database to keep the hierarchical structure safe. And those entities don't have very frequent changes. So, we will persist them in relational database, let's say SQL Server.
  • The application is supposed to hold a large number of products, and it has to filter, list and search products in acceptable performance.The information held in each product may be different. So, for those reasons, we will persist products in NoSQL document database. Here, we will use MongoDB.
  • The application will allow the users to take some notes and share them with other users. Notes are just text that can be listed, searched and filtered. For Note, text will be stored in Lucene index.
  • The application will store products images in cloud blob storage as key/ValueNoSQL database.
  • The application will log errors and store change history in cloud table storage as key/Value NoSQL database.

Designing and Creating a Solution

Before we start coding, let's give a brief high level design structure to our application.

Here is the layered diagram of our solution:

By the diagram here above, we can examine the principal layers used in the application:

  • Presentation: ASP.NET MVC5 application with HTML5 AngularJS
  • Distributed Services: MVC 5 Web API application exposes REST services
  • Application management: Manage the business rules
  • Domain Objects: Core Application business definitions, Interfaces, entities, aggregates
  • Data: Repositories and a specific unit of work implementations. Repository for each domain entity or aggregate and Unit of Work for each data store type.
  • Data stores: At the bottom of the figure above, we see the different data stores that we will use in application SQL server, MongoDB, Azure Storage, Lucene index and in-memory fake store

To examine the interactions between components, let's examine the component diagram below:

Presentation: In the presentation layer, we can see that ASP MVC 5 project will serve as SPA with HTML5 angularJS. That can also consume a direct Web API services in the distributed services using HTTP/HTTPS

DistributedServices: Web API exposing Rest Services over HTTP/HTTPS

Application Management: Class library used as .NET reference for Web API and MVC application.

Domain Objects: Class library used as .NET reference for application management and infrastructure data

Infrastructure data: Class library referencing Domain object and some libraries as connectors for data stores

  • Entity framework 6 for SQL Server
  • Mongocsharpdriver for mongoDB
  • Lucene.net for Lucene index
  • Windows Azure graph API for Azure storage

Now let's zoom on the Domain objects layer and see the entities definitions in this map diagram:

You can find this diagram in the attached solution.

Working with the Code

Domain Entities

After the general design description of our application example, let's delve into some lines of code. Mainly focus on the domain objects definitions and the infrastructure data layer. For the other layers, you can download the attached example.

Starting with domain objects definitions: Here, we will first define a base class for all entities witch contains common properties decorated with some libraries attributes. Here is the code of our base class.

/// <summary>
/// Base class for entities
/// </summary>
public abstract class EntityBase
{
    #region Members
      Guid _Id;
    #endregion
   #region Properties

     /// <summary>
    /// Get or set the persisted object identifier
    /// </summary>
    [Key]
    [Field(Key = true)]
    [BsonId]
    public virtual Guid Id
    {
        get
        {
            return _Id;
        }
        set
        {
            _Id = value;
        }
    }
    /// <summary>
    /// Get or set the Date of Creation
    /// </summary>
    [DataType(DataType.DateTime)]
    public DateTime CreationDate { get; set; }
    /// <summary>
    /// Get or set the Date of LastUpdate
    /// </summary>
    [DataType(DataType.DateTime)]
    public DateTime LastUpdateDate { get; set; }
    #endregion
 }
 

This class defines the Idproperty decorated by some attributes:

  • Key: Attribute for entity framework so that the property will be mapped to the SQL primary key
  • BsonI: Attribute for mongoDB so the Id property will be mapped as document identifier
  • Filed(Key=true): This for Lucene.net to be used as document identifier on the index

Now this is for entities. Let’s see how to define a contract to our repositories. Here is the definition:

/// <summary>
    /// Base interface for implement a "Repository Pattern", for
    /// </summary>
    /// <remarks>
    /// </remarks>
    /// <typeparam name="TEntity">Type of entity for this repository </typeparam>
    public interface IRepository<TEntity> 
        where TEntity : EntityBase
      {
        /// <summary>
        /// Get the unit of work in this repository
        /// </summary>
        IUnitOfWork UnitOfWork { get; }
       
         /// <summary>
        /// Add item into repository
        /// </summary>
        /// <param name="item">Item to add to repository</param>
        Task AddAsync(TEntity item);

        /// <summary>
        /// Delete item 
        /// </summary>
        /// <param name="item">Item to delete</param>
        Task RemoveAsync(TEntity item);

        /// <summary>
        /// Get element by entity key
        /// </summary>
        /// <param name="id">entity key values, the order the are same of order in mapping.</param>
        /// <returns></returns>
        Task<TEntity> GetElementByIdAsync(Guid id,
                                       CancellationToken cancellationToken = default(CancellationToken));

        /// <summary>
        /// Get all elements of type {TEntity} in repository
        /// </summary>
        /// <param name="pageIndex">Page index</param>
        /// <param name="pageCount">Number of elements in each page</param>
        /// <param name="orderBy">Order by expression for this query</param>
        /// <param name="ascending">Specify if order is ascending</param>
        /// <returns>List of selected elements</returns>
        Task<IEnumerable<TEntity>> GetPagedElementsAsync<T>(int pageIndex, int pageCount,
                             Expression<Func<TEntity, T>> orderBy, bool ascending,
                             CancellationToken cancellationToken = default(CancellationToken));
}
 

All entities and aggregates used in the application will define a repository class that implement the contract above as we can see the repository define its unit of work that will be injected at run time. So for each store, we will define its unit of work and inject it to the repository. Here is a contract for unit of work.

public interface IUnitOfWork
        : IDisposable
    {
        /// <summary>
        /// Commit all changes made in a container.
        /// </summary>
        ///<remarks>
        /// If the entity have fixed properties and any optimistic concurrency problem exists,  
        /// then an exception is thrown
        ///</remarks>
        void Commit();
   
......
         /// <summary>
        /// Commit all changes made in a container Async.
        /// </summary>
        ///<remarks>
        /// If the entity have fixed properties and any optimistic concurrency problem exists,  
        /// then an exception is thrown
        ///</remarks>
        Task CommitAsync( CancellationToken cancellationToken = default(CancellationToken));
        
        ......

         /// <summary>
        /// Commit all changes made in  a container Async.
        /// </summary>
        ///<remarks>
        /// If the entity have fixed properties and any optimistic concurrency problem exists,
        /// then 'client changes' are refreshed - Client wins
        ///</remarks>
        Task CommitAndRefreshChangesAsync(CancellationToken cancellationToken = default(CancellationToken));
 

Accessing Data Stores

For each data store, we will define its unit of work so we can then inject it in a specific repository. First, in domain object layer, we have defined a contract for unit of work so the unit of work here will implement it.

Starting with relational database here SQL server, maybe it's the most usual. Here, we will use Entity framework and create a unit of work that implements a base contract and inherits entity framework DBContext. Above is the code used in this example.

public class MainBCUnitOfWork : DbContext, IMainBCUnitOfWork 
    { 
        #region Fileds
        private IDbSet<Customer> _customers;
        private IDbSet<Address> _addresses;
        private IDbSet<Department> _departments;
        private IDbSet<Organization> _organizations;
   ....... 
        
         #region Properties
        public IDbSet<Customer> Customers
        {
            get
            {
                if (this._customers == null)
                    this._customers = (IDbSet<Customer>)this.Set<Customer>();
                return this._customers;
            }
        }
 
        public IDbSet<Department> Departments
        {
            get
            {
                if (this._departments == null)
                    this._departments = (IDbSet<Department>)this.Set<Department>();
                return this._departments;
            }
        }

         ..............
         #endregion
.........
       public virtual IQueryable<TEntity> CreateSet<TEntity>() where TEntity : class,new()
        {
            return (IDbSet<TEntity>)this.Set<TEntity>();
        }

        public virtual void Commit()
        {
            try
            {
                this.SaveChanges();
            }
            catch (DbEntityValidationException ex)
            {
                throw this.GetDBValidationExptions(ex);
            }
        }
         ...............

       public async Task CommitAsync(CancellationToken cancellationToken = default(CancellationToken))
        {
            cancellationToken.ThrowIfCancellationRequested();
            try
            {
                await this.SaveChangesAsync(cancellationToken);
            }
            catch (DbEntityValidationException ex)
            {
                throw this.GetDBValidationExptions(ex);
            }
        }
 ........................
 }
 

The full definition of the class can be found in the attached code.

For each entity stored in SQL server database, an IDBSetproperty is defined. The create set method will be used to retrieve data.

By the same way, we will define a unit of work for mongoDB using mongoDB C# driver that you can add as Nuget package. Here is the code used:

 public class MongoUnitOfWork : IMongoUnitOfWork
    {
        #region Fields
        string _dbHostName;
        string _dbName;
        MongoDatabase _database;
       ......

         #region properties
        public string DbName
        {
            get {
                if (string.IsNullOrEmpty(this._dbName))
                {
                    this._dbName = "polyGlotDemo";
                }
                return _dbName;
            }
            set { _dbName = value; }
        }

       public string DbHostName
        {
            get {
                if (string.IsNullOrEmpty(this._dbHostName))
                {
                    this._dbHostName = "127.0.0.1";
                }
                return _dbHostName;
            }
            set { _dbHostName = value; }
        }

        public MongoDatabase Database
        {
            get { return _database; }
            set { _database = value; }
        }

        public IDbSet<DepartmentAggregate> Departments { get; set; }
        #endregion
        #region Ctor
        public MongoUnitOfWork()
        {
            var pack = new ConventionPack();
            pack.Add(new CamelCaseElementNameConvention());
            ConventionRegistry.Register("MongoUnitOfWorkPack", pack, (t) => true);
            string connectionString = "mongodb://" + t
            MongoClientSettings settings = MongoClientSettings.FromUrl(new MongoUrl(connectionString));
            settings.WriteConcern.Journal = true;
            var mongoClient = new MongoClient(settings);
            var mongoServer = mongoClient.GetServer();
           if (!mongoServer.DatabaseExists(this.DbName))
            {
                throw new MongoException(string.Format
                (CultureInfo.CurrentCulture, Messages.DatabaseDoesNotExist, this.DbName));
            }
             this.Database = mongoServer.GetDatabase(this.DbName);
            var coll = this.Database.GetCollection<
            			DepartmentAggregate>("DepartmentAggregate");
            //coll.RemoveAll();
            foreach (var dep in data.DepartmentAggregates)
            {
                if (!coll.AsQueryable().Any(x => x.Id == dep.Id))
                {
                    coll.Insert<DepartmentAggregate>(dep);
                }
            }

           this.Departments = new MemorySet<DepartmentAggregate>();
        }
        #endregion
   .........
}
       #endregion


The full definition of the class can be found in the attached code.

For the other stores and unit of work, you can browse the code. The logic is the same.

Running the Example

Here are some prerequisites to work with the attached example.

  • Download mongoDB from https://www.mongodb.org/downloads
    • Start mongo and create a database and collection.
  • Visual Studio ultimate if you would like open the modeling project
  • Azure SDK 2.5: Emulate Azure storage
  • Run Visual studio as administrator
  • Run Azure solution
  • Use Nuget to restore any missing package

Points of Interest

Application example managing multiple stores and multiple environments using DDD, Windows Azure, MongoDB, Lucene.net, ...

Hope this example will help and any comments are welcome.

LINK: https://www.codeproject.com/Articles/889978/Polyglot-Persistence-Using-DDD-and-Repository-Unit

 

 

Web API Architecture And Dependency Injection Best Practices


This article explains the Web API best practices for architecture and Dependency Injection, using Unity and other options available for Dependency Injection.

From last few years, Web API is becoming very popular and these days, a lot of projects are going on with Web API. Thousands of projects have been developed using Web API.

If someone is working on Web API, then its architecture and best practices are the most important things, which enable the developer to create one of the best applications. There are more than a hundred rules and recommendations for Web API best practices but in this article, I am going to explain only about the architecture and Dependency Injection.

Web API Architecture

Hence, before discussing about Dependency Injection, it would be better to discuss Web API architecture best practices because Dependency Injection is based on the architecture. Here, I am going to refer to one of my previous articles.

It is recommended to go through my previous article because I will be using source code and concepts of my previous article. You can download the source from the attachment section of my previous article.

Step 1

Create Layered Architecture

Generally, Web API has 3 layers, which are given below.

  1. Web API Layer
  2. Business Layer (Business Logic Layer)
  3. Repository Layer (Data Access Layer)

You can see that Business Layer interfaces do not have any dependency on repository layer interfaces. Thus, the repository layer interface will be referenced only in the classes of business layer.

Step 2

Add POCO (Plain Old CLR Object).

Step 3

Add repository layer project to implement Repository layer interfaces and Business layer project to implement Business layer interfaces.

In the Solution Explorer, it will look, as shown below.

Now, I am going to implement Dependency Injection, using Unity. Right click on API project and open NuGet Package Manager and search for unity.

Install the two packages, as shown in the preceding screenshot.

After installing Unity at Web API layer, it will add some files and codes, which are shown below.

UnityConfig.cs 

 
  1. public static class UnityConfig {  
  2.     public static void RegisterComponents() {  
  3.         var container = new UnityContainer();  
  4.         GlobalConfiguration.Configuration.DependencyResolver = new UnityDependencyResolver(container);  
  5.     }  
  6. }   

UnityResolver.cs

 

 
  1. public class UnityResolver: IDependencyResolver  
  2. {  
  3.     protected IUnityContainer container;  
  4.     public UnityResolver(IUnityContainer container) {…………………………}  
  5.     public IDependencyScope BeginScope() {…………………………}  
  6.     public void Dispose() {…………………………}  
  7.     public object GetService(Type serviceType) {…………………………}  
  8.     public IEnumerable < object > GetServices(Type serviceType) {…………………………}  
  9. }  

Add the code given below to the file.

WebApiConfig.cs

 

 
  1. public static void Register(HttpConfiguration config)  
  2. {  
  3.     var container = new UnityContainer();  
  4.     container.RegisterType < IUserRepository, UserRepository > (new HierarchicalLifetimeManager());  
  5.     container.RegisterType < IUserBusiness, UserBusiness > (new HierarchicalLifetimeManager());  
  6.     container.RegisterType < IBaseRepository < User > , BaseRepository < User >> (new HierarchicalLifetimeManager());  
  7.     config.DependencyResolver = new UnityResolver(container);  
  8. }  

I have seen many people add unity at 2 layers i.e. at Business layer and API layer. Write the codes given below.

API Layer

 

 
  1. container.RegisterType<IUserBusiness, UserBusiness>(new HierarchicalLifetimeManager());  

Business Layer

 

 
  1. container.RegisterType<IUserRepository, UserRepository>(new HierarchicalLifetimeManager());container.RegisterType<IBaseRepository<User>, BaseRepository<User>>(new HierarchicalLifetimeManager());  

I do not endorse the preceding approach due to multiple reasons. Now, I am going to compare pros and cons of 2 approaches.

Approach 1

API Layer

 

 
  1. container.RegisterType<IUserBusiness, UserBusiness>(new HierarchicalLifetimeManager());container.RegisterType<IUserRepository, UserRepository>(new HierarchicalLifetimeManager());container.RegisterType<IBaseRepository<User>, BaseRepository<User>>(new HierarchicalLifetimeManager());  

Approach 2

API Layer

 

 
  1. container.RegisterType<IUserBusiness, UserBusiness>(new HierarchicalLifetimeManager());  

Business Layer

 

 
  1. container.RegisterType<IUserRepository, UserRepository>(new HierarchicalLifetimeManager());container.RegisterType<IBaseRepository<User>, BaseRepository<User>>(new HierarchicalLifetimeManager());  

The screenshot given below explains why the first approach is a better approach,

Injecting Dependencies without any third-party framework

It is not necessary to use any other framework for injecting the dependencies. We can inject dependencies without Unity or any other third party framework.

Go to the class startup.cs and inside the method “ConfigureServices(IServiceCollection services)”, add 2 lines of code given below. 

 
  1. services.AddTransient<IUserManager, UserManager>();  
  2. services.AddTransient<IUserRepository, UserRepository>();   

You can also inject those dependencies from web.config file or from appsettings.json files.

LINK: http://www.c-sharpcorner.com/article/web-api-architecture-and-dependency-injection-best-practices/

HTTP Get, Put, Post And Delete Verbs In ASP.NET WEB API

Introduction

In this article, I will explain about Http Get, Put, Post and Delete verbs in ASP.Net Web API in detail. These are very significant verbs in ASP.NET Web API. This article explains how to perform these 4 verbs in ASP.NET Web API.

Steps for performing HTTP verbs using ASP.NET web API

The steps given below explain about how HTTP verbs perform with ASP.NET Web API.

Step 1

Open new project in Visual Studio, select the Web under visual C# and select ASP.NET Web Application. Finally, give the project name and click OK. Now, select Web API from template Window and click OK.

After opening the project solutions, we can see the all folders in Solution Explorer. Everything is same as ASP.NET MVC but we can find one extra controller and this controller is Value controller. Value controller is inherited from APIController. APIController is an abstract class. Before reading this article, please refer to the article, as it helps to know about APIControllers and API routing.\

Step 2

We can find the five methods in ValuesController. ValuesController looks, as shown below.

In this Controller, we create one static variable with the type of list. In this variable, we assign three strings.

Example 

 
  1. static List<string> languages = new List<string>() {   
  2.             "C#","ASP.NET","MVC"  
  3.         };  

Get method returns what are the strings assigned in the languages variable. Get method contains the code, which looks, as shown below. 

 
  1. // GET api/values  
  2.        public IEnumerable<string> Get()  
  3.        {  
  4.            return languages;  
  5.        }   

If need arises, you can get the specified string from the list. We can get it, using Get method with the parameter. Here, Get method contains overload. 

 
  1. // GET api/values/5  
  2.        public string Get(int id)  
  3.        {  
  4.            return languages[id];  
  5.        }   

If need to Add or Save data in the list arises, we can use Post method. For Post method, we use code given below. 

 
  1. // POST api/values  
  2.         public void Post([FromBody]string value)  
  3.         {  
  4.             languages.Add(value);  
  5.         }   

We can update and delete the data in the list, using the code given below. 

 
  1. // PUT api/values/5  
  2.        public void Put(int id, [FromBody]string value)  
  3.        {  
  4.            languages[id] = value;  
  5.        }  
  6.   
  7.        // DELETE api/values/5  
  8.        public void Delete(int id)  
  9.        {  
  10.            languages.RemoveAt(id);  
  11.        }   

Step 3

We are using Fiddler for monitoring Get, Put, Post, Update actions in Web API. We can download it for free from the URL “https://www.telerik.com/download/fiddler” and install it.

Now, build the project solution and run it. I am running URL http://localhost:51860/api/values/. Now, it will call Get action method in ValueController. Put a breakpoint and you can see which method calls in ValueControllers.



When we put above-mentioned output without the parameter, it automatically calls Get method in Web API. After returning an output, we can see the output as an XML, which is shown below.


If we pass index value of the list, we can get a particular string from the list. I am giving http://localhost:51860/api/values/1so, it will return “ASP.NET” because it has an index value 1.


Step 4

Now, open Fiddler and using it, we can monitor POST, PUT and delete. First, run the Application and then copy http://localhost:51860/api/values URL past in Fiddler in composer tab, followed by clicking enter.

Now, double click the latest URL on the left side in Fiddler and you can view the result. This will show the type of result and request header details.

First, refresh the Application in the Browser. Now, go to composer tab, select GET in the drop-down list and click. Track the latest URL from the left side to composer tab. We can see all details like Host, Connection, Catch-Control, User-Agent and Accept details. 



Now, add new string, using post HTTP verb, how to add and again get back from the list, using Fiddler.

Go to composer tab, select POST in the drop-down list and add new string or value in Request body and add Content-Type: application/JSON, followed by clicking Execute button in right side top.

Now, double click the latest URL on the left side of Fiddler and you can see the response of giving Post request in Fiddler. We can see the response is 204 No Content because post action method is void return type.

Now, we can check whether a string is added in a static variable. Now, select get request and delete added string in request body and click execute. Now, double click the latest one on the left side in Fiddler and you can see added string, which looks as shown below.

Now, we can update the last string in a static variable, using PUT HTTP verb in Fiddler. Now, select PUT from the dropdown in composer tab and the type where we want a new string to update the old string, followed by passing the index value to update the string in the URL and click execute button.

Now, refresh the page and get a new one on the left side of Fiddler. Double-click it and we can see the updated string in Fiddler.

Similarly, we can delete the string from a static variable. Similar to selecting Delete HTTP verb, pass an index value of the one, which needs to be deleted from a static variable. After clicking executes, it refreshes our application. For reference, view the screenshot given below.

Conclusion

This step by step article explained about Http verbs in Web API in ASP.NET. I hope it helps freshers and those who are learning ASP.NET Web API.

LINK: http://www.c-sharpcorner.com/article/http-get-put-post-and-delete-verbs-in-asp-net-web-api/http://www.c-sharpcorner.com/article/http-get-put-post-and-delete-verbs-in-asp-net-web-api/