Skip to main content

.NET Core MVC Application File Upload To Physical Location With Streaming Technique(Useful For Large Files) - Part 1


Streaming Technique In File Upload:

  • Streaming Technique is useful in uploading the larger files.
  • Uploading file is achieved as a multipart request where the files are processed directly by the application.
  • In Streaming Technique file uploading can not be done with Model Binding(but in buffered technique, files uploading can be done with Model Binding by using IFormFile type). So the file is accessed directly from the request object(as multipart request) and processed.
  • Since Model Binding will not work here every filed from the form that is like text fields, file uploading fields, etc are received as 'form-data' format in the multipart request.
  • Streaming helps in scaling the application.
  • Streaming doesn't demand memory or disk space(but in buffered technique, file upload highly depends on the RAM, which doesn't work for large file uploading).
Note:
Click here for file uploading using buffer technique

Create A Sample MVC Application:

Let's understand file uploading using a streaming technique by creating a sample .Net Core MVC application. You can start on by using an editor like Visual Studio 2019(supports .NET Core 3.0+) or VisualStudioCode. In this sample, I'm using VisualStudioCode. Click here to get a basic understand to work with Visual Studio Code.

Basic Understanding On multipart/form-data:

multipart/form-data is one of the content-type headers of the post method. multipart/form-data is mainly used for media file uploading. Having this content-type means each value is sent as a block of data.

Content-Disposition is the general header for the multipart/form-data body. This header will be assigned to every subpart(subpart means in a form field like text, area, select, file, etc are send as multiple parts on using multipart/form-data content-type) of the multipart body. That means for every field on a form will have Content-Disposition header and each subpart is separated by 'boundary' defined in the content-type.

A sample to understand multipart/form-data or Content-Disposition or boundary delimiter as below.
Host: mywebsite.com
Content-Type: multipart/form-data;boundary="boundary"

--boundary
Content-Disposition: form-data; name="field1"

value1
--boundary
Content-Disposition: form-data; name="field2"; filename="mypic.jpg"

Create Sample Form To Upload File:

Let's develop an action method to load form uploading file as follow.
Controller/FileUplaodController:
using Microsoft.AspNetCore.Mvc;

namespace StreamFileUpload.App.Controllers
{
    [Route("file-upload")]
    public class FileUploadContrller : Controller
    {
        public FileUploadContrller()
        {

        }

        [Route("add-file")]
        [HttpGet]
        public IActionResult AddFile()
        {
            return View();
        }
    }
}
Let's develop a small form page as below.
Views/FileUpload/AddFile.cshtml: 
<h1>Add file</h1>
<div>
    <form asp-route="file-upload/add-file"  enctype="multipart/form-data" method="POST">
        <div class="form-group">
            <label for="txtName">Name</label>
            <input type="text" class="form-control" id="txtName">
        </div>
        <div class="form-group">
            <label for="txtAge">Age</label>
            <input type="text" class="form-control" id="txtAge">
        </div>
        <div class="form-group">
            <label for="fileUpload">Upload Files</label>
            <input type="file" class="form-control-file" id="fileUpload" >
        </div>
        <button type="submit" class="btn btn-primary">Save</button>
    </form>
</div>
Form created with two text fields and one file upload field. To make the input element to file upload component need assign 'type=" file"'. 'asp-route' is .Net Core MVC attribute to bind the post URL of the form. 'enctype="multipart/form-data"' is media type to post the files to the server.

Now run the application and preview the page as below.

Read Form Data From Request Object:

Let's develop an action method to post the data from the form below.
using Microsoft.AspNetCore.Mvc;
using Microsoft.Net.Http.Headers;
using Microsoft.AspNetCore.WebUtilities;
using System.Threading.Tasks;
using System;

namespace StreamFileUpload.App.Controllers
{
    [Route("file-upload")]
    public class FileUploadController : Controller
    {

        public FileUploadController()
        {

        }

        [Route("add-file")]
        [HttpGet]
        public IActionResult AddFile()
        {
            return View();
        }

        [Route("add-file")]
        [HttpPost]
        public async Task<IActionResult> SaveFileToPhysicalFolder()
        {
            var boundary = HeaderUtilities.RemoveQuotes(
                MediaTypeHeaderValue.Parse(Request.ContentType).Boundary
            ).Value;

            var reader = new MultipartReader(boundary, Request.Body);

            var section = await reader.ReadNextSectionAsync();

            // this variable to test 
            // how many form fields are posted to server
            // newver use this logic in application development
            int fieldsCount = 0;

            while (section != null)
            {
                var hasContentDisposition = ContentDispositionHeaderValue.TryParse(
                    section.ContentDisposition, out var contentDisposition
                );

                if (hasContentDisposition)
                {
                    if (contentDisposition.DispositionType.Equals("form-data") &&
                    (!string.IsNullOrEmpty(contentDisposition.FileName.Value) ||
                    !string.IsNullOrEmpty(contentDisposition.FileNameStar.Value)))
                    {
                        // uploaded files form fileds
                        fieldsCount = fieldsCount + 1;
                    }
                    else
                    {
                        if (!contentDisposition.Name.Value.Equals( "__RequestVerificationToken",StringComparison.OrdinalIgnoreCase))
                        {
                            // Note below if writen to check only form fileds
                            // never implement this in your application code

                            // __RequestVerificationToken an anti forgerty token 
                            // which send as one of the section or field 
                            // on data posting
                            fieldsCount = fieldsCount + 1;
                        }
                        
                    }
                }
                section = await reader.ReadNextSectionAsync();
            }

            return Content(fieldsCount + "");
        }
    }
}

On debugging you can see content-type in the request object with multipart/form-data and boundary delimiter for subpart separation as below.
var boundary = HeaderUtilities.RemoveQuotes(
 MediaTypeHeaderValue.Parse(Request.ContentType).Boundary
).Value;
  • To fetch the 'boundary' value from 'Request.ContentType' we need to use 'Microsoft.Net.Http.Headers.MediaTypeHeader.Parse()' method. 
  • 'Microsoft.Net.Http.Headers.HeaderUtilities.RemoveQuotes' helps to remove the unnecessary quotes on 'boundary' value. 
  • The reason behind to fetch boundary value is it is the separator for each field in the form of post data and using this boundary value we can iterate those form fields.

var reader = new MultipartReader(boundary, Request.Body);
'Microsoft.AspNetCore.WebUtilities.MultipartReader' object to read a stream of subparts from the multipart data(means reading form fields).

var section = await reader.ReadNextSectionAsync();
while (section != null)
{
 var hasContentDisposition = ContentDispositionHeaderValue.TryParse(
  section.ContentDisposition, out var contentDisposition
 );

 if (hasContentDisposition)
 {
  if (contentDisposition.DispositionType.Equals("form-data") &&
  (!string.IsNullOrEmpty(contentDisposition.FileName.Value) ||
  !string.IsNullOrEmpty(contentDisposition.FileNameStar.Value)))
  {
   // uploaded files form fileds
   
  }
  else
  {
   // text , select all other fields 
  }
 }
 section = await reader.ReadNextSectionAsync();
}
'Microsoft.AspNetCore.WebUtilites.MultipartReader.ReadNextSectionAsync()' method reads the stream of form fields as sections. Each section contains 'ContentDisposition' which is a content-type header for each form filed. Using ContentDisposition we can identify that form field is file uploading field or not using 'ContentDisposition.FileName' or 'ContentDispositionFile.FileNameStr'.

Let's test the post action method receiving all the form fields that we can be determined by using count variable 'fieldCount'. In our page form, we have 3 fields like 'Name', 'Age' and 'FileUpload' so our post method expected to return count value 3, but it returns '0' as below.
This shows no field is read by the server. I did this purposefully to expose common mistakes while creating an Html form. So the reason behind this issue is every field element should have Html "name" attribute. The server reads only Html fields which has a "name" attribute to the elements. Let's update our form with the "name" Html attribute as follow.
Views/FileUpload/AddFile.cshtml: 
<h1>Add file</h1>
<div>
    <form asp-route="file-upload/add-file"  enctype="multipart/form-data" method="POST">
        <div class="form-group">
            <label for="txtName">Name</label>
            <input type="text" class="form-control" id="txtName" name="Name">
        </div>
        <div class="form-group">
            <label for="txtAge">Age</label>
            <input type="text" class="form-control" id="txtAge" name="Age">
        </div>
        <div class="form-group">
            <label for="fileUpload">Upload Files</label>
            <input type="file" class="form-control-file" id="fileUpload"  name="FileUpload">
        </div>
        <button type="submit" class="btn btn-primary">Save</button>
    </form>
</div>
Now again post the data to the server and we can see our expected fields count '3' as below.

Save File To Physical Location:

Up to now, we have implemented how to read form fields from the 'Request' object. Now we need to implement the code to save our uploaded file to a Physical Location on the server.

Now let's create an 'images' folder inside of the 'wwwroot' folder( which stores all static files of the application). Now to read the 'wwwroot' path inside the application we need to inject 'Microsoft.AspNetCore.Hosting.IWebHostingEnvironment' in our controller as below.
using Microsoft.AspNetCore.Hosting;

namespace StreamFileUpload.App.Controllers
{
    [Route("file-upload")]
    public class FileUploadController : Controller
    {
        private readonly IWebHostEnvironment _webHostEnvironment;

        public FileUploadController(IWebHostEnvironment webHostEnvironment)
        {
            _webHostEnvironment = webHostEnvironment;
        }
  
  // code hidden for display purpose
    }
}

Now let's try to save the uploaded file into the folder we have created above.
Controller/FileUploadController.cs: 
public async Task SaveFileToPhysicalFolder()
{
 var boundary = HeaderUtilities.RemoveQuotes(
  MediaTypeHeaderValue.Parse(Request.ContentType).Boundary
 ).Value;

 var reader = new MultipartReader(boundary, Request.Body);

 var section = await reader.ReadNextSectionAsync();


 while (section != null)
 {
  var hasContentDisposition = ContentDispositionHeaderValue.TryParse(
   section.ContentDisposition, out var contentDisposition
  );

  if (hasContentDisposition)
  {
   if (contentDisposition.DispositionType.Equals("form-data") &&
   (!string.IsNullOrEmpty(contentDisposition.FileName.Value) ||
   !string.IsNullOrEmpty(contentDisposition.FileNameStar.Value)))
   {
    string fileStoragePath = $"{_webHostEnvironment.WebRootPath}/images/";
    string fileName = Path.GetRandomFileName() + ".jpg";
    // uploaded files form fileds
    byte[] fileByteArray;
    using (var memoryStream = new MemoryStream())
    {
     await section.Body.CopyToAsync(memoryStream);
     fileByteArray = memoryStream.ToArray();
    }
    using (var fileStream = System.IO.File.Create(Path.Combine(fileStoragePath,fileName)))
    {
     await fileStream.WriteAsync(fileByteArray);
    }
   }
   else
   {
    // no file uplading fields
   }
  }
  section = await reader.ReadNextSectionAsync();
 }

 return Content("Uploaded successfully");
}
To read the file as an array of bytes, by copying the form field (file upload field) section body content to the memory stream. Then the byte result is used by 'System.IO.File' to create the upload file into the destination folder.

Let's test the file uploading functionality as follow.

Explicit Model Binding For Form Data:

Till now we have understood the way to save the files. Now we need to bind other form fields data to .Net Core Model Binding explicitly and also validate the form data if needed.

Let's create a .NET model class that represents form fields as below.
Models/Profile.cs:
namespace StreamFileUpload.App.Models
{
    public class Profile
    {
        public string Name { get; set; }
        public int Age { get; set; }
    }
}
let's update the complete file upload post method as follows.
Controllers/FileUploadController.cs: 
using Microsoft.AspNetCore.Mvc;
using Microsoft.Net.Http.Headers;
using Microsoft.AspNetCore.WebUtilities;
using System.Threading.Tasks;
using System;
using Microsoft.AspNetCore.Hosting;
using System.IO;
using System.Text;
using StreamFileUpload.App.Models;
using Microsoft.AspNetCore.Mvc.ModelBinding;
using Microsoft.AspNetCore.Http;
using System.Globalization;

namespace StreamFileUpload.App.Controllers
{
    [Route("file-upload")]
    public class FileUploadController : Controller
    {
		// some code hidden for display purpose
        [Route("add-file")]
        [HttpPost]
        public async Task<IActionResult> SaveFileToPhysicalFolder()
        {
            var boundary = HeaderUtilities.RemoveQuotes(
                MediaTypeHeaderValue.Parse(Request.ContentType).Boundary
            ).Value;

            var reader = new MultipartReader(boundary, Request.Body);

            var section = await reader.ReadNextSectionAsync();

            var formAccumelator = new KeyValueAccumulator();

            while (section != null)
            {
                var hasContentDisposition = ContentDispositionHeaderValue.TryParse(
                    section.ContentDisposition, out var contentDisposition
                );

                if (hasContentDisposition)
                {
                    if (contentDisposition.DispositionType.Equals("form-data") &&
                    (!string.IsNullOrEmpty(contentDisposition.FileName.Value) ||
                    !string.IsNullOrEmpty(contentDisposition.FileNameStar.Value)))
                    {
                        string fileStoragePath = $"{_webHostEnvironment.WebRootPath}/images/";
                        string fileName = Path.GetRandomFileName() + ".jpg";
                        // uploaded files form fileds
                        byte[] fileByteArray;
                        using (var memoryStream = new MemoryStream())
                        {
                            await section.Body.CopyToAsync(memoryStream);
                            fileByteArray = memoryStream.ToArray();
                        }
                        using (var fileStream = System.IO.File.Create(Path.Combine(fileStoragePath,fileName)))
                        {
                            await fileStream.WriteAsync(fileByteArray);
                        }
                    }
                    else
                    {
                        var key = HeaderUtilities.RemoveQuotes(contentDisposition.Name).Value;

                        using(var streamReader = new StreamReader(section.Body,
                        encoding: Encoding.UTF8,
                        detectEncodingFromByteOrderMarks:true,
                        bufferSize:1024,
                        leaveOpen:true)){
                            var value = await streamReader.ReadToEndAsync();
                            if(string.Equals(value, "undefined",StringComparison.OrdinalIgnoreCase)){
                                value = string.Empty;
                            }
                            formAccumelator.Append(key, value);
                        }
                    }
                }
                section = await reader.ReadNextSectionAsync();
            }

            var profile = new Profile();
            var formValueProvidere = new FormValueProvider(
                BindingSource.Form,
                new FormCollection(formAccumelator.GetResults()),
                CultureInfo.CurrentCulture
            );

            var bindindSuccessfully = await TryUpdateModelAsync(profile,"",formValueProvidere);
            if(ModelState.IsValid){
                // write log to save profile data to database
            }

            return Content("Uploaded successfully");
        }
    }
}


var formAccumelator = new KeyValueAccumulator();
'Microsoft.AspNetCore.WebUtilities.KeyValueAccumulator' object is used to save 'key-value' pairs. Internally this object uses a dictionary to store the data as 'Key-Value' pair. Using this object we store our form field values.

var key = HeaderUtilities.RemoveQuotes(contentDisposition.Name).Value;
'contentDisposition.Name' provides Html element name attribute value and value should match with the .NET model created above. 'Microsoft.Net.Http.Headers.HeaderUtilities.RemoveQuotes()' method helps to trim the quotes inside the value.

using(var streamReader = new StreamReader(section.Body,
 encoding: Encoding.UTF8,
 detectEncodingFromByteOrderMarks:true,
 bufferSize:1024,
 leaveOpen:true)){
   var value = await streamReader.ReadToEndAsync();
   if(string.Equals(value, "undefined",StringComparison.OrdinalIgnoreCase)){
      value = string.Empty;
   }
   formAccumelator.Append(key, value);
 }
Using 'System.IO.StreamReader' read the section body content as a stream of data which helps to fetch the form field value. After fetching key-value pairs store it in 'Microsoft.AspNetCore.WebUtilities.KeyValueAccumulator'. 'leaveOpen:true' avoid the destroying of the stream (section.Body) at the time 'Using' block destroy StreamReader object. 
var profile = new Profile();
var formValueProvidere = new FormValueProvider(
	BindingSource.Form,
	new FormCollection(formAccumelator.GetResults()),
	CultureInfo.CurrentCulture
);
var bindindSuccessfully = await TryUpdateModelAsync(profile,"",formValueProvidere);
'Microsoft.AspNetCore.Mvc.ModelBinding.FormValueReader' object reads the key-value pair datafrom 'Microsoft.AspNetCore.WebUtilities.KeyValueAccutmulator'. 'TryUpdateModelAsync()' methods assign the form data to user defined model  class(Profile).

'TryUpdateModelAsync()' method also validates the model object and fires the server-side validation. To test validations firing adds data annotation on model property.

using System.ComponentModel.DataAnnotations;

namespace StreamFileUpload.App.Models
{
    public class Profile
    {
        [Required(ErrorMessage = "Name is required field")]
        public string Name { get; set; }
        public int Age { get; set; }
    }
}


Wrapping Up:

Hopefully, this article will help to understand the Large Files Uploading Using Streaming Technique In .NET Core. I love to have your feedback, suggestions, and better techniques in the comment section.

Refer:


Follow Me:

Labels:

Comments

Post a Comment

Popular posts from this blog

Angular 14 Reactive Forms Example

In this article, we will explore the Angular(14) reactive forms with an example. Reactive Forms: Angular reactive forms support model-driven techniques to handle the form's input values. The reactive forms state is immutable, any form filed change creates a new state for the form. Reactive forms are built around observable streams, where form inputs and values are provided as streams of input values, which can be accessed synchronously. Some key notations that involve in reactive forms are like: FormControl - each input element in the form is 'FormControl'. The 'FormControl' tracks the value and validation status of form fields. FormGroup - Track the value and validate the state of the group of 'FormControl'. FormBuilder - Angular service which can be used to create the 'FormGroup' or FormControl instance quickly. Form Array - That can hold infinite form control, this helps to create dynamic forms. Create An Angular(14) Application: Let'
3 comments
Read more

.NET 7 Web API CRUD Using Entity Framework Core

In this article, we are going to implement a sample .NET 7 Web API CRUD using the Entity Framework Core. Web API: Web API is a framework for building HTTP services that can be accessed from any client like browser, mobile devices, and desktop apps. In simple terminology API(Application Programming Interface) means an interface module that contains programming functions that can be requested via HTTP calls either to fetch or update data for their respective clients. Some of the Key Characteristics of API: Supports HTTP verbs like 'GET', 'POST', 'PUT', 'DELETE', etc. Supports default responses like 'XML' and 'JSON'. Also can define custom responses. Supports self-hosting or individual hosting, so that all different kinds of apps can consume it. Authentication and Authorization are easy to implement. The ideal platform to build the REST full services. Install The SQL Server And SQL Management Studio: Let's install the SQL server on our l
3 comments
Read more

ReactJS(v18) JWT Authentication Using HTTP Only Cookie

In this article, we will implement the ReactJS application authentication using the HTTP-only cookie. HTTP Only Cookie: In a SPA(Single Page Application) Authentication JWT token either can be stored in browser 'LocalStorage' or in 'Cookie'. Storing the JWT token inside of the cookie then the cookie should be HTTP Only. The HTTP-ONly cookie nature is that it will be only accessible by the server application. Client apps like javascript-based apps can't access the HTTP-Only cookie. So if we use the authentication with HTTP-only JWT cookie then we no need to implement the custom logic like adding authorization header or storing token data, etc at our client application. Because once the user authenticated cookie will be automatically sent to the server by the browser on every API call. Authentication API: To authenticate our client application with JWT HTTP-only cookie, I developed a NetJS(which is a node) Mock API. Check the GitHub link and read the document on G
3 comments
Read more

.NET6 Web API CRUD Operation With Entity Framework Core

In this article, we are going to do a small demo on AspNetCore 6 Web API CRUD operations. What Is Web API: Web API is a framework for building HTTP services that can be accessed from any client like browser, mobile devices, desktop apps. In simple terminology API(Application Programming Interface) means an interface module that contains a programming function that can be requested via HTTP calls to save or fetch the data for their respective clients. Some of the key characteristics of API: Supports HTTP verbs like 'GET', 'POST', 'PUT', 'DELETE', etc. Supports default responses like 'XML' and 'JSON'. Also can define custom responses. Supports self-hosting or individual hosting, so that all different kinds of apps can consume it. Authentication and Authorization are easy to implement. The ideal platform to build REST full services. Create A .NET6 Web API Application: Let's create a .Net6 Web API sample application to accomplish our
6 comments
Read more

Angular 14 State Management CRUD Example With NgRx(14)

In this article, we are going to implement the Angular(14) state management CRUD example with NgRx(14) NgRx Store For State Management: In an angular application to share consistent data between multiple components, we use NgRx state management. Using NgRx state helps to avoid unwanted API calls, easy to maintain consistent data, etc. The main building blocks for the NgRx store are: Actions - NgRx actions represents event to trigger the reducers to save the data into the stores. Reducer - Reducer's pure function, which is used to create a new state on data change. Store - The store is the model or entity that holds the data. Selector - Selector to fetch the slices of data from the store to angular components. Effects - Effects deals with external network calls like API. The effect gets executed based the action performed Ngrx State Management flow: The angular component needs data for binding.  So angular component calls an action that is responsible for invoking the API call.  Aft
8 comments
Read more

Unit Testing Asp.NetCore Web API Using xUnit[.NET6]

In this article, we are going to write test cases to an Asp.NetCore Web API(.NET6) application using the xUnit. xUnit For .NET: The xUnit for .Net is a free, open-source, community-focused unit testing tool for .NET applications. By default .Net also provides a xUnit project template to implement test cases. Unit test cases build upon the 'AAA' formula that means 'Arrange', 'Act' and 'Assert' Arrange - Declaring variables, objects, instantiating mocks, etc. Act - Calling or invoking the method that needs to be tested. Assert - The assert ensures that code behaves as expected means yielding expected output. Create An API And Unit Test Projects: Let's create a .Net6 Web API and xUnit sample applications to accomplish our demo. We can use either Visual Studio 2022 or Visual Studio Code(using .NET CLI commands) to create any.Net6 application. For this demo, I'm using the 'Visual Studio Code'(using the .NET CLI command) editor. Create a fo
7 comments
Read more

Angular 14 Crud Example

In this article, we will implement CRUD operation in the Angular 14 application. Angular: Angular is a framework that can be used to build a single-page application. Angular applications are built with components that make our code simple and clean. Angular components compose of 3 files like TypeScript File(*.ts), Html File(*.html), CSS File(*.cs) Components typescript file and HTML file support 2-way binding which means data flow is bi-directional Component typescript file listens for all HTML events from the HTML file. Create Angular(14) Application: Let's create an Angular(14) application to begin our sample. Make sure to install the Angular CLI tool into our local machine because it provides easy CLI commands to play with the angular application. Command To Install Angular CLI npm install -g @angular/cli Run the below command to create the angular application. Command To Create Angular Application ng new name_of_your_app Note: While creating the app, you will see a noti
6 comments
Read more

Part-1 Angular JWT Authentication Using HTTP Only Cookie[Angular V13]

In this article, we are going to implement a sample angular application authentication using HTTP only cookie that contains a JWT token. HTTP Only JWT Cookie: In a SPA(Single Page Application) Authentication JWT token either can be stored in browser 'LocalStorage' or in 'Cookie'. Storing JWT token inside of the cookie then the cookie should be HTTP Only. The HTTP-Only cookie nature is that it will be only accessible by the server application. Client apps like javascript-based apps can't access the HTTP-Only cookie. So if we use authentication with HTTP only JWT cookie then we no need to implement custom logic like adding authorization header or storing token data, etc at our client application. Because once the user authenticated cookie will be automatically sent to the server by the browser on every API call. Authentication API: To implement JWT cookie authentication we need to set up an API. For that, I had created a mock authentication API(Using the NestJS Se
Post a Comment
Read more

ReactJS(v18) Authentication With JWT AccessToken And Refresh Token

In this article, we are going to do ReactJS(v18) application authentication using the JWT Access Token and Refresh Token. JSON Web Token(JWT): JSON Web Token is a digitally signed and secured token for user validation. The JWT is constructed with 3 important parts: Header Payload Signature Create ReactJS Application: Let's create a ReactJS application to accomplish our demo. npx create-react-app name-of-your-app Configure React Bootstrap Library: Let's install the React Bootstrap library npm install react-bootstrap bootstrap Now add the bootstrap CSS reference in 'index.js'. src/index.js: import 'bootstrap/dist/css/bootstrap.min.css' Create A React Component 'Layout': Let's add a React component like 'Layout' in 'components/shared' folders(new folders). src/components/shared/Layout.js: import Navbar from "react-bootstrap/Navbar"; import { Container } from "react-bootstrap"; import Nav from "react-boot
Post a Comment
Read more

A Small Guide On NestJS Queues

NestJS Application Queues helps to deal with application scaling and performance challenges. When To Use Queues?: API request that mostly involves in time taking operations like CPU bound operation, doing them synchronously which will result in thread blocking. So to avoid these issues, it is an appropriate way to make the CPU-bound operation separate background job.  In nestjs one of the best solutions for these kinds of tasks is to implement the Queues. For queueing mechanism in the nestjs application most recommended library is '@nestjs/bull'(Bull is nodejs queue library). The 'Bull' depends on Redis cache for data storage like a job. So in this queueing technique, we will create services like 'Producer' and 'Consumer'. The 'Producer' is used to push our jobs into the Redis stores. The consumer will read those jobs(eg: CPU Bound Operations) and process them. So by using this queues technique user requests processed very fastly because actually
9 comments
Read more