Posts

Showing posts from February, 2019

Spring batch introduction

An open source framework to create batch processing application which can deal with the high volume of data, capable to handle processing of billions of records in a day efficiently. Lightweight framework – consume very less memory. It’s not a scheduling framework Provide out of the box APIs to read, process and write data.   Efficient transaction management and e rror handling . Why does the batch job needed? Automate the complex processing (applying business logic on data/transforming data) of large volumes of data without any user interaction. Spring Batch vs Spring Boot Batch - · Spring batch is a batch processing framework and spring boot help us to do rapid development by using starter-templates (convention over configuration). What are the ideal use cases for the batch job? Reads the high volume of records from the database/file system.   Process the data e.g. Month-end / Weekend calculations Write back processed data / genera...

Spring Boot Batch - Parallel Processing (Running Multiple Jobs Concurrently)

Image
If you are new to Spring batch please check the earlier post about S pring batch introduction In this post we will see Spring boot batch example, Spring boot batch parallel processing example Spring boot batch parallel processing allows multiple batch jobs to run in parallel to minimize the total elapsed batch processing time. This is not a problem as long as the jobs are not sharing the same files, db-tables, or index spaces. If they do, this service should be implemented using partitioned data. We will create the application having the following 2 jobs -  1st Job: Reads the P roduct information from the database and after processing will give the rating to the project. 2nd Job: Reads the Invoice the information the database and update back to the database.  We are not focusing on the business logic here but the processing of data using multiple jobs. Technical Details -  Java 8 Spring Boot 2.x Spring Batch 4.x MySql 5...