Spring Boot Batch - Parallel Processing (Running Multiple Jobs Concurrently)

-

If you are new to Spring batch please check the earlier post about Spring batch introduction

In this post we will see Spring boot batch example, Spring boot batch parallel processing example
  • Spring boot batch parallel processing allows multiple batch jobs to run in parallel to minimize the total elapsed batch processing time.
  • This is not a problem as long as the jobs are not sharing the same files, db-tables, or index spaces. If they do, this service should be implemented using partitioned data.
We will create the application having the following 2 jobs - 
  • 1st Job: Reads the Product information from the database and after processing will give the rating to the project.
  • 2nd Job: Reads the Invoice the information the database and update back to the database. 
We are not focusing on the business logic here but the processing of data using multiple jobs.
Technical Details - 
  1. Java 8
  2. Spring Boot 2.x
  3. Spring Batch 4.x
  4. MySql 5.7.17
  5. IntelliJ-Community Edition  2018-3 (Same code will work in Eclipse as well 😃

Source Code available on GitHub -

Project Structure - 
  1. InvoiceProcessingJob and ProductProcessingJob these are 2 jobs having their own ItemReader, ItemProcessor, and ItemWriter. 
  2. JobLauncherConfig will run both the above job


pom.xml Following are the dependencies -




























































application.propeties - 


By default Spring batch is enabled, it means that as soon as application started, job will launch 


We are invoking our job using scheduler that why we have provided spring.batch.job.enabled=false















Job-1: We have created the job. Notice the various annotations we are using here. @Configuration required to inform Spring container this is a Config file and will hold bean definitions. 







@EnableBatchProcessing - Enables the batch processing.







Chunk Size - Denotes the commit interval. After processing 5 records job will commit the transaction 



















We are using the JdbcCursorItemReader which will read the data from the database - 









































Job-2: This job reads the product information from the database and generates the invoices 





















































We will use the following class to launch the job we have created above. We have autowired both jobs and using job launcher will start them. The launch method will get periodically called as we have annotated with the @Scheduled annotation














Finaly Job Launcher - 


















Finally following is the main class to start our Spring Boot batch application - 

































































Comments









  1. UnknownFebruary 13, 2019 at 10:10 AM
    simply awesome..keep it up 😃
    ReplyDelete
  2. RBzOctober 7, 2019 at 6:21 PM
    This doesn't run parallel. If i create a asynchronous job launcher it do run parallel, but my job gets executed twice. Any idea?
    ReplyDelete











Post a Comment



















Popular posts from this blog









Right way to close HttpsURLConnetion using try with resources in java 8






-















HttpsURLConnection  does not implement the AutoCloseable interface in Java 8 or later versions.   Since HttpsURLConnection does not implement AutoCloseable, you cannot use it directly in a try-with-resources block. However, you can create a wrapper class that implements AutoCloseable and encapsulates the HttpsURLConnection object, and then use the wrapper class in a try-with-resources block.   Here's an example of how to create a wrapper class for HttpsURLConnection that implements AutoCloseable:      public class HttpsURLConnectionWrapper implements AutoCloseable {      private final HttpsURLConnection connection;      public HttpsURLConnectionWrapper(URL url) throws IOException {         connection = (HttpsURLConnection) url.openConnection();     }      public HttpsURLConnection getConnection() {         return connection;     }      @Override     public void close() throws IOException {         connection.disconnect();     } }     In this example, the HttpsURLConnectionWrapper c...








Read more










Git Quick Card






-














Image







     Git – Introduction     There are many popular DVCS systems like Git, Mercurial, Bazaar which has grown maturely to implement distributed revision across their products. Git is widely used by the development teams across the world due to its simplicity, active open source the development team  and frequent releases/patches. Before moving forward with the git concepts, few of them terminologies which will be frequently used in this tutorial are defined below :   Remote Repository : This refers to the specific project space on the central server where the project source code is stored and will be used by the team members for checkout or commit.   Local Repository  (.git directory): There is a local folder “.git” which is automatically created when the project or folder is git tracked. The “.git” folder is like a local database which will contain all information related to branches, commit history, user configurations, and metadata.   Branches / Tags  – Has the same definition as ...








Read more