RE: When to run batch in parallel?

  • From: Abhijeet Rajurkar <Abhijeet.Rajurkar@xxxxxxxxxx>
  • To: "charlottejanehammond@xxxxxxxxx" <charlottejanehammond@xxxxxxxxx>
  • Date: Tue, 25 May 2010 11:05:25 +0530

Hi Charlotte,

We have implemented the batch processing in parallel threads logically 
splitting the data group and it has worked for us. By doing this we could 
reduce the batch time almost 4 times.

Following points need to be addressed while implementing this:

1. Data set must NOT overlap between the parallel jobs/threads.
2. If next batch process is dependent on first then it should not start until 
all parallel jobs gets over.
3. If you are on RAC and sharing some common data in parallel processing then 
please submit the all related threads/jobs on single node otherwise RAC wait 
events will pop in.
4. CPU utilization is the key for deciding on number of parallel threads or 
5. In our case we have turned the table level parallelism off in order to 
process these logical parallel jobs.

Hope this helps.

Kind Regards
Abhijeet Rajurkar

Charlotte Hammond wrote:
> We've got a long running batch job.  Tracing shows 95% of the time is I/O 
> waits (mostly scattered reads steming from hash joins) against our SAN.  Our 
> architect wants to split it into several parallel streams but I'm dubious.  
> Surely the SAN won't serve up data any faster just because we're running in 
> parallel.
> Is there any benefit in this strategy?  I would expect parallelism to come 
> into play if there's significant non-I/O time (e.g. busy on CPU) where other 
> sessions could take advantage of the burstiness, or if the database was 
> accessing multiple disks directly, not virtualised over a big SAN cache.   I 
> reckon we just need a faster SAN (and/or some SQL tuning).
> Any comments?
> Many thanks
> Charlotte
> --


Mastek is in NASSCOM's 'India Top 20' Software Service Exporters List.
In the US, we're called MAJESCOMASTEK

Opinions expressed in this e-mail are those of the individual and not that of 
Mastek Limited, unless specifically indicated to that effect. Mastek Limited 
does not accept any responsibility or liability for it. This e-mail and 
attachments (if any) transmitted with it are confidential and/or privileged and 
solely for the use of the intended person or entity to which it is addressed. 
Any review, re-transmission, dissemination or other use of or taking of any 
action in reliance upon this information by persons or entities other than the 
intended recipient is prohibited. This e-mail and its attachments have been 
scanned for the presence of computer viruses. It is the responsibility of the 
recipient to run the virus check on e-mails and attachments before opening 
them. If you have received this e-mail in error, kindly delete this e-mail from 
desktop and server.


Other related posts: