Performance issues with Publishing jobs concurrently

Rhythmyx Technical Discussion
1

We have a set of nightly editions to run, but if an edition (ed2) starts while another (ed1) is running, it “starts”, but waits to start working until the edition (ed1) that is running finishes. However ed1 seems to take longer when there is another edition (ed2) waiting in the queue behind it. If we run ed1 manually while no other editions are running or waiting to run, it usually finishes in less time than it takes if run through the cron job. When editions run, their “time elapsed” clock starts and runs even if it is waiting behind another edition in the queue, so there is some discrepancy in the run-times of the jobs waiting to run, but for ed1, the edition causing the backup, why does it take longer to run when items are behind it in the queue than when it is run alone? Usually when it is run alone there is more activity on the system due to user interaction, because it is run during the day rather than overnight. We are working to speed up our publishing jobs, and this issue of the queue slowing down all jobs is something we would like to solve so our publishing can work efficiently.

Does anyone have any ideas?

2

We have noticed that when we run a large amount of jobs at the same time that it does put a decent load on the server. So I am assuming that there is some sort of cache or memory pool that gets its resources used up, and possibly not cleared until a later job runs and clears it out. We do try to run our publishing jobs so that they will not queue up over time, but we haven’t found a way to completely do this yet. Since upgrading to 6.7 we did do a cleanup on our content lists, and things have gotten much faster than on 6.5.2. We are hoping that this continues to 7.