Repost of Thread: Concurrent Publishing Runs

In Rhythmyx 6.5.x (and earlier versions), one edition per site will run concurrently. This was done to make it possible for installation to limit content in slots to that content that was actually published (and not content that is supposed to be published). Nobody does this in practice any more, and we felt that greater concurrency would benefit more customers, so we decided to remove this restriction. (No matter how closely you space the cron jobs, the editions will still be queued).


In Rhythmyx 6.6 (and beyond) we will remove this synchronization, and you can have as many concurrent editions as you would like. We’ve also included some better support for aborting an edition, etc.

Dave


According to <a href=“http://forum.percussion.com/showthread.php?t=601” target="_blank">CTO Vern Imrich,</a>, the new Rhythmyx 6.6 will have multi-threaded publishing. But right now, we don’t. I guess you could fake concurrent publishing with closely spaced cron files - I haven’t tried it. Someone said that running multiple publishers might speed things up, but that would mean extra licensing fees.

I would think the number of sites would depend on the number of different template designs and the number of publishing locations. Unless you have a location scheme that makes $sys.pub_path more flexible than it is by default, you would need several (many) different site definitions. Technically, you could use the same one for <a href=“http://www.mtu.edu/business/” target="_blank">http://www.mtu.edu/business/</a> and <a href=“http://www.mtu.edu/” target="_blank">http://www.mtu.edu/</a> and <a href=“http://www.mtu.edu/safety/” target="_blank">http://www.mtu.edu/safety/</a>, but not for <a href=“http://www.admin.mtu.edu/” target="_blank">http://www.admin.mtu.edu/</a>


How many webmasters do you have? I wouldn’t want 30 webmasters and 30 sites in the same Community. One Site per Community is a good rule. On the other hand, I don’t know if I’d want to set up 150 sites by myself. Our university will potentially have hundreds of sites, and the standard solution to that is to limit the number of variations. We’re hoping to make division webmasters responsible for the sites in their divisions.


We’re a university with 6 divisions by colleges and schools. Each division has many departments with large websites (not yet within CMS).

<b>Question:</b> We’re debating the issue of whether to have 6 sites (one for each division) within Rx or whether 150 or more sites (by department) would be more logical? <br />

Is Rx capable of running overlapping publishing runs? Can multiple sites be published concurrently?

All

Sorry, I did some operation that corrupted the original thread. I did a view source on the page to capture the content.

db.

If anyone is interested I could post a jsp page that would let certain users run a on demand publish from a jsp page.

Rx, as part of fast forward contains a jsp for a “Publish Now” menu action that is available to certain communities (I believe the default is the Enterprise Investments). The jsp page that this references is in:
\AppServer\server\rx\deploy\rxapp.ear\rxapp.war\ui\publishing

I hope that is where you started :D. We created our own publish now edition that is only available in the site root folder (http://forum.percussion.com/showpost.php?p=2777&postcount=16) by basing slightly off the code found there (and using PSDbUtils to query the database directly for certain information).

The problem we had with the publish on an individual item was that it would only publish out that particular item and none of the pages that the item would reference (be included in) hence the need to trigger an incremental publish.

I’m sure that someone could use the code, but perhaps we should petition for a section for codes/examples in the forums?

Thanks for the Publish Now information and for reposting our thread. We’ll keep the Publish Now information in mind, since it seems very useful. Before we can use this we need to figure out how we’re going to handle sites and overall publishing.

Let me respond to the first two replies by Michael and Dave. To explain our situation further, we’re having one overarching design for the entire university. Each department may have their own tweaks to the design controlled by CSS or minor variations in templates. Everything will be published to the same base location. We have one community for each department (we’re at #3 of 150+), so if we follow the one site per community rule then we should go with the 150+ sites. One site per community makes sense to us but would this make it difficult to manage and can Rx handle so many sites and publishing those sites?

Dave, does “one edition per site will run concurrently” mean that if we have 150 sites that 150 publishing runs can run at the same time? Has Rx announced a release date for 6.6?

On second thought, you should be able to publish, for example, to /srv/www/, /srv/www/business/, /srv/www/safety/, and /srv/www/admin/ as if they were one site, if they shared the same root. And with some tweaking, dispatch templates could allow you to use different templates as if they were different sites, if you found a way to flag them, such as customizing the Usage field. I’m not recommending this, just saying that it should be possible. Like others, we restrict each site to its own community, so nobody else can even see the site folder, to prevent accidentally creating content with the wrong community permissions. Though I hear this may not be such a problem in future editions.

Publishing has always been multi-threaded. What is changing is how those threads are actually used. Prior to 6.6, as Dave noted, you could run multiple editions concurrently as long as they were from different sites, but only 1 thread was used per edition.

In 6.6, this will no longer be true. All editions will be queued, they will not run concurrently. What will happen instead is that multiple threads will be used to process the items in the queue. This means any qiven edition will now publish much faster. The new editions also have a simple priority mechanism that allows an edition to be placed at the head of the queue (generally used for publish now.)

Gail, if you had 150 editions, they would not run in parallel, all the items would be queued and the queue would be processed with multiple threads.

The max number of threads is controllable by the implementer, but it is unlikely you would ever want to use 150 threads at once (as it would probably be slower than using 1 thread due to context switching.) The actual number is highly dependent on the hardware and how the Rx server is being used.

One correction. Dave pointed out to me that, in fact, the editions can run concurrently, but there is no control over ‘how’ concurrently they run, if at all. Basically, once an edition is started, it starts queueing all items in its clists. If another edition is started while this queueing is taking place, both editions will queue concurrently and their items will be inter-mingled on the queue. (And so on with additional editions starting while other editions are queueing.) The queue is then processed in FIFO order.