We are having a problem with an incremental publish, which hangs the server when we preview the HTML content list or try to publish - it is the only content list that fails to preview. I have been informed that there are two possible solutions to this:
[ol]increase the maximum memory in the file - can’t do this as it is already at the max.
[/ol]
[LIST=2]add a ‘page by’ parameter to the JEXL query[/LIST]
I would like to try the second solution - but can find no documentation on how to do it - what the syntax is etc. Can anyone help? An example would be great.
The parameter is named “maxrowsperpage” and you add it to the URL in the content list definition. For example, the FastForward EI Incremental content list has a URL like this:
This parameter works in the same in all releases of Rhythmyx back at least to 5.0. (Maybe farther, but I didn’t check).
Note that it doesn’t limit the number of items in the content list. Instead it breaks the list up into “chunks” . If you have 1000 pages to publish, and maxrowsperpage is set to 200, the publisher will generate 5 content lists of 200 pages each. The chunks are processed sequentially: chunk #2 will start as soon as chunk #1 is finished.
I don’t know of maximum limit on the total content list. However, if the “chunks” are large enough (or you try to publish too many pages all at once) the publisher will run out of memory.
Thanks for the speedy comments - I can see this is a useful tool. However, this has not resolved the original problem - that the server ‘hangs’ when we try to carry out this incremental publish or when we try to preview the HTML content list. We have tested the JSR-170 query in the test area and that works okay, so I’m beginning to think that this is a bug. Two new content lists we have created for an incremental publish are getting the same problem. Is this a known issue - has anyone else had a similar problem?
I include the URL and JSR query strings below (oh and we are using version 6.1). The same set up works fine for the full publish.
select rx:sys_contentid, rx:sys_folderid from rx:AboutUsDepartmentsPage,rx:AboutUsGenericPage,rx:AboutUsHomePage,rx:AboutUsHubPage,rx:AboutUsListPage,rx:AboutUsServices,rx:AboutUsSingleColumnLayoutPage where jcr:path like ‘//Sites/Home/Aboutus/%’
How large is this content list? When you run the query in the query tester, how many rows come back? Is this the same number of rows (approximately) that you see when preview the same content list with “Type” radio button set to “normal” instead of “incremental” ?
Let’s verify that it is a “hang” and not just an extreme case of the slows. Preview the content list in the browser (from the edition screen). Even for small content lists, this will take several seconds. If it doesn’t come back in 10 or 15 minutes, then it’s probably hung.
There are no bugs logged in the internal tracking systems that report a “hang” in this circumstance but several that talk about “slowness” of incremental publishing in different circumstances.
The “maxrowsperpage” parameter is applied after the content list is generated, so if that generation process is taking a long time, maxrowsperpage won’t help.
I tested the preview this morning and finally killed it and restarted the server after 30 minutes. The server doesn’t seem to hang, it just slows, but responds to commands via the Monitor tab in the Rhythmyx Server Adminstrator. The Content Explorer becomes unusable as it more or less grinds to a halt it is so slow.
We have started an Incremental publish at 2:00am and come into work later the same morning and it has not finished.
This state of affairs can’t be right - we can’t carry out an incremental publish - what should we do?
We’ve pretty much reached the limit of my ability to diagnose this remotely. Clearly, there’s something wrong with your system.
I want to escalate this to Engineering, and that process happens through Tech Support. I’ve asked them to open an incident for you, but you’ll have to work with them to get this resolved. We’ll most likely need to get a copy of the system and/or remote access.
Bear in mind that this functionality (incremental publishing) is used by virtually every customer who has ever bought the product, so it’s very well tested. The probability of a basic bug is pretty small, but there’s something about your system that is causing something to go astray.
Going back to “maximum number of items”. Not sure if we are talking about the same thing. I’m referring to the error “java.sql.SQLException: Prepared or callable statement has more than 2000 parameter markers” (TAR #MA-06-12-0005). Is maxrowsperpage a workaround here?
I’m not sure what the original problem was, but the maxrowsperpage is applied to the list AFTER it is generated from the query. If the initial query is failing (with a SQL Exception) then adding maxrowsperpage won’t change anything.
I suspect that it would be useful for you to understand what takes time in incremental publishing. It is not the query itself, it is the process of updating the modification dates on all the content that is potentially required to be republished on every incremental.
To do this we find all content types that participate with calculated slots. This is normally one or two content types used for autoindeces.
The auto index content items are then used to find active assembly parents. These are marked as modified as they generally need to be published every time. This is because the auto index items are generally included in the parent as a snippet, and the snippet will be potentially different each time.
The further active assembly ancestors are sometimes touched, depending on what kind of use is made of the item. If the snippet has slots then we assume that the displayed content may change.
This process is relatively expensive as it requires multiple db queries.
To answer your question about paginating a content list, the incrementals should paginate as all content lists do by using a maxrowsperpage query parameter in the URL for the content list. This is used to limit the memory needed for very large content list documents, as well as to improve your ability to cancel a running publish.