Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

We already discussed how to get the job logs and the screen shots. These articles can help you get the Apache Logs and Support Tool:

How to send your IBM i Apache logs to Support

How to run the Support Tool on IBM i

This probably seems like a lot of material to collect, and it is. The reason for this is because the only time any information can be collected is when an incident is happening, so it pays to be thorough. All of this may not be needed to analyze every issue, but it is difficult to predict which of these things might be needed until after the analysis is done. So, it is best to get everything.

...

This workaround is also mentioned in the note at the start of the prior section.  As distributed, Zend sets the Apache timeout very high so as not to conflict with the FastCGI and PHP timeouts.  You can set this timeout to be much shorter, so that Apache can possibly end and respawn a frozen or zombie child process, when PHP is not able to do it. 

This article tells how:  Set IBM i Apache Timeout to a shorter time to recover from lock ups faster

Increase FastCGI child processes

This workaround can help if the problem is that PHP child processes are locking up over time, or if there is a need for more PHP children to handle the work load on your site. 

This article tells how: Increase FastCGI child processes on IBM i

...

This issue should not cause your web site to stop responding, but it can make it much slower, which can aggravate other conditions that might contribute to a freeze. 

This article tells how make sure your DNS settings are valid: Invalid DNS slows Zend Server UI on IBM i

Check and adjust Apache threads

This issue is not very common, but it can cause your web site to slow down or stop.  One possible symptom of this issue might be missing entries in the access log during the incident. 

This article tells how check your threads and adjust as needed: Adjust Apache threads on IBM i to improve performance

Disable code optimization

Code optimization can very occasionally cause a problem with some script that can be very hard to trace.  Disabling optimization is a quick way to test for optimization issues.  For IBM i customers, we find that disabling optimization often has no noticeable effect on performance.  Byte code caching remains in effect, and caching is where most of the performance benefit is found.  It is also a good idea to set consistency checking on. 

This article tells how to do it:   Disable optimization for stability issues

Block search bots and other intrusive traffic (Denial of Service)

...

The XMLSERVICE Toolkit provides a way to call RPG programs, run CL commands, and do other things in the IBM i native Library/Object environment.  If one of these processes halts waiting for a reply to a message, the child process will sit forever in a CNDW status, until the message is answered and the job is allowed to complete.  If that job happens to be running stateful, then no other stateful jobs will process while that one message waits for a response.  If that happens, and if the majority of the pages on the site are using the XMLSERVICE Toolkit, it can seem like the whole site has stopped responding.  Of course, with error monitoring there is no real reason why any program needs to halt like this, and if it does, there is no real reason that the message has to sit unanswered in the queue.  IBM i provides tools to manage all of this.  But it happens sometimes anyway, so this is something you can check for: No XML Toolkit jobs will run if one program is stuck in MSGW status 

Another way this can cause an apparent site freeze is that some web browsers will resend a request if no response in a certain amount of time, so the user might just be waiting patiently, or may turn away from the computer to do something else, and their browser can sit and repetitively call the same script with the same post variables, that will cause the same RPG program to be called with the same parameters and crash again at the same statement.  This can create a freeze even when using stateless jobs.

...

This can get pretty ugly as many users get frustrated by the long waits and start hitting their submit buttons over and over, trying to get their web page to respond.  This article offers some discussion and an idea about how to find ways to disallow multiple submissions from the form: IBM i Apache error logs show: [3425] A remote host refused an attempted connect operationInstead of restarting Apache, try a Reset

When your site stops responding, you can almost always get it started back up by restarting Apache (but please not before you collect the information mentioned in the first section of this article!)  However, you have a menu option available to do a Reset.  This will stop Apache, rebuild your Zend IPCs, and then start Apache back up.  If you do happen to try restarting Apache, and your site will not come back up, try the Reset.  If the Reset works when a simple restart will not, that will be important information to know.  It could point to a problem in caching or other Zend components. 

Here is how to do the Reset: Reset Zend Server for IBM i

Upgrade Zend Server to the latest release

...

Bringing your IBM i PTFs up to date insures that any older known bugs that might contribute to your issue will be fixed. 

Here is how to make sure you are up to date:  Verify your IBM i PTF groups are current

Content by Label
showLabelsfalse
max5
spacesSUPPORT
showSpacefalse
sortmodified
reversetrue
typepage
cqllabel in ("apache","freeze") and type = "page" and space = "SUPPORT" and parent = "41618611"
labelszend zend-server ibmi ibm-i apache freeze

...