Amicuk Programming Answers

Child process dies, nfs locks not released, webserver hangs...

2015-12-10   Views:0


Hi, I have Sun One 6.1 sp 11 on a solaris 10 ldom. The server is configured to write logs access and error to /logs which is an NFS mount to a separate solaris 10 box. The logging to an NFS mount is a business requirement. Sun JWS is configured to ha

I have Sun One 6.1 sp 11 on a solaris 10 ldom.
The server is configured to write logs access and error to /logs which is an NFS mount to a separate solaris 10 box. The logging to an NFS mount is a business requirement.
Sun JWS is configured to have two httpd processes and the watchdog to restart them if one should fail.
Every now and then, about once a day (it varies), one of the child processes will die with messages like this in the error log: (1949 is the wdog pid)
[09/Dec/2009:14:19:06] failure ( 1949): CORE3107: Child process closed admin channel
[09/Dec/2009:14:19:06] fine ( 1949): CORE3061: signal_handler_thread: received signal 18
[09/Dec/2009:14:19:06] fine ( 1949): CORE3049: Primordial process detected child 1950 died: status 37
[09/Dec/2009:14:19:06] fine ( 1949): CORE3050: Is our child, will spawn replacement
[09/Dec/2009:14:19:06] fine ( 1949): CORE3062: Unlinking of /tmp/https-wv2-819e4c2d/.cgistub_1950 returned -1
[09/Dec/2009:14:19:06] fine ( 1949): CORE3047: Server spawned worker process 2011
[09/Dec/2009:14:19:06] fine ( 2011): HTTP5169: User authentication cache entries expire in 120 seconds.
[09/Dec/2009:14:19:06] fine ( 2011): HTTP5170: User authentication cache holds 200 users
[09/Dec/2009:14:19:06] fine ( 2011): HTTP5171: Up to 4 groups are cached for each cached user.
[09/Dec/2009:14:19:06] fine ( 2011): HTTP4207: file cache module initialized (API versions 2 through 2)
[09/Dec/2009:14:19:06] fine ( 2011): HTTP4302: file cache has been initialized
[09/Dec/2009:14:19:06] fine ( 2011): HTTP3066: MaxKeepAliveConnections set to 256
[09/Dec/2009:14:19:06] fine ( 2011): Installed configuration 1
[09/Dec/2009:14:19:06] fine ( 2011): HTTP4193: flex-rotate-init: rotate start time is 0h, 0m
At this point the webserver will not respond. The processes (2*httpd, 1*wdog) are running but do not respond. The access log shows a weird lock with output from pfiles:
21: S_IFREG mode:0777 dev:340,10 ino:34988 uid:111 gid:102 size:0
advisory write lock set by system 0x2 process 280
which I think means the new http process is waiting for the lock to be released, but the lock is never freed.
But what I'm really curious about is why the process is dying in the first place. Anyone seen "status 37" before, or know where I can look it up? I couln't google up any reference on what it might mean...
any help appreciated

The replay answer
I found the following in :
In this situation, the SIGLOST signal is posted to the process. The default action for the SIGLOST signal is to terminate the process.
For you to recover from this state, you must restart any applications that had files open at the time of the failure. Note that the following can occur.
- Some processes that did not reopen the file could receive I/O errors.
- Other processes that did reopen the file, or performed the open operation after the recovery failure, are able to access the file without any problems.
Thus, some processes can access a particular file while other processes cannot.
Edited by: Arvind_Srinivasan on Dec 10, 2009 12:33 AM

Go to See the other 5 answers

Child process dies, nfs locks not released, webserver hangs...

Category:DefaultRelease time:2015-12-10Views:130

Hi, I have Sun One 6.1 sp 11 on a solaris 10 ldom. The server is configured to write logs access and error to /logs which is an NFS mount to a separate solaris 10 box. The logging to an NFS mount is a business requirement. Sun JWS is configured to ha[More]

"CORE3107: Child process closed admin channel" in sun webserver 6.1

Category:DefaultRelease time:-0001-11-30Views:130

Hi, Does anyone know of the following error in the Sun Java Webserver 6.1 logs: =========================================== [08/Mar/2010:20:41:12] failure (27868): CORE3107: Child process closed admin channel [08/Mar/2010:20:41:12] info (13932): HTTP[More]

Coldfusion 10 Enterprise with Tomcat + mod_jk and Apache2 experiencing child process hangups

Category:DefaultRelease time:2015-10-11Views:130

I am experiencing the most bizarre thing that so far I am unable to reproduce with my own visits to the site. After restarting Apache2 my cacti graphs show that the child processes increment consistently over the course of a day without dropping back[More]

Apache POST flex2gateway never closes or times out, reaches max child processes

Category:DefaultRelease time:-0001-11-30Views:130

We have been trying to pass an external PCI scan, and noticed some server lockups after starting a scan.  We are scanning a couple hundred IP addresses, which all resolve to the same servers.  The scans are actively looking for vulnerabilities on the[More]

BPEL child process issue

Category:DefaultRelease time:-0001-11-30Views:130

Problem Description: Parent process invoking more than 10 concurrent child processes with non-blocking invoke = true. All the child processes are not invoked at the same time. Some of the child processes are waiting for others to complete and then in[More]

Signal for non-child process death

Category:DefaultRelease time:-0001-11-30Views:130

I am porting an NT system to Solaris. One process (HM) is responsible for starting groups of server processes, monitoring for death of a process, stopping/restarting/recovering the group. I know how to port this using fork/exec to start processes and[More]

SFTP related processes do not die if parent process dies.

Category:DefaultRelease time:-0001-11-30Views:130

Java Version: 1.4.2_05 JRE: build 1.4.2_05-b04. Running on Solaris 9. Problem: If the process using the SFTP API dies, the children processes created by the SFTP API do not end. A side effect of this is that all the sockets the orginal parent had rem[More]

SunMC - Process is forking and reaping child processes. What's that?

Category:DefaultRelease time:-0001-11-30Views:130

Hey folks, Im really new to the sysadmin world, and I think maybe my company really didn't think things well when they've decided to put me doing this, hehehe. I work with a general queue for which my team receives tickets with different kind of prob[More]

What is "child processes" if using OS Fetchlet with "em_metric_timeout"?

Category:DefaultRelease time:-0001-11-30Views:130

On page 330 of document <Oracle® Enterprise Manager Extensibility Guide 10g Release 5 ( B40007-02>, there are a description of using "em_metric_timeout" with OS Fetchlet as below, Parameter : em_metric_timeout Type : integer Descr[More]

Anyone know how to keep a child Process from closing when the main Applicaiton is closed

Category:DefaultRelease time:-0001-11-30Views:130

  I have a Web based application that needs to use an older version of Java to run it properly . I have been able to sequence this and use a shortcut to call on Iexplorer.exe to open the browser in bubble and have the old version of Java run in the s[More]

HTTP Server process dies

Category:DefaultRelease time:-0001-11-30Views:130

Hi We have got Oracle9ias release 9.0.3 with Webcache running on a 4 processor 4GigRam server. JDK is 1.3.1_02. The HTPP server process dies after a while. The entry in the imp.log reveals this: 03/11/10 20:20:34 ipm_hc_get_headers: failed to get rep[More]

Dear All, I have created one G/l acount without selecting the open item management check box, after that i have posted some transactions to this account. now when i try to select the check box it is not allowing me and it is saying that first tranfer [More]
Hello experts,   I got an error while creating a process order.System allowing to create a process order after saving througing an error " Express document "Update was terminated" received from author "GLP Developer" 'Error Info.. [More]
Hi All , I have a proxy service polling for a file . I have setup the service on test environment , where it works fine . When i deploy the same code with only change is the Managed server target for polling , as the production server is clustered . [More]
I have two CMD windows open runnig a LabVew launched commandline program It is two instances of the iperf program and I found this searching here  taskkill /F /IM iperf.exe That works but it kill both windows. I take it I need to find the PID on the [More]
Hi, Is there any standard report which can show me workitem status and user last logon in sap? can anyone suggest me how do i get these two in single report. ThanksHi, Please check the tcode RSUSR200. It will give you the details of last login of use [More]
Hi Soundtrack pro seems to have only 16bit AAF export function. Is there any workaround to be able to export a multitrack session in higher bit (24bit industry standard) resolution, in AAF or OMF format? I know Final Cut has 24bit AAF export funcion, [More]
Hello, I've installed windwos about a year ago on my Macbook Pro via BootCamp. Now I wish to delete it and I feagured out that the only way to do it is also via BootCamp. When I open BootCamp there is only one option available: "Download the latest A [More]
How, (using CVI, NI-IMAQ and IMAQ Vision) can I duplicate the Grab function seen in MAX; in other words, I want to display images at the frame rate of my camera (30 fps). I've come across several examples on this site, the best of which seemed to be [More]
hi! i have a laserjet 2200 dtn monster here since some time. problem: if tray2 is empty, and tray3 full: xp/vista: prints fine! autoselects tray3 and goes on os 10.4: warning light starts blinking, and i have to press the button on the printer to sta [More]
My Macbook works great. It connects to my wifi network everytime. I just turned it on and it will not connect. Every other computer in my house is connected. Is it broke??? or is there a fix? Thanks anyone for your help. SteveMaybe my story can help [More]