Siebel server start-up Error

There are several scenarios where siebel servers are not coming up or siebel services are not running properly.

These scenarios are tested on UNIX/SUN Solaris Servers.

1.            Siebel services are not stopped properly.

2.            Mainwin Processes are stopped.

3.            Unable to allocate shared memory.

4.            Problem with Database connectivity.

5.            And many more scenarios.

Scenario 1:- Siebel services are not stopped properly

If status of a Siebel Server is something like below, Then it seems that Siebel Services are not stopped properly.

Siebel Server "SBLSRVR1" (Enterprise "SBLENTERPRISE")

started at Mon Feb 27 06:04:08 2012, pid: 18732 (Service process is not running), autostart: no

So Siebel Server won't come up as usual, in this scenario we have to follow some steps mentioned below.

1. Stop Siebel Server

 Run stop_server "SERVER NAME"/ stop_server all

2.  Kill Siebel processes still running 

Run ps -ef | grep "siebel owner on server" -- it will list all the siebel processes still running on Siebel Server.

kill -9 PID --It will kill processes with PID given.

3. Delete All the files created at startup, As server has not been stopped automatically. Some files  may not be deleted.

Go to $SIEBEL_ROOT/bin Directory---Delete/Move diccache.dat  file.

Go to $SIEBEL_ROOT/sys Directory---Delete/Move osdf.ENTERPRISENAME.SERVERNAME  file.

Go to $SIEBEL_ROOT/admin Directory---Delete/Move ENTERPRISENEM.SERVERNAME.shm file.

4. Start Siebel Server

 Run start_server "SERVER NAME"/ start_server all

5. You have done with all, Hopefully Your server will be up after sometime.

Check Points

Before do all the steps above make sure that gateway services are up and running fine and you are able to connect to server manager.

                             Scenario 2:- Mainwin Processes are stopped

If Siebel server has been stopped by crash or any other reason then we need to kill processes still running after stopping server.  In that case we grep all the processes and kill them manually. In this process there is a chance that by accidentally we had killed mainwin processes too or there are other reason sue to which mainwin process has been stopped.

In this scenario, we have to manually start mainwin process .

For this we have below mentioned steps to follow

Go to prompt and check status of mainwin process by running below command

$mwadm status

Run-time mode       : professional

Core Services status: stopped

Core Data Directory : USER_HOME/.mw/SERVER_NAME

RPC Port            : dynamic

Then we goto directory

$cd $SIEBEL_ROOT/mw/.mw/core_data/SERVER_NAME/.mw

Delete .bin files from this directory as these files get created once we start main win processes.

bash-3.00$ ls

hkcu.bin         hklm_sunos5.bin

bash-3.00$ del hkcu.bin         hklm_sunos5.bin

Now delete all temporary files as mentioned in previous step

Now start mainwin processes.

bash-3.00$ mwadm start

MainWin System Core Services version 1.0.1 started.

bash-3.00$ mwadm status

Run-time mode       : professional

Core Services status: running

Core Data Directory : /export/home/siebsupp/.mw/gnncrm34

RPC Port            : dynamic

For confirmation check status of processes run by current user using

bash-3.00$ ps -ef | grep siebsupp

siebsupp 11559     1   0 07:21:56 ?           0:00 watchdog -respond 4 /siebapp/siebel/siebsrvr/mw/.mw/core_data//gnncrm34/watchdo

siebsupp 11561 11559   0 07:21:57 ?           0:00 mwrpcss -1 0our desired process is running here.

siebsupp 11613  9596   0 07:22:05 pts/2       0:00 grep siebsupp

siebsupp  9043  8994   0 06:44:21 pts/2       0:00 -sh

siebsupp  8994  8993   0 06:43:52 ?           0:01 /usr/lib/ssh/sshd

siebsupp 11560 11559   0 07:21:56 ?           0:00 regss 5

siebsupp 11612  9596   0 07:22:05 pts/2       0:00 ps –ef

Now start Siebel server by using start_server all.

Hopefully server will start normally.

                            Scenario 3:- Unable to allocate shared memory

At the time of bringing up Siebel server , services will start but later server shows status of service process not running.

Siebel Server "SBLSRVR1" (Enterprise "SBLENTERPRISE")

started at Mon Feb 27 06:04:08 2012, pid: 18732 (Service process is not running), autostart: no

when we check siebsrvr.log file located at $SIEBEL_ROOT/logs, we get below lines in log file.

GenericLog    GenericError    1    0    2013-08-03 06:30:53    (sissrvr.cpp (2979) err=2000026 sys=17) SBL-SVR-00026: Unable to allocate shared memory.

This error is caused by Siebel processes not stopped in a proper manner and due to this .shm files get corrupt. To get this problem resolved we have to delete .shm file located at $SIEBEL_ROOT/admin Directory.

Other than this we have to follow some more steps to make sure this process will not appear again.

Run below command

bash-3.2$ ipcs -b

IPC status from <running system> as of Thu Aug 15 19:03:04 IST 2013

T         ID      KEY        MODE        OWNER    GROUP QBYTES

Message Queues:

T         ID      KEY        MODE        OWNER    GROUP      SEGSZ

Shared Memory:

m  134217766   0x11f43a1  --rw-rw-r--     root     root       4192

m  587202597   0x11f4fcf  --rw-rw-r--     root     root       4192

m 1358954532   0x11f4fc9  --rw-rw-r--     root     root       4192

m 1358954531   0x11f4efb  --rw-rw-r--     root     root       4192

m 1358954530   0x11f4fc1  --rw-rw-r--     root     root       4192

m          9   0x11f4fb7  --rw-rw-r--     root     root       4192

m          7   0x11f42e4  --rw-rw-r--     root     root       4192

m          6   0x11f420e  --rw-rw-r--     root     root       4192

m          2   0x311f40d3 --rw-rw-rw-     root     root     131164

m          1   0xc6629c9  --rw-r-----     root     root     609228

m          0   0x6347849  --rw-rw-rw-     root     root      65544

T         ID      KEY        MODE        OWNER    GROUP NSEMS

Semaphores:

s  251658267   0x11f4140  --ra-ra-r--     root     root     1

s  520093722   0x11f4fc3  --ra-ra-r--     root     root     1

s  520093720   0x11f4fc4  --ra-ra-r--     root     root     1

s  251658263   0x11f4e64  --ra-ra-r--     root     root     1

s  251658262   0x11f4fb2  --ra-ra-r--     root     root     1

s          7   0x11f429b  --ra-ra-r--     root     root     1

s          6   0x11f4004  --ra-ra-r--     root     root     1

s          2   0x11f42a4  --ra-ra-r--     root     root     1

s          1   0x11f42a3  --ra-ra-r--     root     root     1

s          0   0x311f40d3 --ra-ra-ra-     root     root     5

This will display all the share memory & semaphore details for the particular server.

We have to kill all these semaphores and release shared memory by using below command.

$ipcrm –m <Shared_memory_segment_ID>

$ipcrm –s <semaphore_ID>

Now delete all other temporary files as mentioned earlier.

You are almost done. Now run start_server command.

                          Scenario 4:- Problem with Database connectivity.

In case Siebel servers are not coming up , there is a chance of connectivity issue with database servers . For resolution of this type of problem have to follow below steps.

$obbcsql /u DBUSERNAME(SIEBEL) /p PASSWORD /s DSN_NAME

If you are not able to connect

>> Check that DB servers are up and running fine.

>>Your DSN Entries are correct

>> Your license file is correct

>>Check .odbc.ini file for garbage entries in case of any problem with license file.