SBL-SMI-00027: Internal: Invalid request received (%1)

Applies to:

Siebel High Tech and Manufacturing - Version: 8.0.0.9 SIA[20433] and later   [Release: V8 and later ]
Information in this document applies to any platform.

Symptoms

On any application server of the customer's Production environment a period of time can occur where hundreds of Server Manager processes are spawned generating logs for each that have the error SBL-SMI-00027: Internal: Invalid request received (703). This behavior is not continual and can happen on one server, while server manager commands seem to be running fine on the other servers.

Cause

The issue was caused by a third party monitoring tool sending too many requests within a short time interval.

Solution

The customer worked with their third party vendor to increase the time between server manager commands.

Applies to:

Siebel Communications Call Center - Version: 8.1.1.2 SIA[21215] and later   [Release: V8 and later ]
Information in this document applies to any platform.

Symptoms

Not able to access siebel Administration screens including Administration - Server Management and Administration - Server configuration.
When login to srvrmgr console utility it takes a long time to connect.

In the enterprise server log file a lot of ServerMgr processes are started and terminated with error:

ServerLog ProcessCreate 1 0000163c4f159064:0 2012-01-17 13:51:35 Created server process (OS pid = 843958 ) for ServerMgr
ServerLog ProcessExit 1 0000163d4f159064:0 2012-01-17 13:51:44 ServerMgr 843958 SBL-SMI-00027 Process 843958 exited with error - Internal: Invalid request received (%1)

In the enterprise server log file there was originally a ServerMgr session exited with a security error:

ServerLog ProcessCreate 1 0000163c4f159064:0 2012-01-17 15:48:05 Created server process (OS pid = 843958 ) for ServerMgr
ServerLog ProcessExit 1 0000163d4f159064:0 2012-01-17 15:48:08 ServerMgr 843958 SBL-SEC-10009 Process 843958 exited with error - The requested operation has been rejected by the authentication server (such as DBMS or LDAP server) due to insufficient rights. It is most likely that Siebel authentication subsystem or the authentication server is not configured properly. Please contact your system administrator for assistance.

Cause

Environment was configured to use single sign on. The sadmin user had enough privileges to access the server management screens. However the database user configured for this siebel user in the SSO system had not enough rights to access the gateway.

As a result the initial ServerMgr session was initialized but it exited with SBL-SEC-10009 error. However the Administration screen already received a session id and was trying to use it to perform queries on running servers.
These subsequent queries are not finding the ServerMgr session started and they create a new session to process the request. However because the session cookie is passed these new sessions were exiting with the error "SBL-SMI-00027 Process OSPID exited with error - Internal: Invalid request received (%1)"
After this Administration screen is trying for ever to fire the same request entering a loop.

The high number of starting and failing ServerMgr sessions were overloading the gateway resulting in a overall system slowdown.

Solution

Added Siebel Administrator privilege to the database user that is setup in single sign on system (this was a valid siebel user).
This resulted in a successful ServerMgr session start and access to the Administration screen was restored.