CHANGES
CAUSE
Bug 15835734 - RBAL TRACE FILE KEEP ON GROWING DUE TO REPEATED MESSAGES.
The following justifies how the issue is related to this specific customer:
*** 2013-05-28 16:03:37.350
2013-05-28 16:03:37.349: [ default]failed to initialize skgp context
2013-05-28 16:03:37.350: [ default]slos op : sslssreghdlr
2013-05-28 16:03:37.350: [ default]slos dep : Error 0 (0)
2013-05-28 16:03:37.350: [ default]slos loc : sskgpinit1
2013-05-28 16:03:37.350: [ default]slos info:
[ CLWAL]clsw_Initialize: OLR initlevel [30000]
2013-05-28 16:03:37.350: [ default]a_init: Unable to get log name. Retval:[-4]
2013-05-28 16:03:37.366: [ GPNP]clsgpnp_dbmsGetItem_profile: [at clsgpnp_dbms.c:313] Result: (0) CLSGPNP_OK. got ASM-Profile.DiscoveryString='o/*/*'
This is explained in the following bug:
Bug 15835734 - RBAL TRACE FILE KEEP ON GROWING DUE TO REPEATED MESSAGES
On 11.2.0.3 the ASM rebal trace file keep on increasing with the below error messages.
The ASM rbal trace file has the following repeated messages :
*** 2012-10-18 14:12:00.574
2012-10-18 14:12:00.574: [ default]failed to initialize skgp context
2012-10-18 14:12:00.574: [ default]slos op : sslssreghdlr
2012-10-18 14:12:00.574: [ default]slos dep : Error 0 (0)
2012-10-18 14:12:00.574: [ default]slos loc : sskgpinit1
2012-10-18 14:12:00.574: [ default]slos info:
[ CLWAL]clsw_Initialize: OLR initlevel [30000]
2012-10-18 14:12:00.576: [ default]a_init: Unable to get log name. Retval:[-4]
2012-10-18 14:12:00.621: [ GPNP]clsgpnp_dbmsGetItem_profile: [at clsgpnp_dbms.c:313] Result: (0) CLSGPNP_OK. got ASM-Profile.DiscoveryString='++no-value-at-profile-creation--never-updated-through-ASM++'
SOLUTION
Per non-bug 15835734:
The problem is that every time the the GPnP profile is queried to find out the discovery string for ASM, that trace information is written in the RBAL trace file for debugging purposes.
Therefore every time a query on v$asm_disk or v$asm_diskgroup is executed it will cause the ASM to query the GPnP daemon to find the discovery string for available disks and that's when the debug information is dumped into the RBAL trace.
Since you have two crontab jobs querying those views that's the reason the RBAL trace file grows so fast.
You can change those queries to use v$asm_disk_stat and v$asm_diskgroup_stat instead which do not require asking GPnP daemon for the discovery string and that way RBAL trace file will not be updated due to these cron jobs. If there is any monitoring tool like BMC that uses v$asm_disk, v$asm_diskgroup, then contact BMC for a possibility of using v$asm_disk_stat and v$asm_diskgroup_stat.
You will probably still see those same messages in the RBAL trace file, but not that frequent, and only when a process asks GPnP daemon to find the discovery string.
REFERENCES
BUG:15835734 - RBAL TRACE FILE KEEP ON GROWING DUE TO REPEATED MESSAGES