EM12c : Login to GUI with the correct password causes authentication failure

19 May EM12c : Login to GUI with the correct password causes authentication failure

So the other day I was trying to login to my EM12c R4 environment with the SSA_ADMINISTRATOR user, and I got the error :

Authentication failed. If problem persists, contact your system administrator.

I was quite sure that the password that I had was correct, so I tried with the SYSMAN user and had the same error, but still wanted to verify that I had the correct password , so I tried with the SYSMAN user to login to the repository database, and was successful, so I know something was wrong there.

SQL> connect sysman/
Enter password:
Connected.

So I went to the<gc_inst>/em/EMGC_OMS1/sysman/log/emoms.log and saw the following error

2015-05-18 21:22:06,103 [[ACTIVE] ExecuteThread: '15' for queue: 'weblogic.kernel.Default (self-tuning)'] ERROR audit.AuditManager auditLog.368 - Could not Log audit data, Error:java.sql.SQLException: ORA-14400: inserted partition key does not map to any partition
ORA-06512: at "SYSMAN.MGMT_AUDIT", line 492
ORA-06512: at "SYSMAN.MGMT_AUDIT", line 406
ORA-06512: at line 1

Which led me to believe that the JOB_QUEUE_PROCESSES was set to 0, but it wasn’t the case, it was set to 50, but it is actually not a correct limit, so I bumped it up to 1000 as per documentation and tried to rerun the EM12c repository DBMS Scheduler jobs as per the documentation in 1498456.1

SQL> show parameter JOB_QUEUE_PROCESSES

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
job_queue_processes integer 50

SQL> alter system set JOB_QUEUE_PROCESSES=1000 scope = both;

System altered.

SQL> show parameter both
SQL> show parameter job

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
job_queue_processes integer 1000

SQL> connect / as sysdba
Connected.
SQL> alter system set job_queue_processes = 0;

System altered.

SQL> connect sysman/*****
Connected.
SQL> exec emd_maintenance.remove_em_dbms_jobs;

PL/SQL procedure successfully completed.

SQL> exec gc_interval_partition_mgr.partition_maintenance;

PL/SQL procedure successfully completed.

SQL> @$OMS_HOME/sysman/admin/emdrep/sql/core/latest/admin/admin_recompile_invalid.sql SYSMAN
old 11: AND owner = upper('&RECOMPILE_REPOS_USER')
new 11: AND owner = upper('SYSMAN')
old 26: dbms_utility.compile_schema(upper('&RECOMPILE_REPOS_USER'),FALSE);
new 26: dbms_utility.compile_schema(upper('SYSMAN'),FALSE);
old 41: WHERE owner = upper('&RECOMPILE_REPOS_USER')
new 41: WHERE owner = upper('SYSMAN')
old 84: AND owner = upper('&RECOMPILE_REPOS_USER')
new 84: AND owner = upper('SYSMAN')
old 104: AND ds.table_owner = upper('&RECOMPILE_REPOS_USER')
new 104: AND ds.table_owner = upper('SYSMAN')

PL/SQL procedure successfully completed.
PL/SQL procedure successfully completed.

SQL> connect / as sysdba
Connected.
SQL> alter system set job_queue_processes = 1000;

System altered.

SQL> connect sysman/
Enter password:
Connected.
SQL> exec emd_maintenance.submit_em_dbms_jobs;

PL/SQL procedure successfully completed.

SQL> commit;

Commit complete.

After this I bounced the OMS , but still kept getting the same error, though it fixed the scheduler jobs, I was now seeing the following error in the emoms.log

2015-05-18 22:29:09,573 [[ACTIVE] ExecuteThread: '15' for queue: 'weblogic.kernel.Default (self-tuning)'] WARN auth.EMRepLoginFilter doFilter.450 - InvalidEMUserException caught in EMRepLoginFilter: Failed to login using repository authentication for user: SSA_ADMIN
oracle.sysman.emSDK.sec.auth.InvalidEMUserException: Failed to login using repository authentication for user: SSA_ADMIN

So what I did was an update to the SYSMAN.MGMT_AUDIT_MASTER table and run the procedure MGMT_AUDIT_ADMIN.ADD_AUDIT_PARTITION as was stated in document id 1493151.1

oracle $ sqlplus

Enter user-name: sysman
Enter password:

SQL> update mgmt_audit_master set prepopulate_days=5 where prepopulate_days is null;

1 rows updated.

SQL> select count(1) from mgmt_audit_master where prepopulate_days is null;

COUNT(1)
----------
0

SQL> exec mgmt_audit_admin.add_audit_partition;

PL/SQL procedure successfully completed.

SQL> commit;

Commit complete.

Once I did this, I was able to login with all my EM12c administrators without any issues.

[email protected] [emrep] /home/oracle
oracle $ emcli login -username=ssa_admin
Enter password

Login successful

Conclusion

Even though the JOB_QUEUE_PROCESSES was not set to 0, it was the cause that it was failing as it was a low value for this parameter, so just be careful when setting up this parameter , be sure to follow the latest installation guidelines.