JIRA LDAP incremental synchronization is failing with "different number of guids and users"

[DI2E Licensing]

Our once every 15 minute incremental synch with LDAP is failing.  

Full synchronization works fine.

The incremental sync with our other Atlassian tools is working fine.

Any suggestions?

2014-10-15 13:36:18,230 atlassian-scheduler-quartz1.clustered_Worker-1 ERROR ServiceRunner     [atlassian.crowd.directory.DbCachingRemoteDirectory] Incremental synchronisation was unexpectedly interrupted, falling back to a full synchronisation
com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresherIncSyncException: Cache returned different number of guids and users (possible reason is overlapping guids in cache, most likely null/empty values).
        at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.validateAndReturnGuidsToAddAndDelete(UsnChangedCacheRefresher.java:282)
        at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.synchroniseUserChangesGuid(UsnChangedCacheRefresher.java:315)
        at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.synchroniseUserChanges(UsnChangedCacheRefresher.java:375)
        at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.synchroniseChanges(UsnChangedCacheRefresher.java:120)
        at com.atlassian.crowd.directory.DbCachingRemoteDirectory.synchroniseCache(DbCachingRemoteDirectory.java:999)
        at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.synchronise(DirectorySynchroniserImpl.java:75)
        at com.atlassian.jira.crowd.embedded.JiraDirectorySynchroniser.synchronizeDirectory(JiraDirectorySynchroniser.java:96)
        at com.atlassian.jira.crowd.embedded.JiraDirectorySynchroniser.runJob(JiraDirectorySynchroniser.java:60)
        at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:135)
        at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:101)
        at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:80)
        at com.atlassian.scheduler.quartz1.Quartz1Job.execute(Quartz1Job.java:32)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:223)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
{code}
This is recorded both in {{/opt/atlassian/jira/logs/catalina.out}} and {{/opt/atlassian/jira-data/log/atlassian-jira.log}

[Tiago Comasseto]

The incremental synchronization is a feature that should only work with MS AD, I assume you're using AD right?

Also, the message "Cache returned different number of guids and users" leads me to believe it's cache issue, do you see the same problem if you create a new directory in JIRA to the same LDAP server? 

It's always best doing this kind of testing on an staging instance, instead of production.

Cheers

[DI2E Licensing]

Yes. MS AD. Interestingly enough our staging server links to the same AD and it's fine. If the cache is problem, will stopping and restarting JIRA fix the problem>

[Tiago Comasseto]

You may just disable the directory, enable it again and sync, that would be the same as a restart. Cheers

[DI2E Licensing]

Nuts! Because we already did that and it didn't fix it.

[jose_castro]

@DI2E SysAdmin Do any active directory users have issues logging into JIRA? Also are any users part of a sub-domain?