TEST_CASE_NAME to long causing exception

[Magnus Granander]

Hi!

Today we have been starting to see these errors:

org.springframework.dao.DataIntegrityViolationException: Hibernate operation: could not insert: [com.atlassian.bamboo.resultsummary.tests.TestCaseImpl#34577478]; SQL []; Data truncation: Data too long for column 'TEST_CASE_NAME' at row 1; nested exception is java.sql.BatchUpdateException: Data truncation: Data too long for column 'TEST_CASE_NAME' at row 1
Caused by: java.sql.BatchUpdateException: Data truncation: Data too long for column 'TEST_CASE_NAME' at row 1
	at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:2054)
	at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1467)
	at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:1723)
	at net.sf.hibernate.impl.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:54)
	at net.sf.hibernate.impl.BatchingBatcher.addToBatch(BatchingBatcher.java:34)
	at net.sf.hibernate.persister.EntityPersister.insert(EntityPersister.java:462)
	at net.sf.hibernate.persister.EntityPersister.insert(EntityPersister.java:436)
	at net.sf.hibernate.impl.ScheduledInsertion.execute(ScheduledInsertion.java:37)
	at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java:2451)
	at net.sf.hibernate.impl.SessionImpl.executeAll(SessionImpl.java:2437)
	at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java:2394)
	at net.sf.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:1823)
	at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:3654)
	at net.sf.hibernate.impl.CriteriaImpl.list(CriteriaImpl.java:238)
	at net.sf.hibernate.impl.CriteriaImpl.uniqueResult(CriteriaImpl.java:385)
	at com.atlassian.bamboo.resultsummary.BuildResultsSummaryHibernateDao$30.doInHibernate(BuildResultsSummaryHibernateDao.java:1141)
	at org.springframework.orm.hibernate.HibernateTemplate.execute(HibernateTemplate.java:370)
	at org.springframework.orm.hibernate.HibernateTemplate.execute(HibernateTemplate.java:337)
	at com.atlassian.bamboo.resultsummary.BuildResultsSummaryHibernateDao.findFirstResultAfter(BuildResultsSummaryHibernateDao.java:1137)
	at com.atlassian.bamboo.resultsummary.BuildResultsSummaryManagerImpl.findFirstResultAfter(BuildResultsSummaryManagerImpl.java:1175)
	at sun.reflect.GeneratedMethodAccessor1081.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
	at com.atlassian.bamboo.security.acegi.intercept.aopalliance.AuthorityOverrideMethodSecurityInterceptor.invoke(AuthorityOverrideMethodSecurityInterceptor.java:29)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at sun.proxy.$Proxy40.findFirstResultAfter(Unknown Source)
	at com.atlassian.bamboo.plan.PlanStatePersisterImpl.checkAndUpdateDeltaStatesOfFollowingBuild(PlanStatePersisterImpl.java:232)
	at com.atlassian.bamboo.plan.PlanStatePersisterImpl.saveFinishedBuildResult(PlanStatePersisterImpl.java:108)
	at sun.reflect.GeneratedMethodAccessor1124.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
	at com.atlassian.bamboo.author.AuthorCreatorServiceInterceptor.invoke(AuthorCreatorServiceInterceptor.java:45)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at sun.proxy.$Proxy56.saveFinishedBuildResult(Unknown Source)
	at com.atlassian.bamboo.plan.PlanStatePersisterServiceImpl$1.call(PlanStatePersisterServiceImpl.java:40)
	at com.atlassian.bamboo.plan.PlanStatePersisterServiceImpl$1.call(PlanStatePersisterServiceImpl.java:37)
	at com.atlassian.util.concurrent.ManagedLocks$ManagedLockImpl.withLock(ManagedLocks.java:324)
	at com.atlassian.bamboo.plan.PlanStatePersisterServiceImpl.saveFinishedBuildResult(PlanStatePersisterServiceImpl.java:36)
	at com.atlassian.bamboo.v2.build.agent.LocalBuildResultProcessor.processBuildResult(LocalBuildResultProcessor.java:135)
	at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.processBuildResult(BuildAgentControllerImpl.java:168)
	at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:131)
	at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:111)
	at com.atlassian.bamboo.build.pipeline.concurrent.NamedThreadFactory$2.run(NamedThreadFactory.java:52)
	at java.lang.Thread.run(Thread.java:679)
Caused by: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too long for column 'TEST_CASE_NAME' at row 1
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4094)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4028)
	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2490)
	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2651)
	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2734)
	at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
	at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2458)
	at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:2006)
	... 55 more

The job in question seems to finish OK, according to the logs. We've done some investigations but have now got stuck. The message doesn't include the test case name causing the exception to be thrown and that makes it hard to find exactly what is failing. We also investigated the MySQL log directly, trying to find long test case names. The longest we found was just above 200 chars long and that should not be a problem since the DB column is 4000 chars (yes, we checked that to).

We know you can enable more logging in Bamboo, but are unsure what to enable.

Can someone help us further, please?

[Magnus Granander · Accepted solution]

So I've done som further investigations and the result XML file that TestNG produces have no test case names that are to long. It seems the TestNG Bamboo plugin constructs the test case name from the method signature and the parameters, and is not taking into account the column width of TEST_CASE_NAME. To me this is a bug in the plugin.

[ArmenA]

Thanks for your investigation, Magnus. There was already a Improvement request to increase the size of the mentioned column, I changed it to be a bug based on the current Answers post. The bug will be fixed as per our policy.

Armen

[Magnus Granander]

It seems to be TestNG that causes this behaviour when test methods use DataProviders. TestNG uses toString() to generate a methods signature and if one or more objects give a long string we will be in trouble.

Regarding turning on more logging, it didn't help. We are looking into having MySQL auto truncate long names to identify the troublemakers :)

[Gary Donovan]

Assuming that the root cause really is a test case name that is too long, it is probably worthwhile to refactor your tests so that the names are a more manageable size.

For example, some of our test cases have automatically generated names which are lengthy and difficult to read (the test cases are generated by mapping a test function onto a list of input data, and the test case name comes from serialising each set of input data!). This makes them hard to work with and reference, so in some instances we have replaced them with standalone test cases that have a more meaningful name (eg. "test_poetry_charge_of_light_brigade" rather than "test_poetry_half_a_league_half_a_league_half_a_league_onward_...")

Hope that helps.

[ArmenA]

Shut down your Bamboo instance, enable detaild SQL logging, bring Bamboo instance up and try to reproduce the problem. Bamboo will show what exact text it is trying to insert into the TEST_CASE_NAME column (varchar(4000)) of TEST_CASE table. In case you really need that long test case name to be stored in the DB (which I don't think will be the case), you can try to increase that column size:

ALTER TABLE TEST_CASE MODIFY COLUMN TEST_CASE_NAME VARCHAR(8000)