The Ultimate Guide to High Availability with MariaDB

Oozie error after upgrading to 2.4.1

The mariadb connector was upgraded from 1.1.9 to 2.4.1 for oozie (connected to a mysql cluster using galera) and since then we observe exceptions which was not existing before, an example:

2019-06-14 09:01:17,412 ERROR SubmitXCommand:517 - SERVER[server.mycorp.net] - CLUSTER[bar01] USER[goo] GROUP[-] TOKEN[] APP[goo-Reporter-Q-PRD-spark_pricediscrepancies-process_report-spark_pricediscrepancies_15min] JOB[2292565-190613105334848-oozie-mapr-W] ACTION[2292469-190613105334848-oozie-mapr-W@exec_period_0_1560213900_1560214800_subWf] XException,
org.apache.oozie.command.CommandException: E0803: IO error, E0603: SQL error in operation, <openjpa-2.4.1-r422266:1730418 fatal store error> org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back.  See the nested exceptions for details on the errors that occurred.
FailedObject: org.apache.oozie.WorkflowJobBean@1d1b6265
        at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:280)
        at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:76)
        at org.apache.oozie.command.XCommand.call(XCommand.java:287)
        at org.apache.oozie.DagEngine.submitJob(DagEngine.java:114)
        at org.apache.oozie.LocalOozieClient.run(LocalOozieClient.java:188)
        at org.apache.oozie.action.oozie.SubWorkflowActionExecutor.start(SubWorkflowActionExecutor.java:232)
        at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
        at org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:41)
        at org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:30)
        at org.apache.oozie.command.XCommand.call(XCommand.java:287)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.oozie.command.CommandException: E0603: SQL error in operation, <openjpa-2.4.1-r422266:1730418 fatal store error> org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back.  See the nested exceptions for details on the errors that occurred.
FailedObject: org.apache.oozie.WorkflowJobBean@1d1b6265
        at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:236)
        ... 14 more
Caused by: org.apache.oozie.executor.jpa.JPAExecutorException: E0603: SQL error in operation, <openjpa-2.4.1-r422266:1730418 fatal store error> org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back.  See the nested exceptions for details on the errors that occurred.
FailedObject: org.apache.oozie.WorkflowJobBean@1d1b6265
        at org.apache.oozie.service.JPAService.executeBatchInsertUpdateDelete(JPAService.java:439)
        at org.apache.oozie.executor.jpa.BatchQueryExecutor.executeBatchInsertUpdateDelete(BatchQueryExecutor.java:132)
        at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:233)
        ... 14 more
Caused by: <openjpa-2.4.1-r422266:1730418 fatal store error> org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back.  See the nested exceptions for details on the errors that occurred.
FailedObject: org.apache.oozie.WorkflowJobBean@1d1b6265
        at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:595)
        at org.apache.oozie.service.JPAService.executeBatchInsertUpdateDelete(JPAService.java:435)
        ... 16 more
Caused by: <openjpa-2.4.1-r422266:1730418 fatal general error> org.apache.openjpa.persistence.PersistenceException: The transaction has been rolled back.  See the nested exceptions for details on the errors that occurred.
FailedObject: org.apache.oozie.WorkflowJobBean@1d1b6265
        at org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2368)
        at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2205)
        at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2103)
        at org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:2021)
        at org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:81)
        at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1526)
        at org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:932)
        at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:571)
        ... 17 more
Caused by: <openjpa-2.4.1-r422266:1730418 fatal general error> org.apache.openjpa.persistence.PersistenceException: (conn=10371736) Data too long for column 'app_path' at row 1 {prepstmnt 935425409 INSERT INTO WF_JOBS (id, app_name, app_path, conf, created_time, end_time, external_id, group_name, last_modified_time, log_token, parent_id, proto_action_conf, run, sla_xml, start_time, status, user_name, wf_instance) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)} [code=1406, state=22001]
FailedObject: org.apache.oozie.WorkflowJobBean@1d1b6265
        at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:5003)
        at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4963)
        at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:133)
        at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:75)
        at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:144)
        at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:79)
        at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInternal(PreparedStatementManagerImpl.java:100)
        at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(PreparedStatementManagerImpl.java:88)
        at org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:550)
        at org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:106)
        at org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(BatchingConstraintUpdateManager.java:59)
        at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:104)
        at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:77)
        at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:731)
        at org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:131)
        ... 24 more
Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: (conn=10371736) Data too long for column 'app_path' at row 1 {prepstmnt 935425409 INSERT INTO WF_JOBS (id, app_name, app_path, conf, created_time, end_time, external_id, group_name, last_modified_time, log_token, parent_id, proto_action_conf, run, sla_xml, start_time, status, user_name, wf_instance) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)} [code=1406, state=22001]
        at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConnectionDecorator.java:218)
        at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConnectionDecorator.java:194)
        at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$1000(LoggingConnectionDecorator.java:58)
        at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:1133)
        at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:275)
        at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.executeUpdate(JDBCStoreManager.java:1791)
        at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdate(PreparedStatementManagerImpl.java:268)
        at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpdate


it seems this is caused by this error:

Data too long for column 'app_path' at row 1 {prepstmnt 935425409 INSERT INTO WF_JOBS (id, app_name, app_path, conf, created_time, end_time, external_id, group_name, last_modified_time, log_token, parent_id, proto_action_conf, run, sla_xml, start_time, status, user_name, wf_instance) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}

is there any know-issue or workaround for that ?

Answer Answered by Diego Dupin in this comment.

Connector now forces truncation to return not as a warning, but as an error since 1.2 or 1.3 version. before that version, it was server version dependant (SQL_MODE is now set in strict mode since MariaDB 10.2.4).

The best solution would be to avoid this truncation: "Data too long for column 'app_path' at row 1 {prepstmnt 935425409 INSERT INTO WF_JOBS ..."

If wanting to permit truncation (not recommended at all), the strict mode has to be removed from SQL_MODE.

Comments

Comments loading...
Content reproduced on this site is the property of its respective owners, and this content is not reviewed in advance by MariaDB. The views, information and opinions expressed by this content do not necessarily represent those of MariaDB or any other party.