Issue Details (XML | Word | Printable)

Key: COMMUNITY_EQUITY-420
Type: Sub-task Sub-task
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: DmitryRyashchentsev
Reporter: DmitryRyashchentsev
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
community-equity
COMMUNITY_EQUITY-407

Use a separate ID generator table for CEQ math objects

Created: 13/Jan/10 04:09 PM   Updated: 15/Feb/10 10:26 AM   Resolved: 26/Jan/10 10:24 AM
Component/s: CeQ math
Affects Version/s: None
Fix Version/s: Milestone 1.3

Time Tracking:
Not Specified

Tags:


 Description  « Hide

We have 2 types of activiry:
1. Web services and feed import - light DB operations, need max fast access
2. CEQ calculation - intensive DB update, async background process

And these two type of activity update its own set of tables, and only read others. That allows to exclude locks from calculation part.

But, we have GENERATOR table that they both updates implicitly by persistent layer, and we have deadloack exception that are mentioned in COMMUNITY_EQUITY-405 and COMMUNITY_EQUITY-409

We need to provide a separate GENERATOR table for CEQ calculations.



Misu added a comment - 15/Jan/10 09:15 AM

I think that the simple solution will be to let out the GENERATION table (for the CEQ calculations), we can use the auto generated key. This is not so fast like the the TABLE but it is easy to use and it does not create so much problems like the other one.


DmitryRyashchentsev added a comment - 17/Jan/10 10:12 PM

Provided table GENERATOR_CEQ for generating IDs for async calculations


PRE added a comment - 17/Jan/10 11:32 PM

still getting deadlock error when importing large feeds
Local Exception Stack:
Exception [TOPLINK-4002] (Oracle TopLink Essentials - 2.1 (Build b60e-fcs (12/23/2008))): oracle.toplink.essentials.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: Lock wait timeout exceeded; try restarting transaction
Error Code: 1205
Call: UPDATE GENERATOR SET GENERATOR_VALUE = GENERATOR_VALUE + ? WHERE GENERATOR_KEY = ?
bind => [1, INFORMATION]
Query: DataModifyQuery()
at oracle.toplink.essentials.exceptions.DatabaseException.sqlException(DatabaseException.java:311)
at oracle.toplink.essentials.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:654)
at oracle.toplink.essentials.internal.databaseaccess.DatabaseAccessor.executeNoSelect(DatabaseAccessor.java:703)
at oracle.toplink.essentials.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:492)
at oracle.toplink.essentials.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:452)
at oracle.toplink.essentials.internal.sessions.AbstractSession.executeCall(AbstractSession.java:690)
at oracle.toplink.essentials.internal.queryframework.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:228)
at oracle.toplink.essentials.internal.queryframework.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:214)
at oracle.toplink.essentials.internal.queryframework.DatasourceCallQueryMechanism.executeNoSelectCall(DatasourceCallQueryMechanism.java:257)
at oracle.toplink.essentials.internal.queryframework.DatasourceCallQueryMechanism.executeNoSelect(DatasourceCallQueryMechanism.java:237)
at oracle.toplink.essentials.queryframework.DataModifyQuery.executeDatabaseQuery(DataModifyQuery.java:86)
at oracle.toplink.essentials.queryframework.DatabaseQuery.execute(DatabaseQuery.java:628)
at oracle.toplink.essentials.internal.sessions.AbstractSession.internalExecuteQuery(AbstractSession.java:1845)
at oracle.toplink.essentials.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:952)
at oracle.toplink.essentials.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:924)
at oracle.toplink.essentials.sequencing.QuerySequence.update(QuerySequence.java:344)
at oracle.toplink.essentials.sequencing.QuerySequence.updateAndSelectSequence(QuerySequence.java:283)
at oracle.toplink.essentials.sequencing.StandardSequence.getGeneratedVector(StandardSequence.java:96)
at oracle.toplink.essentials.sequencing.Sequence.getGeneratedVector(Sequence.java:281)
at oracle.toplink.essentials.internal.sequencing.SequencingManager$Preallocation_Transaction_NoAccessor_State.getNextValue(SequencingManager.java:420)
at oracle.toplink.essentials.internal.sequencing.SequencingManager.getNextValue(SequencingManager.java:846)
at oracle.toplink.essentials.internal.sequencing.ClientSessionSequencing.getNextValue(ClientSessionSequencing.java:110)
at oracle.toplink.essentials.internal.descriptors.ObjectBuilder.assignSequenceNumber(ObjectBuilder.java:240)
at oracle.toplink.essentials.internal.sessions.UnitOfWorkImpl.assignSequenceNumber(UnitOfWorkImpl.java:355)
at oracle.toplink.essentials.internal.sessions.UnitOfWorkImpl.registerNotRegisteredNewObjectForPersist(UnitOfWorkImpl.java:3266)
at oracle.toplink.essentials.internal.ejb.cmp3.base.RepeatableWriteUnitOfWork.registerNotRegisteredNewObjectForPersist(RepeatableWriteUnitOfWork.java:432)
at oracle.toplink.essentials.internal.sessions.UnitOfWorkImpl.registerNewObjectForPersist(UnitOfWorkImpl.java:3226)
at oracle.toplink.essentials.internal.ejb.cmp3.base.EntityManagerImpl.persist(EntityManagerImpl.java:221)
at com.sun.enterprise.util.EntityManagerWrapper.persist(EntityManagerWrapper.java:238)
at com.sun.ceq.session.InformationServiceImpl.addInformation(InformationServiceImpl.java:411)
at com.sun.ceq.session.InformationServiceImpl.addInformation(InformationServiceImpl.java:369)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.enterprise.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1011)
at com.sun.enterprise.security.SecurityUtil.invoke(SecurityUtil.java:175)
at com.sun.ejb.containers.BaseContainer.invokeTargetBeanMethod(BaseContainer.java:2920)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4011)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:197)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:83)
at $Proxy274.addInformation(Unknown Source)
at com.sun.ceq.data.InformationData.createInformationWithUrl(InformationData.java:104)
at com.sun.ceq.data.InformationData.createInformationWithUrl(InformationData.java:96)
at com.sun.ceq.data.InformationData.getInformationIdByUrl(InformationData.java:84)
at com.sun.ceq.webservices.jersey.jsonp.InformationWebServiceWithPadding.getByUrl(InformationWebServiceWithPadding.java:85)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.jersey.server.impl.model.method.dispatch.EntityParamDispatchProvider$TypeOutInvoker._dispatch(EntityParamDispatchProvider.java:138)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:67)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:154)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:111)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:71)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:111)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:63)
at com


PRE added a comment - 17/Jan/10 11:41 PM - edited

When running aging and feed import get sometimes
Note: seems not to relate to aging - I get this message as well (sometimes) when just importing a feed)

INSERT INTO INFORMATION (INFORMATION_ID, DESCRIPTION, RATING, INFORMATION_KEY, RATING_COUNT, TITLE, VIEW_COUNT, DOWNLOAD_COUNT, URL, CREATED_ON, ARCHIVED, UPDATED_ON, UPDATED_BY, INFORMATION_TYPE_ID, APPLICATION_NAME, MAIN_AUTHOR_ID, PARENT_INFORMATION_ID) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
bind => [2270, null, 0.0, http://identi.ca/notice/19290769, 0, http://identi.ca/notice/19290769, 0, 0, null, 2010-01-18 00:36:31.0, false, 2010-01-18 00:36:31.0, null, 3, StatusNet, 789, null]
EJB5018: An exception was thrown during an ejb invocation on [DataManipulatorImpl]
javax.ejb.TransactionRolledbackLocalException: Exception thrown from bean; nested exception is: javax.persistence.EntityExistsException:
Exception Description: Cannot persist detached object [com.sun.ceq.entity.math.ActionData@671cea1f].
Class> com.sun.ceq.entity.math.ActionData Primary Key> [9304]
javax.persistence.EntityExistsException:
Exception Description: Cannot persist detached object [com.sun.ceq.entity.math.ActionData@671cea1f].
Class> com.sun.ceq.entity.math.ActionData Primary Key> [9304]
at oracle.toplink.essentials.internal.ejb.cmp3.base.EntityManagerImpl.persist(EntityManagerImpl.java:224)
at com.sun.enterprise.util.EntityManagerWrapper.persist(EntityManagerWrapper.java:238)
at com.sun.ceq.math.model.LinearModelImpl.processAction(LinearModelImpl.java:480)
at com.sun.ceq.math.model.DataManipulatorImpl.processAction(DataManipulatorImpl.java:216)
at sun.reflect.GeneratedMethodAccessor839.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.enterprise.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1011)
at com.sun.enterprise.security.SecurityUtil.invoke(SecurityUtil.java:175)
at com.sun.ejb.containers.BaseContainer.invokeTargetBeanMethod(BaseContainer.java:2920)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4011)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:197)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:83)
at $Proxy273.processAction(Unknown Source)
at com.sun.ceq.math.model.DataManipulatorMessageConsumerBean.processMessage(DataManipulatorMessageConsumerBean.java:199)
at com.sun.ceq.math.model.DataManipulatorMessageConsumerBean.onMessage(DataManipulatorMessageConsumerBean.java:91)
at sun.reflect.GeneratedMethodAccessor874.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.enterprise.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1011)
at com.sun.enterprise.security.SecurityUtil.invoke(SecurityUtil.java:175)
at com.sun.ejb.containers.BaseContainer.invokeTargetBeanMethod(BaseContainer.java:2920)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4011)
at com.sun.ejb.containers.MessageBeanContainer.deliverMessage(MessageBeanContainer.java:1111)
at com.sun.ejb.containers.MessageBeanListenerImpl.deliverMessage(MessageBeanListenerImpl.java:74)
at com.sun.enterprise.connectors.inflow.MessageEndpointInvocationHandler.invoke(MessageEndpointInvocationHandler.java:179)
at $Proxy301.onMessage(Unknown Source)
at com.sun.messaging.jms.ra.OnMessageRunner.run(OnMessageRunner.java:258)
at com.sun.enterprise.connectors.work.OneWork.doWork(OneWork.java:76)
at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:555)
Caused by: Exception [TOPLINK-7231] (Oracle TopLink Essentials - 2.1 (Build b60e-fcs (12/23/2008))): oracle.toplink.essentials.exceptions.ValidationException
Exception Description: Cannot persist detached object [com.sun.ceq.entity.math.ActionData@671cea1f].
Class> com.sun.ceq.entity.math.ActionData Primary Key> [9304]
at oracle.toplink.essentials.exceptions.ValidationException.cannotPersistExistingObject(ValidationException.java:2171)
at oracle.toplink.essentials.internal.sessions.UnitOfWorkImpl.registerNotRegisteredNewObjectForPersist(UnitOfWorkImpl.java:3257)
at oracle.toplink.essentials.internal.ejb.cmp3.base.RepeatableWriteUnitOfWork.registerNotRegisteredNewObjectForPersist(RepeatableWriteUnitOfWork.java:432)
at oracle.toplink.essentials.internal.sessions.UnitOfWorkImpl.registerNewObjectForPersist(UnitOfWorkImpl.java:3226)
at oracle.toplink.essentials.internal.ejb.cmp3.base.EntityManagerImpl.persist(EntityManagerImpl.java:221)
... 29 more
javax.ejb.TransactionRolledbackLocalException: Exception thrown from bean; nested exception is: javax.persistence.EntityExistsException:
Exception Description: Cannot persist detached object [com.sun.ceq.entity.math.ActionData@671cea1f].
Class> com.sun.ceq.entity.math.ActionData Primary Key> [9304]
at com.sun.ejb.containers.BaseContainer.checkExceptionClientTx(BaseContainer.java:3753)
at com.sun.ejb.containers.BaseContainer.postInvokeTx(BaseContainer.java:3601)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1379)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1316)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:205)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:83)
at $Proxy273.processAction(Unknown Source)
at com.sun.ceq.math.model.DataManipulatorMessageConsumerBean.processMessage(DataManipulatorMessageConsumerBean.java:199)
at com.sun.ceq.math.model.DataManipulatorMessageConsumerBean.onMessage(DataManipulatorMessageConsumerBean.java:91)
at sun.reflect.GeneratedMethodAccessor874.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.enterprise.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1011)
at com.sun.enterprise.security.SecurityUtil.invoke(SecurityUtil.java:175)
at com.sun.ejb.containers.BaseContainer.invokeTargetBeanMethod(BaseContainer.java:2920)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4011)
at com.sun.ejb.containers.MessageBeanContainer.deliverMessage(MessageBeanContainer.java:1111)
at com.sun.ejb.containers.MessageBeanListenerImpl.deliverMessage(MessageBeanListenerImpl.java:74)
at com.sun.enterprise.connectors.inflow.MessageEndpointInvocationHandler.invoke(MessageEndpointInvocationHandler.java:179)
at $Proxy301.onMessage(Unknown Source)
at com.sun.messaging.jms.ra.OnMessageRunner.run(OnMessageRunner.java:258)
at com.sun.enterprise.connectors.work.OneWork.doWork(OneWork.java:76)
at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:555)
The log message is null.
javax.ejb.EJBTransactionRolledbackException
at com.sun.ejb.containers.BaseContainer.mapBusinessInterfaceException(BaseContainer.java:1507)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1421)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1316)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:205)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:83)
at $Proxy273.processAction(Unknown Source)
at com.sun.ceq.math.model.DataManipulatorMessageConsumerBean.processMessage(DataManipulatorMessageConsumerBean.java:199)
at com.sun.ceq.math.model.DataManipulatorMessageConsumerBean.onMessage(DataManipulatorMessageConsumerBean.java:91)
at sun.reflect.GeneratedMethodAccessor874.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.enterprise.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1011)
at com.sun.enterprise.security.SecurityUtil.invoke(SecurityUtil.java:175)
at com.sun.ejb.containers.BaseContainer.invokeTargetBeanMethod(BaseContainer.java:2920)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4011)
at com.sun.ejb.containers.MessageBeanContainer.deliverMessage(MessageBeanContai
.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:555)
The log message is null.


PRE added a comment - 26/Jan/10 10:24 AM

done


PRE added a comment - 15/Feb/10 10:26 AM

closed all ticket for Milestone Release 1.3