[Neo-report] NEO tests [master:2894-g5d7dc24:py2.7]: 631 Tests, 22 Errors, 17 Failures

neo-report at nexedi.com neo-report at nexedi.com
Tue Jan 3 17:15:19 CET 2012


Title                     : NEO tests [master:2894-g5d7dc24:py2.7]
Date                      : 2012-01-03
Node                      : blade1
Machine                   : x86_64
System                    : Linux
Python                    : 2.7.2+

Directory                 : /tmp/neo_tests/1325606646.423271
Status                    : 93.819%
NEO_TEST_ZODB_FUNCTIONAL  : 1

                               NEO TESTS REPORT

              Test Module |   run   | success |  errors |  fails  |   time   
--------------------------+---------+---------+---------+---------+----------
                ClientApp |    23   |    23   |     .   |     .   |   1.03s   
           ConnectionPool |     5   |     5   |     .   |     .   |   1.00s   
            MasterHandler |    14   |    14   |     .   |     .   |   0.01s   
           StorageHandler |    14   |    14   |     .   |     .   |   0.01s   
--------------------------+---------+---------+---------+---------+----------
         neo.tests.client |         |         |         |         |          
--------------------------+---------+---------+---------+---------+----------
                   Client |    10   |     9   |     1   |     .   |  16.82s   
                  Cluster |     6   |     5   |     1   |     1   |  67.92s   
                   Master |     4   |     .   |     4   |     .   | 122.09s   
                  Storage |    15   |     2   |    13   |    13   | 790.51s   
--------------------------+---------+---------+---------+---------+----------
     neo.tests.functional |         |         |         |         |          
--------------------------+---------+---------+---------+---------+----------
            ClientHandler |     6   |     6   |     .   |     .   |   0.01s   
          ElectionHandler |    21   |    21   |     .   |     .   |   0.03s   
                MasterApp |     2   |     2   |     .   |     .   |   0.02s   
                 MasterPT |     7   |     7   |     .   |     .   |   0.01s   
                 Recovery |     3   |     3   |     .   |     .   |   0.01s   
           StorageHandler |     8   |     8   |     .   |     .   |   0.06s   
             Transactions |     9   |     9   |     .   |     .   |   0.01s   
             Verification |     6   |     6   |     .   |     .   |   0.01s   
--------------------------+---------+---------+---------+---------+----------
         neo.tests.master |         |         |         |         |          
--------------------------+---------+---------+---------+---------+----------
            ClientHandler |    18   |    18   |     .   |     .   |   0.14s   
    IdentificationHandler |     3   |     3   |     .   |     .   |   0.01s   
    InitializationHandler |     2   |     2   |     .   |     .   |   0.02s   
            MasterHandler |    11   |    11   |     .   |     .   |   0.07s   
              Replication |    10   |    10   |     .   |     .   |   2.28s   
       ReplicationHandler |    21   |    21   |     .   |     .   |   0.02s   
               Replicator |     7   |     7   |     .   |     .   |   0.00s   
               StorageApp |     3   |     3   |     .   |     .   |   0.02s   
             StorageBTree |    24   |    24   |     .   |     .   |   0.01s   
           StorageHandler |     9   |     9   |     .   |     .   |   0.06s   
           StorageMySQLdb |    32   |    32   |     .   |     .   |   0.26s   
             Transactions |    19   |    19   |     .   |     .   |   0.01s   
      VerificationHandler |    11   |    11   |     .   |     .   |   0.10s   
--------------------------+---------+---------+---------+---------+----------
        neo.tests.storage |         |         |         |         |          
--------------------------+---------+---------+---------+---------+----------
                Bootstrap |     2   |     2   |     .   |     .   |   0.04s   
               Connection |    75   |    75   |     .   |     .   |   0.08s   
               Dispatcher |     5   |     5   |     .   |     .   |   0.00s   
                    Event |     1   |     1   |     .   |     .   |   0.00s   
                  Handler |     1   |     1   |     .   |     .   |   0.00s   
                    Nodes |    14   |    14   |     .   |     .   |   0.01s   
                       PT |    11   |    11   |     .   |     .   |   0.01s   
                 Protocol |    87   |    87   |     .   |     .   |   0.02s   
                     Util |     4   |     4   |     .   |     .   |   0.00s   
--------------------------+---------+---------+---------+---------+----------
                neo.tests |         |         |         |         |          
--------------------------+---------+---------+---------+---------+----------
                          |    14   |    11   |     3   |     .   |   1.98s   
--------------------------+---------+---------+---------+---------+----------
       neo.tests.threaded |         |         |         |         |          
--------------------------+---------+---------+---------+---------+----------
                    Basic |    16   |    16   |     .   |     .   |  12.63s   
                 Conflict |     5   |     5   |     .   |     .   |   3.87s   
                  History |     1   |     1   |     .   |     .   |   0.80s   
                 Iterator |     9   |     9   |     .   |     .   |   6.94s   
                       MT |     5   |     5   |     .   |     .   |   6.96s   
                     Pack |    14   |    12   |     .   |     2   |  15.63s   
               Persistent |     1   |     1   |     .   |     .   |   0.81s   
                 ReadOnly |     2   |     2   |     .   |     .   |   1.58s   
                 Revision |     7   |     7   |     .   |     .   |   7.64s   
          Synchronization |     7   |     7   |     .   |     .   |   5.23s   
                     Undo |    17   |    16   |     .   |     1   |  15.30s   
                     ZODB |    10   |    10   |     .   |     .   |   8.74s   
--------------------------+---------+---------+---------+---------+----------
           neo.tests.zodb |         |         |         |         |          
--------------------------+---------+---------+---------+---------+----------
                  Summary |   631   |   606   |    22   |    17   | 1090.82s   
--------------------------+---------+---------+---------+---------+----------



ERRORS:
neo.tests.threaded.test.Test.testDeadlockAvoidance
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/threaded/test.py", line 225, in testDeadlockAvoidance
    self.assertEqual(self._testDeadlockAvoidance([1, 3]),
  File "/home/neo/master/neoppod/neo/tests/threaded/test.py", line 199, in _testDeadlockAvoidance
    t.join()
  File "/home/neo/master/neoppod/neo/tests/threaded/__init__.py", line 664, in run
    apply(*self.__target)
  File "/home/neo/master/py2.7/eggs/transaction-1.2.0-py2.7.egg/transaction/_manager.py", line 89, in commit
    return self.get().commit()
  File "/home/neo/master/py2.7/eggs/transaction-1.2.0-py2.7.egg/transaction/_transaction.py", line 342, in commit
    reraise(t, v, tb)
  File "/home/neo/master/py2.7/eggs/transaction-1.2.0-py2.7.egg/transaction/_transaction.py", line 333, in commit
    self._commitResources()
  File "/home/neo/master/py2.7/eggs/transaction-1.2.0-py2.7.egg/transaction/_transaction.py", line 473, in _commitResources
    reraise(t, v, tb)
  File "/home/neo/master/py2.7/eggs/transaction-1.2.0-py2.7.egg/transaction/_transaction.py", line 450, in _commitResources
    rm.tpc_vote(self)
  File "/home/neo/master/py2.7/src/ZODB3/src/ZODB/Connection.py", line 777, in tpc_vote
    s = vote(transaction)
  File "/home/neo/master/neoppod/neo/client/Storage.py", line 34, in wrapped
    return func(self, *args, **kw)
  File "/home/neo/master/neoppod/neo/client/Storage.py", line 127, in tpc_vote
    tryToResolveConflict=self.tryToResolveConflict)
  File "/home/neo/master/neoppod/neo/client/app.py", line 699, in tpc_vote
    result = self.waitStoreResponses(txn_context, tryToResolveConflict)
  File "/home/neo/master/neoppod/neo/client/app.py", line 678, in waitStoreResponses
    tryToResolveConflict)
  File "/home/neo/master/neoppod/neo/client/app.py", line 647, in _handleConflicts
    serial), data=data)
ConflictError: database conflict error (oid 0x01, class neo.tests.threaded.test.PCounterWithResolution, serial this txn started with 0x0393870435489aaa 2012-01-03 16:04:12.488346, serial currently committed 0x03 1900-01-01 00:00:00.000000)
-------------------------------------------------------------

neo.tests.threaded.test.Test.testStorageReconnectDuringTransactionLog
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/threaded/test.py", line 374, in testStorageReconnectDuringTransactionLog
    ZERO_TID, c.root()._p_serial, 10)
  File "/home/neo/master/neoppod/neo/client/app.py", line 959, in transactionLog
    conn = self.cp.getConnForNode(node)
  File "/home/neo/master/neoppod/neo/client/pool.py", line 165, in getConnForNode
    conn = self._initNodeConnection(node)
  File "/home/neo/master/neoppod/neo/client/pool.py", line 69, in _initNodeConnection
    app._ask(conn, p, handler=app.storage_bootstrap_handler)
  File "/home/neo/master/neoppod/neo/client/app.py", line 240, in _ask
    _handlePacket(qconn, qpacket)
  File "/home/neo/master/neoppod/neo/client/app.py", line 177, in _handlePacket
    handler.dispatch(conn, packet)
  File "/home/neo/master/neoppod/neo/lib/handler.py", line 55, in dispatch
    method(conn, *args)
  File "/home/neo/master/neoppod/neo/client/handlers/storage.py", line 106, in answerTIDsFrom
    assert not tids_from.intersection(set(tid_list))
AttributeError: 'NoneType' object has no attribute 'intersection'
-------------------------------------------------------------

neo.tests.threaded.test.Test.testStorageReconnectDuringUndoLog
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/threaded/test.py", line 385, in testStorageReconnectDuringUndoLog
    t1, = cluster.client.undoLog(0, 10)
  File "/home/neo/master/neoppod/neo/client/app.py", line 920, in undoLog
    conn = self.cp.getConnForNode(storage_node)
  File "/home/neo/master/neoppod/neo/client/pool.py", line 165, in getConnForNode
    conn = self._initNodeConnection(node)
  File "/home/neo/master/neoppod/neo/client/pool.py", line 69, in _initNodeConnection
    app._ask(conn, p, handler=app.storage_bootstrap_handler)
  File "/home/neo/master/neoppod/neo/client/app.py", line 240, in _ask
    _handlePacket(qconn, qpacket)
  File "/home/neo/master/neoppod/neo/client/app.py", line 177, in _handlePacket
    handler.dispatch(conn, packet)
  File "/home/neo/master/neoppod/neo/lib/handler.py", line 55, in dispatch
    method(conn, *args)
  File "/home/neo/master/neoppod/neo/client/handlers/storage.py", line 138, in answerTIDs
    self.app.getHandlerData().update(tid_list)
AttributeError: 'NoneType' object has no attribute 'update'
-------------------------------------------------------------

neo.tests.functional.testMaster.MasterTests.testMasterSequentialStart
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testMaster.py", line 32, in setUp
    self.neo.start()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testMaster.MasterTests.testStoppingPrimaryWithOneSecondary
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testMaster.py", line 32, in setUp
    self.neo.start()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testMaster.MasterTests.testStoppingPrimaryWithTwoSecondaries
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testMaster.py", line 32, in setUp
    self.neo.start()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testMaster.MasterTests.testStoppingSecondaryMaster
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testMaster.py", line 32, in setUp
    self.neo.start()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testClient.ClientTests.testExportFileStorageBug
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testClient.py", line 228, in testExportFileStorageBug
    self.testExport(True)
  File "/home/neo/master/neoppod/neo/tests/functional/testClient.py", line 224, in testExport
    self.__checkTree(root['trees'])
  File "/home/neo/master/neoppod/neo/tests/functional/testClient.py", line 165, in __checkTree
    self.__checkTree(tree.right, depth)
  File "/home/neo/master/neoppod/neo/tests/functional/testClient.py", line 166, in __checkTree
    self.__checkTree(tree.left, depth)
AttributeError: 'Tree' object has no attribute 'left'
-------------------------------------------------------------

neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testCluster.py", line 32, in tearDown
    self.neo.stop()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 379, in stop
    raise NodeProcessError('\n'.join(error_list))
NodeProcessError: 'neostorage' {'--cluster': 'neo_87', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo1', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/storage_0.log', '--name': 'storage_0', '--uuid': 'f51995b1844da02b2a9f6af805a7a5fb', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396'} exited with status 1
'neostorage' {'--cluster': 'neo_87', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo2', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/storage_1.log', '--name': 'storage_1', '--uuid': 'dbdc3f032d3e27dde10bc94f5e6f6da7', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396'} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:19614', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_0.log', '--name': 'master_0', '--partitions': 10, '--uuid': '2a8492a2d0af8b8ffaa53d0872baa72f', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:23803', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_1.log', '--name': 'master_1', '--partitions': 10, '--uuid': '4dce905a785a33a747421e74fe770c12', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:23838', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_2.log', '--name': 'master_2', '--partitions': 10, '--uuid': '54f631c165a52a5fcce25b7f4fa003c0', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:24748', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_3.log', '--name': 'master_3', '--partitions': 10, '--uuid': 'b1855bc4b0e6419ea62fd72af1630866', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:20694', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_4.log', '--name': 'master_4', '--partitions': 10, '--uuid': 'd5d25dfab948f477655c98dc904f1459', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:19332', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_5.log', '--name': 'master_5', '--partitions': 10, '--uuid': '2904c41cc1b80c1dafe125d8a1ff62f5', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:29520', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_6.log', '--name': 'master_6', '--partitions': 10, '--uuid': 'bbec814a57267f6a28a825af5d2c253f', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:27212', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_7.log', '--name': 'master_7', '--partitions': 10, '--uuid': '67f0ec7bf26e45db71bf4ffcc961d1d1', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:21757', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_8.log', '--name': 'master_8', '--partitions': 10, '--uuid': '3758e186c549761a841094e1ea3de3e0', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:31970', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_9.log', '--name': 'master_9', '--partitions': 10, '--uuid': '79e20305d5f8b8e48171b35f7a631166', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:31905', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_10.log', '--name': 'master_10', '--partitions': 10, '--uuid': '5eab2bab2b97f98c9819d5af1da0a147', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:24166', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_11.log', '--name': 'master_11', '--partitions': 10, '--uuid': '4e0d006dd4549d4fd0f22e1db290795f', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:23831', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_12.log', '--name': 'master_12', '--partitions': 10, '--uuid': '8ecaba3e26d2dd292694eccdf6f81f06', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:31837', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_13.log', '--name': 'master_13', '--partitions': 10, '--uuid': '61b17fe481cf6a6bd9832652ebbd51ee', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:31718', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_14.log', '--name': 'master_14', '--partitions': 10, '--uuid': '1ef1c0f9499f4de1089d88e05c98b2b1', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:24569', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_15.log', '--name': 'master_15', '--partitions': 10, '--uuid': '380d00cf94ec73b79645d81d536e7e4c', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:29085', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_16.log', '--name': 'master_16', '--partitions': 10, '--uuid': 'ef70b15ee9fae6c84798c67273350f27', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:17173', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_17.log', '--name': 'master_17', '--partitions': 10, '--uuid': '6ada8274205d125ce1b245ddb1cda33b', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:21077', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_18.log', '--name': 'master_18', '--partitions': 10, '--uuid': 'bd42cd6ea22775f1e01006e3821eef21', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_87', '--bind': '127.0.0.1:27396', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/master_19.log', '--name': 'master_19', '--partitions': 10, '--uuid': '889b97a1a0850705b71b3224048bc6a9', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396', '--replicas': 0} exited with status 1
'neoadmin' {'--cluster': 'neo_87', '--bind': '127.0.0.1:28190', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters/admin.log', '--name': 'admin', '--uuid': '809ea869c06bbeb2a8b8fd20c8169662', '--masters': '127.0.0.1:19614/127.0.0.1:23803/127.0.0.1:23838/127.0.0.1:24748/127.0.0.1:20694/127.0.0.1:19332/127.0.0.1:29520/127.0.0.1:27212/127.0.0.1:21757/127.0.0.1:31970/127.0.0.1:31905/127.0.0.1:24166/127.0.0.1:23831/127.0.0.1:31837/127.0.0.1:31718/127.0.0.1:24569/127.0.0.1:29085/127.0.0.1:17173/127.0.0.1:21077/127.0.0.1:27396'} exited with status 1
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testAcceptFirstEmptyStorageAfterStartupAllowed
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 45, in tearDown
    self.neo.stop()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 379, in stop
    raise NodeProcessError('\n'.join(error_list))
NodeProcessError: 'neomaster' {'--cluster': 'neo_13', '--bind': '127.0.0.1:25705', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testAcceptFirstEmptyStorageAfterStartupAllowed/master_0.log', '--name': 'master_0', '--partitions': 10, '--uuid': 'dcb3604d3c4bb703e5a8332a608eeb6d', '--masters': '127.0.0.1:25705/127.0.0.1:29714', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_13', '--bind': '127.0.0.1:29714', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testAcceptFirstEmptyStorageAfterStartupAllowed/master_1.log', '--name': 'master_1', '--partitions': 10, '--uuid': '6be6fc8fb0b52631aa4d437f2b5d939a', '--masters': '127.0.0.1:25705/127.0.0.1:29714', '--replicas': 0} exited with status 1
'neoadmin' {'--cluster': 'neo_13', '--bind': '127.0.0.1:16454', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testAcceptFirstEmptyStorageAfterStartupAllowed/admin.log', '--name': 'admin', '--uuid': '3dcb34beebea62c9faec3fdbedb1f2b0', '--masters': '127.0.0.1:25705/127.0.0.1:29714'} exited with status 1
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testConflictingStorageRejected
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 45, in tearDown
    self.neo.stop()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 379, in stop
    raise NodeProcessError('\n'.join(error_list))
NodeProcessError: 'neostorage' {'--cluster': 'neo_50', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo0', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testConflictingStorageRejected/storage_0.log', '--name': 'storage_0', '--uuid': '74014a2216a9d27fd19c009c25a25303', '--masters': '127.0.0.1:29806/127.0.0.1:30823'} exited with status 1
'neomaster' {'--cluster': 'neo_50', '--bind': '127.0.0.1:29806', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testConflictingStorageRejected/master_0.log', '--name': 'master_0', '--partitions': 10, '--uuid': '2ab27dd09d7da1655b22f8a4decf36a4', '--masters': '127.0.0.1:29806/127.0.0.1:30823', '--replicas': 1} exited with status 1
'neomaster' {'--cluster': 'neo_50', '--bind': '127.0.0.1:30823', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testConflictingStorageRejected/master_1.log', '--name': 'master_1', '--partitions': 10, '--uuid': '728a892795cce8e520c03f2ed9dc94c5', '--masters': '127.0.0.1:29806/127.0.0.1:30823', '--replicas': 1} exited with status 1
'neoadmin' {'--cluster': 'neo_50', '--bind': '127.0.0.1:25656', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testConflictingStorageRejected/admin.log', '--name': 'admin', '--uuid': 'c5a19ca2612fe8dd85f64266a9aa1752', '--masters': '127.0.0.1:29806/127.0.0.1:30823'} exited with status 1
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testDropNodeWithOtherPending
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 45, in tearDown
    self.neo.stop()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 379, in stop
    raise NodeProcessError('\n'.join(error_list))
NodeProcessError: 'neostorage' {'--cluster': 'neo_86', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo0', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testDropNodeWithOtherPending/storage_0.log', '--name': 'storage_0', '--uuid': '765591c9a36a669f587ea4ffda42e600', '--masters': '127.0.0.1:30254/127.0.0.1:21489'} exited with status 1
'neomaster' {'--cluster': 'neo_86', '--bind': '127.0.0.1:30254', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testDropNodeWithOtherPending/master_0.log', '--name': 'master_0', '--partitions': 10, '--uuid': '17f4622548bcf74666bd5c37434c78e5', '--masters': '127.0.0.1:30254/127.0.0.1:21489', '--replicas': 1} exited with status 1
'neomaster' {'--cluster': 'neo_86', '--bind': '127.0.0.1:21489', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testDropNodeWithOtherPending/master_1.log', '--name': 'master_1', '--partitions': 10, '--uuid': '29aa6083006e2a3ffc69fc52f03ba7bd', '--masters': '127.0.0.1:30254/127.0.0.1:21489', '--replicas': 1} exited with status 1
'neoadmin' {'--cluster': 'neo_86', '--bind': '127.0.0.1:25147', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testDropNodeWithOtherPending/admin.log', '--name': 'admin', '--uuid': '5cd125042f1e6653d3920763dbc88601', '--masters': '127.0.0.1:30254/127.0.0.1:21489'} exited with status 1
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testNewNodesInPendingState
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 45, in tearDown
    self.neo.stop()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 379, in stop
    raise NodeProcessError('\n'.join(error_list))
NodeProcessError: 'neostorage' {'--cluster': 'neo_78', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo0', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testNewNodesInPendingState/storage_0.log', '--name': 'storage_0', '--uuid': '6acc8b8942a21d7ee0f587327c07c132', '--masters': '127.0.0.1:32417/127.0.0.1:20412'} exited with status 1
'neomaster' {'--cluster': 'neo_78', '--bind': '127.0.0.1:32417', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testNewNodesInPendingState/master_0.log', '--name': 'master_0', '--partitions': 10, '--uuid': 'ce4793c83dfe2c36b0ff8b1630d99776', '--masters': '127.0.0.1:32417/127.0.0.1:20412', '--replicas': 1} exited with status 1
'neomaster' {'--cluster': 'neo_78', '--bind': '127.0.0.1:20412', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testNewNodesInPendingState/master_1.log', '--name': 'master_1', '--partitions': 10, '--uuid': '00bf42f456e96e1ed8f0f9b4285af39b', '--masters': '127.0.0.1:32417/127.0.0.1:20412', '--replicas': 1} exited with status 1
'neoadmin' {'--cluster': 'neo_78', '--bind': '127.0.0.1:20976', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testNewNodesInPendingState/admin.log', '--name': 'admin', '--uuid': 'b5abd2fed136786ee50a7f4616754266', '--masters': '127.0.0.1:32417/127.0.0.1:20412'} exited with status 1
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testOudatedCellsOnDownStorage
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 45, in tearDown
    self.neo.stop()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 379, in stop
    raise NodeProcessError('\n'.join(error_list))
NodeProcessError: 'neostorage' {'--cluster': 'neo_56', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo0', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testOudatedCellsOnDownStorage/storage_0.log', '--name': 'storage_0', '--uuid': 'ffb9a3ae9147a2f6bb8e2aa46735145d', '--masters': '127.0.0.1:26811/127.0.0.1:19002'} exited with status 1
'neostorage' {'--cluster': 'neo_56', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo1', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testOudatedCellsOnDownStorage/storage_1.log', '--name': 'storage_1', '--uuid': '2bd74bd0a3a5f295110be0e5b31b0c91', '--masters': '127.0.0.1:26811/127.0.0.1:19002'} exited with status 1
'neomaster' {'--cluster': 'neo_56', '--bind': '127.0.0.1:26811', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testOudatedCellsOnDownStorage/master_0.log', '--name': 'master_0', '--partitions': 10, '--uuid': '29fc880430c964d05171b5f8241ce4ea', '--masters': '127.0.0.1:26811/127.0.0.1:19002', '--replicas': 1} exited with status 1
'neomaster' {'--cluster': 'neo_56', '--bind': '127.0.0.1:19002', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testOudatedCellsOnDownStorage/master_1.log', '--name': 'master_1', '--partitions': 10, '--uuid': 'fcc83133253725dda5dcc818534b3ee3', '--masters': '127.0.0.1:26811/127.0.0.1:19002', '--replicas': 1} exited with status 1
'neoadmin' {'--cluster': 'neo_56', '--bind': '127.0.0.1:23823', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testOudatedCellsOnDownStorage/admin.log', '--name': 'admin', '--uuid': 'ff9c9b0ae95dc8cbf316af08cddcfe48', '--masters': '127.0.0.1:26811/127.0.0.1:19002'} exited with status 1
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testPartitionTableReorganizedAfterDrop
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 45, in tearDown
    self.neo.stop()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 379, in stop
    raise NodeProcessError('\n'.join(error_list))
NodeProcessError: 'neostorage' {'--cluster': 'neo_32', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo0', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testPartitionTableReorganizedAfterDrop/storage_0.log', '--name': 'storage_0', '--uuid': '366535ed9fbfbbe4ae774feb43c16d8e', '--masters': '127.0.0.1:31808/127.0.0.1:16455'} exited with status 1
'neostorage' {'--cluster': 'neo_32', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo1', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testPartitionTableReorganizedAfterDrop/storage_1.log', '--name': 'storage_1', '--uuid': '70b4d5146ac972d1f6fa22d93974897b', '--masters': '127.0.0.1:31808/127.0.0.1:16455'} exited with status 1
'neomaster' {'--cluster': 'neo_32', '--bind': '127.0.0.1:31808', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testPartitionTableReorganizedAfterDrop/master_0.log', '--name': 'master_0', '--partitions': 10, '--uuid': '2a600ef97f6798442c109969fc8a9e31', '--masters': '127.0.0.1:31808/127.0.0.1:16455', '--replicas': 1} exited with status 1
'neomaster' {'--cluster': 'neo_32', '--bind': '127.0.0.1:16455', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testPartitionTableReorganizedAfterDrop/master_1.log', '--name': 'master_1', '--partitions': 10, '--uuid': '99e08f4fd65644f1cf27b9af94da9fe4', '--masters': '127.0.0.1:31808/127.0.0.1:16455', '--replicas': 1} exited with status 1
'neoadmin' {'--cluster': 'neo_32', '--bind': '127.0.0.1:19970', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testPartitionTableReorganizedAfterDrop/admin.log', '--name': 'admin', '--uuid': '6b96d07b3a9b0eb092f171ce07e2fa5a', '--masters': '127.0.0.1:31808/127.0.0.1:16455'} exited with status 1
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testPartitionTableReorganizedWithNewStorage
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 45, in tearDown
    self.neo.stop()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 379, in stop
    raise NodeProcessError('\n'.join(error_list))
NodeProcessError: 'neostorage' {'--cluster': 'neo_6', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo0', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testPartitionTableReorganizedWithNewStorage/storage_0.log', '--name': 'storage_0', '--uuid': '8f9870a99e94f2aad728ee5e8d40d3d9', '--masters': '127.0.0.1:29033/127.0.0.1:27711'} exited with status 1
'neomaster' {'--cluster': 'neo_6', '--bind': '127.0.0.1:29033', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testPartitionTableReorganizedWithNewStorage/master_0.log', '--name': 'master_0', '--partitions': 10, '--uuid': 'fe9c366595addedf8ef3b83fc267e4da', '--masters': '127.0.0.1:29033/127.0.0.1:27711', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_6', '--bind': '127.0.0.1:27711', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testPartitionTableReorganizedWithNewStorage/master_1.log', '--name': 'master_1', '--partitions': 10, '--uuid': '187ef8baeb185dbdc30b12258a493965', '--masters': '127.0.0.1:29033/127.0.0.1:27711', '--replicas': 0} exited with status 1
'neoadmin' {'--cluster': 'neo_6', '--bind': '127.0.0.1:21656', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testPartitionTableReorganizedWithNewStorage/admin.log', '--name': 'admin', '--uuid': 'b31dc38c8a13f640ac6b6a7e9eb845a9', '--masters': '127.0.0.1:29033/127.0.0.1:27711'} exited with status 1
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testRecoveryWithMultiplePT
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 45, in tearDown
    self.neo.stop()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 379, in stop
    raise NodeProcessError('\n'.join(error_list))
NodeProcessError: 'neostorage' {'--cluster': 'neo_14', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo0', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testRecoveryWithMultiplePT/storage_0.log', '--name': 'storage_0', '--uuid': '4bc552efe653d08624b7f74a1faaed0d', '--masters': '127.0.0.1:23998/127.0.0.1:19735'} exited with status 1
'neostorage' {'--cluster': 'neo_14', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo1', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testRecoveryWithMultiplePT/storage_1.log', '--name': 'storage_1', '--uuid': 'a212f28c6ef6aa037d2283e67ce53945', '--masters': '127.0.0.1:23998/127.0.0.1:19735'} exited with status 1
'neomaster' {'--cluster': 'neo_14', '--bind': '127.0.0.1:23998', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testRecoveryWithMultiplePT/master_0.log', '--name': 'master_0', '--partitions': 10, '--uuid': 'cdcb40b3b61be0e1591af6e71c2587a1', '--masters': '127.0.0.1:23998/127.0.0.1:19735', '--replicas': 1} exited with status 1
'neomaster' {'--cluster': 'neo_14', '--bind': '127.0.0.1:19735', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testRecoveryWithMultiplePT/master_1.log', '--name': 'master_1', '--partitions': 10, '--uuid': '28ea100e599506cbac0a6d1b756a44ca', '--masters': '127.0.0.1:23998/127.0.0.1:19735', '--replicas': 1} exited with status 1
'neoadmin' {'--cluster': 'neo_14', '--bind': '127.0.0.1:18160', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testRecoveryWithMultiplePT/admin.log', '--name': 'admin', '--uuid': '9f5771cf59e1c3be56be047d758f7578', '--masters': '127.0.0.1:23998/127.0.0.1:19735'} exited with status 1
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testReplicationBlockedByUnfinished
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 45, in tearDown
    self.neo.stop()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 379, in stop
    raise NodeProcessError('\n'.join(error_list))
NodeProcessError: 'neostorage' {'--cluster': 'neo_1', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo0', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testReplicationBlockedByUnfinished/storage_0.log', '--name': 'storage_0', '--uuid': 'e08ce47f071f2c603ca33d10d8a74f03', '--masters': '127.0.0.1:26517/127.0.0.1:31605'} exited with status 1
'neomaster' {'--cluster': 'neo_1', '--bind': '127.0.0.1:26517', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testReplicationBlockedByUnfinished/master_0.log', '--name': 'master_0', '--partitions': 10, '--uuid': 'beb87dd78609ab4f2d220bf12d4656c8', '--masters': '127.0.0.1:26517/127.0.0.1:31605', '--replicas': 1} exited with status 1
'neomaster' {'--cluster': 'neo_1', '--bind': '127.0.0.1:31605', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testReplicationBlockedByUnfinished/master_1.log', '--name': 'master_1', '--partitions': 10, '--uuid': '39124f3a13b8cca38e04fbe91355f2a1', '--masters': '127.0.0.1:26517/127.0.0.1:31605', '--replicas': 1} exited with status 1
'neoadmin' {'--cluster': 'neo_1', '--bind': '127.0.0.1:22276', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testReplicationBlockedByUnfinished/admin.log', '--name': 'admin', '--uuid': '3b554389f40dabac987b97894f6451e1', '--masters': '127.0.0.1:26517/127.0.0.1:31605'} exited with status 1
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testReplicationThenRunningWithReplicas
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 45, in tearDown
    self.neo.stop()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 379, in stop
    raise NodeProcessError('\n'.join(error_list))
NodeProcessError: 'neostorage' {'--cluster': 'neo_16', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo0', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testReplicationThenRunningWithReplicas/storage_0.log', '--name': 'storage_0', '--uuid': '0f018cc3513636b05423d5e6a93b2bb8', '--masters': '127.0.0.1:30187/127.0.0.1:17126'} exited with status 1
'neomaster' {'--cluster': 'neo_16', '--bind': '127.0.0.1:30187', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testReplicationThenRunningWithReplicas/master_0.log', '--name': 'master_0', '--partitions': 10, '--uuid': '29ef45643f271de973a3440a13f77b0a', '--masters': '127.0.0.1:30187/127.0.0.1:17126', '--replicas': 1} exited with status 1
'neomaster' {'--cluster': 'neo_16', '--bind': '127.0.0.1:17126', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testReplicationThenRunningWithReplicas/master_1.log', '--name': 'master_1', '--partitions': 10, '--uuid': '465788ceb0f4d40c536e2c052168e697', '--masters': '127.0.0.1:30187/127.0.0.1:17126', '--replicas': 1} exited with status 1
'neoadmin' {'--cluster': 'neo_16', '--bind': '127.0.0.1:24708', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testReplicationThenRunningWithReplicas/admin.log', '--name': 'admin', '--uuid': '0b1acb121983b5ae82e2af00a7bb5d46', '--masters': '127.0.0.1:30187/127.0.0.1:17126'} exited with status 1
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testReplicationWithNewStorage
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 45, in tearDown
    self.neo.stop()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 379, in stop
    raise NodeProcessError('\n'.join(error_list))
NodeProcessError: 'neostorage' {'--cluster': 'neo_46', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo0', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testReplicationWithNewStorage/storage_0.log', '--name': 'storage_0', '--uuid': '6760dc0ab21ee81ce2c698a3c9c4d196', '--masters': '127.0.0.1:16932/127.0.0.1:28559'} exited with status 1
'neomaster' {'--cluster': 'neo_46', '--bind': '127.0.0.1:16932', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testReplicationWithNewStorage/master_0.log', '--name': 'master_0', '--partitions': 10, '--uuid': 'a575d76d1b1c9019422bba1865891b88', '--masters': '127.0.0.1:16932/127.0.0.1:28559', '--replicas': 1} exited with status 1
'neomaster' {'--cluster': 'neo_46', '--bind': '127.0.0.1:28559', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testReplicationWithNewStorage/master_1.log', '--name': 'master_1', '--partitions': 10, '--uuid': 'c988ef82b8acd9a5a7cc1e3fe9473e99', '--masters': '127.0.0.1:16932/127.0.0.1:28559', '--replicas': 1} exited with status 1
'neoadmin' {'--cluster': 'neo_46', '--bind': '127.0.0.1:27989', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testReplicationWithNewStorage/admin.log', '--name': 'admin', '--uuid': 'e1e9aee7a445daea674fd7133b293e79', '--masters': '127.0.0.1:16932/127.0.0.1:28559'} exited with status 1
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testSequentialStorageKill
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 45, in tearDown
    self.neo.stop()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 379, in stop
    raise NodeProcessError('\n'.join(error_list))
NodeProcessError: 'neostorage' {'--cluster': 'neo_92', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo0', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testSequentialStorageKill/storage_0.log', '--name': 'storage_0', '--uuid': '93517dc4b8147ecd219c8f24b7de910b', '--masters': '127.0.0.1:18548/127.0.0.1:23251'} exited with status 1
'neostorage' {'--cluster': 'neo_92', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo1', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testSequentialStorageKill/storage_1.log', '--name': 'storage_1', '--uuid': '88765847b837173500cac1c49dbf82b4', '--masters': '127.0.0.1:18548/127.0.0.1:23251'} exited with status 1
'neostorage' {'--cluster': 'neo_92', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo2', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testSequentialStorageKill/storage_2.log', '--name': 'storage_2', '--uuid': '6ac637ae6692fd2a4a62f1e3441c2066', '--masters': '127.0.0.1:18548/127.0.0.1:23251'} exited with status 1
'neomaster' {'--cluster': 'neo_92', '--bind': '127.0.0.1:18548', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testSequentialStorageKill/master_0.log', '--name': 'master_0', '--partitions': 10, '--uuid': 'e0e7788c567983f12342397e504668be', '--masters': '127.0.0.1:18548/127.0.0.1:23251', '--replicas': 2} exited with status 1
'neomaster' {'--cluster': 'neo_92', '--bind': '127.0.0.1:23251', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testSequentialStorageKill/master_1.log', '--name': 'master_1', '--partitions': 10, '--uuid': '0ad5a21d94e63df19417f0b44536a95c', '--masters': '127.0.0.1:18548/127.0.0.1:23251', '--replicas': 2} exited with status 1
'neoadmin' {'--cluster': 'neo_92', '--bind': '127.0.0.1:18390', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testSequentialStorageKill/admin.log', '--name': 'admin', '--uuid': 'c1710f490cf12f850009294d6ec5deb6', '--masters': '127.0.0.1:18548/127.0.0.1:23251'} exited with status 1
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testVerificationTriggered
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 45, in tearDown
    self.neo.stop()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 379, in stop
    raise NodeProcessError('\n'.join(error_list))
NodeProcessError: 'neostorage' {'--cluster': 'neo_11', '--adapter': 'MySQL', '--bind': '127.0.0.1:0', '--database': 'test:@test_neo0', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testVerificationTriggered/storage_0.log', '--name': 'storage_0', '--uuid': '97b9bd5087e02c6d51fc2a721e3a11ca', '--masters': '127.0.0.1:29773/127.0.0.1:31824'} exited with status 1
'neomaster' {'--cluster': 'neo_11', '--bind': '127.0.0.1:29773', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testVerificationTriggered/master_0.log', '--name': 'master_0', '--partitions': 10, '--uuid': '047e8e128750ac1430b6da630febaeb1', '--masters': '127.0.0.1:29773/127.0.0.1:31824', '--replicas': 0} exited with status 1
'neomaster' {'--cluster': 'neo_11', '--bind': '127.0.0.1:31824', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testVerificationTriggered/master_1.log', '--name': 'master_1', '--partitions': 10, '--uuid': '862be85573671f26372f9c33bb727c52', '--masters': '127.0.0.1:29773/127.0.0.1:31824', '--replicas': 0} exited with status 1
'neoadmin' {'--cluster': 'neo_11', '--bind': '127.0.0.1:26875', '--verbose': True, '--logfile': '/tmp/neo_tests/1325606646.423271/neo.tests.functional.testStorage.StorageTests.testVerificationTriggered/admin.log', '--name': 'admin', '--uuid': '1e8bed9c1afdfa9cfaa5e2a78fa81fe5', '--masters': '127.0.0.1:29773/127.0.0.1:31824'} exited with status 1
-------------------------------------------------------------


FAILURES:
neo.tests.functional.testCluster.ClusterTests.testElectionWithManyMasters
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testCluster.py", line 76, in testElectionWithManyMasters
    self.neo.start()
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testAcceptFirstEmptyStorageAfterStartupAllowed
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 438, in testAcceptFirstEmptyStorageAfterStartupAllowed
    pending_number=1, partitions=10)
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 70, in __setup
    self.neo.start(except_storages=stopped_processes)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testConflictingStorageRejected
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 275, in testConflictingStorageRejected
    (started, stopped) = self.__setup(storage_number=2, pending_number=1)
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 70, in __setup
    self.neo.start(except_storages=stopped_processes)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testDropNodeWithOtherPending
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 452, in testDropNodeWithOtherPending
    pending_number=1, partitions=10)
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 70, in __setup
    self.neo.start(except_storages=stopped_processes)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testNewNodesInPendingState
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 145, in testNewNodesInPendingState
    processes = self.__setup(storage_number=3, replicas=1, pending_number=2)
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 70, in __setup
    self.neo.start(except_storages=stopped_processes)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testOudatedCellsOnDownStorage
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 192, in testOudatedCellsOnDownStorage
    (started, _) = self.__setup(storage_number=2, replicas=1)
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 70, in __setup
    self.neo.start(except_storages=stopped_processes)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testPartitionTableReorganizedAfterDrop
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 326, in testPartitionTableReorganizedAfterDrop
    partitions=10, pending_number=0)
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 70, in __setup
    self.neo.start(except_storages=stopped_processes)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testPartitionTableReorganizedWithNewStorage
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 301, in testPartitionTableReorganizedWithNewStorage
    partitions=10, replicas=0)
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 70, in __setup
    self.neo.start(except_storages=stopped_processes)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testRecoveryWithMultiplePT
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 468, in testRecoveryWithMultiplePT
    pending_number=0, partitions=10)
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 70, in __setup
    self.neo.start(except_storages=stopped_processes)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testReplicationBlockedByUnfinished
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 502, in testReplicationBlockedByUnfinished
    pending_number=1, partitions=10)
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 70, in __setup
    self.neo.start(except_storages=stopped_processes)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testReplicationThenRunningWithReplicas
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 353, in testReplicationThenRunningWithReplicas
    pending_number=1, partitions=10)
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 70, in __setup
    self.neo.start(except_storages=stopped_processes)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testReplicationWithNewStorage
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 165, in testReplicationWithNewStorage
    partitions=10)
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 70, in __setup
    self.neo.start(except_storages=stopped_processes)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testSequentialStorageKill
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 239, in testSequentialStorageKill
    (started, _) = self.__setup(replicas=2, storage_number=3, partitions=10)
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 70, in __setup
    self.neo.start(except_storages=stopped_processes)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.functional.testStorage.StorageTests.testVerificationTriggered
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 211, in testVerificationTriggered
    (started, _) = self.__setup(replicas=0, storage_number=1)
  File "/home/neo/master/neoppod/neo/tests/functional/testStorage.py", line 70, in __setup
    self.neo.start(except_storages=stopped_processes)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 344, in start
    self.run(except_storages=except_storages)
  File "/home/neo/master/neoppod/neo/tests/functional/__init__.py", line 339, in run
    raise AssertionError('Timeout when starting cluster')
AssertionError: Timeout when starting cluster
-------------------------------------------------------------

neo.tests.zodb.testPack.PackableTests.checkPackAllRevisions
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/py2.7/src/ZODB3/src/ZODB/tests/PackableStorage.py", line 360, in checkPackAllRevisions
    raises(KeyError, self._storage.loadSerial, oid, revid3)
AssertionError: KeyError not raised
-------------------------------------------------------------

neo.tests.zodb.testPack.PackableTests.checkPackUndoLog
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/py2.7/src/ZODB3/src/ZODB/tests/PackableStorage.py", line 655, in checkPackUndoLog
    self.assertEqual(1,len(self._storage.undoLog()))
  File "/home/neo/master/neoppod/neo/tests/__init__.py", line 144, in assertEqual
    return super(NeoTestBase, self).assertEqual(first, second, msg=msg)
AssertionError: 1 != 3
-------------------------------------------------------------

neo.tests.zodb.testUndo.UndoTests.checkTransactionalUndoAfterPack
-------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neo/master/py2.7/src/ZODB3/src/ZODB/tests/TransactionalUndoStorage.py", line 463, in checkTransactionalUndoAfterPack
    self.assertEqual(len(info2), 2)
  File "/home/neo/master/neoppod/neo/tests/__init__.py", line 144, in assertEqual
    return super(NeoTestBase, self).assertEqual(first, second, msg=msg)
AssertionError: 4 != 2
-------------------------------------------------------------






More information about the Neo-report mailing list