Fatal errors after upgrade to ZeXtras 1.8.17
Results 1 to 9 of 9

Thread: Fatal errors after upgrade to ZeXtras 1.8.17

  1. #1
    Senior Member
    Join Date
    Oct 2013
    Posts
    70

    Fatal errors after upgrade to ZeXtras 1.8.17

    We have a multi-server installation running CentOS 6 x86_64 on 17 Zimbra servers of which 9 are mailbox servers.
    We run Zimbra 8.0.4 OSE and ZeXtras 1.8.15. Last night we had planned an upgrade of ZeXtras to 1.8.17.

    We started the upgrade on one mailbox server and everything completed as expected (no errors on console).
    But when we try to login on the admin web interface, we get "Unable to connect to ZeXtras Server System".

    zxsuite cli returns Module not found for everything..

    [zimbra@zm-embox-01 ~]$ zxsuite core getServices

    Unknown Exception com.zextras.lib.Error.ModuleNotFoundError: Module not
    found: {"module":"ZxCore","action":"getServices"}
    [zimbra@zm-embox-01 ~]$ zxsuite mobile getServices

    Unknown Exception com.zextras.lib.Error.ModuleNotFoundError: Module not
    found: {"module":"ZxMobile","action":"getServices"
    Also, there are fatal errors in mailbox.log.
    2014-08-18 23:11:10,533 ERROR [ZeXtras Activity Thread] [] extensions - ZeXtras Activity: Exception: com.google.inject.ProvisionException: Guice provision errors:

    1) Error injecting constructor, java.lang.IllegalArgumentException: total (sync + async) map backup count must be less than 6
    at com.zextras.core.services.HazelcastService.Hazelca stThread.<init>(Unknown Source)
    at com.zextras.core.services.HazelcastService.Hazelca stThread.class(Unknown Source)
    while locating com.zextras.core.services.HazelcastService.Hazelca stThread
    for parameter 0 at com.zextras.core.services.HazelcastService.Hazelca stService.<init>(Unknown Source)
    at com.zextras.core.services.HazelcastService.Hazelca stService.class(Unknown Source)
    while locating com.zextras.core.services.HazelcastService.Hazelca stService
    for parameter 4 at com.zextras.core.services.CoreModuleSwitchesImpl.< init>(Unknown Source)
    at com.zextras.core.services.CoreModuleSwitchesImpl.c lass(Unknown Source)
    while locating com.zextras.core.services.CoreModuleSwitchesImpl
    while locating com.zextras.core.services.CoreModuleSwitches
    for parameter 1 at com.zextras.lib.switches.CoreModuleStarter.<init>( Unknown Source)
    while locating com.zextras.lib.switches.CoreModuleStarter
    for parameter 5 at com.zextras.core.CoreModule.<init>(Unknown Source)
    at com.zextras.core.CoreModule.class(Unknown Source)
    while locating com.zextras.core.CoreModule
    Caused by: java.lang.IllegalArgumentException: total (sync + async) map backup count must be less than 6
    ...
    2) Error injecting constructor, java.lang.IllegalArgumentException: total (sync + async) map backup count must be less than 6
    at com.zextras.core.services.HazelcastService.Hazelca stThread.<init>(Unknown Source)
    at com.zextras.core.services.HazelcastService.Hazelca stThread.class(Unknown Source)
    while locating com.zextras.core.services.HazelcastService.Hazelca stThread
    for parameter 0 at com.zextras.core.services.HazelcastService.Hazelca stService.<init>(Unknown Source)
    ...
    3) Error injecting constructor, java.lang.IllegalArgumentException: total (sync + async) map backup count must be less than 6
    at com.zextras.core.services.HazelcastService.Hazelca stThread.<init>(Unknown Source)
    at com.zextras.core.services.HazelcastService.Hazelca stThread.class(Unknown Source)
    while locating com.zextras.core.services.HazelcastService.Hazelca stThread
    for parameter 0 at com.zextras.core.services.HazelcastService.Hazelca stService.<init>(Unknown Source)
    ...
    3 errors
    at com.google.inject.internal.InjectorImpl$4.get ( InjectorImpl.java:987 )
    at com.google.inject.internal.InjectorImpl.getInstanc e ( InjectorImpl.java:1013 )
    at com.zextras.core.CoreModule.getInstance ( CoreModule.java:80 )
    at com.zextras.license.LicenseManager$2.doActivity ( LicenseManager.java:346 )
    at com.zextras.lib.activities.ActivityThread.run ( ActivityThread.java:118 )
    2014-08-18 23:11:10,559 INFO [main] [] extensions - Creating Mobile Module
    2014-08-18 23:11:10,592 INFO [main] [] extensions - Creating Admin Module
    2014-08-18 23:11:10,593 INFO [main] [] extensions - Using saved license.
    2014-08-18 23:11:10,594 INFO [ZeXtras Activity Thread] [] extensions - ZeXtras Activity: Starting new thread
    2014-08-18 23:11:10,596 INFO [ZeXtras Real Time Notifier Thread] [] extensions - ZeXtras Real Time Notifier: Started
    2014-08-18 23:11:10,597 INFO [ZeXtras Real Time Notifier Thread] [] extensions - ZeXtras Real Time Notifier: Removing redolog tmp: redolog-66.zetmp
    2014-08-18 23:11:10,597 INFO [main] [] extensions - registered handler at /zextras
    2014-08-18 23:11:10,631 FATAL [main] [] extensions - Unable to initialize zextras: com.google.inject.ProvisionException: Guice provision errors:
    ...
    ZxMobile module is not working and this is visible to the backend users.
    We did not proceed with upgrade on the other mailbox servers.
    We did not proceed with downgrade as documentation states it is not supported (ZeXtras Suite Installation Guide - ZeXtras Suite Wiki).

    Please help.

  2. #2
    The CEO ZeXtras Employee
    Participant
    Zeus's Avatar
    Join Date
    Apr 2011
    Posts
    57
    Hi stsimb !
    We made some investigations on you specific issue and we made a custom buid to solve it !
    Here you can download and install it.

    Let us know!
    Best Regards.
    Zeus.
    ZeXtras Website # ZeXtras Wiki # ZeXtras Store

    ZeXtras CEO. Like a Boss.

  3. #3
    Senior Member
    Join Date
    Oct 2013
    Posts
    70
    Quote Originally Posted by Zeus View Post
    Hi stsimb !
    We made some investigations on you specific issue and we made a custom buid to solve it !
    Here you can download and install it.

    Let us know!
    Best Regards.
    Zeus.
    Thanks for the fix!
    We installed it and we now have zxmobile working, so no visible downtime for our users.
    zxsuite on cli is working again.
    I can also login to the administrative web interface, but have some errors (although I flushed the zimbra cache).

    We also got a notification email saying "Service ZxCore/module could not be started, since one or more mandatory sub-services failed to start."
    Code:
    [zimbra@zm-embox-01 ~]$ zxsuite core getServices
    
            services
                module
                    could_start                                                 true
                    could_stop                                                  false
                    running                                                     false
                license-synchronizer
                    could_start                                                 false
                    could_stop                                                  false
                    running                                                     true
                account-config-migration
                    could_start                                                 false
                    could_stop                                                  true
                    running                                                     true
                task-scheduler
                    could_start                                                 false
                    could_stop                                                  false
                    running                                                     true
                error-soap-handler
                    could_start                                                 false
                    could_stop                                                  true
                    running                                                     true
                operation-log
                    could_start                                                 false
                    could_stop                                                  true
                    running                                                     true
                operation-manager
                    could_start                                                 false
                    could_stop                                                  true
                    running                                                     true
    Running getServices on individual modules returnes running=true for all services.

    I manually executed "zxsuite core doStartService module" and it now reports module with running=true as well.

  4. #4
    Senior Member
    Join Date
    Oct 2013
    Posts
    70
    On the admin web interface, I get an error on every ZeXtras link I click for the server we upgraded (the others are still 1.8.15)..

    JavaScript error encounted in method
    ZaController.prototype._setView

    2014-08-19-154343_496x353_scrot.jpg

    Tried with firefox & chrome with nothing in cache (cleared everything & restarted).

  5. #5
    Jay
    Jay is offline
    Code Master ZeXtras Employee
    Join Date
    May 2011
    Posts
    26
    Hi stsimb,

    Could you please post the complete notification about service start failure and the exception in mailboxd.log during zimbra startup?
    The exception should start with "Unable to start the whole service: ..."

    Best Regards.
    ZeXtras Website # ZeXtras Wiki # ZeXtras Store

    Head of ZeXtras Suite Developers

  6. #6
    Senior Member
    Join Date
    Oct 2013
    Posts
    70
    Notification mail was this
    Code:
    Date: Tue, 19 Aug 2014 14:16:44 +0300 (EEST)
    From: ZeXtras <admin@***>
    To: admin@***
    Message-ID: <1592942884.0.1408447005216.JavaMail.zimbra@zm-embox-01.***>
    Subject: Service ZxCore start failure From server zm-embox-01.***
    MIME-Version: 1.0
    Content-Type: text/plain; charset=utf8
    Content-Transfer-Encoding: 7bit
    
    Service ZxCore/module could not be started, since one or more mandatory sub-services failed to start.
    and mailbox.log can be found here:
    ZeroBin

  7. #7
    Senior Member
    Join Date
    Oct 2013
    Posts
    70
    and we also get these huge stacktraces now ZeroBin.
    cpu load is 100% (normally it was only about 10%) .. all consumed by the java mailboxd process..

  8. #8
    The CEO ZeXtras Employee
    Participant
    Zeus's Avatar
    Join Date
    Apr 2011
    Posts
    57
    After performed some test on the customer's environment, we discovered the load issue was caused by a mixed version infrastructure (only one node had the 1.8.17 version).
    With full infrastructure's upgrade the system doesn't have load issue nor warning messages anymore .
    During this test, in this particular corner case, we found a couple of minor issues (false-negative status on a loaded module and first check update issue using zimbra http proxy).
    This issues will be fixed in hours and will be included in next minor release or in next hotfix if needed.
    I want to thanks stsimb for cooperation :-)

    Zeus
    ZeXtras Website # ZeXtras Wiki # ZeXtras Store

    ZeXtras CEO. Like a Boss.

  9. #9
    Senior Member
    Join Date
    Oct 2013
    Posts
    70
    I confirm the above.
    We also deployed a new version ot the admin zimlet to fix the JavaScript error.

    Thanks to all the people at ZeXtras for the support :-)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •