2nd Server Disaster in over a decade!

Azzer

Administrator
Staff member
Administrator
Joined
Dec 13, 2007
Messages
1,215
I'm really sorry to say that just when things were starting to be worked on in small quantities towards Bushtarion getting some fresh blood, some updates, and possibly a new admin, the hard-drives on the server itself had a fatal crash.

The RAID drives crashed severely (full physical meltdown) and local backups were lost, so we are having to rollback around 48 hours to the last stable remote backup. The server has had it's hard-drives replaced with brand new ones, and most of the day has been sent setting all the server software back up again.

Server has been rolled back to 19th April 2015, the early hours of Sunday morning server time.

Ticks should start up again at 20:30 GMT (that's 21:30 UK time or BST) today (Wednesday the 22nd of April), if everything remains stable.

Note that ticks starting up again depend solely on the reliability of the new server. If there are problems with the new setup the ticks may get delayed further, please watch this space!

To help speed developments along, the development multiplier has been dropped from 1.6 to 1.2. Please note that this is not retroactive - if you had a development already started, it'll still be at whatever research/construct time it was at the point you started it - only new developments will have the new heavily reduced times (exactly how the dev multiplier always works anyway, just with a rather bigger and more sudden drop). Be interesting to see if this enabled any of the lower-ranked players suddenly get their high-level developments finished before the current top alliance/players does and get a surprising advantage!


All users who made purchases after the rollback date will be given their credits back, but nothing else from the last 48 hours can be salvaged, I'm sorry! :(


Our new super-moderator Benneh has been handling a lot of the server re-configuring, a bit of a saviour, and he may have some things to add to this post, or may answer any questions anyone asks here if I don't first!


Sidenote: I was intending to make a post in the not-too distant future explaining my more recent absence/personal life, I may have to speed that up since I imagine people have some questions, but please be patient with me, I'm an old man now!
 

Azzer

Administrator
Staff member
Administrator
Joined
Dec 13, 2007
Messages
1,215
Are we going to get anything back like troops, land, developments etc?

There's no way to restore things in a rollback like this - unfortunately without the original files (which are lost, and are the cause of the 48 hour rollback), there's no way to know what happened in that 48 hours.

This means some people will be better off, some people will be worse off, but all are in the same situation - they have "timewarped" back 48 hours or so, and ticks will be on hold for now.
 

No-Dachi

Official Helper
Joined
Dec 15, 2007
Messages
975
Location
Oslo, Norway
You are aware that you effectively cut ViruS member-base in half, right? I demand bots joining to fill up the empty space!

On a more serious note, this is obviously not your fault, and I'm happy you're around to sort it out :)

Oh, and the round having gone as it is, with rank 1 firmly in lead; this isn't really an issue. If it were in the middle of a big war it would suck, but now I don't think it matters.
 

Nibles

Weeder
Joined
Jul 6, 2008
Messages
22
Location
Bracknell
Good to hear you are still kicking about Azzer, even if you are now an old man ***128518;

Can we get some expectation of what you are planning? This game has needed some refreshing changes/moderations/unit balances/ generally changes or challenges to a game that we all know too well.

I look forward to Benneh bring some new blood and a fresh POV on this game, which I still cannot believe I have been playing since I was 11 years of age... 14 years later I'm still here #BushtarionMinion

Hope you are recovering Az,
 

Dark_Angel

Landscape Designer
Super Moderator
Community Operator
Joined
Jan 11, 2008
Messages
1,978
Location
UK
The server crash has thrown up a bit of an anomoly with the dev multiplier.

My SAs are currently 23% complete from starting them at 1am on Saturday but if someone starts them when the server goes back up, they'll actually finish before me (and at least £100bn more cheaply!)

[edit] Azzer, can you clarify whether the current multiplier of 1.237 is what the multiplier would have been on Sunday? Or is the multplier treating the game like there hasn't been a crash? At the moment things look seriously out to me.

[edit 2] I've done a bit of digging and apparently the current multiplier of 1.237 is not what it was at 8am Sunday (I know of someone who started a tech at *1.6 at midday on Sunday). So the current dev multiplier does need to be highered to correct things, I think.
 
Last edited:

Benneh

Tree Surgeon
Super Moderator
Joined
Dec 14, 2007
Messages
764
Location
Cheltenham, Gloucestershire, UK.
The server crash has thrown up a bit of an anomoly with the dev multiplier.

My SAs are currently 23% complete from starting them at 1am on Saturday but if someone starts them when the server goes back up, they'll actually finish before me (and at least £100bn more cheaply!)

[edit] Azzer, can you clarify whether the current multiplier of 1.237 is what the multiplier would have been on Sunday? Or is the multplier treating the game like there hasn't been a crash? At the moment things look seriously out to me.

[edit 2] I've done a bit of digging and apparently the current multiplier of 1.237 is not what it was at 8am Sunday (I know of someone who started a tech at *1.6 at midday on Sunday). So the current dev multiplier does need to be highered to correct things, I think.

The multiplyer was actually around 1.8, we reducued it to make it what it would have been now, didn't think of current techs though, this may need to be moved back to what it would have been Sunday.

We'll see what Azzer says.
 

Nibles

Weeder
Joined
Jul 6, 2008
Messages
22
Location
Bracknell
I'm really sorry to say that just when things were starting to be worked on in small quantities towards Bushtarion getting some fresh blood, some updates, and possibly a new admin, the hard-drives on the server itself had a fatal crash.

The RAID drives crashed severely (full physical meltdown) and local backups were lost, so we are having to rollback around 48 hours to the last stable remote backup. The server has had it's hard-drives replaced with brand new ones, and most of the day has been sent setting all the server software back up again.

Server has been rolled back to 19th April 2015, the early hours of Sunday morning server time.

Ticks should start up again at 19:00 GMT (that's 20:00 UK time or BST) today (Wednesday the 22nd of April), if everything remains stable.

Note that ticks starting up again depend solely on the reliability of the new server. If there are problems with the new setup the ticks may get delayed further, please watch this space!

Can you confirm what time the round will resume? On the mainpage currently it states
"This world is not currently ticking, as it is the start of a new round.
The ticks for this new round will begin on Wednesday the 22nd of April, 2015 at 20:00:00 GMT." - that would be 21:00 UK time?

Thanks,
 

Benneh

Tree Surgeon
Super Moderator
Joined
Dec 14, 2007
Messages
764
Location
Cheltenham, Gloucestershire, UK.
I have a question. Is there anything I need to do to receive my credits back? My account is now short 26 of them. I cashed in 13 of them and gave 13 to my brother to cash in and there all gone now. Another concern of mine is that even though the credits are gone and I no longer have the game cash from it my cool down time from cashing in is still high. What if anything do I need to do to resolve this??

Mail ID 1 with any relevant info and Azzer will get looking into it. Although I do know that he has already reimbursed players who bought credits and then lost them due to the server crash.

Nibles

We had a few issues with the server during the early hours of this morning. Hence the delay even further of the restart, these have now been resolved.

Stick to the time stated in-game. Check back now and then incase of any changes. I doubt there will be though.
 

Nibles

Weeder
Joined
Jul 6, 2008
Messages
22
Location
Bracknell
Nibles

We had a few issues with the server during the early hours of this morning. Hence the delay even further of the restart, these have now been resolved.

Stick to the time stated in-game. Check back now and then incase of any changes. I doubt there will be though.

Thanks Benneh,

Appreciate your assitance with this - Im impressed how quickly you both managed to get the systems back up so quickly after (as described) old systematic data etc - great effort!
 

Azzer

Administrator
Staff member
Administrator
Joined
Dec 13, 2007
Messages
1,215
I have a question. Is there anything I need to do to receive my credits back? My account is now short 26 of them. I cashed in 13 of them and gave 13 to my brother to cash in and there all gone now. Another concern of mine is that even though the credits are gone and I no longer have the game cash from it my cool down time from cashing in is still high. What if anything do I need to do to resolve this??

Hi Shades (and anyone else who thinks Credits may have gone astray!). I've gone through all the purchases via the logs WorldPay and PayPal give me and manually upped the Bush Credits for all the user accounts who purchased at a time following the rollback point - if anyone thinks they are missing credits they purchased due to the rollback, please mail me (in-game ID 1) or email admin AT bushtarion.com with the purchase details (your account email address, the amount of credits, and any info you have from the emails you received from either WorldPay or PayPal when you purchased - eg transaction ID's from those emails is very helpful to tracking down payments within the WOrldPay or PayPal websites!).
 

Azzer

Administrator
Staff member
Administrator
Joined
Dec 13, 2007
Messages
1,215
The server crash has thrown up a bit of an anomoly with the dev multiplier.

My SAs are currently 23% complete from starting them at 1am on Saturday but if someone starts them when the server goes back up, they'll actually finish before me (and at least £100bn more cheaply!)

[edit] Azzer, can you clarify whether the current multiplier of 1.237 is what the multiplier would have been on Sunday? Or is the multplier treating the game like there hasn't been a crash? At the moment things look seriously out to me.

[edit 2] I've done a bit of digging and apparently the current multiplier of 1.237 is not what it was at 8am Sunday (I know of someone who started a tech at *1.6 at midday on Sunday). So the current dev multiplier does need to be highered to correct things, I think.

I dropped the dev multiplier to make up for the lost time (3 days total), down to 1.2 rather than the 1.65 it rolled back to, to help people speed up on newly started techs.

Not really an awful lot I can do to get people who were "mid-way" through a tech to the correct time (ie to let people who had already started a tech to take advantage of the newly decreased time) I'm afraid - so there's going to be a bit of oddity with people who were already mid-way through things.

I've added the following to the original post;

To help speed developments along, the development multiplier has been dropped from 1.6 to 1.2. Please note that this is not retroactive - if you had a development already started, it'll still be at whatever research/construct time it was at the point you started it - only new developments will have the new heavily reduced times (exactly how the dev multiplier always works anyway, just with a rather bigger and more sudden drop). Be interesting to see if this enabled any of the lower-ranked players suddenly get their high-level developments finished before the current top alliance/players does and get a surprising advantage!
 

Azzer

Administrator
Staff member
Administrator
Joined
Dec 13, 2007
Messages
1,215
How far into the round is it? Just out of curiosity.

The rollback set the game back to;
"World: Time gone: 15.36 days. Time left: 60.64 days. "

Though I have manually adjusted "ticks gone" so the round ends on the correct day, so in-game it technically says (and is):
"World: Time gone: 18.36 days. Time left: 57.64 days. "
 

Azzer

Administrator
Staff member
Administrator
Joined
Dec 13, 2007
Messages
1,215
Note that there are still some issues with the new server config making ticks get delayed - I don't want to run them until I'm confident there won't be a sudden spike of ticks all in one go or anything nasty like that. Please be patient, it is being worked on!
 

Azzer

Administrator
Staff member
Administrator
Joined
Dec 13, 2007
Messages
1,215
Everyone pray to the server gods, ticks should be starting in 9 minutes all being well...
 
Top