Server Catastrophe (Resolved 3/3/2010)

Azzer · 03-03-2010, 11:42 AM

What The Hell Happened?:
Due to a situation beyond anybody's control, the server that was hosting Bushtarion over in Chicago (I live in the UK) had a catastrophe - it had been configured incorrectly for a long time without my knowledge, and a series of "dodgy" hardware setups led to it breaking in a horrendous manner - the entire RAID array failed and corrupted, and the backup systems had been disabled. A lot of this was down to the server having no active cooling systems, and being crammed in to a cabinet half the size it should have been - nothing I could have known about or seen myself

I am no longer with the previous hosts. I am now hosted directly by the data-centre itself. Technician's have been working around the clock to get a new server setup and installed at extremely short notice.

The data from the last round was lost, and a new round will have to be started. All server data - forums and accounts - has been rolled back to the start of January, an old local backup.

This has been an absolute disaster for myself, and I have been doing everything I humanly can to get things sorted and try to fix the situation. I'm really sorry this has happened - I'm a gamer myself, so I know what this must be like for you guys. I appreciate everybody that can show their support and understanding through this <3

Unable To Login?:
Are you getting a "Session timeout" every time you try to login? Simply close all Bushtarion related pages, then close your browser, re-open it, and log back in to Bushtarion.

Are you getting an error with your username/password? Perhaps you changed your password since the start of January, your password will have reverted back to the old one. Try an older password, or submit a "lost password" request from the login page.

Was your account a new account, created in the last 6 weeks or so? If so, I'm afraid it is now lost, and you will have to create a new account. Sorry

New Round:
A new round will begin ticking at 18:00 GMT on Friday the 12th March. The round will run as normal. A snapshot of where the rankings stood before the crash will be used as a manual "portal history entry" to remember "the big crash" by.

I purchased credits last round...?:
If you purchased credits last round that were unused, used for your blueprint, or used for cash purchases in the week before the crash, then you have the option to retrieve them as credits to your account (they can only go to the person that purchased the credits).

Please send me a forum PM here on the forums, or drop me an account mail from your portal page after logging in, with your name, email address, and telling me how you purchased (PayPal, WorldPay, SMS) - and any other information that will help me track down the payments correctly (do not give me any credit card numbers/passwords, I do not need them - but I may need info such as your name/address/any rough times of purchase/method of purchase).

Any further problems, or if you want to discuss any aspect of this or have any concerns at all, log on to the IRC room and leave me a PM - I may not be able to respond immediately, but leave me any questions/information, and I'll try to get back to you as soon as I can - or send me a forum PM here.

Could this happen again?:
The new hosts I'm with are a huge organisation, they own the data-centre itself, have 24/7 staff on hand for any problems, and have set up an automated remote backup system for Bushtarion, not to mention ensured that the server hardware is configured correctly, has sufficient cooling, etc. - the server is also on a much more stable network and infrastructure now. Nothing of this magnitude can happen again, and this is the only time anything this bad has happened in 8 years of running.

Casualties Of War:
The Casualties of war are:
* The Bush Wiki. This was simply unrecoverable in any state from any backup. The nature of the hot the Wiki database works is such that it was simply unrecoverable. I'm so sorry guys. There was a lot of content on there. I'm reverting to the old style Bushtarion manual, and granting certain helpers the ability to update and modify the manual pages from their helper accounts, to keep it up to date.
* The last 7 weeks worth of forum posts and forum registrations (anyone that had signed up in the past 6 weeks will need to sign up again).
* The last 7 weeks worth of game account and ID data (any new account signups in the last 7 weeks will need to sign up again, any password changes will have been undone, etc.)

Other Questions/Info:
Anything else you need help with, please ask here on the forums, or in the #bushtarioN IRC channel (click the "Live Chat" link from the in-game menu after logging in to your ID), or send a "Help Request" ticket in-game for a helper to answer you as best they can.

You can discuss this announcement here:
http://www.bushtarion.com/forums/showthread.php?t=4028

dave · 03-03-2010, 12:19 PM

Quote:

Originally Posted by Azzer

[placeholder copy & paste, details to come]
All purchases anybody makes are logged on both WorldPay/PayPal servers and are always safe, secure, and available - these are never lost.
- Azzer.

includin people who text to get theyre credits?
how are records of this kept

Markb · 03-03-2010, 12:29 PM

SMS Kambi keep records on their servers, so SMS credit purchasers are ok too

dave · 03-03-2010, 12:29 PM

awesome

Twigley · 03-03-2010, 12:58 PM

Oh hi there.

srgmjrten · 03-03-2010, 03:16 PM

Ugh, in your opinion, think you'll be able to get it back and running?

webvictim · 03-03-2010, 05:12 PM

It's already back and running, but the data cannot be recovered.

rainbowape · 03-03-2010, 05:56 PM

Sorry to hear that, it was a shock and a bit of a worry. Spent last night wondering what to do with myself.

I did get the p-unit last round but I will be purchasing it again, I urge everyone who did have them too. I realise this may be not possible for some but those that can afford it should, it would help Azzer and the game in the long run, im sure.

Cheers Azzer for the update, sure this must be a headache for you and if there's anything i could help with let me know...

Azzer · 03-03-2010, 06:49 PM

Quote:

Originally Posted by rainbowape

Sorry to hear that, it was a shock and a bit of a worry. Spent last night wondering what to do with myself.
I did get the p-unit last round but I will be purchasing it again, I urge everyone who did have them too. I realise this may be not possible for some but those that can afford it should, it would help Azzer and the game in the long run, im sure.
Cheers Azzer for the update, sure this must be a headache for you and if there's anything i could help with let me know...

You're a star

antney00 · 03-03-2010, 07:12 PM

Quote:

Originally Posted by rainbowape

Sorry to hear that, it was a shock and a bit of a worry. Spent last night wondering what to do with myself.

I did get the p-unit last round but I will be purchasing it again, I urge everyone who did have them too. I realise this may be not possible for some but those that can afford it should, it would help Azzer and the game in the long run, im sure.

Cheers Azzer for the update, sure this must be a headache for you and if there's anything i could help with let me know...

I also purchased p-unit, as I always do and always will. No problem there, but was wondering about the other 24 creds I had purchased.Also having trouble still logging in and can't get it to reset my pass.

atsanjose · 03-03-2010, 07:21 PM

Quote:

Originally Posted by antney00

Quote:

Originally Posted by rainbowape

Sorry to hear that, it was a shock and a bit of a worry. Spent last night wondering what to do with myself.

I did get the p-unit last round but I will be purchasing it again, I urge everyone who did have them too. I realise this may be not possible for some but those that can afford it should, it would help Azzer and the game in the long run, im sure.

Cheers Azzer for the update, sure this must be a headache for you and if there's anything i could help with let me know...

I also purchased p-unit, as I always do and always will. No problem there, but was wondering about the other 24 creds I had purchased.Also having trouble still logging in and can't get it to reset my pass.

get in touch with Azzer in pm or in irc, just try to remember that lots of people will be contacting him the upcoming week(s) so it can take a while to get response but iam sure you'll be compensated in everything at your request.

Martin · 03-03-2010, 07:29 PM

Quote:

Originally Posted by rainbowape

Sorry to hear that, it was a shock and a bit of a worry. Spent last night wondering what to do with myself.

I did get the p-unit last round but I will be purchasing it again, I urge everyone who did have them too. I realise this may be not possible for some but those that can afford it should, it would help Azzer and the game in the long run, im sure.

Cheers Azzer for the update, sure this must be a headache for you and if there's anything i could help with let me know...

I will also be buying the P-Unit again. The ony thing which I was dissapointed about was that I lost my award for bribing which Azzer has said (out of kindness and due to me doing so very publically) he will add into my awards. In order to allow the 'good-faith' to continue, I will still buy a P-Unit for myself and a couple of others.

Well done on the fast response, last round was over as far as I was concerned anyway

Ezekiel · 03-03-2010, 09:53 PM

as long as i get those credits back from last round, ive got no problem buying this round. need to keep the game going and all that, especial now this ordeal has cost a fortune.

srgmjrten · 03-03-2010, 11:15 PM

I'll continue to purchase credits as always, I know this ain't cheap for ya' man. Glad to see that you got everything together as quickly as you could.

Ranzou · 03-03-2010, 11:57 PM

When did the new signup period start? Have people voted on DOFG twice already?

03-03-2010, 11:42 AM	#1
Azzer Administrator Garden Designer Join Date: Dec 2007 Location: Huddersfield, UK Posts: 1,169	Server Catastrophe (Resolved 3/3/2010) What The Hell Happened?: Due to a situation beyond anybody's control, the server that was hosting Bushtarion over in Chicago (I live in the UK) had a catastrophe - it had been configured incorrectly for a long time without my knowledge, and a series of "dodgy" hardware setups led to it breaking in a horrendous manner - the entire RAID array failed and corrupted, and the backup systems had been disabled. A lot of this was down to the server having no active cooling systems, and being crammed in to a cabinet half the size it should have been - nothing I could have known about or seen myself I am no longer with the previous hosts. I am now hosted directly by the data-centre itself. Technician's have been working around the clock to get a new server setup and installed at extremely short notice. The data from the last round was lost, and a new round will have to be started. All server data - forums and accounts - has been rolled back to the start of January, an old local backup. This has been an absolute disaster for myself, and I have been doing everything I humanly can to get things sorted and try to fix the situation. I'm really sorry this has happened - I'm a gamer myself, so I know what this must be like for you guys. I appreciate everybody that can show their support and understanding through this <3 Unable To Login?: Are you getting a "Session timeout" every time you try to login? Simply close all Bushtarion related pages, then close your browser, re-open it, and log back in to Bushtarion. Are you getting an error with your username/password? Perhaps you changed your password since the start of January, your password will have reverted back to the old one. Try an older password, or submit a "lost password" request from the login page. Was your account a new account, created in the last 6 weeks or so? If so, I'm afraid it is now lost, and you will have to create a new account. Sorry New Round: A new round will begin ticking at 18:00 GMT on Friday the 12th March. The round will run as normal. A snapshot of where the rankings stood before the crash will be used as a manual "portal history entry" to remember "the big crash" by. I purchased credits last round...?: If you purchased credits last round that were unused, used for your blueprint, or used for cash purchases in the week before the crash, then you have the option to retrieve them as credits to your account (they can only go to the person that purchased the credits). Please send me a forum PM here on the forums, or drop me an account mail from your portal page after logging in, with your name, email address, and telling me how you purchased (PayPal, WorldPay, SMS) - and any other information that will help me track down the payments correctly (do not give me any credit card numbers/passwords, I do not need them - but I may need info such as your name/address/any rough times of purchase/method of purchase). Any further problems, or if you want to discuss any aspect of this or have any concerns at all, log on to the IRC room and leave me a PM - I may not be able to respond immediately, but leave me any questions/information, and I'll try to get back to you as soon as I can - or send me a forum PM here. Could this happen again?: The new hosts I'm with are a huge organisation, they own the data-centre itself, have 24/7 staff on hand for any problems, and have set up an automated remote backup system for Bushtarion, not to mention ensured that the server hardware is configured correctly, has sufficient cooling, etc. - the server is also on a much more stable network and infrastructure now. Nothing of this magnitude can happen again, and this is the only time anything this bad has happened in 8 years of running. Casualties Of War: The Casualties of war are: * The Bush Wiki. This was simply unrecoverable in any state from any backup. The nature of the hot the Wiki database works is such that it was simply unrecoverable. I'm so sorry guys. There was a lot of content on there. I'm reverting to the old style Bushtarion manual, and granting certain helpers the ability to update and modify the manual pages from their helper accounts, to keep it up to date. * The last 7 weeks worth of forum posts and forum registrations (anyone that had signed up in the past 6 weeks will need to sign up again). * The last 7 weeks worth of game account and ID data (any new account signups in the last 7 weeks will need to sign up again, any password changes will have been undone, etc.) Other Questions/Info: Anything else you need help with, please ask here on the forums, or in the #bushtarioN IRC channel (click the "Live Chat" link from the in-game menu after logging in to your ID), or send a "Help Request" ticket in-game for a helper to answer you as best they can. You can discuss this announcement here: http://www.bushtarion.com/forums/showthread.php?t=4028 __________________ - Azzer. www.bushtarion.com

03-03-2010, 12:29 PM	#3
Markb IRC Operator Harvester Join Date: Dec 2007 Location: /home/mark/ Posts: 127	SMS Kambi keep records on their servers, so SMS credit purchasers are ok too __________________ NetGamers CSC Senior Administrator Planetarion Team Member #Bushtarion Channel Operator IRC/Bushmail: Markb

03-03-2010, 03:16 PM	#6
srgmjrten Beginner Join Date: Mar 2010 Location: Fort Bliss, Texas Posts: 2	Ugh, in your opinion, think you'll be able to get it back and running? __________________ "That's funny... your daughter calls ME her daddy, too."

03-03-2010, 09:53 PM	#13
Ezekiel Harvester Join Date: Jul 2009 Posts: 186	as long as i get those credits back from last round, ive got no problem buying this round. need to keep the game going and all that, especial now this ordeal has cost a fortune. Last edited by Ezekiel; 03-03-2010 at 11:11 PM.

03-03-2010, 11:15 PM	#14
srgmjrten Beginner Join Date: Mar 2010 Location: Fort Bliss, Texas Posts: 2	I'll continue to purchase credits as always, I know this ain't cheap for ya' man. Glad to see that you got everything together as quickly as you could. __________________ "That's funny... your daughter calls ME her daddy, too."

03-03-2010, 12:29 PM	#4
dave Harvester Join Date: Jan 2010 Location: Northants, England Posts: 180	awesome

03-03-2010, 12:58 PM	#5
Twigley Landscape Designer Join Date: Dec 2007 Location: UK Posts: 2,217	Oh hi there.

03-03-2010, 05:12 PM	#7
webvictim Pruner Join Date: Nov 2009 Location: UK Posts: 90	It's already back and running, but the data cannot be recovered.

03-03-2010, 05:56 PM	#8
rainbowape Planter Join Date: Mar 2008 Location: E. Yorkshire Posts: 32	Sorry to hear that, it was a shock and a bit of a worry. Spent last night wondering what to do with myself. I did get the p-unit last round but I will be purchasing it again, I urge everyone who did have them too. I realise this may be not possible for some but those that can afford it should, it would help Azzer and the game in the long run, im sure. Cheers Azzer for the update, sure this must be a headache for you and if there's anything i could help with let me know...

03-03-2010, 11:57 PM	#15
Ranzou Harvester Join Date: Dec 2007 Location: London, England Posts: 107	When did the new signup period start? Have people voted on DOFG twice already? __________________ My User Profile