BotB Site Hosting Upgrade R+D
BotB Academy Bug Reports and Feature Requests
Level 27 Hostist
post #101141 :: 2018.06.23 5:22pm :: edit 2018.06.23 5:28pm
  dpc, MiDoRi, Robyn and Slimeball liēkd this
  b00daw hæitd this
We've been having serious lag issues with page loads for a couple of weeks or so. I've been tailing my custom page load benchmarker log for a few days now (screenshot tweet
). It's not good. Most requests load in under 2 seconds but then... The mysql calls don't seem to be the culprit; it's the php runtime. This is most likely caused by other users on BotB's shared host account. My present theory is that some new site in this space is doing dumbs. But...

I've been with Dreamhost for about 15 years now. I have received, over the past couple years, on occasion, an email stating my user has exceeded memory limits over a thousand times. These emails give me options i.e. 1) ignore it; 2) optimize yer shit; 3) upgrade to VPS! and, so, maybe it's time for BotB to put its big boy pants on (sheesh the kid is 13 already PUT ON SOME PANTS!! :U) and move to a more isolated space.

I'm looking at Dreamhost's Basic VPS
plan. Plenty of RAM! W0W!~!! Not really info on the CPU power (and some understateed info about CPU minutes
wtf are those? what if i want websockets w/ node?). The real problem is the storage: BotB's battle/entry media is 100gb in size and said Basic Plan only offers 30gb. Boo. So I took a look at their Dream Objects
which is 2.5¢/gb/month so about $2.50/month. But then there's the bandwidth cost of 5¢/gb/month where BotB seems to average around 4.7 TERABYTES/month so $235/month. SERIOUS BOO. The VPS has unlimited bandwidth so let's go back there
: Pro VPS would hold everything but could be outgrown in a couple years. Enterprise VPS is $110/month so fuck that.

BEST CHEAPER SOLUTION?: go with the Business VPS which has 60gb of space and then cache commonly accessed media there copied from the archive cloudy guy. This sucks. It'd be awesome if it works! When will I have time? How long will the site be down while this is figured out? How much could I have finished before a switch over? How long would it take in the player to start if the VPS has to grab the media from the archive cloudy guy? Oh, and managing uploads of new media to the archive cloudy guy! D:
Level 27 Renderist
post #101142 :: 2018.06.23 5:28pm :: edit 2018.06.23 5:30pm
  anewuser, Slimeball and puke7 liēkd this
so it doesn't seem to be an option. crowdsourcing was a fail last time. botb crowdsourcing was like "occupy" movement. no real clear, concise point and goal. n00bz again won't have the money. sorry, bro. i can't give you shit this year; i'm buying a house.

just ride this bullshit out. keep optimizing the code. make more of it burdening client-side. unless you wanna ride electron00b's horse a few times.
Level 27 Pixelist
post #101146 :: 2018.06.23 8:07pm
  Slimeball liēkd this
if the 4.7 TB/mo figure is correct it's probably going outweigh all other considerations, without doing some major caching/storage overhauls like you said.
digitalocean's VPSs have bandwidth prices at least 5 times cheaper than that (e.g. $20 for 4TB + 1 cent/gb)

on the other hand, AWS bandwidth is almost 10 cents/gb/mo regardless of which fancy cache guy you use (?!)
Level 23 Mixist
post #101147 :: 2018.06.23 8:13pm :: edit 2018.06.23 9:05pm
  Slimeball liēkd this
a suggestion worth looking at (given that changing providers is an option):

scaleway isn't going to carve a hole in whoever's wallet is paying. storage wise (if you use a baremetal instance), it's just a matter of adding more disks to it, if you want to avoid rewriting code to work with the archive cloudy guy.

this solution would (at its very cheapest) be €1.99 + €2.99 + (€1 x 50gb storage)

i used digital ocean before but even that was overpriced, given the alternatives where one doesn't have to worry about any bandwidth figures.

if you want i can give you a C1 with 150gb to test it out for 3 months (or for forever, there's virtually no costs worth mentioning, regardless of how much the data transfer will be)
Level 29 Chipist
post #101160 :: 2018.06.24 8:46am
what's your internet speed at home? if it's a lot, you could maybe try hosting from home and allocating bandwidth for your server so your home computer doesn't get bogged down when it's Churnin Bits
Level 23 Chipist
post #101174 :: 2018.06.24 3:55pm :: edit 2018.06.24 3:56pm
  sleeparrow, Robyn, MiDoRi, xracecar, malmen and kleeder liēkd this
  anewuser hæitd this
lets switch to messenger pigeons

huge packet loss but if data arrives it arrives in HUGE quantities
Level 22 Renderist
post #101177 :: 2018.06.24 6:34pm :: edit 2018.06.24 6:36pm
4.7 Terabytes/month? having a media database of 100GB makes this quite possible I suppose. Is that for streaming radio or file downloads?

Nothing unlimited online (paid or free) is ever real. The torrent guys know best.

offtopic on topic, this year I spent 10$ buying food when crossing the border, and after I ate I literally shat...not a good investment...I'd rather donate to botb but what I have available it's too little to help with the costs (around 10$ which would be send via paypal/venmo)

Otherwise enjoy the logs to help ride this one out. Fun if you are into data analysis and figuring out where optimize ;D/
Level 27 Hostist
post #101188 :: 2018.06.25 8:22am :: edit 2018.06.25 10:02am
  sleeparrow, Doxic, MiDoRi, VinCMG, Sintel and Slimeball liēkd this
I think its interesting that most of the lag happens during USA business/office hours. :thinking:
Level 27 Renderist
post #101190 :: 2018.06.25 9:07am
  sleeparrow, mk7, Modus Ponens and Slimeball liēkd this
That's called "white collar productivity."

Someone came forward with a free hosting solution for us with all the resources BotB needs and more! This was about two months ago, and I was a bit too busy to really deal with the move. Since I have a 3 day weekend coming up with no plans I will start working on this migration. No promises we'll be on the new space by Tuesday, but that is presently the goal. :D/
Level 17 Chipist
post #103614 :: 2018.08.31 5:22pm
  sleeparrow, RazerBlue6 and Xyz liēkd this
So like do I need to pack or something...?
Level 26 Chipist
post #103616 :: 2018.08.31 6:50pm
Oh no I hope this doesn't fail like did.
Level 22 Renderist
post #103617 :: 2018.08.31 9:59pm
  Doxic, Modus Ponens, Savestate, RazerBlue6 and b00daw liēkd this
well, puke7 is an certified expert with duck tape.

On the other hand, nothing is forever and if he moves on he moves on. Enjoy the ride while it lasts o/

I'm glad he's doing the migration, more for his cv.
Level 27 Chipist
post #103618 :: 2018.09.01 4:18am
  Razerek, YQN, anewuser, Savestate, Robyn and RazerBlue6 liēkd this
good luck with the move : ) id say 'have fun' but that is a tad optimistic
Level 27 Renderist
post #103671 :: 2018.09.05 4:55pm :: edit 2018.09.05 4:56pm
would be very useful with this new move that botbrs at specific levels can upload dev/code related materials without prejudice. i keep seeing rainwarrior's posts time to time on nesdev being hosted on mediafire, etc. in relation to audio recordings and development thereof. plus it gives good publicity.
Level 29 Mixist
post #103673 :: 2018.09.05 5:30pm
whatever happened to
Level 27 Renderist
post #103676 :: 2018.09.05 6:17pm
i still have mine.
Level 23 Mixist
post #103694 :: 2018.09.06 3:11pm
  raphaelgoulart liēkd this
that's awesome news! kudos to the unmentioned hero
Level 27 Renderist
post #103696 :: 2018.09.06 5:23pm
is it electronoob?
Level 16 Chipist
post #103819 :: 2018.09.13 5:34am :: edit 2018.09.13 5:34am
R3M is the host. I haven't been able to get in contact however, if someone else could then that would be great!
Level 20 Mixist
post #103820 :: 2018.09.13 7:39am
R3M is the host of*
Level 27 Renderist
post #103827 :: 2018.09.13 12:50pm
Electronoob took it over
Level 27 Hostist
post #103828 :: 2018.09.13 1:19pm
I have been spending time here and there working on this. Still sorting some details out with the host. For example, if I restart a VM then it seems to fail hooking up to the network. I could try relying on the linux vm never going down, but that seems pretty dumb. ;)

LOGIN or REGISTER to add your own comments!