Server migration

Tell us what to do, or volunteer to do it.
Forum rules
  • The Committee of Action is an apparatus of struggle.
  • There is no sense in guessing beforehand precisely what strata of the toilers will be attracted to the creation of Committees of Action: the lines of demarcation in the struggling masses will be established during the struggle itself.
  • Real mass elections of the Committees of Action would automatically eject the bourgeois middlemen from the ranks of the People’s Front and thus blow to smithereens the criminal policy dictated by Moscow.
User avatar
Karl_
Nyaaaaaaa~!
Nyaaaaaaa~!
Joined in 2008
Contact:

PostServer migration
by Karl_ » Tue May 21, 2019 9:01 pm

We are currently on a £40/mo physical server with OVH [4/8 cores/threads; 32 GiB RAM; 2 TiB HDD (x2 RAID)]. It's a legacy server they no longer offer, but it's comparable to a SoYouStart XEON-SAT-1-32.

We plan to downgrade to a £20/mo virtual server, a Hetzner CX41 [4/4 cores/threads; 16 GiB RAM; 160 GiB SSD (x2 RAID)]. It's worse overall, but cheaper, and the SSD is actually an upgrade.

People who know about hosting: Are we getting the best deal we can? I think £20/mo should be our budget, though we could go a few quid over for significant improvements.

User avatar
Green Gecko
Treasurer
Joined in 2008
Location: Sussex
Contact:

PostRe: Server migration
by Green Gecko » Tue May 21, 2019 11:35 pm

I would agree that the current server is overkill as we barely skim more than a few % CPU usage even when peak times are taken into account. I'm prepared to put fetishism for a dedicated server aside for a virtualised instance, provided it's not too hard to migrate back if there are significant performance drawbacks;- usually some resource scaling is offered by virtualised (cloud) hosting to address any shortcomings, so we should be able to take advantage of that reduced cost/efficiency.

♥ gaems | t: @GRcade | FB: GRcadeUK | YT: GRcadeVideo | Twitch: GRcadeUK
Image
Image
User avatar
Winckle
Technician
Joined in 2008
Location: Liverpool

PostRe: Server migration
by Winckle » Wed May 22, 2019 4:19 pm

It's intended more for developers, but what about something like linode?

We should migrate GRcade to Discourse.
User avatar
Green Gecko
Treasurer
Joined in 2008
Location: Sussex
Contact:

PostRe: Server migration
by Green Gecko » Wed May 22, 2019 4:49 pm

On that front, I've also considered AWS, so that our costs are flexible based on what we actually use, but that might be to much of a headache. The nice thing about Amazon is that you can pay basically nothing during downtimes (so during the night), but the variable nature of it makes it difficult to cost (I tried doing it within a company to come up with a pricing structure for our product and it was a strawberry floating nightmare). Say for example some thread goes viral and we get rammed, cost might go up by 1000% but we've already made savings to cover it. So it needs something like a small overdraft/cash reserve for contingency rarther than "oh look this is only costing us $10 a month, let's spend that donation here or that add revenue here"). Which isn't something I've ever done but something to bear in mind.

Basically you sign up for EC2 to run your server instances and AWS to host your files and the two interact with each other using this concept called "buckets" (i.e. buckets of data), but it's a bit weird and nothing really explained. But once it's set up it behaves like a normal server.

There's also WebFaction.com which is oriented more around running web-apps and is popular with programmers I know building startup webapps/services at low cost but starts at $10 per month and is fully managed, cloud server is $20 per month. There environments are CentOS 7 with DNS, e-mail, database and whatever script packages you want so PHP Python etc (I might be interested in doing some basic Python development some time). There's loads of room to move around resources-wise whether that's paying an extra fee/add-on or moving package.

Image

The problem with those kinds of environments seems to be variable RAM usage with some CPU peaks, so we can end up paying for that overhead that we never really need.

The "cloud hosting" type setups, if we're going to go into virtualisation rather than dedicated hardware anyway, are an option too. UK company Krystal (you can actually go to free staff/customer party once a year which is nice) are offering some reasonable packages for £30/m or less (with the benefit we don't pay currency exchange fees and could theoretically claim back 20% VAT without much trouble) and they're pretty hot on the optimisation front, and are all SSD clusters to. With a UK customer we're also less likely to see sudden price changes due to Brexit / weak pound which has already happened with OVH/SoYouStart (an extra quid a month so not much but it happened).

https://krystal.co.uk/cloud-hosting

or if we top that out

https://krystal.co.uk/pci-compliant-hosting

I've tested their roundtrip latency which is pretty good and they mention 1.5 tbps of DDoS protection which is handy if we get hit by a scriptkiddy little banana split again (that still servers as my Worst GRcade Time of All Time). They limit it to 100 sites per server. The package is more for eCommerce but may be worth speaking to them - I have an account so can send some messages.

Of course the advantage of having a traditional web server whatever the actual infrastructure we're on is we do know what we're doing and what our resource usage is like, whereas we don't always fit into the infrastructure like the old VPS and VDS we had which caved in during lots of consecutive users or searches etc (which include's "view my own posts" etc.) Largely due to our database being strawberry floating massive, but there's another topic for that.

The pain with a fixed package is we tend to be paying for more than we need the majority of the time and scaling is difficult. Cloud infrastructures have come a long day since the days of "pay A get XYZ or otherwise upgrade and strawberry float around to move to B", with options to adjust resource limits on the fly. That's all thanks to virtualisation and many-core setups spread across a datacentre, with the benefit it's greener and cheaper too.

♥ gaems | t: @GRcade | FB: GRcadeUK | YT: GRcadeVideo | Twitch: GRcadeUK
Image
Image
User avatar
Karl_
Nyaaaaaaa~!
Nyaaaaaaa~!
Joined in 2008
Contact:

PostRe: Server migration
by Karl_ » Wed May 22, 2019 6:39 pm

@Winckle: Aye, I have a little dev instance with Linode and they're usually really good. I had a little downtime a month or so ago, but apart from that it's been great.

@Gecko: Thanks for the input! I think a good place to start might be listing what we can get for our cash at each of the providers. Let's all add to this list.

Package -- Price -- Type -- Cores/Threads -- Memory -- Storage -- Comments
Amazon t3a.small + 50 GiB SSD -- ~£20/mo -- virtual -- 1/2 burst -- 2 GiB -- 50 GiB SSD cloud storage -- need to be careful of weird CPU credit system
Hetzner CX41 -- ~£20/mo -- virtual -- 4/4 shared -- 16 GiB -- 160 GiB SSD RAID -- a budget company with fair-to-mixed reviews
Linode 4GB -- ~£20/mo -- virtual -- 2/2 shared -- 4 GiB - 80 GiB SSD RAID -- a more reputable company
WebFaction Cloud 2GB -- ~£30/mo -- virtual -- 2/2 dedicated -- 4 GiB -- 45 GiB SSD RAID -- dedicated cores is a plus, but part of GoDaddy which is kind of yuck*
Krystal Sapphire -- ~£30/mo -- virtual -- 2/2 shared -- 2 GiB -- 50 GiB SSD RAID -- fully managed, free domain name, good support

* (EDIT: or at least their reputation was bad a few years back, I don't know what they're like now)

User avatar
Green Gecko
Treasurer
Joined in 2008
Location: Sussex
Contact:

PostRe: Server migration
by Green Gecko » Wed May 22, 2019 10:37 pm

I do know one thing bad about webfaction - if you exceed the limits on the package you're on, they freeze the service immediately and require you to upgrade. But maybe my director just wasn't on top of things (by which I mean they definitely weren't), and nobody was monitoring as they weren't being paid to :)

I haven't looked at it for several years so I wouldn't place too much weight on that one, just an idea of the sort of deals you can get for developer/fast environment deployment sort of offerings.

Krystal on the other hand had excellent reviews when I lookd into it just 3 months ago. They're based on cPanel so a bit run of the mill and maybe restrictive but easy for anyone to dip into. There are very few UK/European hosting companies around that seem to give a gooseberry fool and haven't been bought by Go Daddy (Paragon Internet t/a tsohost, WebFusion) and aren't charging ridiculous money for having a datacentre in London (Serverstream which was register1).

It's certainly worth bearing in mind we were with Hetzner for about 12-24 months I can't recall any problems. Which does raise the question, why did we leave? Was it to get the price down from £70-ish?

♥ gaems | t: @GRcade | FB: GRcadeUK | YT: GRcadeVideo | Twitch: GRcadeUK
Image
Image
User avatar
Winckle
Technician
Joined in 2008
Location: Liverpool

PostRe: Server migration
by Winckle » Wed May 22, 2019 11:40 pm

If I recall Hetzner had frequent routing issues

We should migrate GRcade to Discourse.
User avatar
Karl_
Nyaaaaaaa~!
Nyaaaaaaa~!
Joined in 2008
Contact:

PostRe: Server migration
by Karl_ » Wed May 29, 2019 11:34 pm

Cross-posting part of a conversation I had about this:

Minty14 wrote:Oh boy, when do you think the board will be unlocked? I work for a cloud provider and would be happy to chip in :)
[...]
Oh don't worry, I wouldn't recommend [my company] in this case, we're a business focused provider and I wouldn't consider us suitable in this case (it's GTT in case you were curious :P ). However, I do have an unlimited test account should you want dev space, or anything else that doesn't need to be too permanent.

Hopefully I can chip in with decent advice and I'll happily help with any of the migrations etc. I'd say you're on the right track already to be honest and it's definitely wise to move away from dedicated hardware. Don't think about AWS / Azure etc, the management overheads far outweigh the benefits for this use case. I'd definitely add Amazon Lightsail to your considerations though, the pricing is similar to what you've found so far and the AWS tech underpinning it makes it reliable.

EDIT: Meant to include a link to Lightsail pricing: https://aws.amazon.com/lightsail/pricing/

Thanks for your thoughts Minty14.

User avatar
Minty14
Member
Joined in 2008
Location: Nottinghamshire
Contact:

PostRe: Server migration
by Minty14 » Thu May 30, 2019 12:04 am

I'll post more verbose thoughts after a good night's sleep. :)

User avatar
Minty14
Member
Joined in 2008
Location: Nottinghamshire
Contact:

PostRe: Server migration
by Minty14 » Thu May 30, 2019 11:20 am

The way I see it, you have a few key areas to look at when considering your options.

Cost - You're very unlikely to beat £20pm for 4cpu/16RAM/160GB in terms of pure cost unless maybe you signed a long term contract which would be irresponsible. For example, with Amazon Lightsail you get 2cpu/4RAM/80GB for $20 dollars (and even a step up to $40 doesn't get you equal specs to the Hetzner VM.

Management - Being realistic, this isn't a full time job for anyone and is volunteer run. Using a more granular service like AWS would introduce a set of management overheads that I don't see as justified for this type of platform. Services like DigitalOcean, Lightsail etc provide the easiest management and can be spun up and down quickly and easily. I don't have much experience with the more traditional (fair to say more 'static'?) providers like Hetzner, so can't comment there.

Security - Most services will provide the tools you need, we just need to make sure that whatever is chosen has robust and easy to manage options to protect the platform.

Support/Reliability - The bigger names will generally be the most robust, but again, I have no experience with Hetzner, so Karl et al will know what their support service is like generally and what level of reliability they provide. Back up tools and strategy are key here too - I don't know what's in place at the moment, but we should be looking to maintain or exceed whatever level of recovery we currently have.

I'd also consider a hybrid approach too - for example, you could back off media and object storage or backups to a service like S3 fairly economically and easily.

I'd say Hetzner is your best bet due to the low cost, and from the looks of their website, the service looks pretty good. The most important thing is moving away from dedicated hardware, which is overkill and far less scalable and portable.

User avatar
False
COOL DUDE
Joined in 2008

PostRe: Server migration
by False » Fri May 31, 2019 11:17 am

aws is a hassle - Ive used it personally and in a corporate space and its a hassle

the thing with server space is that you have to remember we used to run a lot of dedi gaming servers and we just dont anymore, this is wasted capacity

I would personally lean to the hetzner thing but Ive never had issues with linode

part of moving over Id nuke all of the game server stuff because its just not utilised at all, but leave space to put a couple of shitty ones in like minecraft or whatever - also mumble is still utilised and has features that people (myself included) prefer over discord so that should remain

t. literal pro server boi

Image
User avatar
Winckle
Technician
Joined in 2008
Location: Liverpool

PostRe: Server migration
by Winckle » Fri May 31, 2019 11:37 am

False wrote:aws is a hassle - Ive used it personally and in a corporate space and its a hassle

the thing with server space is that you have to remember we used to run a lot of dedi gaming servers and we just dont anymore, this is wasted capacity

I would personally lean to the hetzner thing but Ive never had issues with linode

part of moving over Id nuke all of the game server stuff because its just not utilised at all, but leave space to put a couple of shitty ones in like minecraft or whatever - also mumble is still utilised and has features that people (myself included) prefer over discord so that should remain

t. literal pro server boi

Mumble could easily run on a linode, and barely affect anything, minecraft however is very heavy going.
Agreed on getting rid of the game server stuff to save on resourcing.

We should migrate GRcade to Discourse.
User avatar
False
COOL DUDE
Joined in 2008

PostRe: Server migration
by False » Fri May 31, 2019 12:32 pm

we've run a minecraft instance on a nuc recently.. its not so bad - I was just using it as an example of something small and temporary anywho

Image
User avatar
Minty14
Member
Joined in 2008
Location: Nottinghamshire
Contact:

PostRe: Server migration
by Minty14 » Fri May 31, 2019 1:07 pm

I think for performance, management and security's sake, it's probably better to dedicate any new host to the forum alone (+mumble), and if the demand is there, build and manage a separate instance for game servers.

User avatar
Karl_
Nyaaaaaaa~!
Nyaaaaaaa~!
Joined in 2008
Contact:

PostRe: Server migration
by Karl_ » Mon Jun 03, 2019 2:44 pm

Winckle wrote:.
Minty14 wrote:.
False wrote:.
Green Gecko wrote:.
OrangeRKN wrote:.

No pressure on anyone, but hypothetically if the six of us put ~£3.30 each in, we could set up a Hetzner CX41 instance for a month and actually test how well it handles everything. Happy for all of us to have sudo access. Amongst other things, we can do some stress tests, and set up a logger to spot routing issues (I know Winckle has had some concerns about their network infrastructure). If it's good we can keep the instance up and start migrating there, and if it's crap at least we know we need to use a different provider.

Not suggesting this is the only way forward (please discuss!) but might be worth it to make a bit of progress and it'd only be a few quid each.

User avatar
Karl_
Nyaaaaaaa~!
Nyaaaaaaa~!
Joined in 2008
Contact:

PostRe: Server migration
by Karl_ » Mon Jun 03, 2019 3:15 pm

Off the top of my head I am happy for Jiggles and/or Errkal and/or Earfolds to help out with this too. I think that's GRcade's tech illuminati tagged into this.
jiggles wrote:.
Errkal wrote:.
Earfolds wrote:.

Again, literally zero pressure on any individual, just trying to build some momentum.

FAO Errkal: I realise you've stopped doing tech stuff on here and are taking a break, possibly entirely done with it -- nonetheless I wanted to tag you just in case. Don't want you to feel like you couldn't one day help out again in the future, be that with this ongoing effort, or something else if it catches your eye. :)

EDIT: Oh, maybe Laga and Qikz? Can't remember how Linux-y / programming-y their work is. Just in case:
Lagamorph wrote:.
Qikz wrote:.

User avatar
Lagamorph
Member ♥
Joined in 2010

PostRe: Server migration
by Lagamorph » Mon Jun 03, 2019 3:21 pm

Karl_ wrote:
Lagamorph wrote:.

I generally work with Powershell and a bit of Javascript lately. Not very Linux-y I'm afraid.

Lagamorph's Underwater Photography Thread
Zellery wrote:Good post Lagamorph.
Turboman wrote:Lagomorph..... Is ..... Right
User avatar
Karl_
Nyaaaaaaa~!
Nyaaaaaaa~!
Joined in 2008
Contact:

PostRe: Server migration
by Karl_ » Mon Jun 03, 2019 3:22 pm

Fair, thanks anyway!

User avatar
Lagamorph
Member ♥
Joined in 2010

PostRe: Server migration
by Lagamorph » Mon Jun 03, 2019 3:25 pm

I have been meaning to get more into Linux for years now, and it's really something I should do given that our vRealise appliances all run on Unix.

Lagamorph's Underwater Photography Thread
Zellery wrote:Good post Lagamorph.
Turboman wrote:Lagomorph..... Is ..... Right
User avatar
Karl_
Nyaaaaaaa~!
Nyaaaaaaa~!
Joined in 2008
Contact:

PostRe: Server migration
by Karl_ » Mon Jun 03, 2019 3:27 pm

Well if you spot something in this forum and think "oh, I wouldn't mind learning how to do that", just post and say so -- there's a lot to get done so I think it'd be good to have the extra hands!


Return to “Committee of Action”

Who is online

Users browsing this forum: No registered users and 1 guest