11 replies to this topic

[bobsi18]

Hey all,
Not really sure where this belongs, but am after a bit of a gauge on how big my site really should be...

The site has been running for about 4 years, and has about 2500 individual products. The database is about 50MB in size, and the website takes up about 330MB (the rest of my site, the email, awstats, other subdomains etc equal up to 1.2GB)

Is this reasonable? I'm looking at moving my hosting around, my current hosting environment is quite reasonable for 15GB bandwidth and 2GB data, but all the "good" webhosts that I've been recommended seem to suggest that a 1-2GB site is an extremely large site. I don't consider my site to be all that "large" but wanted to get a bit of an idea on what kind of sizes other peoples sites are?

[web-project]

Quote

Is this reasonable? I'm looking at moving my hosting around, my current hosting environment is quite reasonable for 15GB bandwidth and 2GB data, but all the "good" webhosts that I've been recommended seem to suggest that a 1-2GB site is an extremely large site. I don't consider my site to be all that "large" but wanted to get a bit of an idea on what kind of sizes other peoples sites are?

with sort of amount of products, yes it's definitely reasonable. You can ask new provider to help you to move your website, as it can be very easy using cPanel to cPanel server and without loosing any data.

[Java Roasters]

You should look at removing some of the log files instead of moving your site.

[bobsi18]

Java Roasters, on 13 September 2010, 22:30, said:

You should look at removing some of the log files instead of moving your site.

Thanks for the responses. Yeh, it's not just the size of the site I'm worried about, the host has been having problems, and I'm wondering if it's time to go to the next "grade" of hosting.

I am trying to sort out just how much space my site takes up before I do that though, as the next grade seems to have a fair bit less included space. Other things I'm looking at are to split up the domains I have on the account (I have the oscommerce one plus a few addons that are just personal sites) and to use google apps for all the email. Anyone using google apps?

We're currently running a promotion on our site, and the site is sooo slow, server load is huge, average is about 15, shooting up to 70 at times and crashing the site. I have tried to ascertain if it's my site causing the problems (there was an issue with bad coding about 3 years ago that caused problems), but my host is unable to tell me if it's my site or anothers. I'm thinking at this stage it is someone elses (because the load seems to get high for no reason - not when I have more people on the site, or when they're doing a particular action such as checking out etc), but I just don't know. I don't think the numbers that I'm getting on the site should be causing problems, it's not astronomical numbers!

[mdtaylorlrim]

The database takes up 0 space in your web file space (although your cPanel may report the database size.) Although you should monitor the log files and keep them to a reasonable size, also monitor the database backups. They do take up file space in your web site.

My rather large site contains 17,000+ products and taps out at 700+ mb in a tgz file including the sql backups of 50mb each. Usually have 3 backups on the site.

There are log files in the file space, and cache files, cache tables, sessions tables, and customers out of data baskets to look for.

[bobsi18]

mdtaylorlrim, on 15 September 2010, 04:17, said:

The database takes up 0 space in your web file space (although your cPanel may report the database size.) Although you should monitor the log files and keep them to a reasonable size, also monitor the database backups. They do take up file space in your web site.

Really? My host has cpanel, and according to "Disk Space Usage" graphs, it says Total: 1004.52 MB (includes files at this directory depth). Under "MySQL Databases", the size of all the databases (I have a few other dbs) adds to 252 MB. My "overall Disk Space Usage" is 1219.93/2000 MB. Whenever I clear out any of the databases, the usage goes down...

[mdtaylorlrim]

bobsi18, on 15 September 2010, 09:42, said:

Really? My host has cpanel, and according to "Disk Space Usage" graphs, it says Total: 1004.52 MB (includes files at this directory depth). Under "MySQL Databases", the size of all the databases (I have a few other dbs) adds to 252 MB. My "overall Disk Space Usage" is 1219.93/2000 MB. Whenever I clear out any of the databases, the usage goes down...

The sql database itself is most likely not in your file space. cPanel will report the usage because it reports disk usage, not only web space usage. Hosts limit or charge for file space so cPanel will report ALL disk space. If you backup your entire site files it will not include the database unless you used osC to do a backup, and it will include the backup, not the live site.


For example, my sql database actually resides in /var/lib/mysql/username/tablename

Edited by mdtaylorlrim, 15 September 2010, 13:02.

[bobsi18]

Ok, a slightly different question, what kind of hosting setup do you have?

I'm currently using a shared hosting environment, but am well within the limits set by my host (using 1200/2000 MB space and approx 10/25GB of bandwidth). However the server seems to be falling over regularly. I know that a shared hosting environment means that there are other sites on there that I don't have control over etc, but my site (and presumably everyone elses) slows to a crawl twice a day or so, this week (with our promo going) it's actually failed (database errors) a few times. I've been with my host for about 4 years, and consider them to be a decent host, but they don't seem concerned about this. I moved to the current server only a few months ago, the old server that was decommissioned never had issues like this. When I rang up today, they suggested that I need a dedicated server.

I tend to think this is a bit extreme, but how do I figure this out? Are there any stats etc I could compare? As I said earlier, I'm thinking of moving up to the "business grade" of hosting (would consider current hosting to be "premium budget"), but am hesitant to stay with this company.

[mdtaylorlrim]

Well, what I would do and what you would do is going to vary greatly as I have my own servers in-house. I do have a couple of customers with hosting accounts though and there are a few thing to consider...

The disk usage is no indication of performance. Outside of a massively fragmented file space the size of a site in disk usage is no indication of anything except how big your backups are going to be.

Bandwidth throttling. A lot of hosts do this to make shared hosting 'fair.' Find out if your box has this implemented. Sites will suffer when hosts are doing maintenance so if your performance hit is at the same time every day this may be the cause.

Grossly undersized memory. Nothing will cause worse performance than not having enough memory and the amount of paging going on is dragging down the box. Nothing you can do. Don't think there is anything you can use to see if this is the cause either. Not as a shared hosting customer.

Check your database tables. Any with a lot of overhead? Optimizing tables may help some.

The worse offender is a poorly written sql statement. You can look at some of your logging options in your site configuration. See if any sql statements are dogging things down.

And finally, find out other sites on your host and see if their performance suffers as your does at the same time. In your case you will probably do this first.


Surely these are only some of the things to do. Hopefully others will add much more.

Edited by mdtaylorlrim, 16 September 2010, 13:08.

[bobsi18]

mdtaylorlrim, on 16 September 2010, 13:06, said:

Well, what I would do and what you would do is going to vary greatly as I have my own servers in-house...

Thanks for your response, I will go through your post carefully and do what I can to see what's going on... Yet more problems with my site yesterday, I had an email from them regarding the high server load, saying:

Quote

I've found that your site wasn't using the inbuilt caching system that PHP on our server runs. This would have been causing your site to use more CPU resources than it had to.

I've enabled this for you now, and your site appears to be running quite well. The load on the server is also quite minimal at the moment.

I have noticed that one particular page on your site uses up a very large amount of resources each time it's run. The page is admin/whos_online.php, and when it's run the CPU usage sit around 100% for a few seconds, every time.

After this email, my whos_online page no longer works, neither did the product attributes. The attributes have been fixed, but the whos_online hasn't, and probably won't be until next week now, even though they have "24/7" support, it is my experience that nothing gets fixed on a weekend, grrrrr!!!

[hambonz]

50 meg database seems a little large for only having 2500 products...you may want to clean your database up a bit...

One of my websites has over 25,000 products on it and the database is only 53 meg, I use supertracker so it makes the database larger over time so I remove old data from it every few weeks or so, that cleans my database and that makes it much smaller when I do backups...

I had to move that site to a dedicated server because of the size, traffic and queries being made on a shared host...shared hosting is not good for larger OSC sites.

Moving it should be a snap, but always backup, backup and did I say BACKUP!

[bobsi18]

hambonz, on 23 September 2010, 06:17, said:

50 meg database seems a little large for only having 2500 products...you may want to clean your database up a bit...

Hmm, ok, I'll take a look at it, see what I can clean up. To be honest, I want to move the site over to a newer version, (it's on MS2), I was all set to begin the process this week, but then I found out that v2.3 isn't far off. So once again I'm waiting to see how far along it is - I'm going to need to do an entire re-install, not an upgrade, because the site has so much custom coding. It was all done a long time ago, and I'm hoping that I'm better at it now, so I may be able to make the site a bit more streamlined (I'm sure it's too bloated at the moment).

I have made the decision to change hosts, and have found one that seems to have good support, who has said that if there are indeed problems with the coding causing server load issues, they'll help me sort it out. Fingers crossed!

Quote

Moving it should be a snap, but always backup, backup and did I say BACKUP!

Spent yesterday doing this (had to do a bit of mucking around to get my site under "half full" so it could do a backup) - but yes, completely agree, always important to backup!!