I've had to make a few changes to the site regarding the guest users. I've pulled the Search Box and the Activity Stream functions. Most of you might ask why? The thing is that both functions are server heavy. The problem is that the bots and spiders are also using these functions as well, increasing the server load needlessly. Both Activity Stream and Search box are heavy CPU and hard drive hitters. This meant I had to pull the two functions from the guest users since there is no way for me to stop the spider and bots from using the functions. Average traffic on the site can range anywhere from 150-300 connections at any one time.

 

These functions are given to registered members and above now. This has reduced the amount of server overhead for the moment. 

 

This all stems from the June 12th when the site went down for short time. This was caused by excessive server load and the server basically gave up. Now systematically rebuilding and tuning from the ground up again I found that there is an issue with the new search engine (ElasticSearch) and ended up reverting back to the old school MySQL server search. I've not given up yet but researching the problem with the software to resolve it and bring it back online quickly. 

Edited June 17, 20186 yr by Mopar1973Man

 

UPDATE

 

As I've been doing my research of the issue of the server going down I found that Elasticsearch was overstepping the results. This was creating huge server loads. I'm going to neck the results down to about 5 pages worth. There is 25 results per page and 5 pages will give 125 result links. Elasticsearch is back up again and functional.

I was wondering what happened the other day. We are so wide open on the job that I forgot about it when I got home that evening.

I've got the Elasticsearch necked down to 125 results or 5 pages worth of results. Really did reduce the amount of load on the server. Orignally, it was creating 10,000 to 12,000 results or 40 pages. The max limit of the Elasticsearch is 10,000 results so when it exceeded 10k limit it would create an error in the logs and then hang the server in heavy load condition. 

 

I've got a bit of memory configuration to still deal with but all software is back up and running. I'm still below 50% server load (45% or 0.90 1-minute) with everything running but getting a memory warning. (Blue text). I've got 182 connections on the website and still holding good.