My experiences on page speed
Hello everybody for the last 2 weeks I was dealing with some tweaks to increase UNA page speed. I would like to share my experiences, for the non professionals it may answer some frequent questions.
Web Server & PHP
I tested 3 web servers in this period APACHE, NGINX, LITESPEED
PHP was 7.2 in all variants.
The tested identical Homepage has timelinemany pictures videos etc. it is full as can be. cache setting is closed from developer >pages>homepage>settings>cache =0
test server is 1 GB 1 vCPU
PHP 7.1 was faster then 7.2, but there were possible setting problems with images. (From studio use GD library did the trick for php7.1)
PHP 7.3 doesnt have the XML module in the repository, it doesnt work properly with UNA yet.
APACHE
With mod-php it is the slowest, with php-fpm faster, it uses .htaccess files for directory specific settings. memory consumption is quite high with mod-php.
homepage with timeline reload (F5) with mod-pgp 24 seconds, second load with clicking to homepage link 18 seconds
didnt test apache with php-fpm @Baloo may have an idea on it.
NGINX
Well known for being faster and lighter.
It uses php-fpm, doesnt use .htaccess files,
modrewrite syntax is a little different you have to translate them to NGINX style.
Memory and CPU consumption dropped drastically.
If you do correct settings to opcache it will show its positive effect.
homepage with timeline reload (F5) with php-fpm15 seconds, second load with clicking to homepage link 9,3 seconds
LITESPEED
This is a new web server comparing to Apache or NGINX.
It can use the apache conf files and .htaccess files.
Apache compatible. Uses its own php fpm.
(there is a openlitespeed free version but this doesnt have apache compability.)
İt has a free enterprise version for servers up to 2 GB, I tested with this one.
it complies different php versions, needs a bit of learning the admin interface.
homepage with timeline reload (F5) with 12 seconds, second load with clicking to homepage link 6,2 seconds, quite impressive.
HARDWARE TESTS
The first surprising result for the hardware was a 1 GB 1 vCPU server may open UNA pages faster then a 32 GB 8 vCPU server. I know unbelievable but this was what I got on Digital Ocean servers.
This was mind blowing, I tested again and again with different servers same results...
Then I started to research...
I found that php is not multitreated! It has somehow multireading for the cgi mode only.
So many vCPU is not benefiting php, it uses one of them for a certain connection.
And when I checked the vCPU cores it was solved.
The CPU frequency was almost same but the only difference was:
in the 1 vCPU server the L2 cache is 1 MB, while in the multiCPU ones it was only 256 KB.
So as PHP is using only one core, that one with 1 MB L2 cache beats all others alone :)
Though more memory and CPU may help for more instant traffic and mysql part.
MYSQL => Mariadb
Mysql uses more resources then php and web server especially CPU on a php-fpm setting.
Making the correct cache settings is important. Read from reliable sources and apply.
First thing first I replace mysql with mariadb.Most important values are
innodb_buffer_pool_size = to be ~70 % of total memory
key_buffer = not very sure but to be ~1/8-1/4 of innodb_buffer_pool_size gives good results
While checking things with mysqltuner I noticed that inno caches are not used at all.
I disabled inno:
skip-innodb or innodb = off
comment all innodb settings
selected Aria (advanced version of MyIsam) as the default db engine reorginize the settings.
+php7,1 +nginx
homepage with timeline reload (F5) with 11,7 seconds, second load with clicking to homepage link 7,18 seconds
SEPERATE MYSQL SERVER SCENARİO
To see how will be the seperate mysql server I used the same 1 GB 1 vCPU servers one like above and one with only mysql.
In this scenario yes CPU and memory usage dropped, but the private network between servers was the bottleneck.
homepage with timeline reload (F5) with 25 seconds, second load with clicking to homepage link 18 seconds. For the slow network issue I have a ticket and waiting the answer. I will update here.
-----------------------------------------------------
It is seen that the network connection is not the problem but the settings in php and mysql is important.
time mysql -h 10.133.78.242 -u'una' -p'XXXX' -e'show status'
real 0m 0.073s
user 0m 0.011s
sys 0m 0.003s
-----------------------------------------------------
After doing the above mentioned changes on Mariadb like disabling innodb completely, I got 13 seconds from timeline instead of 18.
I dont have an idea if disabling innodb is suitable for una or not??? LeonidS
Of course if you dont want to deal with all these, you have the UNA ready hosting from boonex and no worries :)
If you have any questions about my tests please feel free to ask here. But I have to mention that I am not a professional I will try answer within the limits of my knowledge and small experience.
-
I did the same thing, we take advantage of the calm to put things in order and improve the details.
For my part I dropped the timer because the loading time of the home page varies greatly depending on the data it displays. For example, a post in the feed with embedly takes time. Also depends on the pictures posted etc ... So I'm just based on my feelings.
There is a moment that I try to run php5 and php7 simultaneously on the same server, the reason is that I have a Dolphin site and even if the kernel and boonex modules have been updated for php7, the modules third parties did not follow.
Finally I got to know php-fpm because it is a good solution to run several versions simultaneously. Unfortunately I lost a lot of time doing tests with php7.2-fpm with which I got results much slower than php5.6 against any logic. Finally, it seems that 7.2-fpm + relates to a problem with SQL queries. Which forced me to install 7.1 in the end.
So I would just say this. On the same server, so I left php5.6 + apache, which gave me a speed that I would qualify as acceptable without being exceptional. Then I went to php5.6-fpm. Already nothing that is a huge gain, I would say load time largely divided by two. Then I switched to php7.1-fpm, still on the same machine and there I would say loading time still divided by 3. Of course like Cem, I also refined the config of mysql, opcache and fpm.
Well I would say that now it is moving and what is fantastic compared to my old basic config is that now I can load the page with a lot of content and queries, we do not feel the difference. Before I had to be very careful. The difference is incredible!
Cpu: 1 x i7
Ram: 24 GbNo SSD
-----------------
OpCache
opcache.enable = 1
opcache.interned_strings_buffer = 32
opcache.max_accelerated_files = 100000
opcache.max_wasted_percentage = 15
opcache.validate_timestamps = 1
opcache.revalidate_freq = 1
------------------------------------
Mysql
key_buffer_size = 64M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 192
max_connections = 250
table_open_cache = 100000
max_heap_table_size = 128M
tmp_table_size = 1024M
innodb_buffer_pool_size = 10GB
query_cache_limit = 2M
query_cache_size = 32M---------------------------------------
PHP
memory_limit = 1024M
---------------------------------------
Pool
pm = dynamic
pm.max_children = 20
pm.start_servers = 7
pm.min_spare_servers = 7
pm.max_spare_servers = 15
pm.max_requests = 500---------------------------------
Do you think that ssd can still improve? I may be testing soon...
-
Thank you for sharing such info.
SSD will definitely improve the speed, however if you wisely use your memory (since you have it enough) then the speed improvement will not be very significant.
I would suggest to use mysqltuner.pl to check your mysql settings, also performance tips from phpMyAdmin may help.
-
thanks Alex :)
-
Comment by unknown is hidden. -
Thanks Alex, I'm going to try all that to fine tune
-
my 2 cent and two questions probably for Alex T⚜️ or LeonidS
- why is UNA still using APC instead APCu? (so needs the APC-bc module)
- why is it loading a platform.js from cdn.embedly.com on every page even if I have disabled Embedly (and all other) in the settings?So back to technical speed things. I've tried to speed up as much as possible as described below:
using a quadcore server atm from the older generation (Intel Xeon E3-1220 @3.1 GHz) with 4 GB RAM (new server is coming soon) for UNA 10.0.0-B2. Running Ubuntu 16.04 with PHP 7.3 FPM Apache. As frontend there is a nginx proxy, serving images and static files directly.
Also lately added APCu (with APC-bc) as cache, enabled all caches in UNA.As result getting on Google Pagespeed on my UNA site lousy 22 points on Mobile and mediocre 74 on Desktop.
It has 77 request and 13 MB volume (damn pictures!) loading in 3-4 seconds. Worst thing to load are images and album pages, depends on who is online and how many are browsing images, but it can be veeeery slow. No real idea how to cache this huge image blobs from UNA good. It was better with single files in Dolphin IMHO.
So beside from getting a new server I don't see much more options to get more speed.
-
why is UNA still using APC instead APCu? (so needs the APC-bc module)
thank you for noticing this - https://github.com/unaio/una/issues/2277
why is it loading a platform.js from cdn.embedly.com on every page even if I have disabled Embedly (and all other) in the settings?
please specify how you've disabled Embedly ?
No real idea how to cache this huge image blobs
Images are cached by default for 30 days, it's specified in cache_control field in sys_objects_storage table.
It was better with single files in Dolphin IMHO.
Could you please describe it in more details, what you mean here ? and why you think it was better in Dolphin ?
-
Hi Alex T⚜️
Thank you for considering APCu!
Embedly: In Studio > Settings > General > Embeds Provider is set to None. So from my understanding I wouldn't expect any sideload from Embedly.
Images: I've checked again and currently I am not completely sure how you save the images. My first impression was you put them in some mega "binary blobs", but now I just find some binary files in /storage/ tree.. so might be mistaken on that part.
-
trunky you are welcome
Embedly: Froala loads Embedly JS, since it's possible to insert Embedly via Froala editor in any form where Froala is used, so it can appear on any page, this is the reason it's loaded on every page. If you disable Froala editor then Embedly JS will not be loaded.
Images: I suspect that it maybe you've enabled "Enable cache for images" setting in Developer module and maybe increased "Cached image max size (in Kb)" value, it will cause that images which are smaller than cached image max size will be displayed as base64 inline in HTML.
-
Thanks Alex T⚜️ for the input... I've removed all embedly references in the Froala module and now it looks much better, no more loading embedly scripts.
I am back from vacation, new server is ready now so let's see what the speed says :)
-
Ok, some more on the topic... new server is here, 8 cores, 16 GB ram... all working fine, APC(u) is up and running.
Had to rush switching from Dolphin to Una now for reasons, so Una is live now. All working well, server is mostly idle, cache is up to 1.5 GB of ram, all fine. But the community (=UNA) is partly completely unresponsive. It's like clicking a link and then 20-30 seconds nothing happens except browser showing a spinner. And we have around 6-10 people online from 8k.
After that "dead" time which seems to happen for all users at the same time (server is at 0% load meanwhile) it shortly goes up to like 90% on all cores for a second (probably executing all the commands been on hold so far?) and then it works smooth and fast again until the next hickup. Can't say when, maybe next click, maybe in a few mins.
Alex T⚜️ Any idea what that could be? Where to check what?
-
One of the reasons when it may happen if you switch to InnoDB and there are deadlocks, if you have set deadlock timeout in MySQL to 30 seconds then some pages could take 30 seconds or more to execute, I've written about MySQL settings for InnoDB here:
-
Was there an option you changed to make it use multithreaded or forced it to use one? If so, where did you do that.
-
No unfortunately there is no option for that.
-
So at this point I have to just deal with freezing occasionally or switch to a single core. :P I guess at the end of the day the other processes are still using all the cores. I will probably take a look into Litespeed on payday either way to see if it gets me a little more speed to make up for the random freezing.
-
for litespeed I recommend scalahosting company, they are very good.
Each php process will use a free core of course, but it seems like they are not free that much :) -
At this point would you recommend upgrading to MariaDB 10.3? Is there anything you can share on the changes made and if it affected your site at all. Does WHM give the option to change from Inno to Aria or will I have to do that in root? When it comes to databases I am still learning a lot. Right now on MySQL 5.7. A fear I have is when I installed Froala it wouldnt work and I had to put a code to convert all my darn tables. So I hope they would convert over the same and with no issues. They are in utf8mb4_unicode_ci.
-
you will need root permissions.
if you dont shift to mariadb there is no need to change the engine to Aria.My site works on Mariadb + Aria engine now, this was the best setup for me.
But mysql+inno is perfectly fine, just you need to convert UNA tables from Myisam to Inno for best performance. -
When converting table type, has anyone ran into problems where it messed with anything on their existing website?
-
if you are using apache with mod-php the biggest positive impact could be changing to apache + phpfpm or nginx + pgpfpm
-
I didnt try it.
-
I am using phpfpm, I ran into problems in the beginning thankfully which caused me to switch to it instead and made a big improvement.
-
Please help with this. I know this is old, but I am preparing a server transfer and I want my UNA site to be as fast as possible while still allowing video uploads.
1. Should I use a...
a.) VPS cloud server with:
40 GBPS network in
2 GBPS network out
8 vCore processor
8 TB bandwidth per month
256 SSD drive
16 GB ram
UNA's Amazon S3 plugin to use Wasabi S3 compatible storage
b.) dedicated server with:
1 GBPS network in
1 GBPS network out
16 Core / 32 thread processor
unmetered bandwidth per month
256 GB SSD drive
256 GB ram
UNA's Amazon S3 plugin to use Wasabi S3 compatible storage
2.) Then also, which is the better setup idea to use...
a.) cpanel, Lightspeed server, cloudflare
b.) cpanel, apache, engintron, cloudflare
c.) ISP config with either of the above two setups vs Cpanel?
I tried Webmin / Virtualmin so many times and it just kept giving me migraines, but I don't know about ISP config yet.
I know that more ram and more ssd storage is better, but I still need to know if the 40 GBPS is worth the hardware sacrifice when using UNA, live video messenger, and autoplay timeline video streams. (100 mb max uploads because of cloudflare)
-
start with a minimal setup and as you will be using virtual servers, you may increase as needed in time.
for storage to be cost effective you may better use object based S3 solutions.
I am using my own virtual servers but my recommandation for a new beginner to UNA is using UNA's hosting options with S3 and having no headaches.