0

we have a large Woocommerce website hosted on Google VM servers (E2 4vCPU + 16 GB RAM + 100 GB SSD Storage), recently we created a mobile application using flutter and WordPress -Woocommerce- API, but we faced a three issues

  1. if two users open the app at the same time, data loading synchronously not parallel, it seems the server dealing with a single request only at the same time
  2. fetching data is very slow, to load a product maybe it will take more than 5 seconds on a HighSpeed Internet connection
  3. if the user using the app the website will take more time to load, it seems the server dealing with a single request only at the same time

Environment Details

  • server NGINX + MySQL + PHP-FPM
  • PHP 7.4
  • OS: Centos 8
  • All tables Engin - InnoDB

we did the following

  • increased PHP.ini memory limit to 15GB
  • increased PHP.ini timeout limit to 3000 sec
  • Linked website with cloudflare. - Installed Cache Plugin W3 Total Cache
  • Increase NGINX MAX Clients to 150

I am looking for a suggestion on how I can allow to server/MySQL handle a lot of requests?

PHP-FPM Config

pm.max_children = 200
pm.start_servers = 50
pm.min_spare_servers = 50
pm.max_spare_servers = 150
;pm.max_requests = 500
;php_admin_value[memory_limit] = 128M
;request_terminate_timeout = 0
;rlimit_core = 0
;rlimit_files = 1024
;pm.process_idle_timeout = 10s;
;pm.max_requests = 500
pm = dynamic
; process.priority = -19

PHP ini

;user_ini.cache_ttl = 300
implicit_flush = Off
;unserialize_max_depth = 4096
;realpath_cache_size = 4096k
;realpath_cache_ttl = 120
zend.exception_ignore_args = On
max_input_time = 6000
max_execution_time = 3000
;max_input_nesting_level = 64
memory_limit = 15000M
post_max_size = 800M
;mysqlnd.mempool_default_size = 16000
;mysqlnd.net_read_timeout = 31536000
;mysqlnd.net_cmd_buffer_size = 2048
soap.wsdl_cache_enabled=1
soap.wsdl_cache_ttl=86400

PHP-FPM www-status, when the app opening

pool:                 www
process manager:      ondemand
start time:           03/Feb/2023:11:59:36 +0000
start since:          25801
accepted conn:        14613
listen queue:         0
max listen queue:     0
listen queue len:     0
idle processes:       0
active processes:     20
total processes:      20
max active processes: 20
max children reached: 53
slow requests:        0

************************
pid:                  14270
state:                Running
start time:           03/Feb/2023:19:08:39 +0000
start since:          58
requests:             17
request duration:     563199
request method:       GET
request URI:          /index.php?consumer_key=xxx&consumer_secret=yyy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14272
state:                Running
start time:           03/Feb/2023:19:09:03 +0000
start since:          34
requests:             12
request duration:     267198
request method:       GET
request URI:          /index.php
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14273
state:                Finishing
start time:           03/Feb/2023:19:09:03 +0000
start since:          34
requests:             7
request duration:     5577206
request method:       GET
request URI:          /index.php?consumer_key=xxx&consumer_secret=yyy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14274
state:                Running
start time:           03/Feb/2023:19:09:03 +0000
start since:          34
requests:             7
request duration:     2475191
request method:       GET
request URI:          /index.php?consumer_key=xx&consumer_secret=yyy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14275
state:                Running
start time:           03/Feb/2023:19:09:03 +0000
start since:          34
requests:             8
request duration:     2731158
request method:       GET
request URI:          /index.php?status=publish&category=189&orderby=popularity&per_page=5&consumer_key=xxx&consumer_secret=yy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14276
state:                Running
start time:           03/Feb/2023:19:09:04 +0000
start since:          33
requests:             8
request duration:     934026
request method:       GET
request URI:          /index.php?consumer_key=xxx&consumer_secret=yyy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14277
state:                Running
start time:           03/Feb/2023:19:09:07 +0000
start since:          30
requests:             6
request duration:     3243188
request method:       GET
request URI:          /index.php?consumer_key=xxx&consumer_secret=yyy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14278
state:                Finishing
start time:           03/Feb/2023:19:09:07 +0000
start since:          30
requests:             6
request duration:     5822217
request method:       GET
request URI:          /index.php?consumer_key=xx&consumer_secret=yyy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14279
state:                Running
start time:           03/Feb/2023:19:09:07 +0000
start since:          30
requests:             7
request duration:     220
request method:       GET
request URI:          /www-status?full
content length:       0
user:                 -
script:               -
last request cpu:     0.00
last request memory:  0

************************
pid:                  14280
state:                Running
start time:           03/Feb/2023:19:09:07 +0000
start since:          30
requests:             6
request duration:     5309224
request method:       GET
request URI:          /index.php?status=publish&per_page=20&page=1&skip_cache=1&stock_status=instock&consumer_key=xxx&consumer_secret=yy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14281
state:                Running
start time:           03/Feb/2023:19:09:07 +0000
start since:          30
requests:             7
request duration:     2048190
request method:       GET
request URI:          /index.php?consumer_key=xx&consumer_secret=yyy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14282
state:                Running
start time:           03/Feb/2023:19:09:07 +0000
start since:          30
requests:             6
request duration:     5482457
request method:       GET
request URI:          /index.php?consumer_key=xx&consumer_secret=yy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14283
state:                Running
start time:           03/Feb/2023:19:09:07 +0000
start since:          30
requests:             6
request duration:     5262082
request method:       GET
request URI:          /index.php?consumer_key=xx&consumer_secret=yy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14288
state:                Running
start time:           03/Feb/2023:19:09:07 +0000
start since:          30
requests:             7
request duration:     330128
request method:       GET
request URI:          /index.php?per_page=100&page=1&per_page=10&consumer_key=xxx&consumer_secret=yy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14291
state:                Running
start time:           03/Feb/2023:19:09:07 +0000
start since:          30
requests:             6
request duration:     4372096
request method:       GET
request URI:          /index.php?consumer_key=xxx&consumer_secret=yyy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14293
state:                Running
start time:           03/Feb/2023:19:09:10 +0000
start since:          27
requests:             6
request duration:     710233
request method:       GET
request URI:          /index.php?consumer_key=xxx&consumer_secret=yyy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14296
state:                Running
start time:           03/Feb/2023:19:09:12 +0000
start since:          25
requests:             5
request duration:     4356106
request method:       GET
request URI:          /index.php?consumer_key=xxx&consumer_secret=yyy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14297
state:                Running
start time:           03/Feb/2023:19:09:13 +0000
start since:          24
requests:             5
request duration:     3993473
request method:       GET
request URI:          /index.php?consumer_key=xxx&consumer_secret=yyy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14299
state:                Running
start time:           03/Feb/2023:19:09:15 +0000
start since:          22
requests:             4
request duration:     5846079
request method:       GET
request URI:          /index.php?status=publish&category=190&orderby=popularity&per_page=5&consumer_key=xxx&consumer_secret=yyy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0

************************
pid:                  14269
state:                Running
start time:           03/Feb/2023:19:08:37 +0000
start since:          60
requests:             15
request duration:     355854
request method:       GET
request URI:          /index.php?consumer_key=xxx&consumer_secret=yyy
content length:       0
user:                 -
script:               /var/www/mydomain.com/html/index.php
last request cpu:     0.00
last request memory:  0
Improve this question
5
  • What does the Lighthouse feature in Chrome devtools tell you? What does the Query Monitor plugin tell you? Is your MySql server overburdened? Sounds like it to me. Paradoxically, sometimes reducing concurrency (reducing max clients) can improve responsiveness and request throughput. Commented Feb 3, 2023 at 1:14
  • You php process looks too high for your CPUs and you don't inform any MySQL configuration. You can use php-fpm slow log or xdebug + kcachegrind to analyze the profile data as explained in xdebug.org/docs/profiler. Commented Feb 3, 2023 at 1:18
  • "loading synchronously" -- please provide details of what is loaded and how. If it is MySQL, provide the SQL and SHOW CREATE TABLE. Commented Feb 3, 2023 at 18:28
  • I added PHP-PFM Status @RickJames Commented Feb 3, 2023 at 20:06
  • Please add SHOW PROCESSLIST (preferrably without the "Sleep" entries). Commented Feb 3, 2023 at 23:45

1 Answer 1

Reset to default
1

First of all make sure your FPM-CONFIG is actually read

you have set pm.max_children to 200 , and with a memory_limit of 15GB

to have this config, you better have more than 3000GB of RAM (200 x 15GB/15000M) - which you dont.

My suggestion is -

reduce max memory usage to 512MB php.ini

memory_limit = 512M

fpm-config -

set PM to ondemand

pm=ondemand
pm.max_children = 20
pm.start_servers = 4
pm.min_spare_servers = 5
pm.max_spare_servers = 10
pm.process_idle_timeout = 10

Debugging further

add this to fpm-config

pm.status_path = /www-status

nginx conf - (you might need to adjust this to fit yours)

        location ~ ^/(www-status)$ {
                fastcgi_pass 127.0.0.1:9000; # replace this or use the unix socket
                fastcgi_param SCRIPT_FILENAME 
                $document_root$fastcgi_script_name;
                fastcgi_param SCRIPT_NAME $fastcgi_script_name;
                fastcgi_index index.php;
                include fastcgi_params;
        }

head onto site.com/www-status and youll see the fpm usage info, with this you can find out how much you actually need.

also try to use Nginx caching, this will be a huge help for your server use NGINX cahing - https://docs.nginx.com/nginx/admin-guide/content-cache/content-caching

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Thanks for your effort, I did all your steps but still facing the same issue can you help me more? I added www-status on the question please check it @janithcooray
see the part that says max children reached, this means the maximum capacity has been reached 53 times. you seem to be getting 1.7 connections / user vists on average per second. so about 6.5K visitors an hour. you can further reduce your memory limit to 128MB and set max_children to 80. if that doesn’t work your only options are 1. Increase vCPUs ,2. Increase Caching, 3.Increase Server RAM and max_children, 4. Add another server as a load balanced setup.
also this question belongs to serverfault, best move it before this gets downvoted and removed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.