uwsig prefork vs lazy-apps

2023-03-14 作者 admin

问题背景

在工作中，发现一个问题，使用uwsgi+flask 这对搭档时，引入了apollo动态配置管理服务，在启动uwsgi的过程中会去链接apollo服务器（如代码所示），这时候会发现虽然看上去uwsgi启动成功了，但是却不能提供服务。

import json
import time
from flask import Flask
from doom.utils.apollo import apollo_get_value
from doom.utils.apollo import configure_apollo


def create_app():
    app = Flask(__name__)
    app.config['APOLLO_APPID'] = "apolllo_appid"
    app.config['APOLLO_NAMESPACE'] = "ns.python.json"
    app.config['APOLLO_CONFIG_SERVER_URL'] = "http://apollo_server.com:7088"
    configure_apollo(app)

    # time.sleep(1)

    @app.route("/", methods=['GET'])
    def index():
        print("-" * 20)
        print(json.dumps(apollo_get_value(app), ensure_ascii=False, indent=4))
        print("+" * 20)
        return "pong\n"

    return app


app = create_app()


if __name__ == "__main__":
    app.run()

uwsgi --http-socket 0.0.0.0:5001 --process 4 --threads 4 --plugin python3 --wsgi-file manage.py --callable app --stats 0.0.0.0:9191

如代码和启动方式所示，uwsgi启动后，使用curl测试时，会hang住。

分析记录

uwsgi在启动后，家在python代码后，会fork出来其他的worker进程（这样进加载一次代码即可），（一下分析不确定是否正确），猜测在fork时，尚未链接至apollo，导致fork出来的worker不能和apollo正确交互。（需要详细分析）

解决办法

使用uwsgi的lazy-apps模式

uwsgi --http-socket 0.0.0.0:5001 --process 4 --threads 4 --plugin python3 --wsgi-file manage.py --callable app --stats 0.0.0.0:9191 --lazy-apps

参考摘录文档

Lazy apps VS prefork

One of the controversial design choices of uWSGI is “preforking by default”.

It means your app is loaded on startup and then fork() is called for each worker.

While this is the common approach in the UNIX world and it is an expected behaviour for a Perl developer (that is historically more near to the UNIX world) it is totally unknown and unexpected by a Python (and maybe Ruby) one.

So one of the choices you need to make when building a uWSGI-based service is how to manage the fork() behaviour.

If you are unsure let me tell you one thing: with preforking behaviour you will make some user very happy, and lot of users completely lost. With –lazy-apps you will have all of your users totally unconcerned. Trust me, few happy users cannot make you happy too when you have angry customers too.

So, uWSGI default fork() behaviour is generally wrong for massive hosting, so add –lazy-apps and eventually give the advanced users the freedom to change it when needed.

参考资料