What if?

What if 100,000 users should send a request to my flask app simultaneously? How will the app behave in such conditions? What if my sqlalchemy db table was filled with 100,000 rows, how long would db queries take?

I am new to flask (2 months), built 2 projects but have never had so many persons visit my web apps. Anyone with these experience should please share

7 thoughts on “What if?”

  1. I’m not 100% certain about this, but I’m pretty sure your web host’s system will handle queuing up the requests to send to your app. It’s not like all 100,000 will hit your site at once. Bandwidth will come into play for something like this.

    Reply
  2. The connections are handled by your HTTP server in front of your Flask app such as NGNIX. This is why you absolutely have to have front your app by a web server in production. NGINX will receive the 100,000 connections and try to pass them out to as many instances of your app you have configured and running. In reality, handling 100,000 simultaneous connections would require some more advanced load balancing and scaling.

    More realistically if you say have four threads of your app running and you receive 20 connections NGINX will do its best to hold 16 connections open and service them as flask instances complete or it will return 504 error code to the client if it can’t service them fast enough.

    Your DB question is 100% dependent on the performance of your host and what database you’re using. If you’re running off an SD card on a RaspberryPi the query time is going to be slower than running on an SSD drive on a server. Same for say sqlite vs postgresql. The only thing you can do is benchmark, test, compare and decide what is good enough.

    Reply
  3. I do that for a living, stressing web services/pages.

    It all resorts to your architecture really, your host will send traffic to your site until your weekly/monthly quota is reached. After that everything will be blocked for a time or until you upgrade your hosting plan.

    As to your architecture, if you expect such a load of visits you will want to use load balancing and/or have a web server with queuing capabilities (like heroku, gunicorn, nginx plus, etc).

    Reply
  4. Ok, I have experience with this. I used to have the same doubts but seeing stuff in production cleared a lot of doubts.

    While developing, we just run the single flask app on some random port and use sqlalchemy with sqlite3 backend. All of this is for simplicity of development.

    In production, a lot of factors come to picture: vertical and horizontal scaling. Flask is just a web framework. It comes with a tiny web server (werkzeug) which is NOT capable of production needs. You need to have a strong web server sitting in front. Most of the online guides will recommend Nginx for this.

    So you usually have this:

    Browser –> Nginx running on 80/443 –> redirect request to URL “/” to Flask running on 8000 –> Flask handles the request. Response follows reverse cycle.

    So if you looked closely, Nginx directs the request based on the URL (one way among several other ways). In one of my projects, I had a flask server and nodejs server running. In this case, any request to “/flask” would go to flask running on 8000 port and requests to “/node” would go to nodejs server running on 3000 port. This is why Nginx is commonly used as a “reverse proxy”. Here are some [Nginx stats](https://www.nginx.com/blog/testing-the-performance-of-nginx-and-nginx-plus-web-servers/). Have a look at the “Connections per second” column especially, they answer your question.

    Now, coming to sqlalchemy, your reservations about it is not entirely valid. sqlalchemy is just an adaptor library for several SQL backends. If you choose a good production DBMS like MySQL/PostgreSQL then you are set. Don’t use sqlite3 for production. For starters, sqlite3 locks the entire database for writing which is terrible. Sure, using an adaptor is an overhead (as is all abstraction), but you are using a python server so it’s safe to say that speed is not of priority.

    Now, this just solves ONE deployment of “Nginx+Flask”. But what if you need to scale it geographically? How to handle concurrency across systems? (in case two users are ordering the same item when there’s only 1 in stock) What about redundancy? What happens in case one of the instances (nginx instance/flask app) goes down?

    These are age old questions of distributed systems! I can give a few hints, but THIS is the time to explore a few good talks on Youtube. Especially those by engineers who work in large scale distributed systems companies (AWS, Fastly, google etc.). You will learn about Global Load Balancers, Endpoint Caching, WAN optimisers etc. It will be fun to know how a single request to google.com jumped through so many servers and routers to eventually reach an endpoint owned by Google Inc.!

    There are a LOT of technologies today which makes deploying distributed systems easier than before: docker+kubernetes, consensus protocols, blockchain etc.

    In case you are really interested in what happens if you just send 100000 requests to your flask app, use some tool like “httperf” or “iperf” to test them.

    Happy stress testing! 🙂

    Reply
  5. I have spent the last couple of years building, deploying, rebuilding, and redeploying Flask projects in different configurations. I have found that nginx -> gunicorn -> flask is by far the best way to do it. I started with very simple servers, even mistakenly deploying the “development” Werkzeug server into production at first, and have worked my way into this configuration through trial and error.

    A good guide here: [https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-gunicorn-and-nginx-on-ubuntu-18-04](https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-gunicorn-and-nginx-on-ubuntu-18-04)

    The benefit here comes from gunicorn “workers” which allow you to spin up multiple processes to handle requests as well as give each process multiple threads if your app is IO intensive. If you can fan this out on your server enough, you will end up making the DB the bottleneck rather than the API which is ideal from an application standpoint. At that point, you can speed the queries up with good indexes. I also like the nginx portion just for the reverse proxy, pointing to the configured gunicorn endpoint.

    u/spryion went into some of the details of multi-server architecture which is a little bit more complicated. In applications where I wasn’t dealing with a database, I would use nginx as my load balancer. In the application where I do have a database, I do not yet know how I’ll handle load as it grows… although I’m in a \~100 person firm so I probably won’t even get to that point.

    Good luck!

    Reply
  6. In our product, we use nginx as a reverse proxy and uWSGI as the interface to the flask application. This allows for concurrency on flask to improve performance and throughput. Basically, uWSGI exposes a unix socket and all requests received by nginx are sent through that socket. Both nginx and uWSGI natively use the uwsgi protocol and it works “out of the box”.

    Reply

Leave a Comment