How to delete hundreds of documents to a Django Web App?

Question

I developed a Django application and i have to process 500 documents (I have to upload them using a .zip/.rar and to process them one by one using NLP). The problem is that when i try to upload all the 500 documents my app takes so much time, around 2 hours. I would like to know what is the best way to upload these documents using the Django Framework? How can i upload them one by one so i will not have time out problems or too big file errors?

P.D The user wants to upload the 500 documents compressed in .zip/.rar file. He wants to upload this file once so the system has to process the 500 documents once. So i have to find a way to upload them without overloading my Web App. I tryied to upload them once, but the server process takes so much time uploading 500 documents and i can find http time out errors or too big entity errors.

Answer 1

Anyone answered so i will answer myself. I used Celery + Django. Celery let you create asynchronous tasks so you can create a process that is independent from your web app and it can work in the background. You can use RabbitMQ or Redis to send messages between your independent process and your application. Just search on Google : Celery + Django.