What do people think are the most important issues when developing an application that is going to allow users to upload video and images to a server and have them transcoded by FFMPEG and stored in Amazon S3? I have a couple of options;
1) install FFMPEG on the same server that handles file uploads, when a video is uploaded and stored on EC2 instance, call FFMPEG to convert it then when done, write the file to S3 bucket and dispose of the original.
How scalable is this? What happens when many users upload at the same time? How do I manage multiple processes at once? How do I know when to start another instance and load balance this configuration?
2) Have one server for processing uploads (updating database, renaming files etc) and one server for doing transcoding. Again what is the best way to manage multiple processes? should I be looking at Amazon SQS for this? Can I tell the transcoding server to get the file from the upload server or should I copy the file to the transcoding server? Should I just store all files on S3 and SQS can read from there. I am trying to have as little traffic as possible.
I am running a Linux box as the upload server and have FFMPEG running on this.
Any advice on best practices for setting up such a configuration would be appreciated.