Django app that can provide user friendly, multiple / mass file upload functionality to other apps

Question

Hi,

I'm going to be honest: this is a question I asked on the Django-Users mailinglist last week. Since I didn't get any replies there yet, I'm reposting it on Stack Overflow in the hope that it gets more attention here.

I want to create an app that makes it easy to do user friendly, multiple / mass file upload in your own apps. With user friendly I mean upload like Gmail, Flickr, ... where the user can select multiple files at once in the browse file dialog. The files are then uploaded sequentially or in parallel and a nice overview of the selected files is shown on the page with a progress bar next to them. A 'Cancel' upload button is also a possible option.

All that niceness is usually solved by using a Flash object. Complete solutions are out there for the client side, like: SWFUpload http://swfupload.org/ , FancyUpload http://digitarald.de/project/fancyupload/ , YUI 2 Uploader http://developer.yahoo.com/yui/uploader/ and probably many more.

Ofcourse the trick is getting those solutions integrated in your project. Especially in a framework like Django, double so if you want it to be reusable.

So, I have a few ideas, but I'm neither an expert on Django nor on Flash based upload solutions. I'll share my ideas here in the hope of getting some feedback from more knowledgeable and experienced people. (Or even just some 'I want this too!' replies :) )

You will notice that I make a few assumptions: this is to keep the (initial) scope of the application under control. These assumptions are of course debatable:

All right, my idea's so far:

• If you want to mass upload multiple files, you are going to have a model to contain each file in. I.e. the model will contain one FileField or one ImageField. Models with multiple (but ofcourse finite) amount of FileFields/ ImageFields are not in need of easy mass uploading imho: if you have a model with 100 FileFields you are doing something wrong :) Examples where you would want my envisioned kind of mass upload:

  • An app that has just one model 'Brochure' with a file field, a title field (dynamically created from the filename) and a date_added field.
  • A photo gallery app with models 'Gallery' and 'Photo'. You pick a Gallery to add pictures to, upload the pictures and new Photo objects are created and foreign keys set to the chosen Gallery.

• It would be nice to be able to configure or extend the app for your favorite Flash upload solution. We can pick one of the three above as a default, but implement the app so that people can easily add additional implementations (kinda like Django can use multiple databases). Let it be agnostic to any particular client side solution.

• If we need to pick one to start with, maybe pick the one with the smallest footprint? (smallest download of client side stuff)

• The Flash based solutions asynchronously (and either sequentially or in parallel) POST the files to a url. I suggest that url to be local to our generic app (so it's the same for every app where you use our app in). That url will go to a view provided by our generic app.

• The view will do the following: create a new model instance, add the file, OPTIONALLY DO EXTRA STUFF and save the instance.

• DO EXTRA STUFF is code that the app that uses our app wants to run. It doesn't have to provide any extra code, if the model has just a FileField/ImageField the standard view code will do the job. But most app will want to do extra stuff I think, like filling in the other fields: title, date_added, foreignkeys, manytomany, ...

• I have not yet thought about a mechanism for DO EXTRA STUFF. Just wrapping the generic app view came to mind, but that is not developer friendly, since you would have to write your own url pattern and your own view. Then you have to tell the Flash solutions to use a new url etc... I think something like signals could be used here?

• Forms/Admin: I'm still very sketchy on how all this could best be integrated in the Admin or generic Django forms/widgets/... (and this is were my lack of Django experience shows):

  • In the case of the Gallery/Photo app: You could provide a mass Photo upload widget on the Gallery detail form. But what if the Gallery instance is not saved yet? The file upload view won't be able to set the foreignkeys on the Photo instances. I see that the auth app, when you create a user, first asks for username and password and only then provides you with a bigger form to fill in emailadres, pick roles etc. We could do something like that.
  • In the case of an app with just one model: How do you provide a form in the Django admin to do your mass upload? You can't do it with the detail form of your model, that's just for one model instance.

There's probably dozens more questions that need to be answered before I can even start on this app. So please tell me what you think! Give me input! What do you like? What not? What would you do different? Is this idea solid? Where is it not?

Thank you!

Answer 1

I just released a simple app for this about a month ago: django-uploadify.

It's basically a Django template tag that acts as a wrapper for the very nifty Uploadify (requires jQuery). Using it is as simple as adding this to your template...

{% load uploadify_tags }{% multi_file_upload ‘/upload/complete/url/’ %}

The tag will fire events (1 per file) on both the client-side and server-side (Django signal) to indicate when an incoming file has been received.

For example, assuming you have a model 'Media' that handles all user-uploaded files...

def upload_received_handler(sender, data, **kwargs):
    if file:
        new_media = Media.objects.create(
            file = data,
            new_upload = True,
        )
        new_media.save()

upload_recieved.connect(upload_received_handler, dispatch_uid=‘whatever.upload_received’)

Check out the wiki for info on how to set it up and create the signal handlers (client/server).

---

About your conceptual implementation from above, here's a few points of consideration:

• Having the app automatically create the "File Model" instance probably isn't as robust as people may already have their own models they're working with
• If you want to implement any type of security or authentication, you need an open system and less of an 'auto-create' type
• I really think signals/events are the way to handle this, and also handle the 'DO OTHER STUFF' part of what you mentioned.
• My conclusion was that multi-upload can never really be a form widget in the sense that Django implements form widgets. 1 file will most likely be represented by 1 model instance (with some exceptions), which means that we end up with a situation where 1 widget can represent N model instances. However Django is setup so that a widget represents 1 value for 1 field in 1 instance. It just doesn't fit for the majority of use-cases to have it as a widget (hence why I went the template tag route).

Answer 2

Am not able to get django-uploadify working in my system. Do you have any simple django project as an example? (Would be great to have a working example project)