Best fast authentication mechanism for embedded devices with Nginx & Apache

Question

Hello All,

We have a project deployed with Nginx, Apache, Django and Postgres.

The project has large number of embedded devices login in to the server(https) in 5 minutes intervals and send a file to the server. Also, the WebUI face of the project has normal users login in and performing different functions.

Currently, system doesn't differentiates between the a embedded device and normal(human) user. When the devices number was low, this mechanism was working fine with the required speed. But overtime with increased number of devices the load on the database increased a lot. We observed around 60000 django_session entries per day. The import script that we run to process the files sent from the devices is hardly keeping up with the incoming data and database is heavily loaded.

I want to implement a minimal authentication mechanism which can just authenticate the device very fast and allows it to send the text file. I want to avoid the django authentication way but want to keep using the auth_user table for the username-password for the devices. Given performance requirement, I need some way, which uses nginx and doesn't really bring apache into picture and accepts the file.

What you think is the best way to achieve this? Also, what this large embedded devices related projects use for such type of mechanism?

Answer 1

So as i understand it each embedded device has a user account setup in django and you wish to avoid this overhead.

So first a question, how does the back end know when its talking to a embedded device or a human user? I would have a assume this information is embedded in django system.

If so what you need to do is intercept the user name and password before it gets passed to django, check for the type of account in the database and if its an embedded device pass it to a handling app that does nothing but accept the files and process them. How you intercept this is up to you, you could do it in almost any 'web' language you wish and just use redirects when you've decided how to handle the traffic.

If you get the opportunity to rewrite this further, why even have a user name password for the embedded device? assuming it is an embedded device (single purpose minimal interface) then isn't checking its SSL certificate against a list of known certificates enough to identify it? This could remove any interaction with the django system at all as all you need to do is check the cert against a bank of known ok certs.

Answer 2

Can you write a compiled cgi script (in C or C++?) that authenticates the embedded devices against the Postgres database directly, and stores the file in an appropriate manner? You obviously don't need sessions, since the embedded device makes a one-time connection every 5 minutes.

I don't know much about Django (picked this up from the embedded tag), so I don't know what's responsible for its overhead.

It actually isn't too hard to write a CGI script in C (assuming you or someone on your team knows C) for this simple task.