I am looking for a comparison of django and flask for a project that will live for a long time, and will need to be maintained, built upon and grow as the months progress. I am considering Flask + SQLAlchemy or django. I do not need batteries, as I usually end up having to modify them, so it is fine if I have to re-implement a couple o...
This question is about design decision. I am currently working on a web project that will have 40K users to start with and in couple of month expected to grow 50M users (not concurrent users though). I would like to have a architecture that can be scaled out easily without much effort. In order to explain, I would like to use a trivial ...
Hi, We have very complicated code(still well written) which performs lot of stuff for a particular operation. Currently, this is performed in a service which talks to database directly. I have two goals for this service. I want to simplify the organization of the code. I want to be able to add more machines to scale in the future. T...
Currently I'm working with an in house white label cms that we resell to multiple clients and it all runs from the same box/db. I'm just looking at converting this to have an ecommerce version that we'll run alongside it. I'm wondering whether there will be an issue keeping all the products/categories/orders in one db or whether it wou...
Typically I use a database such as MySQL or PostGreSQL on the same machine as the application using it, which makes access easy and secure. I'm just now building the first site that will have a separate physical database server (later this year it will). I'm wondering 3 things: (security) What things should I look into for starters per...
I am trying to find the best solution of servicing a user of a HTTP service, that requires a huge amount of data at once. The idea is that we have an extremely big table which can get to around 80 million records - it might happen that some clients request the whole data (and yes, they need it all at once); the data gets updated only onc...
Is there a simple way to load test an asp.net application? I've been reading on some microsoft pattern and practices documents that tell you to use ACT which seems kind of hard to use for complex requests. ...
I have these entity kinds: Molecule Atom MoleculeAtom Given a list(molecule_ids) whose lengths is in the hundreds, I need to get a dict of the form {molecule_id: list(atom_ids)}. Likewise, given a list(atom_ids) whose length is in the hunreds, I need to get a dict of the form {atom_id: list(molecule_ids)}. Both of these bulk lookups...
I'm wondering about managing collections of data in memcached. I'm confused about how I maintain cache freshness/integrity when I have many levels of caching at work. Let's assume I have a hierarchy of cached data objects, for example, A represents the most atomic element of data, B represents a collection of 10 As C represents ...
I am planning to use open source, SQL/NoSQL? MVC? CQRS? caching? hadoop? Bigtable? S3? By scalable I mean supporting +1M users This is a social-change venture (not-for-profit), including geo-location and social networking features. ...
I have a web aaplication which currently has a bunch of users and use Asp.net Membership to create and manage users. I want to add OAuth functionality to my application and want to still using Asp.net Membership (login , etc ). ( and I also want my user login to application with any public social network ID , if that network use a OAut...
We are developing a website quite similar with ebay.com and in order to upgrade/maintain it without much effort we decided to split/isolate different parts of the website like ebay does too (e.g the item page/application will be served from cgi.domain.com , signin application from signin.domain.com, shopping cart application from offer.d...
Ladies and Gentlemen, This question might have been asked just too many times nevertheless its never bad to get fresh perspective on When developing a web application be it "SAAS" model or just a "Product" model, what are the best practises in structuring the application ? And Why do you think so ? Point to remember : Scalability is a...
Let's say I have an Oracle database table of 40MM 9-digit ZIP codes which includes the 4-digit ZIP code for each. I have a list of 800 5-digit ZIP codes and I need to find all the associated 9-digit ZIP codes. There are 40K 5-digit ZIP codes altogether. Assume we have indexed the 5-digit ZIP code field with a B*Tree (conventional) ind...
Hi folks, I am in the process of building/architecting a business social network web application that has a component that I think will lead to major scalability issues and I'd like to get some feedback/thoughts on the best way forward. The application has a User object. The idea is, that every time a new user joins the system he ranks...
I have a web app that stores a large amount of text data. The db is currently increasing by 1GB a week. I expect this to grow exponentially as we get more customers, so 1GB this week, 2GB next week, 4GB the following week, then 8GB, etc... Right now this data is stored in a single MS SQL 2008 database that 10GB in size. Performance is ...
This is a rather broad question..but throwing it out there. So let's say someone comes up with a decent web service idea, build an app on open source framework. Hosts it on a VPS for less than $100/month and things pick up, users start showing up. 1 - Assumming the above happens, how does one grow from a VPS to a full fledged internet c...
I am aware of event driven I/O like select, poll, epoll, etc allow someone to build say a highly scalable web server, but I am confused by the details. If there is only one thread of execution and one process running for the server, then when the server is running its "processing" routine for the ready clients, isn't this done in a seria...
I'm building a server application that needs to perform a lot of http requests to a couple other servers on an ongoing basis. Currently, I'm basically setting up about 30 threads and continuously running HttpWebRequests synchronously on each thread, achieving a throughput of about 30 requests per second. I am indeed setting the Servi...
I need to create a web form to collect user data (survey) that is expected to peak at 100-200k concurrent users, maybe more. I'm more experienced with Java & PHP solutions with RDBMS for persistence, and with those platforms I'd need a ton of VMs and serious hardware load distribution to handle this kind of traffic. Since this might not...