Can anyone point me to a reference or provide a high level overview of how companies like Facebook, Yahoo, Google, etc al perform the large scale (e.g. multi-TB range) log analysis that they do for operations and especially web analytics? Focusing on web analytics in particular, I'm interested in two closely-related aspects: query perf...
I am looking for a reasonably well tested library+server to store a persistent distributed hash table. I am hesistant to use SQL-based solutions as the data is highly document oriented, consisting of millions of ~64KB blobs with only a single index (computed by hash of said BLOB) - and needs to be able to be distributed for long term s...
I'm working on a piece of software that creates a contract and captures the client's signature via gif and applies it to the contract (and spits out a pdf). Now, we're storing the data from the contract but when it comes to the signature, I'm not sure if I should. Store it: Pros: If the pdf document is lost, I can reconstruct the docum...
Suppose I have any data stored in bytes. For example: 0110001100010101100101110101101 How can I store it as printable text? The obvious way would be to convert every 0 to the character '0' and every 1 to the character '1'. In fact this is what I'm currently doing. I'd like to know how I could pack them more tightly, without losi...
Hello everyone, For SQL Server 2005/2008, after some self-study, I am not sure whether my understanding about how character data type is physically stored is correct. Please help to review and comment. Here is my understanding, please review and feel free to correct me. when using unicode character type (e.g. nvarchar), underlying cha...
I'm trying to set data in long-term storage in a GreaseMonkey script, except that GM_setValue() seems to fail silently: $("a#linkid").click(function() { GM_setValue("foo", 123); // doesn't work, but does not generate error }); GM_setValue("bar", 123); // works properly, value is set ...
In a native(c++) windows mobile app. There are ways to be notified of low memory, WM_HIBERNATE, and low power, RequestPowerNotifications(). Is there any way to be notified when storage space is running low? Or must an app just poll regularly with GetDiskFreeSpaceEx()? ...
Prior to Vista (2003, XP) Windows had a key in the registry HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\dmio\Boot Info\Primary Disk Group which indicated which dynamic disk group is imported/active on this installation. Since Vista, this key is gone. Still, the OS needs to store that info in some way, since the same disk gro...
I'm trying to programatically determine the cluster size of a storage card, using C# / .NET Compact Framework, on Widows Mobile. For desktop Windows there's the GetDiskFreeSpace() function, but it doesn't exist in coredll.dll/Windows Mobile. Is there any other way I could find out the size of the cluster for a storage card? ...
This might be a really trivial one. Is File storage OS dependant ? Why do text Files change when moved from Dos to Unix, is it that the Editor in Unix interpret certain characters differently or does the File itself change when moved from Dos to Unix and hence the utility Dos to Unix. Why a Java Class File can be moved from Dos to Un...
since we suffer from creeping degradation in our web application we decided to monitor our application performance and measure individual actions. for example we will measure the duration of each request, the duration of individual actions like editing a customer or creating an appointment, searching for a contract. in most cases the d...
If storing multiple (10+) values on a large number of divs, is it more optimal to store them all in a single object, or as separate values? Single Object: $("#div_id").data("data", {foo:1, bar:2}); Separate Values: $("#div_id") .data("foo", 1) .data("bar", 2); What are the trade-offs of each method? Some of these attributes wi...
Does the Microsoft "Extensible Storage Engine" expose enough access to the storage manager to allow me to write a custom access method, for example a GiST? Alternatively, is there a basic log manager / buffer pool manager project for Windows that I could extend to play around with GiST without re-inventing the entire wheel? (Access meth...
Hello All, I'm looking to let users of my web application upload videos to my site, however i believe i will be constrained by how much storage i will actually have. Is there any alternatives such as being able to store videos on Youtube via the api and recall them when needed. Any help for the design of how videos should be stored woul...
Hello, I am wondering whats more efficient, to store temporary data (related to that session) in a session using say $_SESSION variable in PHP or store and retrieve from an SQL database? Thank you for your time. ...
I am creating a simple application and getting stuck with data storage option. To begin with I would like to use SQL Server as my data storage. I will not be using any special features of SQL Server, its pure tables with CRUD operations. Now I should be able to switch the underlying data store to either SQL Data Services or Amazon S3 by...
I'm considering optimal open source solution for storing xml documents with further querying on them effectively. Amount of data will be small. As far as I understand native xml databases might form a proper solution for my case. They obviously store xml documents in highly efficient way. It would be great to learn your experience. Any s...
Does anyone know what subset of the iPhone's overall storage capacity is reserved for applications and application data? I am not talking about RAM, but rather disk space. I remember reading that only a portion of the iPhone's capacity was used for application data, while the remainder was used for music and media, however I cannot remem...
Every sample I come across has the entities and properties in the storage model named exactly the same as in the conceptual model. So in the mapping section, I can't tell whether an entity or property is from the storage model or conceptual model. This is a snippet of an Entity Framework diagram. Which "ID" fields are from the database ...
Hi, I am new to programming USB. I want to develop a simulator in Linux for USB mass storage devices. I would be thankful to you if you share links. ...