views:

33

answers:

1

Hi,

I'm trying to find a tool and library to edit, write and read data in a hierarchical structure, similar to an LDAP tree, a Windows registry or a Berkeley DB structure. The keys should represent some hierarchy, and the values should have a relatively flexible format (typing is optional, but could be useful). Here is an example:

Items/item_1/shape = "rectangle"
Items/item_1/top = 10
Items/item_1/left = 10
Items/item_1/width = 30
Items/item_1/height = 40

Items/item_2/shape = "square"
Items/item_2/top = 10
Items/item_2/left = 10
Items/item_2/width = 30

Items/item_3/shape = "circle"
Items/item_3/centre_x = 40
Items/item_3/centre_y = 50
Items/item_3/radius = 20
Items/item_3/colour = blue

The use-case would be:

  1. Edit the data store via a convenient GUI. This could look like Windows Regedit or Apache Directory Studio (LDAP Browser).

  2. Save that data into some store (e.g. a file).

  3. Load this store from another application, which would be able to query it from an API. The library for this would ideally be callable from Python.

I'd like these operations to be reasonably fast for reading, but not have it all loaded in memory in advance. The data store would be updated into the application more or less manually, much less often than the data is read. Being able to write to that data from the API would be a plus, but it's not a strict requirement.

Queries would be good. For example (in pseudo query), "List Items/* where top == 10" would return:

  • Items/item_1
  • Items/item_2

Ease of edition (via a good GUI) is one of the most important features.

I've considered a few options:

  1. An LDAP server answers most requirements (especially with the help of Apache Directory Studio). However, deploying an LDAP server just for that is too heavy. However good Apache Directory Studio is, the user still needs a reasonable understanding of LDAP (more than just explaining the tree hierachy). I'd also like some flexibility in the creation of schemas (or no schemas at all) rather than having to rely on an administrator to do this.

  2. Windows Regedit. I'm not sure if it's possible, but I guess it's conceivable to have a registry-like file that has nothing to do with the actual windows registry, editable with custom content via Regedit. I would however like this GUI application to be available on non-Windows platforms and I'm not even sure there's a Python API to read a Windows registry file.

  3. RDF. That could work. I'm sure there are fairly good Python libraries for semantic web. However, I don't need any reasoning capabilities. I'd rather have something fast and that doesn't use much memory. I'm not sure there are any good GUI tools to view and edit the tree (since it's geared towards webs and graphs).

There are certainly ways to build this sort of data structures on top of existing systems like SQLite (for example), and this would be fine but I'm not sure whether there's a good GUI that would come with it.

I'd appreciate comments and suggestions, thank you.

A: 

Forgive my densitosity, but if your hierarchy is anything but the roughest kind of example, there must be hugely compelling, overpowering reasons for your choosing that over, say, JSON or even (gulp!) XML:

<items>
  <item>
   <number>1</number>
   <shape>rectangle</shape>
   <top>10</top>
   <left>10</left>
   <width>30</width>
   <height>40</height>
  </item>
  <item>
   <number>2</number>
   <shape>triangle</shape>
   <top>20</top>
   <left>50</left>
   <width>30></width>
   <height>40</height>
  </item>
</items>

For XML, you know there are tons of editors that answer your needs. Same with JSON. And I think there are also XML-> MySQL -> XML libraries.

Why not take that kind of approach?

Pete Wilson
Thanks for the suggestion. I've considered XML too, but I'd like to avoid the overhead of loading the DOM document every time I need to read something from it (memory restrictions). This would end up being a large file (small in RDBMS terms, but around 5-10MB). That would be fine for the exchange format, though.I think the speed and memory requirements would vary substantially depending on the way the SQL engine indexes the XML doc, if it has some native XML support (MySQL, PostgreSQL). I must admit I had something more embedded in mind, and SQLite doesn't seem to support XML operations.
Bruno
"more embedded" : That would be great. Like you, though, I don't know of anything that would let you wriggle out of writing at least some code. Could you staple all the pieces together with a shell script? Would it help to consider running the app in a browser window?
Pete Wilson
I don't mind writing some code for this, but preferably not for the GUI. What I had in mind with "more embedded" was something like SQLite, that doesn't require to run another server.If that's not possible, I suppose I could set up a MySQL or PostgreSQL server, but I'd rather write code that's not dependent on a particular SQL flavour. I don't mind the small SQL tweaks, but in my experience, XML support in SQL databases not only differs in the syntax but also in what's supported.
Bruno