Need help on selecting items in simple database in python.

Question

I'm trying to write a circuit schematic drawing tool in Python. I'm creating a simple database based on dictionaries which holds all the components and their properties. I'm also trying to create a simple query language where you can select, for example, all resistors with value>100ohms or footprint='0402'

So far, I can select things using some primitive predicate search and then union and intersection for more complex things.

However, I'm having trouble defining what the semantics ought to be for a purely negative search. For instance, something like

footprint != '0402'

should select all items with footprint not equal to 0402. But doing this with only intersection gives me a blank result. For this to work I need to select all components, then intersect with "not 0402" to remove the ones I don't want.

But this seems kind of brute force and seems like a fishy solution. I'm not interested in using a "real" database and query language, so please don't suggest that. I'm looking for the proper engineering rationale here, not necessarily a workaround to the problem.

This problem was covered to some degree in SICP book, but I got very confused, since I think they were also using continuations and stuff, which I don't get yet.

Can someone please explain what the 'proper' usage ought to be for negatively selecting stuff. I tried this in a commercial cad tool and it worked as expected, but then I've seen some SQL query examples (I think), which selected things first, then removed the unwanted ones.

thanks
michael

Answer 1

It really depends on how you've implemented this "simple database based on dictionaries" as to why straight negation isn't working - you'll have to give us some more clues.

There's no reason why a straightforward negation match shouldn't work in Python, e.g.:

components = [
    { 'name': 'resistor', 'footprint': '0402', },
    { 'name': 'LED', 'footprint': '0100', },
    { 'name': 'speaker', 'footprint': '2000', },
]

[comp for comp in components if comp['footprint'] != '0402']
# [{'footprint': '0100', 'name': 'LED'}, {'footprint': '2000', 'name': 'speaker'}]

At a stretch, this is a simple database based on dictionaries - the specific capabilities are really going to depend on your actual implementation.

I'm resisting the temptation to suggest use of a real database and query language as I assume this is a learning exercise. It is a learning exercise, right? :)

Answer 2

You'll be a lot happier with a proper database.

You have SQLite with your Python distribution. Simply define tables instead of dictionaries and use SQL.

If you need more power and sophistication, you can add SQLAlchemy (or SQLObject) and you won't struggle with these problems.

Answer 3

I agree with S.Lott that you would be happier with a real DB. SQLite is really light and fast, almost no drawbacks on using it.

if you really need to expose a simpler, yet complete, query language to the user, check JSONPath or JSONQuery (the second is a superset of the first). The exact syntax is (obviously) thought for JavaScript; but it should give you some ideas.

also, try to check similarities and differences with XPath and XQuery. it helps to see what is useful and what's culture-specific.

Answer 4

Can one of you DB-fluent people translate this google-like query into SQLite for me please, here with dotdict fields in the PyPI xmlrpc db --

summary: this that - butnotthis  version: > 0.1  author: James | Javier

i.e. d.summary ~ "this" and d.summary ~ "that" and d.summary !~ "butnotthis" and d.version > "0.1" and d.author ~ "James|Javier"

Aand, does anyone have a grammar and parser for such google-like queries ?

thanks, cheers -- denis