Is there a way to pickle a class definition? What I'd like to do is pickle the definition (which may created dynamically), and then send it over a TCP connection so that an instance can be created on the other end. I understand that there may be dependencies, like modules and global variables that the class relies on. I'd like to bundl...
I am using json and jsonpickle sometimes to serialize objects to files, using the following function: def json_serialize(obj, filename, use_jsonpickle=True): f = open(filename, 'w') if use_jsonpickle: import jsonpickle json_obj = jsonpickle.encode(obj) f.write(json_obj) else: simplejson.dump(obj, f) f.cl...
I have a set of data points, each described by a dictionary. The processing of each data point is independent and I submit each one as a separate job to a cluster. Each data point has a unique name, and my cluster submission wrapper simply calls a script that takes a data point's name and a file describing all the data points. That sc...
I'm using PYML to construct a multiclass linear support vector machine (SVM). After training the SVM, I would like to be able to save the classifier, so that on subsequent runs I can use the classifier right away without retraining. Unfortunately, the .save() function is not implemented for that classifier, and attempting to pickle it (b...
I want my class to implement Save and Load functions which simply do a pickle of the class. But apparently you cannot use 'self' in the fashion below. How can you do this? self = cPickle.load(f) cPickle.dump(self,f,2) ...
I'm unpickling a NetworkX object that's about 1GB in size on disk. Although I saved it in the binary format (using protocol 2), it is taking a very long time to unpickle this file---at least half an hour. The system I'm running on has plenty of system memory (128 GB), so that's not the bottleneck. I've read here that pickling can be spe...
It's taking me up to an hour to read a 1-gigabyte NetworkX graph data structure using cPickle (its 1-GB when stored on disk as a binary pickle file). Note that the file quickly loads into memory. In other words, if I run: import cPickle as pickle f = open("bigNetworkXGraph.pickle","rb") binary_data = f.read() # This part doesn't take...
Does anyone know if pickle is compatible with jruby? I just installed cucumber and cucumber-rails. I then tried gem install pickle and it installs, but a script/generate pickle yields Couldn't find 'pickle' generator I did everything according to the readme no dice. Anyone have any experience with this: specs jruby-1.4.0 rail...
I am getting below error when i am trying to run my cucumber features. Using the default profile... WARNING: Nokogiri was built against LibXML version 2.7.5, but has dynamically loaded 2.7.6 Strategy is : truncation and ORM is active_record regular expression too big: /^((?:(?:)|(?:(?:a|an|another|the|that) )?(?:(?:(?:(?:first|last|...
i'm trying to pickle a big class and getting "TypeError: can't pickle module objects". despite looking around the web, i can't exactly figure out what this means. and i'm not sure which "module object" is causing the trouble. is there a way to find the culprit? the stack trace doesn't seem to indicate anything. ...
In order to save time moving data I pickled some models and dumped them to file. I then reloaded them into another database using the same exact model. The save worked fine and the objects kept their old id which is what I wanted. However, when saving new objects I run into nextval errors. Not being very adept with postgres, I'm no...
I am building music app, where user can do several tasks including but not limited to listening song, like song, recommend song to a friend and extra. currently I have this model: class Activity(models.Model): activity = models.TextField() user = models.ForeignKey(User) date = models.DateTimeField(auto_now=True) so far I t...
I know this isn't exactly how the pickle module was intended to be used, but I would have thought this would work. I'm using Python 3.1.2 Here's the background code: import pickle FILEPATH='/tmp/tempfile' class HistoryFile(): """ Persistent store of a history file Each line should be a separate Python object Usually...
I've been getting RuntimeError: maximum recursion depth exceeded when trying to pickle a highly-recursive tree object. Much like this asker here. He solved his problem by setting the recursion limit higher with sys.setrecursionlimit. But I don't want to do that: I think that's more of a workaround than a solution. Because I want to be a...
What is a correct way to pickle an object from a class with slots, when this object references itself through one of its attributes? Here is a simple example, with my current implementation, which I'm not sure is 100 % correct: import weakref import pickle class my_class(object): __slots__ = ('an_int', 'ref_to_self', '__weakref__...
I would like to pickle an unbound method in Python 3.x. I'm getting this error: >>> class A: ... def m(self): ... pass >>> import pickle >>> pickle.dumps(A.m) Traceback (most recent call last): File "<pyshell#3>", line 1, in <module> pickle.dumps(A.m) File "C:\Python31\lib\pickle.py", line 1358, in dumps Pickler(...
(Keep in mind I'm working in Python 3, so a solution needs to work in Python 3.) I would like to use the copyreg module to teach Python how to pickle functions. When I tried to do it, the _Pickler object would still try to pickle functions using the save_global function. (Which doesn't work for unbound methods, and that's the motivation...
I'm having trouble unpickling objects from Google App Engine. I am running Windows 7. Here is the procedure: Create a CSV with one of the fields being pickle.dumps([[('CS', 2110), ('CS', 3300), ('CS', 3140)]]), or some similar argument. The CSV looks something like this: INFO,2210,"CS 2110, 3300, 3140","(lp0 (lp1 (S'CS' p2 I2110 tp3...
I'm writing a mapping class which persists to the disk. I am currently allowing only str keys but it would be nice if I could use a couple more types: hopefully up to anything that is hashable (ie. same requirements as the builtin dict), but more reasonable I would accept string, unicode, int, and tuples of these types. To that end I wo...
So I have a function which creates a dynamic model. I accomplish this in a way very similar to AuditTrail (see django wiki). Sample of code is here: https://gist.github.com/0212845ae00891efe555 Is there any way I can make a dynamically-generated class pickle-able? Ideally something thats not a crazy monkeypatch/hack? ...