And to Mike and Ian for their continued help and advice.
So, what do we think about this? It's pretty close to Jason's work, but slightly different in that no patch to Paste is required, and since I use pip rather than easy_install, things work out a little differently for sys.path. Also, I have managed to get the DataStore working on the local standard Pylons install.
> And to Mike and Ian for their continued help and advice.
> So, what do we think about this? It's pretty close to Jason's work, but > slightly different in that no patch to Paste is required, and since I use > pip rather than easy_install, things work out a little differently for > sys.path. Also, I have managed to get the DataStore working on the local > standard Pylons install.
just tried the wiki page
i got the following probs:
File "lib/setuptools-0.6c11-py2.5.egg/pkg_resources.py", line 24, in
<module>
from os import utime, rename, unlink, mkdir
ImportError: cannot import name mkdir
INFO 2009-10-29 19:56:30,793 dev_appser
reco
On Oct 22, 1:57 pm, Matt Feifarek <matt.feifa...@gmail.com> wrote:
> > And to Mike and Ian for their continued help and advice.
> > So, what do we think about this? It's pretty close to Jason's work, but
> > slightly different in that no patch to Paste is required, and since I use
> > pip rather than easy_install, things work out a little differently for
> > sys.path. Also, I have managed to get the DataStore working on the local
> > standard Pylons install.
On Thu, Oct 29, 2009 at 3:19 PM, reco <r...@nex9.com> wrote: > just tried the wiki page > i got the following probs:
> File "lib/setuptools-0.6c11-py2.5.egg/pkg_resources.py", line 24, in > <module> > from os import utime, rename, unlink, mkdir > ImportError: cannot import name mkdir > INFO 2009-10-29 19:56:30,793 dev_appser
Yeah, I'm now getting this too. I think that the 1.2.7 release of GAE broke this. I'm going to look into it.
This is the kind of thing I'm worried about with trying to hack our way into running GAE apps on Paster. Everything works fine, then Google does something mysterious and then a fundamental breakage occurs.
I'm going to see if I can figure this out, but I think that we should take Ian Bicking's advice and put our efforts into getting the Pylons stack running on the dev appserver from Google.
On Sun, Nov 1, 2009 at 9:06 AM, Matt Feifarek <matt.feifa...@gmail.com> wrote: > On Thu, Oct 29, 2009 at 3:19 PM, reco <r...@nex9.com> wrote:
>> just tried the wiki page >> i got the following probs:
>> File "lib/setuptools-0.6c11-py2.5.egg/pkg_resources.py", line 24, in >> <module> >> from os import utime, rename, unlink, mkdir >> ImportError: cannot import name mkdir >> INFO 2009-10-29 19:56:30,793 dev_appser
> Yeah, I'm now getting this too. I think that the 1.2.7 release of GAE broke > this. I'm going to look into it. > This is the kind of thing I'm worried about with trying to hack our way into > running GAE apps on Paster. Everything works fine, then Google does > something mysterious and then a fundamental breakage occurs. > I'm going to see if I can figure this out, but I think that we should take > Ian Bicking's advice and put our efforts into getting the Pylons stack > running on the dev appserver from Google.
mkdir is one of those operations that's prohibited on App Engine. The development server is faithfully mimicking the production server in this regard. Appengine Monkey puts stub functions into the os module to make this work. (You can't actually create a directory, but you can import a module that imports the functions.) I don't know how Google implemented support for Setuptools, whether they did it this way or not.
So the question is, what's the difference between Matt's code and the appengine-homedir code that accounts for this exception?
This is why I wish Pylons/Paste didn't use Setuptools at runtime, because it's hard to trace why things like this are happening. I think runner.py in appengine-homedir instantiates the application and HTTPServer manually, bypassing pkg_resources precisely to avoid problems like this.
Also, make sure you're not using an old Appengine SDK.
> File "lib/setuptools-0.6c11-py2.5.egg/pkg_resources.py", line 24, in
> <module>
> from os import utime, rename, unlink, mkdir
> ImportError: cannot import name mkdir
> INFO 2009-10-29 19:56:30,793 dev_appser
Actually, the latest setuptools is what changed. There didn't used to
be a specific import of mkdir.
I solved it in my app.py by setting os.mkdir to a value before
importing anything else. Hackish, but it works fine. This leaves you
open to breakage from something else the next time you upgrade any
third-party deps, of course. But an attempt to preemptively handle all
such errors would itself be prone to breakage if the supported set of
modules/functions changed in the future, so it's probably better just
to spread the idea of setting such functions equal to a value before
other imports that use them.
As for what will raise errors, the GAE Python runtime docs make the oh-
so-helpful statement that "the os module is available, but with
unsupported features disabled," without mentioning exact function
names. I generated a list, though, and it's pretty long. Another issue
is that while some will raise ImportError/AttributeError (e.g.
os.mkdir), others will import fine but not work (tempfile.mkstemp).
Has anyone published a list of standard library functions that result
in ImportError/AttributeError when accessed under GAE? Otherwise, I
might create an automated process for locating them dynamically, for,
eh, fun or something.
>> File "lib/setuptools-0.6c11-py2.5.egg/pkg_resources.py", line 24, in >> <module> >> from os import utime, rename, unlink, mkdir >> ImportError: cannot import name mkdir >> INFO 2009-10-29 19:56:30,793 dev_appser
> Actually, the latest setuptools is what changed. There didn't used to > be a specific import of mkdir.
This is an App Engine bug then and should be raised on is issue tracker. They specifically made App Engine compatible with Setuptools, so they;'re probably not aware the latest version broke. (They don't use Setuptools themselves.)
> This leaves you > open to breakage from something else the next time you upgrade any > third-party deps, of course.
This is one of the general limitations of App Engine. It works best with packages that are written for it, or at least don't do any disk I/O.
> As for what will raise errors, the GAE Python runtime docs make the oh- > so-helpful statement that "the os module is available, but with > unsupported features disabled," without mentioning exact function > names. I generated a list, though, and it's pretty long. Another issue > is that while some will raise ImportError/AttributeError (e.g. > os.mkdir), others will import fine but not work (tempfile.mkstemp).
> Has anyone published a list of standard library functions that result > in ImportError/AttributeError when accessed under GAE? Otherwise, I > might create an automated process for locating them dynamically, for, > eh, fun or something.
There is probably a list somewhere. If not, you can write a quick webapp to test all of them. I don't think they'll be changing the list. They're pretty smart about backward compatibility, which is why runtime version 1 is sticking with Python 2.5.
The main Pylons-Appengine article needs a rewrite to include all this new research and findings. If something works for you and you've tested it, it's worth featuring. We need to keep the current advice in one central place so it doesn't get lost. If you're unsure about your approach, it's still worth mentioning; just list what you're unsure about so the reader can make up their own mind.
> On Thu, Oct 29, 2009 at 3:19 PM, reco <r...@nex9.com> wrote:
>> just tried the wiki page >> i got the following probs:
>> File "lib/setuptools-0.6c11-py2.5.egg/pkg_resources.py", line 24, in >> <module> >> from os import utime, rename, unlink, mkdir >> ImportError: cannot import name mkdir >> INFO 2009-10-29 19:56:30,793 dev_appser
> Yeah, I'm now getting this too. I think that the 1.2.7 release of GAE broke > this. I'm going to look into it.
Actually, it turns out I have a *different* error; I'm getting a problem with os.makedirs. I patched it thusly in my runpylons.py:
# fix the setuptools bug introduced in GAE 1.2.7 def dummyfunc(*args, **kwargs): pass os.makedirs = dummyfunc
That allows the app to be run under the dev_appserver. Reco, what version of the SDK are you running when you have this problem?
i just modified the code in file appengine_monkey.py, add a line of
code after line 37:
Missing.patch(os, 'mkdir')
because of the implementation of Ian Bicking's "filesystem.py" module,
i think, it works.
but i do not know the risk behind that change, someone knows?
On Nov 3, 12:12 am, Matt Feifarek <matt.feifa...@gmail.com> wrote:
|
