Mike Meyer <m...@mired.org> writes: > Note that the quoted article only applies to *writing* attributes. It > doesn't say anything about needing accessors to *read* a > variable. This encourages me that the convention I use - adopted from > Eiffel, where the compiler enforces it - of freeling reading > attributes, but providing methods to write them - is a right way todo > things.
Generally that sounds reasonable. Obviously there are other examples when (e.g. for security) you have to make sure that variables can't be read by other classes, e.g. you have a class that stores a capability (or a password) in an instance variable, and uses it for privileged operations.
>>>to be able to share private variables with other classes under certain >>>circumstances, it's better to use something like C++'s "friend" >>>declaration, where you can export the variables to a specific other class.
>>That assumes that you always know for sure that the given variable >>will always be used as a private/friend variable in the lifecycle of >>the software.
> Obviously if you find you need to use it in another place later, you > update the declaration. The idea is for you (or an automatic tool) to
If its your code this is possible, but if not and the maintainer is not willing or able to change it, then you have a problem.
The fact that in other languages there are wired tricks to get to private instance variables is a strong indicator that there is a need for that.
Gregor Horvath <g.horv...@gmx.at> writes: > If its your code this is possible, but if not and the maintainer is > not willing or able to change it, then you have a problem.
Perhaps the maintainer has good reason for not wanting to change it. After all, he's maintaining the code and you're not. In the course of that maintenance he might change the internal usage of the private variable that messes up the way you use it. He's supposed to be allowed to do that--that's why the variable is private. Is he supposed to get your permission every time he wants to change how the private variables in his class work?
> The fact that in other languages there are wired tricks to get to > private instance variables is a strong indicator that there is a need > for that.
Java has some JVM switches that open ways to get at private variables so things like debuggers can work. In production code you'd turn that stuff off. And it's certainly not intended for normal classes in an application to get at the private variables of other classes.
> Guided freedom is better than enforced authority.
The obvious way to provide that guidance is to declare variables to be private if other classes shouldn't mess with the variables. There is still the freedom to change the declaration if necessary. Unless of course you don't have the source code. Where there is no source code, there is no freedom, no matter what features the language has.
Paul Rubin <http://phr...@NOSPAM.invalid> writes: > Gregor Horvath <g.horv...@gmx.at> writes: >> > to be able to share private variables with other classes under certain >> > circumstances, it's better to use something like C++'s "friend" >> > declaration, where you can export the variables to a specific other class.
>> That assumes that you always know for sure that the given variable >> will always be used as a private/friend variable in the lifecycle of >> the software.
> Obviously if you find you need to use it in another place later, you > update the declaration. The idea is for you (or an automatic tool) to > be able to find all the uses of some instance variable. In Python > (because of things like setattr), that can't be done.
One of the advantages of avoiding boilerplate is that you avoid the need to maintain it.
It's not clear that tracking all uses of a variable in Python can't be done. It's clear that it's *difficult* - you have to track the objects bound to both arguments for ever setattr call - but that's not the same thing as impossible. My gut says "halting problem", but I haven't proved it. Most other languages aren't that different - unless you place serious restrictions on them. For instance, most languages will let me pass a reference to an instance variable to a method of another object. If that method then stores the reference in a global (or class, or instance) variable, you suddenly have to track the value of that new variable to figure out if the variable of interest is being used whenever the new one is used. Which is pretty much the same place you get with setattr.
<mike -- Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Paul Rubin <http://phr...@NOSPAM.invalid> writes: > Mike Meyer <m...@mired.org> writes: >> Note that the quoted article only applies to *writing* attributes. It >> doesn't say anything about needing accessors to *read* a >> variable. This encourages me that the convention I use - adopted from >> Eiffel, where the compiler enforces it - of freeling reading >> attributes, but providing methods to write them - is a right way todo >> things.
> Generally that sounds reasonable. Obviously there are other examples > when (e.g. for security) you have to make sure that variables can't be > read by other classes, e.g. you have a class that stores a capability > (or a password) in an instance variable, and uses it for privileged > operations.
If you can't trust the code that shares your address space, you're in a world of hurt for security. Compile-time restrictions don't matter for squat - you need serious restrictions on what the program can do at runtime.
<mike -- Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
On Wed, 28 Sep 2005 15:23:07 -0700, Paul Rubin wrote: > Gregor Horvath <g.horv...@gmx.at> writes: >> If its your code this is possible, but if not and the maintainer is >> not willing or able to change it, then you have a problem.
> Perhaps the maintainer has good reason for not wanting to change it. > After all, he's maintaining the code and you're not. In the course of > that maintenance he might change the internal usage of the private > variable that messes up the way you use it. He's supposed to be > allowed to do that--that's why the variable is private. Is he > supposed to get your permission every time he wants to change how the > private variables in his class work?
No, but that is precisely why Python's semi-private variables are usually better. Names like _X and class.__X are warnings to the developer "use these at your own risk", without preventing developers who need them from using them. You have most of the benefits of private variables with none of the disadvantages.
Steven D'Aprano <st...@REMOVETHIScyber.com.au> writes: > No, but that is precisely why Python's semi-private variables are > usually better. Names like _X and class.__X are warnings to the developer > "use these at your own risk", without preventing developers who need them > from using them. You have most of the benefits of private variables with > none of the disadvantages.
I'm sorry but I thought the idea was to actually reduce the risk of bugs, not merely attach the risk to the right person. If changing the way a class uses its own private variables breaks an application because another class was using the private variable unexpectedly, then that's bad, regardless of whether the other class's author was notified or not. It's better to let the compiler automatically flag these things, than to depend on conventions.
Mike Meyer <m...@mired.org> writes: > > Generally that sounds reasonable. Obviously there are other examples > > when (e.g. for security) you have to make sure that variables can't be > > read by other classes, e.g. you have a class that stores a capability > > (or a password) in an instance variable, and uses it for privileged > > operations.
> If you can't trust the code that shares your address space, you're in > a world of hurt for security. Compile-time restrictions don't matter > for squat - you need serious restrictions on what the program can do > at runtime.
>>> I thought about it, but I didn't mention it in the end because this >>> feature ("name mangling") isn't intended as a mechanism for making >>> things private - it's intended to prevent namespace clashes when >>> doing >>> multiple inheritance.
>> There is limited support for class-private identifiers. >> [...] >> Name mangling is intended to give classes an easy way to define >> ``private'' instance variables and methods, >> [...] >> """
> the sentence you're quoting the first part of continues:
> without having to worry about instance variables defined by > derived > classes
That elaborates on the intent, it doesn't change it. The sentence clearly says that the intent is to easily define private variables, whereas Simon said that it the intent was not to provide a mechanism for making variables private.
> and the paragraph later says:
> Note that the mangling rules are designed mostly to avoid > accidents
That's explaining why you can still access the variables if you want to, not saying that this feature isn't meant to be used to indicate that a variable is private.
> and both sentences are from the *tutorial*, which doesn't exactly > qualify as a design document.
A tutorial should not encourage users to use a feature in the wrong way, though. If leading underscore(s) were not meant to indicate privateness, then the tutorial should not state that - this is where a large proportion of users learn Python; it's nonsensical to teach them something that's incorrect.
> if you want more rationale, here's the > post that led to the current design:
> "In my version, private data has it's own scope, so that name > clashes > will not occur if a private with the same name is used in a > subclass."
> see the rest of that thread for more about the history of __.
Disagreeing with someone who actually took part in the discussion may not be a sensible idea but...
It's pretty clear to me from that thread that using a single/double underscore with variables *is* intended to indicate that the variable is private in some fashion. As Guido pointed out:
'*leading* underscore has a long tradition (in the C world, anyway, but we don't really want to ignore that here) of meaning "internal use" (of some kind)'
The name mangling lets one use private variables without having to worry about name clashes, but the purpose of the leading underscore (s) is to indicate that the variable is private
"private seems to be useful because it offers a form of data hiding that's currently absent in Python"
as many people in that thread indicated that they were doing before this was proposed as an addition.
The OP wanted to know how to use private (and protected) variables in Python. Even the FAQ has this answer:
'Variables with double leading underscore are "mangled" to provide a simple but effective way to define class private variables. [...] This doesn't guarantee privacy: an outside user can still deliberately access the "_classname__spam" attribute, and private values are visible in the object's __dict__."
Which is basically what I said in my reply to the OP. If this isn't the intent, then there's a lot of incorrect documentation, and a lot of incorrect c.l.p answers to this question in the past*.
=Tony.Meyer
* Well, there are no doubt a lot of incorrect c.l.p answers to any question :). But anyway...
Paul Rubin <http://phr...@NOSPAM.invalid> writes: > Mike Meyer <m...@mired.org> writes: >> > Generally that sounds reasonable. Obviously there are other examples >> > when (e.g. for security) you have to make sure that variables can't be >> > read by other classes, e.g. you have a class that stores a capability >> > (or a password) in an instance variable, and uses it for privileged >> > operations.
>> If you can't trust the code that shares your address space, you're in >> a world of hurt for security. Compile-time restrictions don't matter >> for squat - you need serious restrictions on what the program can do >> at runtime.
> You need both.
Yup. Any language besides Java even *try* to provide both for a production environment? Lots of languages do runtime checking that can be disabled for production compilation - which makes it's worthless in this case.
Of course, at this point you're no longer talking about a general purpose programming environment. Language design decisions that are correct for this environment aren't necessarily correct for general purpose programming languages. Trying to tweak some exiting general purpose language to make it suitable for use in the kind of environment where you can't trust the code you share your address space with is the wrong way to go about it. You want to design such a language to fit your secure environment, *after* you've designed that environment.
At that point, things which are unrelated to the security of the environment may be more attractive than they would be in a general purpose programming language. A number of runtime checks have to be in place to insure that semantics of the language stay "correct". Since we're going to have those, I would like constructs I can use to ensure that the semantics of the program stay correct, like function entry/exit conditions, loop and object invariants, and so on. Basically, the whole DbC thing.
<mike -- Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
> allowed to do that--that's why the variable is private. Is he > supposed to get your permission every time he wants to change how the > private variables in his class work?
No, but the assumption here is that the maintainer / designer of a class alaways knows everything and things are static. Unfortunatly this is wrong in real live.
When you use a _ variable in python you know that this may break in the future because of changing interfaces. Thats a typical engineering Trade Off on my side. One that is in other languages not even possible. A clear disadvantage. Python gives me power and does not take it away like the others.
> stuff off. And it's certainly not intended for normal classes in an > application to get at the private variables of other classes.
So is in Python. It is certainly not intended to use _ variables. But you can do it, at your own risk. I do not need an authority that forbiddes things, but one that guides in the correct direction without coming in my way. The engineering decisions regarding my application should be on my side, not the language lawyers.
Gregor Horvath <g.horv...@gmx.at> writes: > No, but the assumption here is that the maintainer / designer of a > class alaways knows everything and things are static. Unfortunatly > this is wrong in real live.
I'd say it's pretty far removed from how multi-person software projects actually work.
> When you use a _ variable in python you know that this may break in > the future because of changing interfaces. Thats a typical engineering > Trade Off on my side. > One that is in other languages not even possible. A clear disadvantage. > Python gives me power and does not take it away like the others.
Huh? If my car has a "feature" that lets someone blow it to smithereens from hundreds of miles away without even intending to, that's somehow an advantage?
> The engineering decisions regarding my application should be on my > side, not the language lawyers.
No problem, but any decision like that should be expressed in "writing", by re-declaring the variable so it's no longer private.
Tony Meyer wrote: > That elaborates on the intent, it doesn't change it. The sentence > clearly says that the intent is to easily define private variables, > whereas Simon said that it the intent was not to provide a mechanism > for making variables private.
Are you aware of the fact that computer terms might have slightly different meanings in different languages, due to differences in language details and semantics? Of course they're "private variables", but they're "private" in the Python sense, and they were added to Python to solve problems that were observed in Python, not because someone thought it was important to cater to confused C++ or Java programmers.
And as Simon said, and the original thread showed, the problem they were (and are) intended to address is accidental namespace collisions when sub- classing.
If we'd really needed "true" private variables, don't you think we would have been able to come up with a design that provided that? It just wasn't important, because Python is Python.
> Huh? If my car has a "feature" that lets someone blow it to > smithereens from hundreds of miles away without even intending to, > that's somehow an advantage?
I would not accept a car that does constraint my driving directions. I want to drive for myself, because its fun.
> No problem, but any decision like that should be expressed in > "writing", by re-declaring the variable so it's no longer private.
Remember, its Python not Java. Tastes are different and so are programming languages, but please do not make Python to a Java Clone.
I love python because of its simplicity, freedom, power and because its fun to program. One reason is that it does not restrict the programer to tight und has genuine simple solutions, like the one with "private" instance variables.
Gregor Horvath <g.horv...@gmx.at> writes: > One reason is that it does not restrict the programer to tight und has > genuine simple solutions, like the one with "private" instance > variables.
If you don't want the compiler to make sure your private instance variables stay private, then don't declare them that way. You're the one asking for less flexibility.
Steve Holden wrote: > could ildg wrote: > > Python is wonderful except that it has no real private and protected > > properties and methods. > > Every py object has dict so that you can easily find what fields and > > methods an obj has, > > this is very convenient, but because of this, py is very hard to support > > real private and > > protected? > > If private and protected is supported, python will be perfect.
> You only say that because you assume private and protected give you a > security that they actually don't. They certainly make it more difficult > to *spot* the security errors.
Honestly I like to use private/protect/public modifiers in C++ for the sake of code documentation. I like to know which attributes are dedicated to be known by other objects, which ones are for internal use only and which ones should be at least publicly accessible within a class hierarchy. This helps structuring code in the large and spotting attention. Code becomes easier accessible. But if we have Sunday or I get asked by serious programmers I also know the right technical answers about protection rights, security etc.
> Ah, but in the way of your code -- it is not "your car"... It is the > car you supplied to someone "hundreds of miles away"... And they are > perfectly free to open the hood... tamper with the engine programming, etc.
I don't understand what you're getting at. If they're free to edit the code, then why should they complain about private variables? They can change the declaration if they need to.
> If you don't want the compiler to make sure your private instance > variables stay private, then don't declare them that way. You're the > one asking for less flexibility.
I want to declare them as privat, but want to give the flexibilty to access them at the users own risk.
Declaring everything as public is nonsene, because there should be a garanteed stable interface.
Gregor Horvath <g.horv...@gmx.at> writes: > > If you don't want the compiler to make sure your private instance > > variables stay private, then don't declare them that way. You're the > > one asking for less flexibility.
> I want to declare them as private, but want to give the flexibilty to > access them at the users own risk.
What else do you want to let users do at their own risk? Treat integers as pointers, like in C? Both are violations of type safety.
> Declaring everything as public is nonsene, because there should be a > garanteed stable interface.
You could have a "friend" declaration like in C++, if you want to let some class see the private instance variables of another class.
I can't think of a single time that I've ever seen a legitimate use of name mangling to reach from one class into another in a Python application (I don't count something like a debugger). If you're got some concrete examples I wouldn't mind looking.
Chris Gonnerman wrote: > There are two philosophies about programming:
> -- Make it hard to do wrong.
> -- Make it easy to do right.
> What you are promoting is the first philosophy: Tie the programmer's > hands so he can't do wrong. Python for the most part follows the > second philosophy,
So it is for the very this reason there is no assignment operator in the Python?
> You could have a "friend" declaration like in C++, if you want to let > some class see the private instance variables of another class.
Everything is said on this topic. There are two ligitimate solutions to the problem of private instance variables. Its a matter of taste, and mine is the pythonic one.
Gregor Horvath <g.horv...@gmx.at> writes: > Everything is said on this topic. There are two ligitimate solutions > to the problem of private instance variables. Its a matter of taste, > and mine is the pythonic one.
The Pythonic solution is to have both solutions available, and Python in fact used to have both (name mangling and the Bastion class). But Bastion turned out not to work, and it's not easy to fix that under CPython, so it was removed. Now the Bastion class is gone, and only name mangling is left. Maybe Bastion can come back under Pypy.
en.karpac...@ospaz.ru wrote: > On Wed, 28 Sep 2005 08:14:50 -0500 > Chris Gonnerman wrote:
>>There are two philosophies about programming:
>>-- Make it hard to do wrong.
>>-- Make it easy to do right.
>>What you are promoting is the first philosophy: Tie the programmer's >>hands so he can't do wrong. Python for the most part follows the >>second philosophy,
> So it is for the very this reason there is no assignment operator in the > Python?
If you are really asking whether assignment was deliberately designed as a statement rather than an operator, the answer is "yes".
Paul Rubin wrote: > Gregor Horvath <g.horv...@gmx.at> writes:
>>>If you don't want the compiler to make sure your private instance >>>variables stay private, then don't declare them that way. You're the >>>one asking for less flexibility.
>>I want to declare them as private, but want to give the flexibilty to >>access them at the users own risk.
> What else do you want to let users do at their own risk? Treat > integers as pointers, like in C? Both are violations of type safety.
>>Declaring everything as public is nonsene, because there should be a >>garanteed stable interface.
> You could have a "friend" declaration like in C++, if you want to let > some class see the private instance variables of another class.
> I can't think of a single time that I've ever seen a legitimate use of > name mangling to reach from one class into another in a Python > application (I don't count something like a debugger). If you're got > some concrete examples I wouldn't mind looking.
[psst ... do you think they've stopped now?] -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.pycon.org
Paul Rubin wrote: > If changing the way a class uses its own private variables breaks an application > because another class was using the private variable unexpectedly, > then that's bad, regardless of whether the other class's author was > notified or not. It's better to let the compiler automatically flag > these things, than to depend on conventions.
The problem with Python's compiler as it is today is that it can't generally detect private attribute access: information about an object/class being accessed is only guaranteed to be available at run-time. Of course, Bastion and its peers [1] "solve" this by introducing run-time environments which seek to guarantee attribute privacy.
As for the merits of private/protected/final attributes and methods, I've seen enough time-wasting and premature "security" optimisations to know that such facilities can also waste a tremendous amount of time in multi-person projects.
Paul
[1] At EuroPython there was a talk about protecting object attributes in Ubuntu Launchpad [2], for which the slides are apparently unavailable, and while chatting about this later on I noted that I'd seen something like it before. The realisation that mxProxy [3] was this apparently often overlooked thing occurred more or less simultaneously with the realisation that I was talking to its author. :-)
|
