Content Generation Using Key Words Python
- Content Generation Using Key Words Python Download
- Next Keyword Python
- Content Generation Using Keywords Python In Excel
Python Docstrings In this tutorial, we will learn about Python docstrings. More specifically, we will learn how and why docstrings are used with the help of examples. Python Keywords Previous Next. To end a function, returns a generator. But we cannot warrant full correctness of all content. While using this site. Generator objects are used either by calling the next method on the generator object or using the generator object in a “for in” loop (as shown in the above program). # A Python program to demonstrate use of. # generator object with next # A generator function. Def simpleGeneratorFun: yield 1.
Welcome! Are you completely new to programming?If not then we presume you will be looking for information aboutwhy and how to get started with Python.Fortunately anexperienced programmer in any programming language (whatever it may be)can pick up Python very quickly.It's also easy for beginners to use and learn, sojump in!
Installing Python is generally easy, and nowadaysmany Linux and UNIX distributions include a recent Python.Evensome Windows computers (notably those from HP) now come with Pythonalready installed.If you do need to install Python and aren't confident about thetask you can finda few notes on theBeginnersGuide/Downloadwiki page, but installation is unremarkable on most platforms.
Before getting started, you may want to find out which IDEs and texteditors are tailored to makePython editing easy, browse the list of introductory books, or look at code samples that you might findhelpful.
Content Analysis in Python. This page is currently not much more than an extended advertisment for doing content analysis in Python. In time it might expand to a full tutorial, should anyone express interest in reading one. In the meantime it'll hopefully just whet your appetite. Sep 25, 2019 The use of multiple Python yield statements can be leveraged as far as your creativity allows. Using Advanced Generator Methods. You’ve seen the most common uses and constructions of generators, but there are a few more tricks to cover. In addition to yield, generator objects can make use of the following methods.send.throw.close. How to change a set in Python? Sets are mutable. But since they are unordered, indexing have no meaning. We cannot access or change an element of set using indexing or slicing. Set does not support it. We can add single element using the add method and multiple elements using the update method. Mar 06, 2018 Examples of text generation include machines writing entire chapters of popular novels like Game of Thrones and Harry Potter, with varying degrees of success. In this article, we will use python and the concept of text generation to build a machine learning model that can write sonnets in the style of William Shakespeare. Let’s get into it!
There is a list of tutorials suitable for experienced programmers on theBeginnersGuide/Tutorialspage. There is also a list ofresources in other languageswhich might be useful if English is not your first language.
The online documentationis your first port of call for definitive information.There is a fairly brieftutorialthat gives you basic information about the language andgets you started. You can follow this by looking at thelibrary referencefor a full description of Python's many libraries and thelanguage reference fora complete (though somewhat dry) explanation of Python's syntax.If you are looking for common Python recipes and patterns, youcan browse the ActiveState Python Cookbook
If you want to know whether a particular application, or a librarywith particular functionality, is available in Python there are anumber of possible sources of information. The Python web siteprovides aPython Package Index(also known as the Cheese Shop, a reference to the Monty Pythonscript of that name).There is also asearch page for a number of sources of Python-relatedinformation. Failing that, justGoogle for a phrase including the word 'python'and you may well get the result you need.If all else fails, ask on thepython newsgroupand there's a good chance someone will put you on the right track.
If you have a question, it's a good idea to try theFAQ, which answers the most commonlyasked questions about Python.
If you want to help to develop Python, take a look at thedeveloper area for further information.Please note that you don't have to be an expert programmerto help. The documentation is just as important as thecompiler, and still needs plenty of work!
| PEP: | 343 |
|---|---|
| Title: | The 'with' Statement |
| Author: | Guido van Rossum, Nick Coghlan |
| Status: | Final |
| Type: | Standards Track |
| Created: | 13-May-2005 |
| Python-Version: | 2.5 |
| Post-History: | 2-Jun-2005, 16-Oct-2005, 29-Oct-2005, 23-Apr-2006, 1-May-2006, 30-Jul-2006 |
Contents
This PEP adds a new statement 'with' to the Python language to makeit possible to factor out standard uses of try/finally statements.
State of decay 2 portable generator location. In this PEP, context managers provide __enter__() and __exit__()methods that are invoked on entry to and exit from the body of thewith statement.
This PEP was originally written in first person by Guido, andsubsequently updated by Nick Coghlan to reflect later discussionon python-dev. Any first person references are from Guido'soriginal.
Python's alpha release cycle revealed terminology problems in thisPEP and in the associated documentation and implementation [14].The PEP stabilised around the time of the first Python 2.5 betarelease.
Yes, the verb tense is messed up in a few places. We've beenworking on this PEP for over a year now, so things that wereoriginally in the future are now in the past :)
After a lot of discussion about PEP 340 and alternatives, Idecided to withdraw PEP 340 and proposed a slight variant on PEP310. After more discussion, I have added back a mechanism forraising an exception in a suspended generator using a throw()method, and a close() method which throws a new GeneratorExitexception; these additions were first proposed on python-dev in[2] and universally approved of. I'm also changing the keyword to'with'.
After acceptance of this PEP, the following PEPs were rejected dueto overlap:
- PEP 310, Reliable Acquisition/Release Pairs. This is theoriginal with-statement proposal.
- PEP 319, Python Synchronize/Asynchronize Block. Its use casescan be covered by the current PEP by providing suitablewith-statement controllers: for 'synchronize' we can use the'locking' template from example 1; for 'asynchronize' we can usea similar 'unlocking' template. I don't think having an'anonymous' lock associated with a code block is all thatimportant; in fact it may be better to always be explicit aboutthe mutex being used.
PEP 340 and PEP 346 also overlapped with this PEP, but werevoluntarily withdrawn when this PEP was submitted.
Some discussion of earlier incarnations of this PEP took place onthe Python Wiki [3].
PEP 340, Anonymous Block Statements, combined many powerful ideas:using generators as block templates, adding exception handling andfinalization to generators, and more. Besides praise it receiveda lot of opposition from people who didn't like the fact that itwas, under the covers, a (potential) looping construct. Thismeant that break and continue in a block-statement would break orcontinue the block-statement, even if it was used as a non-loopingresource management tool.
But the final blow came when I read Raymond Chen's rant aboutflow-control macros [1]. Raymond argues convincingly that hidingflow control in macros makes your code inscrutable, and I findthat his argument applies to Python as well as to C. I realizedthat PEP 340 templates can hide all sorts of control flow; forexample, its example 4 (auto_retry()) catches exceptions andrepeats the block up to three times.
However, the with-statement of PEP 310 does not hide controlflow, in my view: while a finally-suite temporarily suspends thecontrol flow, in the end, the control flow resumes as if thefinally-suite wasn't there at all.
Remember, PEP 310 proposes roughly this syntax (the 'VAR =' part isoptional):
which roughly translates into this:
Now consider this example:
Here, just as if the first line was 'if True' instead, we knowthat if BLOCK1 completes without an exception, BLOCK2 will bereached; and if BLOCK1 raises an exception or executes a non-localgoto (a break, continue or return), BLOCK2 is not reached. Themagic added by the with-statement at the end doesn't affect this.
(You may ask, what if a bug in the __exit__() method causes anexception? Then all is lost -- but this is no worse than withother exceptions; the nature of exceptions is that they can happenanywhere, and you just have to live with that. Even if youwrite bug-free code, a KeyboardInterrupt exception can still causeit to exit between any two virtual machine opcodes.)
This argument almost led me to endorse PEP 310, but I had one idealeft from the PEP 340 euphoria that I wasn't ready to drop: usinggenerators as 'templates' for abstractions like acquiring andreleasing a lock or opening and closing a file is a powerful idea,as can be seen by looking at the examples in that PEP.
Inspired by a counter-proposal to PEP 340 by Phillip Eby I triedto create a decorator that would turn a suitable generator into anobject with the necessary __enter__() and __exit__() methods.Here I ran into a snag: while it wasn't too hard for the lockingexample, it was impossible to do this for the opening example.The idea was to define the template like this:
and used it like this:
The problem is that in PEP 310, the result of calling EXPR isassigned directly to VAR, and then VAR's __exit__() method iscalled upon exit from BLOCK1. But here, VAR clearly needs toreceive the opened file, and that would mean that __exit__() wouldhave to be a method on the file.
While this can be solved using a proxy class, this is awkward andmade me realize that a slightly different translation would makewriting the desired decorator a piece of cake: let VAR receive theresult from calling the __enter__() method, and save the value ofEXPR to call its __exit__() method later. Then the decorator canreturn an instance of a wrapper class whose __enter__() methodcalls the generator's next() method and returns whatever next()returns; the wrapper instance's __exit__() method calls next()again but expects it to raise StopIteration. (Details below inthe section Optional Generator Decorator.)
So now the final hurdle was that the PEP 310 syntax:
would be deceptive, since VAR does not receive the value ofEXPR. Borrowing from PEP 340, it was an easy step to:
Additional discussion showed that people really liked being ableto 'see' the exception in the generator, even if it was only tolog it; the generator is not allowed to yield another value, sincethe with-statement should not be usable as a loop (raising adifferent exception is marginally acceptable). To enable this, anew throw() method for generators is proposed, which takes one tothree arguments representing an exception in the usual fashion(type, value, traceback) and raises it at the point where thegenerator is suspended.
Once we have this, it is a small step to proposing anothergenerator method, close(), which calls throw() with a specialexception, GeneratorExit. This tells the generator to exit, andfrom there it's another small step to proposing that close() becalled automatically when the generator is garbage-collected.
Then, finally, we can allow a yield-statement inside a try-finallystatement, since we can now guarantee that the finally-clause will(eventually) be executed. The usual cautions about finalizationapply -- the process may be terminated abruptly without finalizingany objects, and objects may be kept alive forever by cycles ormemory leaks in the application (as opposed to cycles or leaks inthe Python implementation, which are taken care of by GC).
Note that we're not guaranteeing that the finally-clause isexecuted immediately after the generator object becomes unused,even though this is how it will work in CPython. This is similarto auto-closing files: while a reference-counting implementationlike CPython deallocates an object as soon as the last referenceto it goes away, implementations that use other GC algorithms donot make the same guarantee. This applies to Jython, IronPython,and probably to Python running on Parrot.
(The details of the changes made to generators can now be found inPEP 342 rather than in the current PEP)
A new statement is proposed with the syntax:
Here, 'with' and 'as' are new keywords; EXPR is an arbitraryexpression (but not an expression-list) and VAR is a singleassignment target. It can not be a comma-separated sequence ofvariables, but it can be a parenthesized comma-separatedsequence of variables. (This restriction makes a future extensionpossible of the syntax to have multiple comma-separated resources,each with its own optional as-clause.)
The 'as VAR' part is optional.
The translation of the above statement is:
Here, the lowercase variables (mgr, exit, value, exc) are internalvariables and not accessible to the user; they will most likely beimplemented as special registers or stack positions.
The details of the above translation are intended to prescribe theexact semantics. If either of the relevant methods are not foundas expected, the interpreter will raise AttributeError, in theorder that they are tried (__exit__, __enter__).Similarly, if any of the calls raises an exception, the effect isexactly as it would be in the above code. Finally, if BLOCKcontains a break, continue or return statement, the __exit__()method is called with three None arguments just as if BLOCKcompleted normally. (I.e. these 'pseudo-exceptions' are not seenas exceptions by __exit__().)
If the 'as VAR' part of the syntax is omitted, the 'VAR =' part ofthe translation is omitted (but mgr.__enter__() is still called).
Content Generation Using Key Words Python Download
The calling convention for mgr.__exit__() is as follows. If thefinally-suite was reached through normal completion of BLOCK orthrough a non-local goto (a break, continue or return statement inBLOCK), mgr.__exit__() is called with three None arguments. Ifthe finally-suite was reached through an exception raised inBLOCK, mgr.__exit__() is called with three arguments representingthe exception type, value, and traceback.
IMPORTANT: if mgr.__exit__() returns a 'true' value, the exceptionis 'swallowed'. That is, if it returns 'true', executioncontinues at the next statement after the with-statement, even ifan exception happened inside the with-statement. However, if thewith-statement was left via a non-local goto (break, continue orreturn), this non-local return is resumed when mgr.__exit__()returns regardless of the return value. The motivation for thisdetail is to make it possible for mgr.__exit__() to swallowexceptions, without making it too easy (since the default returnvalue, None, is false and this causes the exception to bere-raised). The main use case for swallowing exceptions is tomake it possible to write the @contextmanager decorator sothat a try/except block in a decorated generator behaves exactlyas if the body of the generator were expanded in-line at the placeof the with-statement.
The motivation for passing the exception details to __exit__(), asopposed to the argument-less __exit__() from PEP 310, was given bythe transactional() use case, example 3 below. The template inthat example must commit or roll back the transaction depending onwhether an exception occurred or not. Rather than just having aboolean flag indicating whether an exception occurred, we pass thecomplete exception information, for the benefit of anexception-logging facility for example. Relying on sys.exc_info()to get at the exception information was rejected; sys.exc_info()has very complex semantics and it is perfectly possible that itreturns the exception information for an exception that was caughtages ago. It was also proposed to add an additional boolean todistinguish between reaching the end of BLOCK and a non-localgoto. This was rejected as too complex and unnecessary; anon-local goto should be considered unexceptional for the purposesof a database transaction roll-back decision.
To facilitate chaining of contexts in Python code that directlymanipulates context managers, __exit__() methods should notre-raise the error that is passed in to them. It is always theresponsibility of the caller of the __exit__() method to do anyreraising in that case.
That way, if the caller needs to tell whether the __exit__()invocation failed (as opposed to successfully cleaning up beforepropagating the original error), it can do so.
If __exit__() returns without an error, this can then beinterpreted as success of the __exit__() method itself (regardlessof whether or not the original error is to be propagated orsuppressed).
However, if __exit__() propagates an exception to its caller, thismeans that __exit__()itself has failed. Thus, __exit__()methods should avoid raising errors unless they have actuallyfailed. (And allowing the original error to proceed isn't afailure.)
In Python 2.5, the new syntax will only be recognized if a futurestatement is present:
This will make both 'with' and 'as' keywords. Without the futurestatement, using 'with' or 'as' as an identifier will cause aWarning to be issued to stderr.
In Python 2.6, the new syntax will always be recognized; 'with'and 'as' are always keywords.
With PEP 342 accepted, it is possible to write a decoratorthat makes it possible to use a generator that yields exactly onceto control a with-statement. Here's a sketch of such a decorator:
This decorator could be used as follows:
A robust implementation of this decorator will be madepart of the standard library.
It would be possible to endow certain objects, like files,sockets, and locks, with __enter__() and __exit__() methods sothat instead of writing:
one could write simply:
I think we should be careful with this; it could lead to mistakeslike:
which does not do what one might think (f is closed before BLOCK2is entered).
OTOH such mistakes are easily diagnosed; for example, thegenerator context decorator above raises RuntimeError when asecond with-statement calls f.__enter__() again. A similar errorcan be raised if __enter__ is invoked on a closed file object.
For Python 2.5, the following types have been identified ascontext managers:
A context manager will also be added to the decimal module tosupport using a local decimal arithmetic context within the bodyof a with statement, automatically restoring the original contextwhen the with statement is exited.
This PEP proposes that the protocol consisting of the __enter__()and __exit__() methods be known as the 'context management protocol',and that objects that implement that protocol be known as 'contextmanagers'. [4]
The expression immediately following the with keyword in thestatement is a 'context expression' as that expression provides themain clue as to the runtime environment the context managerestablishes for the duration of the statement body.
The code in the body of the with statement and the variable name(or names) after the as keyword don't really have special terms atthis point in time. The general terms 'statement body' and 'targetlist' can be used, prefixing with 'with' or 'with statement' if theterms would otherwise be unclear.
Given the existence of objects such as the decimal module'sarithmetic context, the term 'context' is unfortunately ambiguous.If necessary, it can be made more specific by using the terms'context manager' for the concrete object created by the contextexpression and 'runtime context' or (preferably) 'runtimeenvironment' for the actual state modifications made by the contextmanager. When simply discussing use of the with statement, theambiguity shouldn't matter too much as the context expression fullydefines the changes made to the runtime environment.The distinction is more important when discussing the mechanics ofthe with statement itself and how to go about actually implementingcontext managers.
Many context managers (such as files and generator-based contexts)will be single-use objects. Once the __exit__() method has beencalled, the context manager will no longer be in a usable state(e.g. the file has been closed, or the underlying generator hasfinished execution).
Requiring a fresh manager object for each with statement is theeasiest way to avoid problems with multi-threaded code and nestedwith statements trying to use the same context manager. It isn'tcoincidental that all of the standard library context managersthat support reuse come from the threading module - they're allalready designed to deal with the problems created by threadedand nested usage.
This means that in order to save a context manager with particularinitialisation arguments to be used in multiple with statements, itwill typically be necessary to store it in a zero-argument callablethat is then called in the context expression of each statementrather than caching the context manager directly.
When this restriction does not apply, the documentation of theaffected context manager should make that clear.
The following issues were resolved by BDFL approval (and a lackof any major objections on python-dev).
What exception should GeneratorContextManager raise when theunderlying generator-iterator misbehaves? The following quote isthe reason behind Guido's choice of RuntimeError for both thisand for the generator close() method in PEP 342 (from [8]):
'I'd rather not introduce a new exception class just for thispurpose, since it's not an exception that I want people to catch:I want it to turn into a traceback which is seen by theprogrammer who then fixes the code. So now I believe theyshould both raise RuntimeError.There are some precedents for that: it's raised by the corePython code in situations where endless recursion is detected,and for uninitialized objects (and for a variety ofmiscellaneous conditions).'
It is fine to raise AttributeError instead of TypeError if therelevant methods aren't present on a class involved in a withstatement. The fact that the abstract object C API raisesTypeError rather than AttributeError is an accident of history,rather than a deliberate design decision [11].
Objects with __enter__/__exit__ methods are called 'contextmanagers' and the decorator to convert a generator functioninto a context manager factory is contextlib.contextmanager.There were some other suggestions [16] during the 2.5 releasecycle but no compelling arguments for switching away from theterms that had been used in the PEP implementation were made.
For several months, the PEP prohibited suppression of exceptionsin order to avoid hidden flow control. Implementationrevealed this to be a right royal pain, so Guido restored theability [13].
Another aspect of the PEP that caused no end of questions andterminology debates was providing a __context__() method thatwas analogous to an iterable's __iter__() method [5][7][9].The ongoing problems [10][13] with explaining what it was and whyit was and how it was meant to work eventually lead to Guidokilling the concept outright [15] (and there was much rejoicing!).
The notion of using the PEP 342 generator API directly to definethe with statement was also briefly entertained [6], but quicklydismissed as making it too difficult to write non-generatorbased context managers.
The generator based examples rely on PEP 342. Also, some of theexamples are unnecessary in practice, as the appropriate objects,such as threading.RLock, are able to be used directly in withstatements.
The tense used in the names of the example contexts is notarbitrary. Past tense ('-ed') is used when the name refers to anaction which is done in the __enter__ method and undone in the__exit__ method. Progressive tense ('-ing') is used when the namerefers to an action which is to be done in the __exit__ method.
A template for ensuring that a lock, acquired at the start of ablock, is released when the block is left:
Used as follows:
A template for opening a file that ensures the file is closedwhen the block is left:
Used as follows:
A template for committing or rolling back a databasetransaction:
Example 1 rewritten without a generator:
(This example is easily modified to implement the otherrelatively stateless examples; it shows that it is easy to avoidthe need for a generator if no special state needs to bepreserved.)
Redirect stdout temporarily:
Used as follows:
This isn't thread-safe, of course, but neither is doing thissame dance manually. In single-threaded programs (for example,in scripts) it is a popular way of doing things.
A variant on opened() that also returns an error condition:
Used as follows:
Another useful example would be an operation that blockssignals. The use could be like this:
An optional argument might be a list of signals to be blocked;by default all signals are blocked. The implementation is leftas an exercise to the reader.
Another use for this feature is the Decimal context. Here's asimple example, after one posted by Michael Chermside:
Sample usage (adapted from the Python Library Reference):
Here's a simple context manager for the decimal module:
Sample usage:
A generic 'object-closing' context manager:
This can be used to deterministically close anything with aclose method, be it file, generator, or something else. Itcan even be used when the object isn't guaranteed to requireclosing (e.g., a function that accepts an arbitraryiterable):
(Python 2.5's contextlib module contains a versionof this context manager)
PEP 319 gives a use case for also having a released()context to temporarily release a previously acquired lock;this can be written very similarly to the locked contextmanager above by swapping the acquire() and release() calls:
Sample usage:
A 'nested' context manager that automatically nests thesupplied contexts from left-to-right to avoid excessiveindentation:
Sample usage:
Is equivalent to:
(Python 2.5's contextlib module contains a versionof this context manager)
This PEP was first accepted by Guido at his EuroPythonkeynote, 27 June 2005.It was accepted again later, with the __context__ method added.The PEP was implemented in Subversion for Python 2.5a1The __context__() method will be removed in Python 2.5a3
Many people contributed to the ideas and concepts in this PEP,including all those mentioned in the acknowledgements for PEP 340and PEP 346.
Additional thanks goes to (in no meaningful order): Paul Moore,Phillip J. Eby, Greg Ewing, Jason Orendorff, Michael Hudson,Raymond Hettinger, Walter Dörwald, Aahz, Georg Brandl, Terry Reedy,A.M. Kuchling, Brett Cannon, and all those that participated in thediscussions on python-dev.
| [1] | Raymond Chen's article on hidden flow controlhttps://devblogs.microsoft.com/oldnewthing/20050106-00/?p=36783 |
| [2] | Guido suggests some generator changes that ended up in PEP 342https://mail.python.org/pipermail/python-dev/2005-May/053885.html |
| [3] | Wiki discussion of PEP 343http://wiki.python.org/moin/WithStatement |
| [4] | Early draft of some documentation for the with statementhttps://mail.python.org/pipermail/python-dev/2005-July/054658.html |
| [5] | Proposal to add the __with__ methodhttps://mail.python.org/pipermail/python-dev/2005-October/056947.html |
| [6] | Proposal to use the PEP 342 enhanced generator API directlyhttps://mail.python.org/pipermail/python-dev/2005-October/056969.html |
| [7] | Guido lets me (Nick Coghlan) talk him into a bad idea ;)https://mail.python.org/pipermail/python-dev/2005-October/057018.html |
| [8] | Guido raises some exception handling questionshttps://mail.python.org/pipermail/python-dev/2005-June/054064.html |
| [9] | Guido answers some questions about the __context__ methodhttps://mail.python.org/pipermail/python-dev/2005-October/057520.html |
| [10] | Guido answers more questions about the __context__ methodhttps://mail.python.org/pipermail/python-dev/2005-October/057535.html |
| [11] | Guido says AttributeError is fine for missing special methodshttps://mail.python.org/pipermail/python-dev/2005-October/057625.html |
| [12] | Original PEP 342 implementation patchhttp://sourceforge.net/tracker/index.php?func=detail&aid=1223381&group_id=5470&atid=305470 |
| [13] | (1, 2) Guido restores the ability to suppress exceptionshttps://mail.python.org/pipermail/python-dev/2006-February/061909.html |
| [14] | A simple question kickstarts a thorough review of PEP 343https://mail.python.org/pipermail/python-dev/2006-April/063859.html |
| [15] | Guido kills the __context__() methodhttps://mail.python.org/pipermail/python-dev/2006-April/064632.html |
| [16] | Proposal to use 'context guard' instead of 'context manager'https://mail.python.org/pipermail/python-dev/2006-May/064676.html |
Next Keyword Python
This document has been placed in the public domain.