10 Awesome Python 3.9 Features by Farhad Malik ¶
See also
Farhad Malik ¶
My personal blog, aiming to explain complex mathematical, financial and technological concepts in simple terms.
Contact: FarhadMalik84 @ googlemail . com
Introduction ¶
The latest Python 3.9.0 final version is out on the Monday, 2020–10–05
Just like most of the Python fans, I am super excited to explore and use the latest features.
This article will provide an overview of the must-know features of Python 3.9.
This is again an exciting time for the Python programmers.
I read through the Python 3.9 release notes and the associated discussions.
Based on the information, I wanted to write a comprehensive guide so everyone can get a glimpse of the features along with their detailed workings.
Before I begin, I have to say, I am very excited to explore version 3.9 as some of the features are definitely going to be used in my applications.
In A Nutshell ¶
From the dictionary update/merge to the addition of new string methods to the introduction of zoneinfo library, a number of new features have been added.
Furthermore, a new stable and high-performant parser has been introduced.
The standard library is updated with numerous new features along with the addition of new modules, zoneinfo and graphlib.
A number of modules have been improved too, such as ast, asyncio, concurrent.futures, multiprocessing, xml amongst others.
This release has further stabilized the Python standard library.
Let’s explore Python 3.9 features now.
2. Feature New Flexible High Performant PEG-Based Parser (PEP-0617) ¶
The Python 3.9 version is proposing to replace the current LL(1) based Python parser with a new PEG-based parser which is high-performant and stable.
Detailed Explanation ¶
The current CPython parser is LL(1) based. Subsequently, the grammar is LL(1) based to allow it to be parsed by the LL(1) parser.
The LL(1) parser is a top-down parser. Furthermore, it parses the inputs from left to right. The current grammar is context-free grammar hence the context of the tokens is not taken into account .
The Python 3.9 version is proposing to replace it with a new PEG-based parser which means it will lift the current LL(1) grammar Python restrictions .
Additionally, the current parser has been patched with a number of hacks that are going to be removed. As a result, it will reduce the maintenance cost in long run .
As an instance, although the LL(1) parsers and grammars are simple to implement, the restrictions do not allow them to express common constructs in a natural way to the language designer and the reader.
The parser only looks at one token ahead to distinguish possibilities.
The choice operator | is ordered. For an instance, if the following rule is written:
rule: A|B|C
The LL(1) parser, a context-free grammar parser, will generate constructions such that given an input string will deduce whether it needs to expand A or B or C.
The PEG parser is different . It will check if the first alternative succeeds. If it fails only then it will continue with the second or the third.
The PEG parser generates exactly one valid-tree for a string. Hence it’s not ambiguous like the LL(1) parser .
The PEG parser also directly generates the AST nodes for a rule via grammar actions. This means it avoids the generation of the intermediate steps .
The key point to take is that the PEG parser has been extensively tested and validated. The PEG parser performance is fine-tuned.
As a result, for most instructions, it comes within 10% of the memory and speed consumption of the current parser. This is mainly because the intermediate syntax tree is not constructed.
I am eliminating the mention of the low-level details for the sake of keeping the article simple and readable. The link is provided at the bottom if more information is required to be understood.
6. Feature: Ability To Cancel Concurrent Futures (issue 39349, Add “cancel_futures” parameter to concurrent.futures.Executor.shutdown()) ¶
See also
A new parameter cancel_futures have been added to the concurrent.futures.Executor.shutdown().
This parameter cancels all of the pending futures that have not started.
Prior to version 3.9, the process would wait for them to complete before shutting down the executor.
Explanation ¶
The new parameter cancel_futures have been added to both ThreadPoolExecutor and ProcessPoolExecutor.
The way it works is when the value of the parameter is True then all pending futures would be canceled when the shutdown() function is called.
In a nutshell, when the shutdown() is executed, the interpreter checks if the executor is not garbage collected. If it is still in memory then it gets all of the pending worker items and then cancels the futures.
Once there are no pending work items then it shuts down the worker. Link: https://bugs.python.org/issue39349
7. Feature: AsyncIO and multiprocessing Improvements (issue 30966, Add multiprocessing.queues.SimpleQueue.close() ) ¶
See also
A number of improvements have been made to the asyncio and multiprocessing library in this release.
As an instance,
-
The reuse_address parameter of asyncio.loop.create_datagram_endpoint() is no longer supported due to significant security concerns.
-
New coroutines, shutdown_default_executor() and coroutine asyncio.to_thread() have been added. The shutdown_default_executor schedules a shutdown for the default executor that waits on the ThreadPoolExecutor to finish closing.
The asyncio.to_thread() is mainly used for running IO-bound functions in a separate thread to avoid blocking the event loop.
With regards to the multiprocessing library improvements, a new method close() has been added to the multiprocessing.SimpleQueue class. This method explicitly closes the queue. This will ensure that the queue is closed and does not stay around for longer than expected.
The key to remember is that the methods get(), put(), empty() must not be called once the queue is closed.
8. Feature: Consistent Package Import Errors (issue 37444) ¶
See also
The main issue with importing Python libraries prior to the 3.9 release was the inconsistent import behavior in Python when the relative import went past its top-level package.
The builtins.__import__() raises ValueError while importlib.__import__() raises ImportError.
It has been fixed now. The __import__() now raises ImportError instead of ValueError.
For Further Information: https://bugs.python.org/issue37444