From cd3ce5c0e21a3b84e1e04baa07d85efd3f4d273b Mon Sep 17 00:00:00 2001 From: Nils Philippsen Date: Jun 07 2017 10:07:54 +0000 Subject: [PATCH 1/5] fix too short title underline --- diff --git a/source/design/constructing.rst b/source/design/constructing.rst index 41e2593..943d034 100644 --- a/source/design/constructing.rst +++ b/source/design/constructing.rst @@ -1,5 +1,5 @@ Constructing a modular distribution -========= +=================================== The fundamental objective of the Modularity effort is to From 90c09ca9e96c64d7de468877079eff522378bdce Mon Sep 17 00:00:00 2001 From: Nils Philippsen Date: Jun 07 2017 10:10:29 +0000 Subject: [PATCH 2/5] fix or get rid of dangling links --- diff --git a/source/design/constructing.rst b/source/design/constructing.rst index 943d034..9266547 100644 --- a/source/design/constructing.rst +++ b/source/design/constructing.rst @@ -37,7 +37,7 @@ Once we have parts of the distribution on different release cycles, we also have the issue of how to maintain different versions branching on different criteria and different schedules. Branching and versioning is relevant here, but is a complex topic in its own right and is covered under -:doc:`/architecture/versioning`. +:doc:`/design/versioning`. .. rubric:: TOC diff --git a/source/development/get-involved.rst b/source/development/get-involved.rst index cb46a9c..2b4237a 100644 --- a/source/development/get-involved.rst +++ b/source/development/get-involved.rst @@ -83,11 +83,6 @@ for higher-level stuff ("epics"). Technical details ----------------- -Architecture -~~~~~~~~~~~~ - -Refer to :doc:`/architecture/infrastructure` - Code repositories ~~~~~~~~~~~~~~~~~ @@ -97,8 +92,4 @@ drafts and even a couple of proof-of-concept modules. Repositories typically start with the *fm-* prefix and are open to all members of the `Pagure @modularity group `__. -Services and tools -~~~~~~~~~~~~~~~~~~ - -Refer to :doc:`../prototype/developer-notes` From 2275398c5246ce63d432882a0ca25907eae610c3 Mon Sep 17 00:00:00 2001 From: Nils Philippsen Date: Jun 07 2017 10:11:07 +0000 Subject: [PATCH 3/5] resurrect coding style and grooming docs --- diff --git a/source/development/get-involved.rst b/source/development/get-involved.rst index 2b4237a..b934721 100644 --- a/source/development/get-involved.rst +++ b/source/development/get-involved.rst @@ -92,4 +92,13 @@ drafts and even a couple of proof-of-concept modules. Repositories typically start with the *fm-* prefix and are open to all members of the `Pagure @modularity group `__. +Integration of changes +~~~~~~~~~~~~~~~~~~~~~~ +In order so that our code base is always clean and maintainable, we have to enforce certain rules on how code is written or formatted, how changes are broken up into commits and how pull requests are handled. + +.. toctree:: + :maxdepth: 1 + + integration/coding-style + integration/grooming diff --git a/source/development/integration/coding-style.rst b/source/development/integration/coding-style.rst new file mode 100644 index 0000000..8915dc9 --- /dev/null +++ b/source/development/integration/coding-style.rst @@ -0,0 +1,503 @@ +Coding Style +============ + +Most of our code is written in Python, so this document will concentrate +on it. + +Upstream guidelines +------------------- + +Fortunately, with PEP 8 there's an extensive official `Style Guide for +Python Code `__. All new +Python code you submit should conform to it, unless you have good +reasons to deviate from it, `for instance +readability `__. + +Keep PEP 20, the `Zen of +Python `__, under your +pillow. + +Keep It Simple +-------------- + +The code you write now probably needs to be touched by someone else down +the road, and that someone else might be less experienced than you, or +have a terrible headache and be under pressure of time. So while a +particular construct may be a clever way of doing something, a simple +way of doing the same thing can be and often is preferrable. If (when) +complexity can't be avoided, try to isolate it: put a difficult +operation into its own function, method or class, add comments. If +complexity can be hidden from upper layers of the code, do so. + +Comments and Docstrings +----------------------- + +Be generous when it comes to commenting your code, it's better to have a +superfluous comment than if one were necessary but is missing. However, +if there is a comment it should be correct and agree with the code, +otherwise people have to guess if the comment or the code needs to be +straightened out. + +Adding `docstrings `__ to +modules, classes, methods and functions is encouraged. If you use the +`Sphinx +format `__ +to describe parameters, return values, etc., even better! + +Python 2 and 3 +-------------- + +Python comes in two major versions nowadays: + +- The legacy version 2, of which the `first release 2.0 came out in + October 2000 `__. The + Python project `will maintain its final minor release 2.7 until + 2020 `__. + +- The current version 3, its `first release 3.0 was published in + December 2008 `__. At + the time of writing, the current minor release is version 3.5, to be + superseded by 3.6 around the end of 2016. + +Version 3 is not backwards compatible to version 2. While we mainly +target "the future", there are some components we have to work with that +haven't yet been ported over the Python 3, most notably +`koji `__. Additionally, we may also +want to support the "user tools" we create on legacy systems, so we +can't write code that uses all the latest features. Fortunately, many of +the original Python 3 features have been back-ported to Python 2.7, so +we can and should write code that is very close to writing idiomatic +Python 3 but can still be run on version 2.7. Targeting older minor +releases (Python 2.6 and earlier) is much more of a balancing act, so we +won't aim for it. + +The following sections cover areas that require some attention. The +Python project itself has a great `Porting Python 2 Code to Python +3 `__ document which +goes into much detail about the differences and is worth a read, even +though it mainly addresses existing Python 2 code bases. + +Absolute and relative imports +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In Python 2, importing modules can be ambiguous when a module of that +name exists in the same package and elsewhere in the module search path +``sys.path``. To work around this ambiguity, programmers often resorted +to adding paths private to the project to the beginning of ``sys.path`` +to force loading modules from a project-internal location (which adds +unwanted noise and can make e.g. testing code that isn't installed +difficult). Python 3 introduces new syntax for import statements which +makes both cases distinct, this is available since version 2.5 from the +``__future__`` module: + +:: + + from __future__ import absolute_import + + # Import the sys module from the module search path + import sys + + # Import the foo module from the same directory + from . import foo + + # Import snafu from the bar module one directory above + from ..bar import snafu + +Print function +~~~~~~~~~~~~~~ + +Python 3 did away with ``print`` as a statement and introduced it as a +function. In order to use it the same way in Python 2.7, add the +following to the top of source code files where you use ``print``: + +:: + + from __future__ import print_function + +Numbers +~~~~~~~ + +Python 2 has two integer types, \`int\` which is whatever integer-type +is native to the system (which has certain maximal and minimal values +and can overflow) and \`long\` which can store arbitrary integer +numbers. Python 3 only the latter type, but it's called ``int``. + +Dividing integer numbers using ``/`` truncates the result to an integer +in Python 2 by default, but yields a floating point number in Python 3. +In order for code to do the same thing on either version, include the +following line at the top of your source files where you divide numbers, +and use ``/`` for normal divisions and ``//`` for divisions that should +truncate the result: + +:: + + from __future__ import division + +Strings +~~~~~~~ + +Some consider this the main difference between Python 2 and 3: Both +versions have a type for strings of bytes and strings of Unicode +character points. They are called ``str`` and ``unicode`` in version 2 +and ``bytes`` and ``str`` in version 3, respectively. + +String Literals +^^^^^^^^^^^^^^^ + +Python 2 and 3 use different ways of marking literals of the different +types by default. Byte strings can have no prefix or ``b`` in Python +2.7, but must be prefixed in Python 3, and text strings must have the +``u`` prefix in Python 2 which can be and usually is omitted in Python +3: + +:: + + # a byte string in Python 2 and 3 + string1 = b"abc" + + # a byte string in Python 2, but a text string in Python 3 + string2 = "def" + + # a text string in Python 2 and 3 + string3 = u"ghi" + +In order to ease writing code that is compatible between the versions, +you can switch Python 2 to treat unprefixed string literals as +``unicode``, the text string type, by adding this snippet to the top of +the relevant source code files: + +:: + + from __future__ import unicode_literals + +Explicit Encoding and Decoding +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In Python 2, the byte and text string types are exchangeable in many +places, taking the user's or system default locale into account (and +sometimes failing, when the locale didn't match up with encoded data). +Apart from the change in type names and how literals look like, Python 3 +requires you to explicitly encode ``str`` and decode ``bytes`` objects +if you need them cast into the respective other string type. It is good +practice to exclusively use text strings for strings that represent text +in a program and decode byte strings as early and encode text strings as +late as possible at interfaces that produce or consume encoded data. + +.. raw:: mediawiki + + {{admon/note|Implicit string type conversion in Python 2|Python 2 lets you attempt to replace a str substring in a unicode object (or vice versa) and would attempt to cast the one into the other by encoding or decoding on the fly as needed. This piece of code won't work in Python 3:}} + +:: + + from __future__ import print_function + text_string = u"Hello, world!" + print(text_string.replace("world", "gang")) + +.. raw:: mediawiki + + {{admon/tip|Explicit string type conversion in Python 2 and 3|Python 3 requires explicit encoding/decoding to cast between byte and text strings. This also works in Python 2 and is preferred of course.}} + +:: + + from __future__ import print_function, unicode_literals + text_string = "Hello, world!" + print(text_string.replace(b"world".decode('utf-8'), b"gang".decode('ascii'))) + +String formatting +^^^^^^^^^^^^^^^^^ + +With version 3.6 around the corner, there are four ways to format +strings in Python now: + +#. using the ``%`` operator +#. using ``string.Template`` of `PEP + 292 `__ +#. with the ``str.format()`` method +#. using `PEP 498 literal string + interpolation `__ + +The last method isn't available yet in a stable Python release and will +never be in Python 2, so it's not suitable for our purposes. The other +three variants work in all Python versions we're interested in, +formatting with ``string.Template`` is very rarely done however. The +remaining two ways, commonly called old-style (``%`` operator) and +new-style (``str.format()``), are both in wide-spread use, `here's a +site showcasing the differences between +them `__. New-style formatting is more powerful +and often easier to read, but on the other hand can be a little more to +type. From a technical point of view, this is a case of "use what works +for you", but for consistency sake the new-style ``str.format()`` way is +preferrable if you're comfortable with using it. If not, others can +convert old-style to new-style formatting for you during review or when +happening across it. At any rate, consistently use one way or the other +in what you submit. + +Old- and New-style Classes +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Python 2 and earlier knows two types of classes, old-style which have no +base class, and new-style which have ``object`` as the base class. +Because their behavior is slightly different in some places, and some +things can't be done with old-style classes, we want to stick to +new-style classes wherever possible. + +The syntactical difference is that new-style classes have to explicitly +be derived from ``object`` or another new-style class. + +:: + + # old-style classes + class OldFoo: + pass + + class OldBar(OldFoo): + pass + + # new-style classes + class NewFoo(object): + pass + + class NewBar(NewFoo): + pass + +Python 3 only knows new-style classes and the requirement to explicitly +derive from ``object`` was dropped. In projects that will only ever run +on Python 3, it's acceptable not to explicitly derive classes without +parents from ``object``, but if in doubt, do it just the same. + +Idiomatic code +-------------- + +In Python, it's easy to inadvertently emulate idiomatic styles of other +languages like C/C++ or Java. In cases where there are constructs +"native" to the language, it's preferrable to use them. + +Literals and Comprehensions +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Python has special syntax for literals for a couple of built-in compound +data types: lists, tuples, dictionaries, strings, sets. It's customary +to use that syntax instead of the class constructor to create objects +for these data types unless you have good reason not to. Apart from how +it looks, the literal syntax is performing a little bit better (because +it doesn't have to look up the class name in the current scope). NB: Set +literals are peculiar in that you can't create empty ones­—they would +look the same as empty dicts. + ++-------------+-------------------------------------+----------------------------------------+ +| Data Type | Good | Bad | ++=============+=====================================+========================================+ +| ``str`` | | ``a_str = "abc"`` | ``empty_str = str()`` | +| | | ``empty_str = ""`` | | ++-------------+-------------------------------------+----------------------------------------+ +| ``list`` | | ``a_list = [1, 2]`` | | ``a_list = list((1, 2))`` | +| | | ``empty_str = []`` | | ``empty_list = list()`` | ++-------------+-------------------------------------+----------------------------------------+ +| ``tuple`` | | ``a_tuple = ('a', 'b', 3)`` | | ``a_tuple = tuple(['a', 'b', 3])`` | +| | | ``empty_tuple = ()`` | | ``empty_tuple = tuple()`` | ++-------------+-------------------------------------+----------------------------------------+ +| ``dict`` | | ``a_dict = {'a': 1}`` | | ``a_dict = dict(('a', 1))`` | +| | | ``empty_dict = {}`` | | ``empty_dict = dict()`` | ++-------------+-------------------------------------+----------------------------------------+ +| ``set`` | | ``a_set = {"banana", "apple"}`` | ``a_set = set(["banana", "apple"])`` | +| | | **``empty_set = set()``** | | ++-------------+-------------------------------------+----------------------------------------+ + +Table: Creating compound objects + +Often the initial contents of a compound object are only known when it's +created at runtime. For simple cases like mere type conversions, calling +the class constructors are the way to go: + +- Converting a tuple to a list or vice versa: + +| ``   a_tuple = (1, 2, 3)`` +| ``   ...`` +| ``   a_list = list(a_tuple)`` +| ``   ...`` +| ``   another_list = [4, 5, 6]`` +| ``   ...`` +| ``   another_tuple = tuple(another_list)`` + +- Convert a list to a set, e.g. to filter out duplicates: + +| ``   a_list = [1, 2, 3, 2]`` +| ``   ...`` +| ``   a_set = set(a_list)`` + +For more involved cases, say some values need to be filtered or a +specific attribute of the objects is wanted, Python has so-called +comprehensions to create compound objects in a syntactically "nice" way. +These largely supersede the old (ugly) way of using ``map()`` and +``filter()`` in conjunction with class constructors. + ++-------------------------------------------------------------------------------------------------------+-------------+------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Comprehension type | Data Type | Example | Remarks | ++=======================================================================================================+=============+================================================+=============================================================================================================================================================================+ +| `List Comprehension `__ | ``list`` | ``a_list = [x for x in range(20) if x % 2]`` | Put all odd numbers smaller than 20 into a list. | ++-------------------------------------------------------------------------------------------------------+-------------+------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| `Dict Comprehension `__ | ``dict`` | | ``a_dict = {k: getattr(an_obj, k)`` | Fill a dict with those attribute names and values of an object that aren't considered "protected" or "private" (names with one or two leading underscores, respectively). | +| | | | ``    for k in dir(an_obj)`` | | +| | | | ``    if not k.startswith("_")}`` | | ++-------------------------------------------------------------------------------------------------------+-------------+------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| `Set Comprehension `__ | ``set`` | ``a_set = {o.name for o in a_list}`` | Create a set containing the value of the attribute ``name`` of objects in a list. | ++-------------------------------------------------------------------------------------------------------+-------------+------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +Table: Using comprehensions to create compound objects + +Looping +~~~~~~~ + +Languages like C normally use incremented indices to loop over arrays: + +:: + + float pixels[NUMBER_OF_PIXELS] = [...]; + + for (int i = 0; i < NUMBER_OF_PIXELS; i++) + { + do_something_with_a_pixel(pixels[i]); + } + +.. raw:: mediawiki + + {{admon/warning|Looping C-style in Python|Avoid looping over indices of sequences, rather than the sequences themselves in Python.}} + +Implementing the loop like this would give away that you've programmed +in C or a similar language before: + +:: + + pixels = [...] + + for i in range(len(pixels)): + do_something_with_a_pixel(pixels[i]) + +.. raw:: mediawiki + + {{admon/note|Looping over iterables in Python|In Python, you can simply iterate over many non-scalar data types.}} + +Here's the "native" way to implement the above loop: + +:: + + pixels = [...] + + for p in pixels: + do_something_with_a_pixel(p) + +.. raw:: mediawiki + + {{admon/tip|Using enumerate()|If you need to keep track of the current count of looped-over items, use the enumerate() built-in.}} + +It yields pairs of count (starting at 0 by default) and the current +value like this: + +:: + + pixels = [...] + + for p_no, p in enumerate(pixels, 1): + print("Working on pixel no. {}".format(p_no)) + do_something_with_a_pixel(p) + +Properties rather than explicit accessor methods +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In order to allow future changes in how object attributes (member +variables) are set, some languages encourage always using getter and/or +setter methods. This is unnecessary in Python, as you can intercept +access to an attribute by wrapping it into a +`property `__ +if and when this becomes necessary. Properties allow having accessor +methods without making the user of the class have to use them +explicitly. This way you can validate values when an attribute is set, +or translate back and forth between the interface used on the attribute +and an internal representation. + +Validating a value when setting an attribute +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To ensure that an ``Employee`` object only has positive values for its +``salary`` attribute, you'd put a property in its place which checks +values before storing them in an attribute called e.g. ``_salary``: + +:: + + class Employee(object): + + @property + def salary(self): + return self._salary + + @salary.setter + def salary(self, salary): + if salary <= 0: + raise ValueError("Salary must be positive.") + self._salary = salary + +.. raw:: mediawiki + + {{admon/caution|Avoid recursion|In order to avoid endless recursion, you must use a different attribute than the one using the property to store actual values.}} + +Translating between attribute interface and internal representation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Take these classes of geometric primitives, ``Point`` and ``Circle``: + +:: + + class Point(object): + def __init__(self, x, y): + self.x = x + self.y = y + + class Circle(object): + def __init__(self, point, radius): + self.point = point + self.radius = radius + +If you wanted to add a ``diameter`` attribute to ``Circle``, you can do +so as a property which translates back and forth between it and the +existing ``radius`` attribute: + +:: + + ... + class Circle(object): + def __init__(self, point, radius=None, diameter=None): + self.point = point + if (radius is None) == (diameter is None): + raise ValueError("Exactly one of radius or diameter must be set") + if radius is not None: + self.radius = radius + else: + self.diameter = diameter + + @property + def diameter(self): + return self.radius * 2 + + @diameter.setter + def diameter(self, diameter): + self.radius = diameter / 2.0 + ... + +Even setting ``self.diameter`` in the constructor goes by way of the +property and therefore the setter method. + +External links +-------------- + +- `Python Design and History + FAQ `__ +- `PEP8: Style Guide for Python + Code `__ +- `PEP20: The Zen of + Python `__ +- `PyFormat: Using ``%`` and ``.format()`` for great + good! `__ +- `Sphinx Info field + lists `__ + for docstrings + diff --git a/source/development/integration/grooming.rst b/source/development/integration/grooming.rst new file mode 100644 index 0000000..3f7994e --- /dev/null +++ b/source/development/integration/grooming.rst @@ -0,0 +1,174 @@ +Grooming Your Changes +===================== + +Apart from :doc:`coding-style`, +there are some things that you should keep in mind regarding the changes +you submit. Normally you'd develop your changes in a private branch on +your fork of a repository and, when you're done, submit them as pull +requests ("PR") against a public branch of the repository. The following +guidelines concentrate on changes in this format, their goal is to +enable you to groom the commits forming your pull request so that +another person can review it without great effort, that the changes can +be integrated well with the existing code and can be easily debugged +later if necessary. + +Pull Requests +------------- + +Scope +~~~~~ + +One pull request should really be about implementing one feature or +solving one problem. For instance, when developing your changes you +might spot a bug in existing code and fix it. Mixing these changes with +your new feature make reviewing them more work because the person doing +it needs to assess if a chunk of your changes is related to the feature, +or the bug fix. Similarly, if the review of your feature drags out, the +bug fix might take that much longer before it's available to others. In +most cases you should therefore create separate pull requests for both +sets of changes. + +As an exception to that, merely janitorial changes to the parts of the +code your pull request touches anyway—say, fixing trailing whitespace or +indentation, superficial changes that make the code you worked on better +to read or understand—are acceptable as long as you put these changes in +a commit or commits of their own, ideally put before your "real" changes +in the commit order. This makes it easier to cope with other PRs that +might fix the same things. + +Describing your changes +~~~~~~~~~~~~~~~~~~~~~~~ + +The bigger the changes you submit are, the more important it is to give +the reviewer a high level summary of what it is they are reviewing. If a +pull request consists only of one commit, then its commit log should be +sufficient in most cases and the forges hosting our repositories (Pagure +and GitHub) use it as the default description text on submission. If it +is longer, you may need to condense the individual changes of your +commits, and maybe lose some comments about the problem you wanted to +solve and your approach. If you are unsure about parts of your changes, +this is also the place give the reviewer a heads-up. + +Linear History and Rebasing +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The changes you submit for reviewing should be a linear string of +commits, please don't have merges in there. Therefore, in order to track +upstream changes while you are still developing in a private branch, you +should rebase it on top of the upstream branch you track. You can of +course do that manually, but it's easier to tell git to automatically +attempt to rebase your changes on top of the branch from which you pull +(replace ``$branchname`` with the actual name of your local branch): + +``   git config branch.$branchname.rebase true`` + +You can also set this globally for any newly created branch (you'd have +to do the above for all existing branches, though): + +``   git config --global branch.autosetuprebase always`` + +Set up this way, pulling from upstream will attempt to apply your +private commits in order on top of the new upstream ones, one after +another. If that fails at some point, e.g. because of conflicts, it'll +interrupt the rebasing process, so that you can resolve the issue, and +continue with ``git rebase --cont``. Alternatively, you could also +restore the previous state by running ``git rebase --abort``, e.g. to +assess the differences between your (unrebased) branch and upstream +before giving it a go again. + +The Review +~~~~~~~~~~ + +When you've submitted your changes as a pull request, hopefully someone +will pick it up soon (if not, poke some people on IRC: +`#fedora-modularity on +Freenode `__) and give you +feedback in form of comments, questions or suggestions. The comment +section of a pull request isn't very suitable for longer discussions, so +you might switch to email, IRC or another medium to discuss a topic, and +then summarize in the PR. Consulting other contributors is encouraged, +if additional opinions are needed. The job of a reviewer is not just to +act as a gatekeeper for the project, but also to assist you in getting +your changes into an acceptable state. This can go as far as making +minor fixes on the fly rather than asking you to do it, or bringing the +stack of commits "into shape" before merging the pull request. + +Individual commits +------------------ + +Commit Scope and Size +~~~~~~~~~~~~~~~~~~~~~ + +Like a pull request itself, a commit should also be about just one +thing. For example, you should split the implementation of a new class +from where existing code is converted to use it, as well as removing the +legacy code it replaces. The reverse also holds true—one concern should +be dealt with in one commit: if you discover bugs in a newly introduced +piece of code while you're still developing it, the buggy commit +introducing it and the fix should be rolled into one. This keeps the +number of broken commits down which e.g. makes it easier to use +``git bisect`` at a later point. + +.. raw:: mediawiki + + {{admon/note|"Commit early, commit often."|It's much easier to merge smaller commits into larger ones if they belong together, rather than disassembling a commit that actually addresses more than one concern.}} + +Commit Log Messages +~~~~~~~~~~~~~~~~~~~ + +The purpose of a commit log message is to briefly summarize the changes +in the commit, but it's also where background information should be put, +e.g. why some approach was used and not another. + +Format +^^^^^^ + +A commit log should consist of a short summary line (<50 characters, +also called "title"), optionally followed by a blank line and a more +thorough description. The summary should tersely describe the objective +of the commit, while the description would go into detail about the +actual implementation. + +Building a Commit +~~~~~~~~~~~~~~~~~ + +Often you'll want to pick only parts of your uncommitted changes, in +order to follow these guidelines, or to leave out debugging statements +which you don't want to submit. You can select the parts in your changes +you want to commit by using ``git add --patch`` which presents the +differences as hunks in unified diff format and lets you choose which +ones to add to the staging area and which to skip. After committing +these staged changes, you can repeat the process until all changes you +want to submit are taken care of. There are ways to separate a large +commit into smaller ones, but this approach is often more difficult one +of the two. + +Tools +----- + +- Adding using patch mode: With ``git add --patch ...`` you can pick + which changes you want to commit. +- Interactive rebasing: Use ``git rebase -i ... @{u}`` to reorder your + commits, reword their commit messages, merge or amend them. It's + important to not do this to upstream commits, therefore ``@{u}`` + specifies the point where your branch split off from upstream. + +.. raw:: mediawiki + + {{admon/important|If all else fails:|GIT remembers the history of revisions you had checked out in your repository, refer to the output of git reflog to find a "known good" one.}} + +.. raw:: mediawiki + + {{admon/caution|Using git reset|You can use git reset [--hard] $some_sha1_commit to bring you back to a known good state. Be careful, though: using the --hard option will lose any changes made to files under the control of GIT.}} + +See also +-------- + +- The `Pro Git book `__ + + - The `"Rewriting + History" `__ + chapter for more detailed information about amending, interactive + rebasing, and other advanced ways of screwing up your repository + ;) + From bffaef819371a0d4fe60e1e531d03adca5821260 Mon Sep 17 00:00:00 2001 From: Nils Philippsen Date: Jun 07 2017 12:19:10 +0000 Subject: [PATCH 4/5] better reflect how much Python code we produce --- diff --git a/source/development/get-involved.rst b/source/development/get-involved.rst index b934721..2103084 100644 --- a/source/development/get-involved.rst +++ b/source/development/get-involved.rst @@ -100,5 +100,5 @@ In order so that our code base is always clean and maintainable, we have to enfo .. toctree:: :maxdepth: 1 - integration/coding-style + integration/coding-style-python integration/grooming diff --git a/source/development/integration/coding-style-python.rst b/source/development/integration/coding-style-python.rst new file mode 100644 index 0000000..f047f92 --- /dev/null +++ b/source/development/integration/coding-style-python.rst @@ -0,0 +1,502 @@ +Coding Style (Python) +===================== + +Read this if you plan to contribute code written in the Python language. + +Upstream guidelines +------------------- + +Fortunately, with PEP 8 there's an extensive official `Style Guide for +Python Code `__. All new +Python code you submit should conform to it, unless you have good +reasons to deviate from it, `for instance +readability `__. + +Keep PEP 20, the `Zen of +Python `__, under your +pillow. + +Keep It Simple +-------------- + +The code you write now probably needs to be touched by someone else down +the road, and that someone else might be less experienced than you, or +have a terrible headache and be under pressure of time. So while a +particular construct may be a clever way of doing something, a simple +way of doing the same thing can be and often is preferrable. If (when) +complexity can't be avoided, try to isolate it: put a difficult +operation into its own function, method or class, add comments. If +complexity can be hidden from upper layers of the code, do so. + +Comments and Docstrings +----------------------- + +Be generous when it comes to commenting your code, it's better to have a +superfluous comment than if one were necessary but is missing. However, +if there is a comment it should be correct and agree with the code, +otherwise people have to guess if the comment or the code needs to be +straightened out. + +Adding `docstrings `__ to +modules, classes, methods and functions is encouraged. If you use the +`Sphinx +format `__ +to describe parameters, return values, etc., even better! + +Python 2 and 3 +-------------- + +Python comes in two major versions nowadays: + +- The legacy version 2, of which the `first release 2.0 came out in + October 2000 `__. The + Python project `will maintain its final minor release 2.7 until + 2020 `__. + +- The current version 3, its `first release 3.0 was published in + December 2008 `__. At + the time of writing, the current minor release is version 3.5, to be + superseded by 3.6 around the end of 2016. + +Version 3 is not backwards compatible to version 2. While we mainly +target "the future", there are some components we have to work with that +haven't yet been ported over the Python 3, most notably +`koji `__. Additionally, we may also +want to support the "user tools" we create on legacy systems, so we +can't write code that uses all the latest features. Fortunately, many of +the original Python 3 features have been back-ported to Python 2.7, so +we can and should write code that is very close to writing idiomatic +Python 3 but can still be run on version 2.7. Targeting older minor +releases (Python 2.6 and earlier) is much more of a balancing act, so we +won't aim for it. + +The following sections cover areas that require some attention. The +Python project itself has a great `Porting Python 2 Code to Python +3 `__ document which +goes into much detail about the differences and is worth a read, even +though it mainly addresses existing Python 2 code bases. + +Absolute and relative imports +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In Python 2, importing modules can be ambiguous when a module of that +name exists in the same package and elsewhere in the module search path +``sys.path``. To work around this ambiguity, programmers often resorted +to adding paths private to the project to the beginning of ``sys.path`` +to force loading modules from a project-internal location (which adds +unwanted noise and can make e.g. testing code that isn't installed +difficult). Python 3 introduces new syntax for import statements which +makes both cases distinct, this is available since version 2.5 from the +``__future__`` module: + +:: + + from __future__ import absolute_import + + # Import the sys module from the module search path + import sys + + # Import the foo module from the same directory + from . import foo + + # Import snafu from the bar module one directory above + from ..bar import snafu + +Print function +~~~~~~~~~~~~~~ + +Python 3 did away with ``print`` as a statement and introduced it as a +function. In order to use it the same way in Python 2.7, add the +following to the top of source code files where you use ``print``: + +:: + + from __future__ import print_function + +Numbers +~~~~~~~ + +Python 2 has two integer types, \`int\` which is whatever integer-type +is native to the system (which has certain maximal and minimal values +and can overflow) and \`long\` which can store arbitrary integer +numbers. Python 3 only the latter type, but it's called ``int``. + +Dividing integer numbers using ``/`` truncates the result to an integer +in Python 2 by default, but yields a floating point number in Python 3. +In order for code to do the same thing on either version, include the +following line at the top of your source files where you divide numbers, +and use ``/`` for normal divisions and ``//`` for divisions that should +truncate the result: + +:: + + from __future__ import division + +Strings +~~~~~~~ + +Some consider this the main difference between Python 2 and 3: Both +versions have a type for strings of bytes and strings of Unicode +character points. They are called ``str`` and ``unicode`` in version 2 +and ``bytes`` and ``str`` in version 3, respectively. + +String Literals +^^^^^^^^^^^^^^^ + +Python 2 and 3 use different ways of marking literals of the different +types by default. Byte strings can have no prefix or ``b`` in Python +2.7, but must be prefixed in Python 3, and text strings must have the +``u`` prefix in Python 2 which can be and usually is omitted in Python +3: + +:: + + # a byte string in Python 2 and 3 + string1 = b"abc" + + # a byte string in Python 2, but a text string in Python 3 + string2 = "def" + + # a text string in Python 2 and 3 + string3 = u"ghi" + +In order to ease writing code that is compatible between the versions, +you can switch Python 2 to treat unprefixed string literals as +``unicode``, the text string type, by adding this snippet to the top of +the relevant source code files: + +:: + + from __future__ import unicode_literals + +Explicit Encoding and Decoding +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In Python 2, the byte and text string types are exchangeable in many +places, taking the user's or system default locale into account (and +sometimes failing, when the locale didn't match up with encoded data). +Apart from the change in type names and how literals look like, Python 3 +requires you to explicitly encode ``str`` and decode ``bytes`` objects +if you need them cast into the respective other string type. It is good +practice to exclusively use text strings for strings that represent text +in a program and decode byte strings as early and encode text strings as +late as possible at interfaces that produce or consume encoded data. + +.. raw:: mediawiki + + {{admon/note|Implicit string type conversion in Python 2|Python 2 lets you attempt to replace a str substring in a unicode object (or vice versa) and would attempt to cast the one into the other by encoding or decoding on the fly as needed. This piece of code won't work in Python 3:}} + +:: + + from __future__ import print_function + text_string = u"Hello, world!" + print(text_string.replace("world", "gang")) + +.. raw:: mediawiki + + {{admon/tip|Explicit string type conversion in Python 2 and 3|Python 3 requires explicit encoding/decoding to cast between byte and text strings. This also works in Python 2 and is preferred of course.}} + +:: + + from __future__ import print_function, unicode_literals + text_string = "Hello, world!" + print(text_string.replace(b"world".decode('utf-8'), b"gang".decode('ascii'))) + +String formatting +^^^^^^^^^^^^^^^^^ + +With version 3.6 around the corner, there are four ways to format +strings in Python now: + +#. using the ``%`` operator +#. using ``string.Template`` of `PEP + 292 `__ +#. with the ``str.format()`` method +#. using `PEP 498 literal string + interpolation `__ + +The last method isn't available yet in a stable Python release and will +never be in Python 2, so it's not suitable for our purposes. The other +three variants work in all Python versions we're interested in, +formatting with ``string.Template`` is very rarely done however. The +remaining two ways, commonly called old-style (``%`` operator) and +new-style (``str.format()``), are both in wide-spread use, `here's a +site showcasing the differences between +them `__. New-style formatting is more powerful +and often easier to read, but on the other hand can be a little more to +type. From a technical point of view, this is a case of "use what works +for you", but for consistency sake the new-style ``str.format()`` way is +preferrable if you're comfortable with using it. If not, others can +convert old-style to new-style formatting for you during review or when +happening across it. At any rate, consistently use one way or the other +in what you submit. + +Old- and New-style Classes +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Python 2 and earlier knows two types of classes, old-style which have no +base class, and new-style which have ``object`` as the base class. +Because their behavior is slightly different in some places, and some +things can't be done with old-style classes, we want to stick to +new-style classes wherever possible. + +The syntactical difference is that new-style classes have to explicitly +be derived from ``object`` or another new-style class. + +:: + + # old-style classes + class OldFoo: + pass + + class OldBar(OldFoo): + pass + + # new-style classes + class NewFoo(object): + pass + + class NewBar(NewFoo): + pass + +Python 3 only knows new-style classes and the requirement to explicitly +derive from ``object`` was dropped. In projects that will only ever run +on Python 3, it's acceptable not to explicitly derive classes without +parents from ``object``, but if in doubt, do it just the same. + +Idiomatic code +-------------- + +In Python, it's easy to inadvertently emulate idiomatic styles of other +languages like C/C++ or Java. In cases where there are constructs +"native" to the language, it's preferrable to use them. + +Literals and Comprehensions +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Python has special syntax for literals for a couple of built-in compound +data types: lists, tuples, dictionaries, strings, sets. It's customary +to use that syntax instead of the class constructor to create objects +for these data types unless you have good reason not to. Apart from how +it looks, the literal syntax is performing a little bit better (because +it doesn't have to look up the class name in the current scope). NB: Set +literals are peculiar in that you can't create empty ones­—they would +look the same as empty dicts. + ++-------------+-------------------------------------+----------------------------------------+ +| Data Type | Good | Bad | ++=============+=====================================+========================================+ +| ``str`` | | ``a_str = "abc"`` | ``empty_str = str()`` | +| | | ``empty_str = ""`` | | ++-------------+-------------------------------------+----------------------------------------+ +| ``list`` | | ``a_list = [1, 2]`` | | ``a_list = list((1, 2))`` | +| | | ``empty_str = []`` | | ``empty_list = list()`` | ++-------------+-------------------------------------+----------------------------------------+ +| ``tuple`` | | ``a_tuple = ('a', 'b', 3)`` | | ``a_tuple = tuple(['a', 'b', 3])`` | +| | | ``empty_tuple = ()`` | | ``empty_tuple = tuple()`` | ++-------------+-------------------------------------+----------------------------------------+ +| ``dict`` | | ``a_dict = {'a': 1}`` | | ``a_dict = dict(('a', 1))`` | +| | | ``empty_dict = {}`` | | ``empty_dict = dict()`` | ++-------------+-------------------------------------+----------------------------------------+ +| ``set`` | | ``a_set = {"banana", "apple"}`` | ``a_set = set(["banana", "apple"])`` | +| | | **``empty_set = set()``** | | ++-------------+-------------------------------------+----------------------------------------+ + +Table: Creating compound objects + +Often the initial contents of a compound object are only known when it's +created at runtime. For simple cases like mere type conversions, calling +the class constructors are the way to go: + +- Converting a tuple to a list or vice versa: + +| ``   a_tuple = (1, 2, 3)`` +| ``   ...`` +| ``   a_list = list(a_tuple)`` +| ``   ...`` +| ``   another_list = [4, 5, 6]`` +| ``   ...`` +| ``   another_tuple = tuple(another_list)`` + +- Convert a list to a set, e.g. to filter out duplicates: + +| ``   a_list = [1, 2, 3, 2]`` +| ``   ...`` +| ``   a_set = set(a_list)`` + +For more involved cases, say some values need to be filtered or a +specific attribute of the objects is wanted, Python has so-called +comprehensions to create compound objects in a syntactically "nice" way. +These largely supersede the old (ugly) way of using ``map()`` and +``filter()`` in conjunction with class constructors. + ++-------------------------------------------------------------------------------------------------------+-------------+------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Comprehension type | Data Type | Example | Remarks | ++=======================================================================================================+=============+================================================+=============================================================================================================================================================================+ +| `List Comprehension `__ | ``list`` | ``a_list = [x for x in range(20) if x % 2]`` | Put all odd numbers smaller than 20 into a list. | ++-------------------------------------------------------------------------------------------------------+-------------+------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| `Dict Comprehension `__ | ``dict`` | | ``a_dict = {k: getattr(an_obj, k)`` | Fill a dict with those attribute names and values of an object that aren't considered "protected" or "private" (names with one or two leading underscores, respectively). | +| | | | ``    for k in dir(an_obj)`` | | +| | | | ``    if not k.startswith("_")}`` | | ++-------------------------------------------------------------------------------------------------------+-------------+------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| `Set Comprehension `__ | ``set`` | ``a_set = {o.name for o in a_list}`` | Create a set containing the value of the attribute ``name`` of objects in a list. | ++-------------------------------------------------------------------------------------------------------+-------------+------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +Table: Using comprehensions to create compound objects + +Looping +~~~~~~~ + +Languages like C normally use incremented indices to loop over arrays: + +:: + + float pixels[NUMBER_OF_PIXELS] = [...]; + + for (int i = 0; i < NUMBER_OF_PIXELS; i++) + { + do_something_with_a_pixel(pixels[i]); + } + +.. raw:: mediawiki + + {{admon/warning|Looping C-style in Python|Avoid looping over indices of sequences, rather than the sequences themselves in Python.}} + +Implementing the loop like this would give away that you've programmed +in C or a similar language before: + +:: + + pixels = [...] + + for i in range(len(pixels)): + do_something_with_a_pixel(pixels[i]) + +.. raw:: mediawiki + + {{admon/note|Looping over iterables in Python|In Python, you can simply iterate over many non-scalar data types.}} + +Here's the "native" way to implement the above loop: + +:: + + pixels = [...] + + for p in pixels: + do_something_with_a_pixel(p) + +.. raw:: mediawiki + + {{admon/tip|Using enumerate()|If you need to keep track of the current count of looped-over items, use the enumerate() built-in.}} + +It yields pairs of count (starting at 0 by default) and the current +value like this: + +:: + + pixels = [...] + + for p_no, p in enumerate(pixels, 1): + print("Working on pixel no. {}".format(p_no)) + do_something_with_a_pixel(p) + +Properties rather than explicit accessor methods +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In order to allow future changes in how object attributes (member +variables) are set, some languages encourage always using getter and/or +setter methods. This is unnecessary in Python, as you can intercept +access to an attribute by wrapping it into a +`property `__ +if and when this becomes necessary. Properties allow having accessor +methods without making the user of the class have to use them +explicitly. This way you can validate values when an attribute is set, +or translate back and forth between the interface used on the attribute +and an internal representation. + +Validating a value when setting an attribute +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To ensure that an ``Employee`` object only has positive values for its +``salary`` attribute, you'd put a property in its place which checks +values before storing them in an attribute called e.g. ``_salary``: + +:: + + class Employee(object): + + @property + def salary(self): + return self._salary + + @salary.setter + def salary(self, salary): + if salary <= 0: + raise ValueError("Salary must be positive.") + self._salary = salary + +.. raw:: mediawiki + + {{admon/caution|Avoid recursion|In order to avoid endless recursion, you must use a different attribute than the one using the property to store actual values.}} + +Translating between attribute interface and internal representation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Take these classes of geometric primitives, ``Point`` and ``Circle``: + +:: + + class Point(object): + def __init__(self, x, y): + self.x = x + self.y = y + + class Circle(object): + def __init__(self, point, radius): + self.point = point + self.radius = radius + +If you wanted to add a ``diameter`` attribute to ``Circle``, you can do +so as a property which translates back and forth between it and the +existing ``radius`` attribute: + +:: + + ... + class Circle(object): + def __init__(self, point, radius=None, diameter=None): + self.point = point + if (radius is None) == (diameter is None): + raise ValueError("Exactly one of radius or diameter must be set") + if radius is not None: + self.radius = radius + else: + self.diameter = diameter + + @property + def diameter(self): + return self.radius * 2 + + @diameter.setter + def diameter(self, diameter): + self.radius = diameter / 2.0 + ... + +Even setting ``self.diameter`` in the constructor goes by way of the +property and therefore the setter method. + +External links +-------------- + +- `Python Design and History + FAQ `__ +- `PEP8: Style Guide for Python + Code `__ +- `PEP20: The Zen of + Python `__ +- `PyFormat: Using ``%`` and ``.format()`` for great + good! `__ +- `Sphinx Info field + lists `__ + for docstrings + diff --git a/source/development/integration/coding-style.rst b/source/development/integration/coding-style.rst deleted file mode 100644 index 8915dc9..0000000 --- a/source/development/integration/coding-style.rst +++ /dev/null @@ -1,503 +0,0 @@ -Coding Style -============ - -Most of our code is written in Python, so this document will concentrate -on it. - -Upstream guidelines -------------------- - -Fortunately, with PEP 8 there's an extensive official `Style Guide for -Python Code `__. All new -Python code you submit should conform to it, unless you have good -reasons to deviate from it, `for instance -readability `__. - -Keep PEP 20, the `Zen of -Python `__, under your -pillow. - -Keep It Simple --------------- - -The code you write now probably needs to be touched by someone else down -the road, and that someone else might be less experienced than you, or -have a terrible headache and be under pressure of time. So while a -particular construct may be a clever way of doing something, a simple -way of doing the same thing can be and often is preferrable. If (when) -complexity can't be avoided, try to isolate it: put a difficult -operation into its own function, method or class, add comments. If -complexity can be hidden from upper layers of the code, do so. - -Comments and Docstrings ------------------------ - -Be generous when it comes to commenting your code, it's better to have a -superfluous comment than if one were necessary but is missing. However, -if there is a comment it should be correct and agree with the code, -otherwise people have to guess if the comment or the code needs to be -straightened out. - -Adding `docstrings `__ to -modules, classes, methods and functions is encouraged. If you use the -`Sphinx -format `__ -to describe parameters, return values, etc., even better! - -Python 2 and 3 --------------- - -Python comes in two major versions nowadays: - -- The legacy version 2, of which the `first release 2.0 came out in - October 2000 `__. The - Python project `will maintain its final minor release 2.7 until - 2020 `__. - -- The current version 3, its `first release 3.0 was published in - December 2008 `__. At - the time of writing, the current minor release is version 3.5, to be - superseded by 3.6 around the end of 2016. - -Version 3 is not backwards compatible to version 2. While we mainly -target "the future", there are some components we have to work with that -haven't yet been ported over the Python 3, most notably -`koji `__. Additionally, we may also -want to support the "user tools" we create on legacy systems, so we -can't write code that uses all the latest features. Fortunately, many of -the original Python 3 features have been back-ported to Python 2.7, so -we can and should write code that is very close to writing idiomatic -Python 3 but can still be run on version 2.7. Targeting older minor -releases (Python 2.6 and earlier) is much more of a balancing act, so we -won't aim for it. - -The following sections cover areas that require some attention. The -Python project itself has a great `Porting Python 2 Code to Python -3 `__ document which -goes into much detail about the differences and is worth a read, even -though it mainly addresses existing Python 2 code bases. - -Absolute and relative imports -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -In Python 2, importing modules can be ambiguous when a module of that -name exists in the same package and elsewhere in the module search path -``sys.path``. To work around this ambiguity, programmers often resorted -to adding paths private to the project to the beginning of ``sys.path`` -to force loading modules from a project-internal location (which adds -unwanted noise and can make e.g. testing code that isn't installed -difficult). Python 3 introduces new syntax for import statements which -makes both cases distinct, this is available since version 2.5 from the -``__future__`` module: - -:: - - from __future__ import absolute_import - - # Import the sys module from the module search path - import sys - - # Import the foo module from the same directory - from . import foo - - # Import snafu from the bar module one directory above - from ..bar import snafu - -Print function -~~~~~~~~~~~~~~ - -Python 3 did away with ``print`` as a statement and introduced it as a -function. In order to use it the same way in Python 2.7, add the -following to the top of source code files where you use ``print``: - -:: - - from __future__ import print_function - -Numbers -~~~~~~~ - -Python 2 has two integer types, \`int\` which is whatever integer-type -is native to the system (which has certain maximal and minimal values -and can overflow) and \`long\` which can store arbitrary integer -numbers. Python 3 only the latter type, but it's called ``int``. - -Dividing integer numbers using ``/`` truncates the result to an integer -in Python 2 by default, but yields a floating point number in Python 3. -In order for code to do the same thing on either version, include the -following line at the top of your source files where you divide numbers, -and use ``/`` for normal divisions and ``//`` for divisions that should -truncate the result: - -:: - - from __future__ import division - -Strings -~~~~~~~ - -Some consider this the main difference between Python 2 and 3: Both -versions have a type for strings of bytes and strings of Unicode -character points. They are called ``str`` and ``unicode`` in version 2 -and ``bytes`` and ``str`` in version 3, respectively. - -String Literals -^^^^^^^^^^^^^^^ - -Python 2 and 3 use different ways of marking literals of the different -types by default. Byte strings can have no prefix or ``b`` in Python -2.7, but must be prefixed in Python 3, and text strings must have the -``u`` prefix in Python 2 which can be and usually is omitted in Python -3: - -:: - - # a byte string in Python 2 and 3 - string1 = b"abc" - - # a byte string in Python 2, but a text string in Python 3 - string2 = "def" - - # a text string in Python 2 and 3 - string3 = u"ghi" - -In order to ease writing code that is compatible between the versions, -you can switch Python 2 to treat unprefixed string literals as -``unicode``, the text string type, by adding this snippet to the top of -the relevant source code files: - -:: - - from __future__ import unicode_literals - -Explicit Encoding and Decoding -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In Python 2, the byte and text string types are exchangeable in many -places, taking the user's or system default locale into account (and -sometimes failing, when the locale didn't match up with encoded data). -Apart from the change in type names and how literals look like, Python 3 -requires you to explicitly encode ``str`` and decode ``bytes`` objects -if you need them cast into the respective other string type. It is good -practice to exclusively use text strings for strings that represent text -in a program and decode byte strings as early and encode text strings as -late as possible at interfaces that produce or consume encoded data. - -.. raw:: mediawiki - - {{admon/note|Implicit string type conversion in Python 2|Python 2 lets you attempt to replace a str substring in a unicode object (or vice versa) and would attempt to cast the one into the other by encoding or decoding on the fly as needed. This piece of code won't work in Python 3:}} - -:: - - from __future__ import print_function - text_string = u"Hello, world!" - print(text_string.replace("world", "gang")) - -.. raw:: mediawiki - - {{admon/tip|Explicit string type conversion in Python 2 and 3|Python 3 requires explicit encoding/decoding to cast between byte and text strings. This also works in Python 2 and is preferred of course.}} - -:: - - from __future__ import print_function, unicode_literals - text_string = "Hello, world!" - print(text_string.replace(b"world".decode('utf-8'), b"gang".decode('ascii'))) - -String formatting -^^^^^^^^^^^^^^^^^ - -With version 3.6 around the corner, there are four ways to format -strings in Python now: - -#. using the ``%`` operator -#. using ``string.Template`` of `PEP - 292 `__ -#. with the ``str.format()`` method -#. using `PEP 498 literal string - interpolation `__ - -The last method isn't available yet in a stable Python release and will -never be in Python 2, so it's not suitable for our purposes. The other -three variants work in all Python versions we're interested in, -formatting with ``string.Template`` is very rarely done however. The -remaining two ways, commonly called old-style (``%`` operator) and -new-style (``str.format()``), are both in wide-spread use, `here's a -site showcasing the differences between -them `__. New-style formatting is more powerful -and often easier to read, but on the other hand can be a little more to -type. From a technical point of view, this is a case of "use what works -for you", but for consistency sake the new-style ``str.format()`` way is -preferrable if you're comfortable with using it. If not, others can -convert old-style to new-style formatting for you during review or when -happening across it. At any rate, consistently use one way or the other -in what you submit. - -Old- and New-style Classes -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Python 2 and earlier knows two types of classes, old-style which have no -base class, and new-style which have ``object`` as the base class. -Because their behavior is slightly different in some places, and some -things can't be done with old-style classes, we want to stick to -new-style classes wherever possible. - -The syntactical difference is that new-style classes have to explicitly -be derived from ``object`` or another new-style class. - -:: - - # old-style classes - class OldFoo: - pass - - class OldBar(OldFoo): - pass - - # new-style classes - class NewFoo(object): - pass - - class NewBar(NewFoo): - pass - -Python 3 only knows new-style classes and the requirement to explicitly -derive from ``object`` was dropped. In projects that will only ever run -on Python 3, it's acceptable not to explicitly derive classes without -parents from ``object``, but if in doubt, do it just the same. - -Idiomatic code --------------- - -In Python, it's easy to inadvertently emulate idiomatic styles of other -languages like C/C++ or Java. In cases where there are constructs -"native" to the language, it's preferrable to use them. - -Literals and Comprehensions -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Python has special syntax for literals for a couple of built-in compound -data types: lists, tuples, dictionaries, strings, sets. It's customary -to use that syntax instead of the class constructor to create objects -for these data types unless you have good reason not to. Apart from how -it looks, the literal syntax is performing a little bit better (because -it doesn't have to look up the class name in the current scope). NB: Set -literals are peculiar in that you can't create empty ones­—they would -look the same as empty dicts. - -+-------------+-------------------------------------+----------------------------------------+ -| Data Type | Good | Bad | -+=============+=====================================+========================================+ -| ``str`` | | ``a_str = "abc"`` | ``empty_str = str()`` | -| | | ``empty_str = ""`` | | -+-------------+-------------------------------------+----------------------------------------+ -| ``list`` | | ``a_list = [1, 2]`` | | ``a_list = list((1, 2))`` | -| | | ``empty_str = []`` | | ``empty_list = list()`` | -+-------------+-------------------------------------+----------------------------------------+ -| ``tuple`` | | ``a_tuple = ('a', 'b', 3)`` | | ``a_tuple = tuple(['a', 'b', 3])`` | -| | | ``empty_tuple = ()`` | | ``empty_tuple = tuple()`` | -+-------------+-------------------------------------+----------------------------------------+ -| ``dict`` | | ``a_dict = {'a': 1}`` | | ``a_dict = dict(('a', 1))`` | -| | | ``empty_dict = {}`` | | ``empty_dict = dict()`` | -+-------------+-------------------------------------+----------------------------------------+ -| ``set`` | | ``a_set = {"banana", "apple"}`` | ``a_set = set(["banana", "apple"])`` | -| | | **``empty_set = set()``** | | -+-------------+-------------------------------------+----------------------------------------+ - -Table: Creating compound objects - -Often the initial contents of a compound object are only known when it's -created at runtime. For simple cases like mere type conversions, calling -the class constructors are the way to go: - -- Converting a tuple to a list or vice versa: - -| ``   a_tuple = (1, 2, 3)`` -| ``   ...`` -| ``   a_list = list(a_tuple)`` -| ``   ...`` -| ``   another_list = [4, 5, 6]`` -| ``   ...`` -| ``   another_tuple = tuple(another_list)`` - -- Convert a list to a set, e.g. to filter out duplicates: - -| ``   a_list = [1, 2, 3, 2]`` -| ``   ...`` -| ``   a_set = set(a_list)`` - -For more involved cases, say some values need to be filtered or a -specific attribute of the objects is wanted, Python has so-called -comprehensions to create compound objects in a syntactically "nice" way. -These largely supersede the old (ugly) way of using ``map()`` and -``filter()`` in conjunction with class constructors. - -+-------------------------------------------------------------------------------------------------------+-------------+------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Comprehension type | Data Type | Example | Remarks | -+=======================================================================================================+=============+================================================+=============================================================================================================================================================================+ -| `List Comprehension `__ | ``list`` | ``a_list = [x for x in range(20) if x % 2]`` | Put all odd numbers smaller than 20 into a list. | -+-------------------------------------------------------------------------------------------------------+-------------+------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| `Dict Comprehension `__ | ``dict`` | | ``a_dict = {k: getattr(an_obj, k)`` | Fill a dict with those attribute names and values of an object that aren't considered "protected" or "private" (names with one or two leading underscores, respectively). | -| | | | ``    for k in dir(an_obj)`` | | -| | | | ``    if not k.startswith("_")}`` | | -+-------------------------------------------------------------------------------------------------------+-------------+------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| `Set Comprehension `__ | ``set`` | ``a_set = {o.name for o in a_list}`` | Create a set containing the value of the attribute ``name`` of objects in a list. | -+-------------------------------------------------------------------------------------------------------+-------------+------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -Table: Using comprehensions to create compound objects - -Looping -~~~~~~~ - -Languages like C normally use incremented indices to loop over arrays: - -:: - - float pixels[NUMBER_OF_PIXELS] = [...]; - - for (int i = 0; i < NUMBER_OF_PIXELS; i++) - { - do_something_with_a_pixel(pixels[i]); - } - -.. raw:: mediawiki - - {{admon/warning|Looping C-style in Python|Avoid looping over indices of sequences, rather than the sequences themselves in Python.}} - -Implementing the loop like this would give away that you've programmed -in C or a similar language before: - -:: - - pixels = [...] - - for i in range(len(pixels)): - do_something_with_a_pixel(pixels[i]) - -.. raw:: mediawiki - - {{admon/note|Looping over iterables in Python|In Python, you can simply iterate over many non-scalar data types.}} - -Here's the "native" way to implement the above loop: - -:: - - pixels = [...] - - for p in pixels: - do_something_with_a_pixel(p) - -.. raw:: mediawiki - - {{admon/tip|Using enumerate()|If you need to keep track of the current count of looped-over items, use the enumerate() built-in.}} - -It yields pairs of count (starting at 0 by default) and the current -value like this: - -:: - - pixels = [...] - - for p_no, p in enumerate(pixels, 1): - print("Working on pixel no. {}".format(p_no)) - do_something_with_a_pixel(p) - -Properties rather than explicit accessor methods -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -In order to allow future changes in how object attributes (member -variables) are set, some languages encourage always using getter and/or -setter methods. This is unnecessary in Python, as you can intercept -access to an attribute by wrapping it into a -`property `__ -if and when this becomes necessary. Properties allow having accessor -methods without making the user of the class have to use them -explicitly. This way you can validate values when an attribute is set, -or translate back and forth between the interface used on the attribute -and an internal representation. - -Validating a value when setting an attribute -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To ensure that an ``Employee`` object only has positive values for its -``salary`` attribute, you'd put a property in its place which checks -values before storing them in an attribute called e.g. ``_salary``: - -:: - - class Employee(object): - - @property - def salary(self): - return self._salary - - @salary.setter - def salary(self, salary): - if salary <= 0: - raise ValueError("Salary must be positive.") - self._salary = salary - -.. raw:: mediawiki - - {{admon/caution|Avoid recursion|In order to avoid endless recursion, you must use a different attribute than the one using the property to store actual values.}} - -Translating between attribute interface and internal representation -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Take these classes of geometric primitives, ``Point`` and ``Circle``: - -:: - - class Point(object): - def __init__(self, x, y): - self.x = x - self.y = y - - class Circle(object): - def __init__(self, point, radius): - self.point = point - self.radius = radius - -If you wanted to add a ``diameter`` attribute to ``Circle``, you can do -so as a property which translates back and forth between it and the -existing ``radius`` attribute: - -:: - - ... - class Circle(object): - def __init__(self, point, radius=None, diameter=None): - self.point = point - if (radius is None) == (diameter is None): - raise ValueError("Exactly one of radius or diameter must be set") - if radius is not None: - self.radius = radius - else: - self.diameter = diameter - - @property - def diameter(self): - return self.radius * 2 - - @diameter.setter - def diameter(self, diameter): - self.radius = diameter / 2.0 - ... - -Even setting ``self.diameter`` in the constructor goes by way of the -property and therefore the setter method. - -External links --------------- - -- `Python Design and History - FAQ `__ -- `PEP8: Style Guide for Python - Code `__ -- `PEP20: The Zen of - Python `__ -- `PyFormat: Using ``%`` and ``.format()`` for great - good! `__ -- `Sphinx Info field - lists `__ - for docstrings - diff --git a/source/development/integration/grooming.rst b/source/development/integration/grooming.rst index 3f7994e..b290164 100644 --- a/source/development/integration/grooming.rst +++ b/source/development/integration/grooming.rst @@ -1,16 +1,14 @@ Grooming Your Changes ===================== -Apart from :doc:`coding-style`, -there are some things that you should keep in mind regarding the changes -you submit. Normally you'd develop your changes in a private branch on -your fork of a repository and, when you're done, submit them as pull -requests ("PR") against a public branch of the repository. The following -guidelines concentrate on changes in this format, their goal is to -enable you to groom the commits forming your pull request so that -another person can review it without great effort, that the changes can -be integrated well with the existing code and can be easily debugged -later if necessary. +Apart from coding style concerns, there are some things that you should keep in +mind regarding the changes you submit. Normally you'd develop your changes in a +private branch on your fork of a repository and, when you're done, submit them +as pull requests ("PR") against a public branch of the repository. The +following guidelines concentrate on changes in this format, their goal is to +enable you to groom the commits forming your pull request so that another +person can review it without great effort, that the changes can be integrated +well with the existing code and can be easily debugged later if necessary. Pull Requests ------------- From b1b4bc2c6a29071b825250bea5990e06db4e59cd Mon Sep 17 00:00:00 2001 From: Nils Philippsen Date: Jun 07 2017 12:19:45 +0000 Subject: [PATCH 5/5] cope with lack of nested markup in rST --- diff --git a/source/development/integration/coding-style-python.rst b/source/development/integration/coding-style-python.rst index f047f92..f80d85d 100644 --- a/source/development/integration/coding-style-python.rst +++ b/source/development/integration/coding-style-python.rst @@ -494,8 +494,8 @@ External links Code `__ - `PEP20: The Zen of Python `__ -- `PyFormat: Using ``%`` and ``.format()`` for great - good! `__ +- `PyFormat `__: Using ``%`` and ``.format()`` for + great good! - `Sphinx Info field lists `__ for docstrings