Python map (draft)

Error Handling
   
def spam(divideBy):
    try:
        return myIntVar / divideBy
    except ZeroDivisionError as e:
        print('Error: Invalid argument: {}'.format(e))
    finally :
        print('Either there was an error or a division')

Debugging
Raising Exceptions

Exceptions are raised with a raise statement. In code, a raise statement consists of the following:

    The raise keyword
    A call to the Exception() function
    A string with a helpful error message passed to the Exception() function

˃˃˃ raise Exception('This is the error message.')
Traceback (most recent call last):
  File "˂pyshell#191˃", line 1, in ˂module˃
    raise Exception('This is the error message.')
Exception: This is the error message.

Often it’s the code that calls the function, not the function itself, that knows how to handle an expection. So you will commonly see a raise statement inside a function and the try and except statements in the code calling the function.

def box_print(symbol, width, height):
    if len(symbol) != 1:
      raise Exception('Symbol must be a single character string.')
    if width ˂= 2:
      raise Exception('Width must be greater than 2.')
    if height ˂= 2:
      raise Exception('Height must be greater than 2.')
    print(symbol * width)
    for i in range(height - 2):
        print(symbol + (' ' * (width - 2)) + symbol)
    print(symbol * width)
for sym, w, h in (('*', 4, 4), ('O', 20, 5), ('x', 1, 3), ('ZZ', 3, 3)):
    try:
        box_print(sym, w, h)
    except Exception as err:
        print('An exception happened: ' + str(err))

Getting the Traceback as a String

The traceback is displayed by Python whenever a raised exception goes unhandled. But can also obtain it as a string by calling traceback.format_exc(). This function is useful if you want the information from an exception’s traceback but also want an except statement to gracefully handle the exception. You will need to import Python’s traceback module before calling this function.

˃˃˃ import traceback

˃˃˃ try:
˃˃˃      raise Exception('This is the error message.')
˃˃˃ except:
˃˃˃      with open('errorInfo.txt', 'w') as error_file:
˃˃˃          error_file.write(traceback.format_exc())
˃˃˃      print('The traceback info was written to errorInfo.txt.')
116
The traceback info was written to errorInfo.txt.

The 116 is the return value from the write() method, since 116 characters were written to the file. The traceback text was written to errorInfo.txt.

Traceback (most recent call last):
  File "˂pyshell#28˃", line 2, in 
Exception: This is the error message.

Assertions

An assertion is a sanity check to make sure your code isn’t doing something obviously wrong. These sanity checks are performed by assert statements. If the sanity check fails, then an AssertionError exception is raised. In code, an assert statement consists of the following:

    The assert keyword
    A condition (that is, an expression that evaluates to True or False)
    A comma
    A string to display when the condition is False

˃˃˃ pod_bay_door_status = 'open'

˃˃˃ assert pod_bay_door_status == 'open', 'The pod bay doors need to be "open".'

˃˃˃ pod_bay_door_status = 'I\'m sorry, Dave. I\'m afraid I can\'t do that.'

˃˃˃ assert pod_bay_door_status == 'open', 'The pod bay doors need to be "open".'

Traceback (most recent call last):
  File "˂pyshell#10˃", line 1, in ˂module˃
    assert pod_bay_door_status == 'open', 'The pod bay doors need to be "open".'
AssertionError: The pod bay doors need to be "open".

In plain English, an assert statement says, “I assert that this condition holds true, and if not, there is a bug somewhere in the program.” Unlike exceptions, your code should not handle assert statements with try and except; if an assert fails, your program should crash. By failing fast like this, you shorten the time between the original cause of the bug and when you first notice the bug. This will reduce the amount of code you will have to check before finding the code that’s causing the bug.

Disabling Assertions

Assertions can be disabled by passing the -O option when running Python.

Logging
To enable the logging module to display log messages on your screen as your program runs, copy the following to the top of your program (but under the #! python shebang line):

import logging

logging.basicConfig(level=logging.DEBUG, format=' %(asctime)s - %(levelname)s- %(message)s')

Say you wrote a function to calculate the factorial of a number. In mathematics, factorial 4 is 1 × 2 × 3 × 4, or 24. Factorial 7 is 1 × 2 × 3 × 4 × 5 × 6 × 7, or 5,040. Open a new file editor window and enter the following code. It has a bug in it, but you will also enter several log messages to help yourself figure out what is going wrong. Save the program as factorialLog.py.

˃˃˃ import logging
˃˃˃
˃˃˃ logging.basicConfig(level=logging.DEBUG, format=' %(asctime)s - %(levelname)s- %(message)s')
˃˃˃
˃˃˃ logging.debug('Start of program')
˃˃˃
˃˃˃ def factorial(n):
˃˃˃
˃˃˃     logging.debug('Start of factorial(%s)' % (n))
˃˃˃     total = 1
˃˃˃
˃˃˃     for i in range(1, n + 1):
˃˃˃         total *= i
˃˃˃         logging.debug('i is ' + str(i) + ', total is ' + str(total))
˃˃˃
˃˃˃     logging.debug('End of factorial(%s)' % (n))
˃˃˃
˃˃˃     return total
˃˃˃
˃˃˃ print(factorial(5))
˃˃˃ logging.debug('End of program')
2015-05-23 16:20:12,664 - DEBUG - Start of program
2015-05-23 16:20:12,664 - DEBUG - Start of factorial(5)
2015-05-23 16:20:12,665 - DEBUG - i is 0, total is 0
2015-05-23 16:20:12,668 - DEBUG - i is 1, total is 0
2015-05-23 16:20:12,670 - DEBUG - i is 2, total is 0
2015-05-23 16:20:12,673 - DEBUG - i is 3, total is 0
2015-05-23 16:20:12,675 - DEBUG - i is 4, total is 0
2015-05-23 16:20:12,678 - DEBUG - i is 5, total is 0
2015-05-23 16:20:12,680 - DEBUG - End of factorial(5)
0
2015-05-23 16:20:12,684 - DEBUG - End of program

Logging Levels

Logging levels provide a way to categorize your log messages by importance. There are five logging levels, described in Table 10-1 from least to most important. Messages can be logged at each level using a different logging function.
Level   Logging Function    Description
DEBUG   logging.debug()     The lowest level. Used for small details. Usually you care about these messages only when diagnosing problems.
INFO    logging.info()  Used to record information on general events in your program or confirm that things are working at their point in the program.
WARNING     logging.warning()   Used to indicate a potential problem that doesn’t prevent the program from working but might do so in the future.
ERROR   logging.error()     Used to record an error that caused the program to fail to do something.
CRITICAL    logging.critical()  The highest level. Used to indicate a fatal error that has caused or is about to cause the program to stop running entirely.
Disabling Logging

After you’ve debugged your program, you probably don’t want all these log messages cluttering the screen. The logging.disable() function disables these so that you don’t have to go into your program and remove all the logging calls by hand.

˃˃˃ import logging

˃˃˃ logging.basicConfig(level=logging.INFO, format=' %(asctime)s -%(levelname)s - %(message)s')

˃˃˃ logging.critical('Critical error! Critical error!')
2015-05-22 11:10:48,054 - CRITICAL - Critical error! Critical error!

˃˃˃ logging.disable(logging.CRITICAL)

˃˃˃ logging.critical('Critical error! Critical error!')

˃˃˃ logging.error('Error! Error!')

Logging to a File

Instead of displaying the log messages to the screen, you can write them to a text file. The logging.basicConfig() function takes a filename keyword argument, like so:

import logging

logging.basicConfig(filename='myProgramLog.txt', level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')

Context Manager
("with")
ºwithº
- A context manager is an object (file objects, ...) that is notified when a context 
  ("block of code") starts and ends.
- When a context ends, the file object is closed automatically:
- The "with" statement takes care of the notifying.

01 with open(filename) as openFile:
02     file_contents = openFile.read()
03 ...       #  ← here openFile is closed

ºHand writen Context Managerº

import ºcontextlibº
º@contextlib.contextmanagerº
def Bºcontext_manager(num)º: 
    print('OºEnterº')
    yield num + 1    # ← Generator
    print('GºExitº')
    print('')

˃˃˃ with Bºcontext_manager(2)º as cm:
˃˃˃     # the following instructions are run when
˃˃˃     # the 'yield' point of the context
˃˃˃     # manager is reached.
˃˃˃     # 'cm' will have the value that was yielded
˃˃˃     print('QºEnterºRight in the middle')

OºEnterº
QºEnterºRight in the middle
GºExitº

Immutable Data Structures
@[https://opensource.com/article/18/10/functional-programming-python-immutable-data-structures]

Dataclasses
- 3.7+
- Designed for storing data.

- decorator and functions for automatically adding generated special 
  methods such as __init__() and __repr__() to user-defined classes.

Python 2.7                     DataClass (3.7+)

                               from dataclasses import dataclass
                               from typing import Any
                               º@dataclassº
class Number:                  class Number:
  def __init__(self, val):       val: int = 0
    self.val = val                     ^  ^^^
                                       |  (Opt) default value
                                    Type hint (or typing.Any)


˃˃˃ obj = Number(2)            ˃˃˃ obj = Number(2)
˃˃˃ obj.val                    ˃˃˃ obj.val
2                              2

Oficial Doc

Functions
def foo():
    """
    This is a function docstring
    You can also use:
    ''' Function Docstring '''
    """
    ...
    return result

Lambdas
 
add = ºlambdaº x, y: x + y
             ^^^^
             func. args

˃˃˃ (ºlambdaº x, y: x + y)(5, 3)
8

def build_adder(n):          # builder of lambdas
    return lambda x: x + n
plus_3 = build_adder(3)

º (args) and º* (kwargs)

- In funct. declaration:
  - '*' means:
    "pack all remaining positional arguments into a tuple      named "
  - ºº   means:
    "pack all remaining positional arguments into a dictionary named 2


- In funct. call:
  - '*' means:
     "unpack tuple or list named  to positional arguments at this position"
  - 'ºº' means:
     "unpack dictionary    named  to positional arguments at this position"

Example:
- Wrapper function around any other function:
def forward(f, ºargs, º*kwargs):
    return f(ºargs, º*kwargs)

Strings
  
String concatenation:

˃˃˃ 'Alice' 'Bob'
'AliceBob'

Note: Avoid + operator for string concatenation. Prefer string formatting.

String Replication:
˃˃˃ 'Alice' * 5
'AliceAliceAliceAliceAlice'

String format:
˃˃˃ print('It is good to meet you, {}'.format(myName))


ºEscape Charactersº
Escape     Prints as
character    
\'         Single quote
\"         Double quote
\t         Tab
\n         Newline
\\         Backslash

˃˃˃ print(ºrº'That is Carol\'s cat.')  # ºRaw Stringsº: ignores all escape characters 
That is Carol\'s cat.                         mostly used for regex definition


º'''º Multiline String
myMultilineStr = '''My Line 1,
 ...
 my last line''')

str01 = 'Hello world!'           'Hello world!'.ºstartswithº('Hello')   # True
str01[0]     # 'H'               'Hello world!'.ºendswithº  ('world!')  # True
str01[0:5]   # 'Hello'
str01[::-1]  # '!dlrow olleH'

'Hello' in 'Hello World'  # True    'Hello world!'.ºupper()º  # 'HELLO WORLD!'
'HELLO' in 'Hello World'  # False   'Hello world!'.ºlower()º  # 'hello world!'
                                    
                                    'Hello world!'.ºislower()º # False
                                    'Hello world!'.ºisupper()º # False
                                    'HELLO WORLD!'.ºisupper()º # True

myString.ºisalpha()  º #   True if string consists only of letters and is not blank.
myString.ºisalnum()  º #   True if string consists only of lettersand numbers and is not blank.
myString.ºisdecimal()º #   True if string consists only of numeric characters and is not blank.
myString.ºisspace()  º #   True if string consists only of spaces,tabs, and new-lines and is not blank.
myString.ºistitle()  º #   True if string consists only of wordsthat begin with an uppercase letter 
                                followed by onlylowercase letters.
    
', '.ºjoinº(['a', 'b', 'c']) # 'a, b, c'
'a b c'.ºsplit()º            # ['a', 'b', 'c']

Justifying Text:
   '1234'.ºrjust º(10,'+') # '+++++1234'
   '1234'.ºljust º(10,'-') # '1234-----'
   '1234'.ºcenterº(10,'|') # '|||||||Hello|||||||'
                    ^^^
                    Optional(' ' by default)

Removing Whitespace with strip, rstrip, and lstrip

'    Hello World     '.º strip()º  # 'Hello World'
'    Hello World     '.ºlstrip()º  # 'Hello World     '
'    Hello World     '.ºrstrip()º  # '     Hello World'
'Hello World '

Formatting
% operator (Discouraged in favor of str.format)
'Hello %s' % 'Pete'  # "Hello Pete"
'I have %x apples' % num # 

ºstr.format (Python 2.7+) º
"name: {}, age: {}".format('John', 20) # "name: John, age: 20"

Lazy formatting

ºFormatted String Literals f-stringsº Python 3.6+
name = 'Elizabeth'
ºfº'Hello {name}!'   # 'Hello Elizabeth!
ºfº'result: {myIntVar01 + myIntVar02}'


ºTemplate Stringsº
(simpler, less powerful)

from string import Template
name = 'Elizabeth'
t = Template('Hey $name!')
t.substitute(name=name)
'Hey Elizabeth!'
textwrap lib

Regex
- Import the regex module with import re.
- Create a Regex object with the re.compile() function. (Remember to use a raw string.)
- Pass the string you want to search into the Regex object's search() method. 
  This returns a Match object.

import ºreº  # Step 1
phone_num_regex = re.ºcompileº(r'\d\d\d-\d\d\d-\d\d\d\d')  # step 2. Create regex object
                             ^
                             raw string

mo = phone_num_regex.ºsearchº('My number is 415-555-4242.') # step 3.
^^                           ^^^^^^^^^^^^^^^^^^^^^^^^^
Object of type "Match"       Input string to search against

print('Phone number found: {}'.format(mo.ºgroupº()))        # step 4. show results
                                         ^^^^^    
                                         returns actual matched text string
                                         (415-555-4242)

ºGrouping with Parenthesesº
phone_num_regex = re.compile(r'(\d\d\d)-(\d\d\d-\d\d\d\d)')  # step 2
mo = phone_num_regex.search('My number is 415-555-4242.')    # step 3
mo.group(1)                                                  # '415'
mo.group(2)                                                  # '555-4242'
mo.group(0)                                                  # '415-555-4242'
mo.group()                                                   # '415-555-4242'
mo.groups()                                                  # ('415', '555-4242')

Other regext examples:
r'wom(a|e)n'  # | == or        (match woman or women)
r'(wo)?man'   # ? == optional  (match man  or woman )
r'(wo)ºman'   # º == 0 or more (match man  or woman or wowoman or ... )
r'(wo)+man'   # + == 1 or more (match         woman or wowoman or ... )
r'(wo){2}man' # {2} == 2 time  (match                  wowoman )

ºGreedy vs Nongreedy Matchingº



Python’s regular expressions are greedy by default, which means that in ambiguous situations they will match the longest string possible. The non-greedy version of the curly brackets, which matches the shortest string possible, has the closing curly bracket followed by a question mark.

˃˃˃ greedy_ha_regex = re.compile(r'(Ha){3,5}')
˃˃˃ mo1 = greedy_ha_regex.search('HaHaHaHaHa')
˃˃˃ mo1.group()
'HaHaHaHaHa'

˃˃˃ nongreedy_ha_regex = re.compile(r'(Ha){3,5}?')
˃˃˃ mo2 = nongreedy_ha_regex.search('HaHaHaHaHa')
˃˃˃ mo2.group()
'HaHaHa'

The findall Method

In addition to the search() method, Regex objects also have a findall() method. While search() will return a Match object of the first matched text in the searched string, the findall() method will return the strings of every match in the searched string.

˃˃˃ phone_num_regex = re.compile(r'\d\d\d-\d\d\d-\d\d\d\d') # has no groups

˃˃˃ phone_num_regex.findall('Cell: 415-555-9999 Work: 212-555-0000')
['415-555-9999', '212-555-0000']

To summarize what the findall() method returns, remember the following:

    When called on a regex with no groups, such as \d-\d\d\d-\d\d\d\d, the method findall() returns a list of ng matches, such as ['415-555-9999', '212-555-0000'].

    When called on a regex that has groups, such as (\d\d\d)-d\d)-(\d\ d\d\d), the method findall() returns a list of es of strings (one string for each group), such as [('415', ', '9999'), ('212', '555', '0000')].

Making Your Own Character Classes

There are times when you want to match a set of characters but the shorthand character classes (\d, \w, \s, and so on) are too broad. You can define your own character class using square brackets. For example, the character class [aeiouAEIOU] will match any vowel, both lowercase and uppercase.

˃˃˃ vowel_regex = re.compile(r'[aeiouAEIOU]')

˃˃˃ vowel_regex.findall('Robocop eats baby food. BABY FOOD.')
['o', 'o', 'o', 'e', 'a', 'a', 'o', 'o', 'A', 'O', 'O']

You can also include ranges of letters or numbers by using a hyphen. For example, the character class [a-zA-Z0-9] will match all lowercase letters, uppercase letters, and numbers.

By placing a caret character (^) just after the character class’s opening bracket, you can make a negative character class. A negative character class will match all the characters that are not in the character class. For example, enter the following into the interactive shell:

˃˃˃ consonant_regex = re.compile(r'[^aeiouAEIOU]')

˃˃˃ consonant_regex.findall('Robocop eats baby food. BABY FOOD.')
['R', 'b', 'c', 'p', ' ', 't', 's', ' ', 'b', 'b', 'y', ' ', 'f', 'd', '.', '
', 'B', 'B', 'Y', ' ', 'F', 'D', '.']

The Caret and Dollar Sign Characters

    You can also use the caret symbol (^) at the start of a regex to indicate that a match must occur at the beginning of the searched text.

    Likewise, you can put a dollar sign ($) at the end of the regex to indicate the string must end with this regex pattern.

    And you can use the ^ and $ together to indicate that the entire string must match the regex—that is, it’s not enough for a match to be made on some subset of the string.

The r'^Hello' regular expression string matches strings that begin with 'Hello':

˃˃˃ begins_with_hello = re.compile(r'^Hello')

˃˃˃ begins_with_hello.search('Hello world!')
˂_sre.SRE_Match object; span=(0, 5), match='Hello'˃

˃˃˃ begins_with_hello.search('He said hello.') is None
True

The r'\d$' regular expression string matches strings that end with a numeric character from 0 to 9:

˃˃˃ whole_string_is_num = re.compile(r'^\d+$')

˃˃˃ whole_string_is_num.search('1234567890')
˂_sre.SRE_Match object; span=(0, 10), match='1234567890'˃

˃˃˃ whole_string_is_num.search('12345xyz67890') is None
True

˃˃˃ whole_string_is_num.search('12 34567890') is None
True

The Wildcard Character

The . (or dot) character in a regular expression is called a wildcard and will match any character except for a newline:

˃˃˃ at_regex = re.compile(r'.at')

˃˃˃ at_regex.findall('The cat in the hat sat on the flat mat.')
['cat', 'hat', 'sat', 'lat', 'mat']

Matching Everything with Dot-Star

˃˃˃ name_regex = re.compile(r'First Name: (.º) Last Name: (.º)')

˃˃˃ mo = name_regex.search('First Name: Al Last Name: Sweigart')

˃˃˃ mo.group(1)
'Al'

˃˃˃ mo.group(2)
'Sweigart'

The dot-star uses greedy mode: It will always try to match as much text as possible. To match any and all text in a nongreedy fashion, use the dot, star, and question mark (.*?). The question mark tells Python to match in a nongreedy way:

˃˃˃ nongreedy_regex = re.compile(r'˂.*?˃')
˃˃˃ mo = nongreedy_regex.search('˂To serve man˃ for dinner.˃')
˃˃˃ mo.group()
'˂To serve man˃'

˃˃˃ greedy_regex = re.compile(r'˂.*˃')
˃˃˃ mo = greedy_regex.search('˂To serve man˃ for dinner.˃')
˃˃˃ mo.group()
'˂To serve man˃ for dinner.˃'

Matching Newlines with the Dot Character

The dot-star will match everything except a newline. By passing re.DOTALL as the second argument to re.compile(), you can make the dot character match all characters, including the newline character:

˃˃˃ no_newline_regex = re.compile('.*')
˃˃˃ no_newline_regex.search('Serve the public trust.\nProtect the innocent.\nUphold the law.').group()
'Serve the public trust.'

˃˃˃ newline_regex = re.compile('.*', re.DOTALL)
˃˃˃ newline_regex.search('Serve the public trust.\nProtect the innocent.\nUphold the law.').group()
'Serve the public trust.\nProtect the innocent.\nUphold the law.'

Review of Regex Symbols
Symbol  Matches
?   zero or one of the preceding group.
*   zero or more of the preceding group.
+   one or more of the preceding group.
{n}     exactly n of the preceding group.
{n,}    n or more of the preceding group.
{,m}    0 to m of the preceding group.
{n,m}   at least n and at most m of the preceding p.
{n,m}? or *? or +?  performs a nongreedy match of the preceding p.
^spam   means the string must begin with spam.
spam$   means the string must end with spam.
.   any character, except newline characters.
\d, \w, and \s  a digit, word, or space character, resectively.
\D, \W, and \S  anything except a digit, word, or space acter, respectively.
[abc]   any character between the brackets (such as a, b, ).
[^abc]  any character that isn’t between the brackets.
Case-Insensitive Matching

To make your regex case-insensitive, you can pass re.IGNORECASE or re.I as a second argument to re.compile():

˃˃˃ robocop = re.compile(r'robocop', re.I)

˃˃˃ robocop.search('Robocop is part man, part machine, all cop.').group()
'Robocop'

˃˃˃ robocop.search('ROBOCOP protects the innocent.').group()
'ROBOCOP'

˃˃˃ robocop.search('Al, why does your programming book talk about robocop so much?').group()
'robocop'

Substituting Strings with the sub() Method

The sub() method for Regex objects is passed two arguments:

    The first argument is a string to replace any matches.
    The second is the string for the regular expression.

The sub() method returns a string with the substitutions applied:

˃˃˃ names_regex = re.compile(r'Agent \w+')

˃˃˃ names_regex.sub('CENSORED', 'Agent Alice gave the secret documents to Agent Bob.')
'CENSORED gave the secret documents to CENSORED.'

Another example:

˃˃˃ agent_names_regex = re.compile(r'Agent (\w)\w*')

˃˃˃ agent_names_regex.sub(r'\1ºººº', 'Agent Alice told Agent Carol that Agent Eve knew Agent Bob was a double agent.')
Aºººº told Cºººº that Eºººº knew Bºººº was a double agent.'

Managing Complex Regexes

To tell the re.compile() function to ignore whitespace and comments inside the regular expression string, “verbose mode” can be enabled by passing the variable re.VERBOSE as the second argument to re.compile().

Now instead of a hard-to-read regular expression like this:

phone_regex = re.compile(r'((\d{3}|\(\d{3}\))?(\s|-|\.)?\d{3}(\s|-|\.)\d{4}(\s*(ext|x|ext.)\s*\d{2,5})?)')

you can spread the regular expression over multiple lines with comments like this:

phone_regex = re.compile(r'''(
    (\d{3}|\(\d{3}\))?            # area code
    (\s|-|\.)?                    # separator
    \d{3}                         # first 3 digits
    (\s|-|\.)                     # separator
    \d{4}                         # last 4 digits
    (\s*(ext|x|ext.)\s*\d{2,5})?  # extension
    )''', re.VERBOSE)

File and dirs
There are two main modules in Python that deals with path manipulation. One is the os.path module and the other is the pathlib module. The pathlib module was added in Python 3.4, offering an object-oriented way to handle file system paths.
Backslash on Windows and Forward Slash on OS X and Linux

On Windows, paths are written using backslashes () as the separator between folder names. On Unix based operating system such as macOS, Linux, and BSDs, the forward slash (/) is used as the path separator. Joining paths can be a headache if your code needs to work on different platforms.

Fortunately, Python provides easy ways to handle this. We will showcase how to deal with this with both os.path.join and pathlib.Path.joinpath

Using os.path.join on Windows:

˃˃˃ import os

˃˃˃ os.path.join('usr', 'bin', 'spam')
'usr\\bin\\spam'

And using pathlib on *nix:

˃˃˃ from pathlib import Path

˃˃˃ print(Path('usr').joinpath('bin').joinpath('spam'))
usr/bin/spam

pathlib also provides a shortcut to joinpath using the / operator:

˃˃˃ from pathlib import Path

˃˃˃ print(Path('usr') / 'bin' / 'spam')
usr/bin/spam

Notice the path separator is different between Windows and Unix based operating system, that's why you want to use one of the above methods instead of adding strings together to join paths together.

Joining paths is helpful if you need to create different file paths under the same directory.

Using os.path.join on Windows:

˃˃˃ my_files = ['accounts.txt', 'details.csv', 'invite.docx']

˃˃˃ for filename in my_files:
˃˃˃     print(os.path.join('C:\\Users\\asweigart', filename))
C:\Users\asweigart\accounts.txt
C:\Users\asweigart\details.csv
C:\Users\asweigart\invite.docx

Using pathlib on *nix:

˃˃˃ my_files = ['accounts.txt', 'details.csv', 'invite.docx']
˃˃˃ home = Path.home()
˃˃˃ for filename in my_files:
˃˃˃     print(home / filename)
/home/asweigart/accounts.txt
/home/asweigart/details.csv
/home/asweigart/invite.docx

The Current Working Directory

Using os on Windows:

˃˃˃ import os

˃˃˃ os.getcwd()
'C:\\Python34'
˃˃˃ os.chdir('C:\\Windows\\System32')

˃˃˃ os.getcwd()
'C:\\Windows\\System32'

Using pathlib on *nix:

˃˃˃ from pathlib import Path
˃˃˃ from os import chdir

˃˃˃ print(Path.cwd())
/home/asweigart

˃˃˃ chdir('/usr/lib/python3.6')
˃˃˃ print(Path.cwd())
/usr/lib/python3.6

Creating New Folders

Using os on Windows:

˃˃˃ import os
˃˃˃ os.makedirs('C:\\delicious\\walnut\\waffles')

Using pathlib on *nix:

˃˃˃ from pathlib import Path
˃˃˃ cwd = Path.cwd()
˃˃˃ (cwd / 'delicious' / 'walnut' / 'waffles').mkdir()
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.6/pathlib.py", line 1226, in mkdir
    self._accessor.mkdir(self, mode)
  File "/usr/lib/python3.6/pathlib.py", line 387, in wrapped
    return strfunc(str(pathobj), *args)
FileNotFoundError: [Errno 2] No such file or directory: '/home/asweigart/delicious/walnut/waffles'

Oh no, we got a nasty error! The reason is that the 'delicious' directory does not exist, so we cannot make the 'walnut' and the 'waffles' directories under it. To fix this, do:

˃˃˃ from pathlib import Path
˃˃˃ cwd = Path.cwd()
˃˃˃ (cwd / 'delicious' / 'walnut' / 'waffles').mkdir(parents=True)

And all is good :)
Absolute vs. Relative Paths

There are two ways to specify a file path.

    An absolute path, which always begins with the root folder
    A relative path, which is relative to the program’s current working directory

There are also the dot (.) and dot-dot (..) folders. These are not real folders but special names that can be used in a path. A single period (“dot”) for a folder name is shorthand for “this directory.” Two periods (“dot-dot”) means “the parent folder.”
Handling Absolute and Relative Paths

To see if a path is an absolute path:

Using os.path on *nix:

˃˃˃ import os
˃˃˃ os.path.isabs('/')
True
˃˃˃ os.path.isabs('..')
False

Using pathlib on *nix:

˃˃˃ from pathlib import Path
˃˃˃ Path('/').is_absolute()
True
˃˃˃ Path('..').is_absolute()
False

You can extract an absolute path with both os.path and pathlib

Using os.path on *nix:

˃˃˃ import os
˃˃˃ os.getcwd()
'/home/asweigart'
˃˃˃ os.path.abspath('..')
'/home'

Using pathlib on *nix:

from pathlib import Path
print(Path.cwd())
/home/asweigart
print(Path('..').resolve())
/home

You can get a relative path from a starting path to another path.

Using os.path on *nix:

˃˃˃ import os
˃˃˃ os.path.relpath('/etc/passwd', '/')
'etc/passwd'

Using pathlib on *nix:

˃˃˃ from pathlib import Path
˃˃˃ print(Path('/etc/passwd').relative_to('/'))
etc/passwd

Checking Path Validity

Checking if a file/directory exists:

Using os.path on *nix:

import os
˃˃˃ os.path.exists('.')
True
˃˃˃ os.path.exists('setup.py')
True
˃˃˃ os.path.exists('/etc')
True
˃˃˃ os.path.exists('nonexistentfile')
False

Using pathlib on *nix:

from pathlib import Path
˃˃˃ Path('.').exists()
True
˃˃˃ Path('setup.py').exists()
True
˃˃˃ Path('/etc').exists()
True
˃˃˃ Path('nonexistentfile').exists()
False

Checking if a path is a file:

Using os.path on *nix:

˃˃˃ import os
˃˃˃ os.path.isfile('setup.py')
True
˃˃˃ os.path.isfile('/home')
False
˃˃˃ os.path.isfile('nonexistentfile')
False

Using pathlib on *nix:

˃˃˃ from pathlib import Path
˃˃˃ Path('setup.py').is_file()
True
˃˃˃ Path('/home').is_file()
False
˃˃˃ Path('nonexistentfile').is_file()
False

Checking if a path is a directory:

Using os.path on *nix:

˃˃˃ import os
˃˃˃ os.path.isdir('/')
True
˃˃˃ os.path.isdir('setup.py')
False
˃˃˃ os.path.isdir('/spam')
False

Using pathlib on *nix:

˃˃˃ from pathlib import Path
˃˃˃ Path('/').is_dir()
True
˃˃˃ Path('setup.py').is_dir()
False
˃˃˃ Path('/spam').is_dir()
False

Finding File Sizes and Folder Contents

Getting a file's size in bytes:

Using os.path on Windows:

˃˃˃ import os
˃˃˃ os.path.getsize('C:\\Windows\\System32\\calc.exe')
776192

Using pathlib on *nix:

˃˃˃ from pathlib import Path
˃˃˃ stat = Path('/bin/python3.6').stat()
˃˃˃ print(stat) # stat contains some other information about the file as well
os.stat_result(st_mode=33261, st_ino=141087, st_dev=2051, st_nlink=2, st_uid=0,
--snip--
st_gid=0, st_size=10024, st_atime=1517725562, st_mtime=1515119809, st_ctime=1517261276)
˃˃˃ print(stat.st_size) # size in bytes
10024

Listing directory contents using os.listdir on Windows:

˃˃˃ import os
˃˃˃ os.listdir('C:\\Windows\\System32')
['0409', '12520437.cpx', '12520850.cpx', '5U877.ax', 'aaclient.dll',
--snip--
'xwtpdui.dll', 'xwtpw32.dll', 'zh-CN', 'zh-HK', 'zh-TW', 'zipfldr.dll']

Listing directory contents using pathlib on *nix:

˃˃˃ from pathlib import Path
˃˃˃ for f in Path('/usr/bin').iterdir():
˃˃˃     print(f)
...
/usr/bin/tiff2rgba
/usr/bin/iconv
/usr/bin/ldd
/usr/bin/cache_restore
/usr/bin/udiskie
/usr/bin/unix2dos
/usr/bin/t1reencode
/usr/bin/epstopdf
/usr/bin/idle3
...

To find the total size of all the files in this directory:

WARNING: Directories themselves also have a size! So you might want to check for whether a path is a file or directory using the methods in the methods discussed in the above section!

Using os.path.getsize() and os.listdir() together on Windows:

˃˃˃ import os
˃˃˃ total_size = 0

˃˃˃ for filename in os.listdir('C:\\Windows\\System32'):
      total_size = total_size + os.path.getsize(os.path.join('C:\\Windows\\System32', filename))

˃˃˃ print(total_size)
1117846456

Using pathlib on *nix:

˃˃˃ from pathlib import Path
˃˃˃ total_size = 0

˃˃˃ for sub_path in Path('/usr/bin').iterdir():
...     total_size += sub_path.stat().st_size
˃˃˃
˃˃˃ print(total_size)
1903178911

Copying Files and Folders

The shutil module provides functions for copying files, as well as entire folders.

˃˃˃ import shutil, os

˃˃˃ os.chdir('C:\\')

˃˃˃ shutil.copy('C:\\spam.txt', 'C:\\delicious')
   'C:\\delicious\\spam.txt'

˃˃˃ shutil.copy('eggs.txt', 'C:\\delicious\\eggs2.txt')
   'C:\\delicious\\eggs2.txt'

While shutil.copy() will copy a single file, shutil.copytree() will copy an entire folder and every folder and file contained in it:

˃˃˃ import shutil, os

˃˃˃ os.chdir('C:\\')

˃˃˃ shutil.copytree('C:\\bacon', 'C:\\bacon_backup')
'C:\\bacon_backup'

Moving and Renaming Files and Folders

˃˃˃ import shutil
˃˃˃ shutil.move('C:\\bacon.txt', 'C:\\eggs')
'C:\\eggs\\bacon.txt'

The destination path can also specify a filename. In the following example, the source file is moved and renamed:

˃˃˃ shutil.move('C:\\bacon.txt', 'C:\\eggs\\new_bacon.txt')
'C:\\eggs\\new_bacon.txt'

If there is no eggs folder, then move() will rename bacon.txt to a file named eggs.

˃˃˃ shutil.move('C:\\bacon.txt', 'C:\\eggs')
'C:\\eggs'

Permanently Deleting Files and Folders

    Calling os.unlink(path) or Path.unlink() will delete the file at path.

    Calling os.rmdir(path) or Path.rmdir() will delete the folder at path. This folder must be empty of any files or folders.

    Calling shutil.rmtree(path) will remove the folder at path, and all files and folders it contains will also be deleted.

Safe Deletes with the send2trash Module

You can install this module by running pip install send2trash from a Terminal window.

˃˃˃ import send2trash

˃˃˃ with open('bacon.txt', 'a') as bacon_file: # creates the file
...     bacon_file.write('Bacon is not a vegetable.')
25

˃˃˃ send2trash.send2trash('bacon.txt')

Walking a Directory Tree

˃˃˃ import os
˃˃˃
˃˃˃ for folder_name, subfolders, filenames in os.walk('C:\\delicious'):
˃˃˃     print('The current folder is {}'.format(folder_name))
˃˃˃
˃˃˃     for subfolder in subfolders:
˃˃˃         print('SUBFOLDER OF {}: {}'.format(folder_name, subfolder))
˃˃˃     for filename in filenames:
˃˃˃         print('FILE INSIDE {}: {}'.format(folder_name, filename))
˃˃˃
˃˃˃     print('')
The current folder is C:\delicious
SUBFOLDER OF C:\delicious: cats
SUBFOLDER OF C:\delicious: walnut
FILE INSIDE C:\delicious: spam.txt

The current folder is C:\delicious\cats
FILE INSIDE C:\delicious\cats: catnames.txt
FILE INSIDE C:\delicious\cats: zophie.jpg

The current folder is C:\delicious\walnut
SUBFOLDER OF C:\delicious\walnut: waffles

The current folder is C:\delicious\walnut\waffles
FILE INSIDE C:\delicious\walnut\waffles: butter.txt

pathlib provides a lot more functionality than the ones listed above, like getting file name, getting file extension, reading/writing a file without manually opening it, etc. Check out the official documentation if you want to know more!
Reading and Writing Files
The File Reading/Writing Process

To read/write to a file in Python, you will want to use the with statement, which will close the file for you after you are done.
Opening and reading files with the open function

˃˃˃ with open('C:\\Users\\your_home_folder\\hello.txt') as hello_file:
...     hello_content = hello_file.read()
˃˃˃ hello_content
'Hello World!'

˃˃˃ # Alternatively, you can use the ºreadlines()º method to get a list of string values from the file, one string for each line of text:

˃˃˃ with open('sonnet29.txt') as sonnet_file:
...     sonnet_file.readlines()
[When, in disgrace with fortune and men's eyes,\n', ' I all alone beweep my
outcast state,\n', And trouble deaf heaven with my bootless cries,\n', And
look upon myself and curse my fate,']

˃˃˃ # You can also iterate through the file line by line:
˃˃˃ with open('sonnet29.txt') as sonnet_file:
...     for line in sonnet_file: # note the new line character will be included in the line
...         print(line, end='')

When, in disgrace with fortune and men's eyes,
I all alone beweep my outcast state,
And trouble deaf heaven with my bootless cries,
And look upon myself and curse my fate,

Writing to Files

˃˃˃ with open('bacon.txt', 'w') as bacon_file:
...     bacon_file.write('Hello world!\n')
13

˃˃˃ with open('bacon.txt', 'a') as bacon_file:
...     bacon_file.write('Bacon is not a vegetable.')
25

˃˃˃ with open('bacon.txt') as bacon_file:
...     content = bacon_file.read()

˃˃˃ print(content)
Hello world!
Bacon is not a vegetable.

Saving Variables with the shelve Module

To save variables:

˃˃˃ import shelve

˃˃˃ cats = ['Zophie', 'Pooka', 'Simon']
˃˃˃ with shelve.open('mydata') as shelf_file:
...     shelf_file['cats'] = cats

To open and read variables:

˃˃˃ with shelve.open('mydata') as shelf_file:
...     print(type(shelf_file))
...     print(shelf_file['cats'])
˂class 'shelve.DbfilenameShelf'˃
['Zophie', 'Pooka', 'Simon']

Just like dictionaries, shelf values have keys() and values() methods that will return list-like values of the keys and values in the shelf. Since these methods return list-like values instead of true lists, you should pass them to the list() function to get them in list form.

˃˃˃ with shelve.open('mydata') as shelf_file:
...     print(list(shelf_file.keys()))
...     print(list(shelf_file.values()))
['cats']
[['Zophie', 'Pooka', 'Simon']]

Saving Variables with pprint.pformat

˃˃˃ import pprint

˃˃˃ cats = [{'name': 'Zophie', 'desc': 'chubby'}, {'name': 'Pooka', 'desc': 'fluffy'}]

˃˃˃ pprint.pformat(cats)
"[{'desc': 'chubby', 'name': 'Zophie'}, {'desc': 'fluffy', 'name': 'Pooka'}]"

˃˃˃ with open('myCats.py', 'w') as file_obj:
...     file_obj.write('cats = {}\n'.format(pprint.pformat(cats)))
83

Reading ZIP Files

˃˃˃ import zipfile, os

˃˃˃ os.chdir('C:\\')    # move to the folder with example.zip
˃˃˃ with zipfile.ZipFile('example.zip') as example_zip:
...     print(example_zip.namelist())
...     spam_info = example_zip.getinfo('spam.txt')
...     print(spam_info.file_size)
...     print(spam_info.compress_size)
...     print('Compressed file is %sx smaller!' % (round(spam_info.file_size / spam_info.compress_size, 2)))

['spam.txt', 'cats/', 'cats/catnames.txt', 'cats/zophie.jpg']
13908
3828
'Compressed file is 3.63x smaller!'

Extracting from ZIP Files

The extractall() method for ZipFile objects extracts all the files and folders from a ZIP file into the current working directory.

˃˃˃ import zipfile, os

˃˃˃ os.chdir('C:\\')    # move to the folder with example.zip

˃˃˃ with zipfile.ZipFile('example.zip') as example_zip:
...     example_zip.extractall()

The extract() method for ZipFile objects will extract a single file from the ZIP file. Continue the interactive shell example:

˃˃˃ with zipfile.ZipFile('example.zip') as example_zip:
...     print(example_zip.extract('spam.txt'))
...     print(example_zip.extract('spam.txt', 'C:\\some\\new\\folders'))
'C:\\spam.txt'
'C:\\some\\new\\folders\\spam.txt'

Creating and Adding to ZIP Files

˃˃˃ import zipfile

˃˃˃ with zipfile.ZipFile('new.zip', 'w') as new_zip:
...     new_zip.write('spam.txt', compress_type=zipfile.ZIP_DEFLATED)

This code will create a new ZIP file named new.zip that has the compressed contents of spam.txt.
JSON, YAML and configuration files
JSON

Open a JSON file with:

import json
with open("filename.json", "r") as f:
    content = json.loads(f.read())

Write a JSON file with:

import json

content = {"name": "Joe", "age": 20}
with open("filename.json", "w") as f:
    f.write(json.dumps(content, indent=2))

YAML

Compared to JSON, YAML allows a much better humain maintainance and gives ability to add comments. It is a convinient choice for configuration files where human will have to edit.

There are two main librairies allowing to access to YAML files:

    PyYaml
    Ruamel.yaml

Install them using pip install in your virtual environment.

The first one it easier to use but the second one, Ruamel, implements much better the YAML specification, and allow for example to modify a YAML content without altering comments.

Open a YAML file with:

from ruamel.yaml import YAML

with open("filename.yaml") as f:
    yaml=YAML()
    yaml.load(f)

Anyconfig

Anyconfig is a very handy package allowing to abstract completly the underlying configuration file format. It allows to load a Python dictionary from JSON, YAML, TOML, and so on.

Install it with:

pip install anyconfig

Usage:

import anyconfig

conf1 = anyconfig.load("/path/to/foo/conf.d/a.yml")

Handling
modules
/projects

virtualenv
Use Cases for Virt. Enviroments:
  - test python code in encapsulated environments 
  - avoid filling the base Python installation with
    libraries we might use for only one project.

ºInstallº
$ pip install virtualenv             # Linux/...
$ pip install virtualenvwrapper-win  # Windows

ºUsage:º
$ mkvirtualenv HelloWold # ← Make new Virtual Environment
  ^^^^^^^^^^^^^^^^^^^^^^ 
  Anything installed from there one will be specific to
  the HellowWorld project and available to the projects
  we connect to this environment.

$ ºsetprojectdirº .  # ← binds current virtualenv current working dir.

$ ºdeactivateº         # ← Deactivates virtualenv

$ ºworkonº HelloWorld   # ← Reconnect to HelloWorld vir.env.

poetry
(mvn/npm/... 4 Py)
- Packaging and packagin dependency tool.

ºInstallº
$ pip install --user poetry

ºUssageº

$ poetry new my-project  # ← Create new project
Output layout:
 my-project
 ├── Oºpyproject.tomlº  ← orchestrate project and dependencies
 ├── README.rst
 ├── poetry_demo
 │   └── __init__.py
 └── tests
     ├── __init__.py
     └── test_poetry_demo.py

ºpyproject.toml exampleº
| º[tool.poetry]º
| name = "my-project"
| version = "0.1.0"
| description = ""
| authors = ["your name ˂your@mail.com˃"]
| 
| º[tool.poetry.dependencies]º
| python = "*"
| pendulum = "^1.4"              ← Edit manually or use "$ poetry add pendulum"
| 
| º[tool.poetry.dev-dependencies]º
| pytest = "^3.4"

$ poetry ºinstallº      # ← Install declared dependencies

$ poetry ºremoveº pendulum  # ← remove dependencry (or edit pryp

itertools Module
  
The itertools module is a collection of tools intented to be fast and use memory efficiently when handling iterators (like lists or dictionaries).

From the official Python 3.x documentation:

    The module standardizes a core set of fast, memory efficient tools that are useful by themselves or in combination. Together, they form an “iterator algebra” making it possible to construct specialized tools succinctly and efficiently in pure Python.

The itertools module comes in the standard library and must be imported.

The operator module will also be used. This module is not necessary when using itertools, but needed for some of the examples below.
accumulate

Makes an iterator that returns the results of a function.

itertools.accumulate(iterable[, func])

Example:

˃˃˃ data = [1, 2, 3, 4, 5]
˃˃˃ result = itertools.accumulate(data, operator.mul)
˃˃˃ for each in result:
˃˃˃    print(each)
1
2
6
24
120

The operator.mul takes two numbers and multiplies them:

operator.mul(1, 2)
2
operator.mul(2, 3)
6
operator.mul(6, 4)
24
operator.mul(24, 5)
120

Passing a function is optional:

˃˃˃ data = [5, 2, 6, 4, 5, 9, 1]
˃˃˃ result = itertools.accumulate(data)
˃˃˃ for each in result:
˃˃˃    print(each)
5
7
13
17
22
31
32

If no function is designated the items will be summed:

5
5 + 2 = 7
7 + 6 = 13
13 + 4 = 17
17 + 5 = 22
22 + 9 = 31
31 + 1 = 32

combinations

Takes an iterable and a integer. This will create all the unique combination that have r members.

itertools.combinations(iterable, r)

Example:

˃˃˃ shapes = ['circle', 'triangle', 'square',]
˃˃˃ result = itertools.combinations(shapes, 2)
˃˃˃ for each in result:
˃˃˃    print(each)
('circle', 'triangle')
('circle', 'square')
('triangle', 'square')

combinations_with_replacement

Just like combinations(), but allows individual elements to be repeated more than once.

itertools.combinations_with_replacement(iterable, r)

Example:

˃˃˃ shapes = ['circle', 'triangle', 'square']
˃˃˃ result = itertools.combinations_with_replacement(shapes, 2)
˃˃˃ for each in result:
˃˃˃    print(each)
('circle', 'circle')
('circle', 'triangle')
('circle', 'square')
('triangle', 'triangle')
('triangle', 'square')
('square', 'square')

count

Makes an iterator that returns evenly spaced values starting with number start.

itertools.count(start=0, step=1)

Example:

˃˃˃ for i in itertools.count(10,3):
˃˃˃    print(i)
˃˃˃    if i ˃ 20:
˃˃˃        break
10
13
16
19
22

cycle

This function cycles through an iterator endlessly.

itertools.cycle(iterable)

Example:

˃˃˃ colors = ['red', 'orange', 'yellow', 'green', 'blue', 'violet']
˃˃˃ for color in itertools.cycle(colors):
˃˃˃    print(color)
red
orange
yellow
green
blue
violet
red
orange

When reached the end of the iterable it start over again from the beginning.
chain

Take a series of iterables and return them as one long iterable.

itertools.chain(*iterables)

Example:

˃˃˃ colors = ['red', 'orange', 'yellow', 'green', 'blue']
˃˃˃ shapes = ['circle', 'triangle', 'square', 'pentagon']
˃˃˃ result = itertools.chain(colors, shapes)
˃˃˃ for each in result:
˃˃˃    print(each)
red
orange
yellow
green
blue
circle
triangle
square
pentagon

compress

Filters one iterable with another.

itertools.compress(data, selectors)

Example:

˃˃˃ shapes = ['circle', 'triangle', 'square', 'pentagon']
˃˃˃ selections = [True, False, True, False]
˃˃˃ result = itertools.compress(shapes, selections)
˃˃˃ for each in result:
˃˃˃    print(each)
circle
square

dropwhile

Make an iterator that drops elements from the iterable as long as the predicate is true; afterwards, returns every element.

itertools.dropwhile(predicate, iterable)

Example:

˃˃˃ data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 1]
˃˃˃ result = itertools.dropwhile(lambda x: x˂5, data)
˃˃˃ for each in result:
˃˃˃    print(each)
5
6
7
8
9
10
1

filterfalse

Makes an iterator that filters elements from iterable returning only those for which the predicate is False.

itertools.filterfalse(predicate, iterable)

Example:

˃˃˃ data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
˃˃˃ result = itertools.filterfalse(lambda x: x˂5, data)
˃˃˃ for each in result:
˃˃˃    print(each)
5
6
7
8
9
10

groupby

Simply put, this function groups things together.

itertools.groupby(iterable, key=None)

Example:

˃˃˃ robots = [{
    'name': 'blaster',
    'faction': 'autobot'
}, {
    'name': 'galvatron',
    'faction': 'decepticon'
}, {
    'name': 'jazz',
    'faction': 'autobot'
}, {
    'name': 'metroplex',
    'faction': 'autobot'
}, {
    'name': 'megatron',
    'faction': 'decepticon'
}, {
    'name': 'starcream',
    'faction': 'decepticon'
}]
˃˃˃ for key, group in itertools.groupby(robots, key=lambda x: x['faction']):
˃˃˃    print(key)
˃˃˃    print(list(group))
autobot
[{'name': 'blaster', 'faction': 'autobot'}]
decepticon
[{'name': 'galvatron', 'faction': 'decepticon'}]
autobot
[{'name': 'jazz', 'faction': 'autobot'}, {'name': 'metroplex', 'faction': 'autobot'}]
decepticon
[{'name': 'megatron', 'faction': 'decepticon'}, {'name': 'starcream', 'faction': 'decepticon'}]

islice

This function is very much like slices. This allows you to cut out a piece of an iterable.

itertools.islice(iterable, start, stop[, step])

Example:

˃˃˃ colors = ['red', 'orange', 'yellow', 'green', 'blue',]
˃˃˃ few_colors = itertools.islice(colors, 2)
˃˃˃ for each in few_colors:
˃˃˃    print(each)
red
orange

permutations

itertools.permutations(iterable, r=None)

Example:

˃˃˃ alpha_data = ['a', 'b', 'c']
˃˃˃ result = itertools.permutations(alpha_data)
˃˃˃ for each in result:
˃˃˃    print(each)
('a', 'b', 'c')
('a', 'c', 'b')
('b', 'a', 'c')
('b', 'c', 'a')
('c', 'a', 'b')
('c', 'b', 'a')

product

Creates the cartesian products from a series of iterables.

˃˃˃ num_data = [1, 2, 3]
˃˃˃ alpha_data = ['a', 'b', 'c']
˃˃˃ result = itertools.product(num_data, alpha_data)
˃˃˃ for each in result:
    print(each)
(1, 'a')
(1, 'b')
(1, 'c')
(2, 'a')
(2, 'b')
(2, 'c')
(3, 'a')
(3, 'b')
(3, 'c')

repeat

This function will repeat an object over and over again. Unless, there is a times argument.

itertools.repeat(object[, times])

Example:

˃˃˃ for i in itertools.repeat("spam", 3):
    print(i)
spam
spam
spam

starmap

Makes an iterator that computes the function using arguments obtained from the iterable.

itertools.starmap(function, iterable)

Example:

˃˃˃ data = [(2, 6), (8, 4), (7, 3)]
˃˃˃ result = itertools.starmap(operator.mul, data)
˃˃˃ for each in result:
˃˃˃    print(each)
12
32
21

takewhile

The opposite of dropwhile(). Makes an iterator and returns elements from the iterable as long as the predicate is true.

itertools.takewhile(predicate, iterable)

Example:

˃˃˃ data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 1]
˃˃˃ result = itertools.takewhile(lambda x: x˂5, data)
˃˃˃ for each in result:
˃˃˃    print(each)
1
2
3
4

tee

Return n independent iterators from a single iterable.

itertools.tee(iterable, n=2)

Example:

˃˃˃ colors = ['red', 'orange', 'yellow', 'green', 'blue']
˃˃˃ alpha_colors, beta_colors = itertools.tee(colors)
˃˃˃ for each in alpha_colors:
˃˃˃    print(each)
red
orange
yellow
green
blue

˃˃˃ colors = ['red', 'orange', 'yellow', 'green', 'blue']
˃˃˃ alpha_colors, beta_colors = itertools.tee(colors)
˃˃˃ for each in beta_colors:
˃˃˃    print(each)
red
orange
yellow
green
blue

zip_longest

Makes an iterator that aggregates elements from each of the iterables. If the iterables are of uneven length, missing values are filled-in with fillvalue. Iteration continues until the longest iterable is exhausted.

itertools.zip_longest(*iterables, fillvalue=None)

Example:

˃˃˃ colors = ['red', 'orange', 'yellow', 'green', 'blue',]
˃˃˃ data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10,]
˃˃˃ for each in itertools.zip_longest(colors, data, fillvalue=None):
˃˃˃    print(each)
('red', 1)
('orange', 2)
('yellow', 3)
('green', 4)
('blue', 5)
(None, 6)
(None, 7)
(None, 8)
(None, 9)
(None, 10)

Modules
  
alt 1                          alt 2
import random, sys, os, math   from random import *.
print(random.randint(1, 10))   print(randint(1, 10))

º__main__ º
Top-level script environment

__main__ is the name of the scope in which top-level code executes. A module’s name is set equal to __main__ when read from standard input, a script, or from an interactive prompt.

A module can discover whether or not it is running in the main scope by checking its own __name__, which allows a common idiom for conditionally executing code in a module when it is run as a script or with python -m but not when it is imported:

˃˃˃ if __name__ == "__main__":
...     # execute only if run as a script
...     main()

For a package, the same effect can be achieved by including a main.py module, the contents of which will be executed when the module is run with -m.

For example we are developing script which is designed to be used as module, we should do:

˃˃˃ # Python program to execute function directly
˃˃˃ def add(a, b):
...     return a+b
...
˃˃˃ add(10, 20) # we can test it by calling the function save it as calculate.py
30
˃˃˃ # Now if we want to use that module by importing we have to comment out our call,
˃˃˃ # Instead we can write like this in calculate.py
˃˃˃ if __name__ == "__main__":
...     add(3, 5)
...
˃˃˃ import calculate
˃˃˃ calculate.add(3, 5)
8

Advantages

    Every Python module has it’s __name__ defined and if this is __main__, it implies that the module is being run standalone by the user and we can do corresponding appropriate actions.
    If you import this script as a module in another script, the name is set to the name of the script/module.
    Python files can act as either reusable modules, or as standalone programs.
    if __name__ == “main”: is used to execute some code only if the file was run directly, and not imported.


ºsys moduleº
import sys
sys.exit()

setup.py
The setup script is the centre of all activity in building, distributing, and installing modules using the Distutils. The main purpose of the setup script is to describe your module distribution to the Distutils, so that the various commands that operate on your modules do the right thing.

The setup.py file is at the heart of a Python project. It describes all of the metadata about your project. There a quite a few fields you can add to a project to give it a rich set of metadata describing the project. However, there are only three required fields: name, version, and packages. The name field must be unique if you wish to publish your package on the Python Package Index (PyPI). The version field keeps track of different releases of the project. The packages field describes where you’ve put the Python source code within your project.

This allows you to easily install Python packages. Often it's enough to write:

python setup.py install

and module will install itself.

Our initial setup.py will also include information about the license and will re-use the README.txt file for the long_description field. This will look like:

˃˃˃ from distutils.core import setup
˃˃˃ setup(
...    name='pythonCheatsheet',
...    version='0.1',
...    packages=['pipenv',],
...    license='MIT',
...    long_description=open('README.txt').read(),
... )

Find more information visit http://docs.python.org/install/index.html.

pipenv
  
Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world. Windows is a first-class citizen, in our world.

ºInstall pipenvº
- pip install pipenv

Enter your Project directory and install the Packages for your project
$ cd my_project
$ pipenv install 

Pipenv will install your package and create a Pipfile for you in your project’s directory. The Pipfile is used to track which dependencies your project needs in case you need to re-install them.

ºUninstall Packagesº

$ pipenv uninstall 

Activate the Virtual Environment associated with your Python project
$ pipenv shell

Exit the Virtual Environment
$ exit

Find more information and a video in docs.pipenv.org.

Anaconda
Anaconda is another popular tool to manage python packages.

Where packages, notebooks, projects and environments are shared. Your place for free public conda package hosting.

Usage:
- Make a Virtual Environment

  $ conda create -n HelloWorld

  To use the Virtual Environment, activate it by:

  $ conda activate HelloWorld

  Anything installed now will be specific to the project HelloWorld

  $ Exit the Virtual Environment
  $ conda deactivate